On the Causes of Negative Impact of CET-4

Transcription

1 Sino-US English Teaching, ISSN August 2012, Vol. 9, No. 8, D DAVID PUBLISHING On the Causes of Negative Impact of CET-4 WANG Jian-mei, SUN Chun-lei Chongqing University of Technology, Chongqing, China As one of the world well-known high-stakes tests, CET-4 (College English Test Band 4) changed from being generally accepted to being widely questioned in the past 25 years. In spite of a series of reforms of CET-4, the negative impact of CET-4 remains intense. Three main reasons are probed into here. Firstly, the objective of CET-4 is ambiguous. Secondly, the test objective does not accord with teaching objective. Finally, test contents do not match well with the practical language use. To improve the present passive state of CET-4, some suggestions are given. Keywords: CET-4 (College English Test Band 4), negative impact, causes, test objective, teaching objective Introduction Impact refers to any of the effects that a test may have on individuals, politics or practices, within the classroom, the school, the educational system or society as a whole (Wall, 1997, p. 291). The impact can be positive and negative. However, Noble and Smith (1994) found that high-stakes testing could affect teachers more directly and negatively (p. 3). As one of the high-stakes tests, CET-4 (College English Test Band 4) has been developing fast and claims the largest number of test takers in the world today. The past 25 years have witnessed CET-4 being generally accepted to being widely questioned. In the first few years, it was commonly believed that CET-4 helped to enhance the college English teaching and learning greatly by triggering a strong enthusiasm of English learning among students and arousing the awareness of English education among school administrators, parents, and society. As a result, all colleges and universities appropriated more funds to improve the English teaching condition. So far, the positive impact of CET-4 was prominent (YANG, 2000; CHEN, 2004). Since the early 1990s, the negative impact of CET-4 came to be noted: Because of CET4, test-taking teaching and learning dominated college English classes in most colleges and universities; the comprehensive language skills could not be encouraged and checked, etc. (LIU, 1991; HUANG, LIU, & NI, 1996; YE & ZHAO, 1999). Taking these problems into serious consideration, the test administrators made consistent efforts to bring about positive effects on teaching and learning by means of major changes to the test in the following years: Some new subjective testing items were introduced in the middle of 1990s; speaking began to be taken into consideration; the change of result report and the cancelling of certificate; the changing weight on listening and reading, etc.. However, in spite of a series of reforms of CET-4, the negative impact of CET-4 remains intense and the criticism becomes sharp and profound (NIU, 2001; LIU, 2003; ZHANG, 2003; HAN, DAI, & YANG, 2004). What are the main reasons for it? WANG Jian-mei, professor at School of Foreign Languages, Chongqing University of Technology. SUN Chun-lei, associate professor at School of Foreign Languages, Chongqing University of Technology.

2 ON THE CAUSES OF NEGATIVE IMPACT OF CET The Causes of Negative Impact of CET-4 The impact of a high-stakes test is determined by many factors, such as, the quality of test, the features of educational system, the state of society, and even the political patterns. Furthermore, test developers play the decisive role in the effort of encouraging desirable impact of language tests by establishing right test contents, analyzing the characteristics of test-takers, and confiming explicitly the application of test results. The Ambiguous Objective of CET-4 Clarifying the objective of a language test is the premise of a validated language test. The most important consideration in the development and use of language tests is the purpose or purposes for which the particular test is intended (Bachman, 1990, p. 78). Based on McNamara s (2000) study, the process of developing a test can be described as follows: Analysis of demand for test study the limitation of conducting the test deciding test contents choosing test method trial test validation improved test conducting test (the impact of test) (pp ). As shown above, the demand of a test is the initial impetus of the appearance of a language test. The demand may come from the educational or social activities. For example, a school wants to examine students language ability or a company is to select potential employees in an international trade business. In other words, the first step of developing a test is to make clear whether the test is to check the result of teaching and learning, or to decide the language proficiency of students, or to set a kind of threshold, or to select certain talents according to their language proficiency. Therefore, the intention (objective) of the test is the foundation for developing any language test and the most direct and essential causes of the impact of the test. Once the objective of test the cornerstone shakes because of educational, social, or political reasons, the validity of the test is weakened and the negative impact of the test will become intensified. At this moment, the effort of reforming the test content and test method to improve the nature and impact of the test will be in vain, as the test is not what it originally intended to be. In fact, the negative impact of a test is largely brought about by the nature of the test. When CET-4 first appeared more than 20 years ago, its objective was to check whether college students in China had met the objective set in college English teaching syllabus after two-year learning, and it remained the same. College English is one of the compulsive courses for college students now, so a terminative and large-scale test remains the essential way to evaluate the teaching quality (JIN, 2006). Therefore, CET-4 is supposed to be a criteria-referenced test. In a criteria-referenced test the scores of test takers can clearly represent their language ability and indicate what they can do and what they cannot with the language (HAN et al., 2004). However, CET-4 is actually used as a typical norm-referenced test in terms of its application and the treatment of test scores. CET-4 is a large-scale norm-referenced test based on educational measurement technology (YANG & Weir, 1998). The most distinctive feature of a norm-referenced test is the comparison of all the test results and only a very small fraction of students can be qualified as excellent, while most of the students will be classified as middle and below group. That is to say, considerable numbers of students are positioned as fail group of CET-4 no matter how their English is and whether they have met the objective of college English teaching syllabus or not. It is this ambiguity of test that results in the misuse of the results of CET-4 by the school, society, and some administrations, eventually leading to the intense negative impact of CET-4.

3 1368 ON THE CAUSES OF NEGATIVE IMPACT OF CET-4 The Contradiction of Teaching Objectives and Testing Objective As one important part of language teaching and learning, language tests lie in every stage of teaching process. For instance, placement test is employed to decide students starting point of language learning; diagnose test is frequently used in the process of learning to find out learning difficulties or failure and to adjust teaching contents and speed; achievement tests are always conducted at the end of term to evaluate the learning results and spot the problems in language teaching and learning. Buck (1988) summarized the relationship between tests and educational goals as: Most educators would probably agree that the content of classroom instruction should be decided on the basis of clearly understood educational goals, and examinations should try to ascertain whether these goals have been achieved. When the examination does that, it forces students and teachers to concentrate on these goals, and the washback effect on the classroom is very beneficial. (p. 17) In the same way, high-stakes language tests are supposed to back up language teaching and learning. Otherwise, the negative impact of test is bound to intensify. Meanwhile, if the teaching objective contradicts with social demand, the corresponding test will bring about a negative impact to the society. As mentioned above, the objective of CET-4 is to check how well the College English Syllabus is implemented, or whether the students have met the objective of College English teaching after two-year learning. As a matter of fact, it is quite unscientific and unreasonable for more than 1,800 colleges and universities to enforce the implementation of the same syllabus regardless of the district economic difference, the diverse aims and features of school running, and the discrepant teaching resources and teaching conditions. The Ministry of Education has become aware of the irrationality of implementing the same syllabus in all schools. In 2002, the new College English Curricula Teaching Requirement came into being, which provided three levels of English teaching (general requirement, higher requirement, and the highest requirement) for each school to choose according to their practical situation. Meanwhile, the Ministry of Education encourages colleges and universities to make customized and characterized college English syllabus of their own. In recent years, some colleges have already begun reforming the traditional College English course by reducing its credits considerably and adopting flexible credit hour system. Different from the College English teaching reform, the reformed CET-4 still fails to take the differences in regional diversity, teaching resources, and students English foundation in each school. The objective of CET-4 remains checking the teaching results of all schools. There is a big gap between the concrete teaching objective of each school and the objective of CET-4. In that case, of course, the negative impact of CET-4 cannot be diminished. Poor Authenticity of Test Authenticity is an essential factor in any language testing as it may affect the validity of the test, the way of preparing the tests students may adopt, and even the psychology of students. Authenticity is an important quality for language tests for two reasons: (1) It provides a link between test performance and the TLU (target language use) tasks and domain to which we want to generalize; and (2) The way test takers perceive the relative authenticity of test tasks can, potentially, facilitate their test performance. (Bachman & Adrian, 1996, p. 39) Of course, there are some differences between the test performance and actual language use, and it is

4 ON THE CAUSES OF NEGATIVE IMPACT OF CET impossible to realize the complete authenticity of test. However, Testing is about making inferences (McNamara, 2000, p. 7). From a test with high authenticity, we can infer more accurately the students real and comprehensive language ability. Taking this test, students have to command all kinds of language skills relating to language use in real life. Therefore, test authenticity may affect directly students choice of language learning and their language knowledge. For example, in the early 1980s, College English in China was not taken into serious consideration in most colleges and universities, and the overall-language skill development was not valued among students and teachers. In English teaching and learning, great weight was put on reading, and to some extent, speaking, listening, and writing were ignored in a considerable number of schools, while in recent years, listening is weighted in CET-4 (constituting about 35%). As a result, most colleges exert great effort to help students improve their listening ability by enriching listening materials and allocating more time and money to listening practice. It can be safely inferred that the learning contents as well as learning method have much to do with the authenticity of test. With many reforms, CET-4 has not changed substantially in test authenticity. For instance, there have not been any changes in items of oral test and writing, which can well reflect students language ability. As a high-stakes test that can comprehensively test students abilities in language use (YANG, 2003), CET-4 rejects the participation of most students in oral test. Till now, only a small fraction of students who got 550 points and above (out of 710 full marks) in the paper test of CET-4 are qualified to apply for oral test of CET-4. So the construct validity of CET-4 is under question. If the test contents are not in line with the real-life language and the test results will bring great impact on students and teachers, students and teachers then tend to adopt every possible test-taking strategy to improve their test scores. Subsequently, what the students are expected to learn and master is laid aside and the negative impact of test appears. Suggestions on Improving the Negative Impact of CET-4 Redefining the Nature of CET-4 As discussed above, CET-4 should be a criterion-referenced test in view of its original aim of development. However, based on the way of processing test scores and the use of test results, it is actually a norm-referenced test. Students, teachers, schools, and society are more concerned about the student s position in the norm. The test administration focuses on the test validity and discrimination, pursues the beautiful normal distribution pattern, and selects the top students leaving the majority of test takers failed (about 30% of test takers can pass CET-4 every time) (CAI, 2007). It is unfeasible for such a test to be designed on account of various teaching contents in all schools. Therefore, a solution for test administration is to reflect on the nature of CET-4 and to make the test objective clear. Another solution is to adapt CET-4 into a proficiency test. Then the conflict between CET-4 and College English teaching and learning at different schools can be diminished, and students can freely choose to take the test or not. What is more, each school may set its objective of English teaching and learning according to the school-running target and the students real situation. As a result, a desirable state of a hundred flowers blossom and a hundred schools of thought contend may come true in College English teaching in China.

5 1370 ON THE CAUSES OF NEGATIVE IMPACT OF CET-4 Developing a Variety of Language Tests and Providing Multiple Choices Every year there are millions of students signing for CET-4. Yet, the intention of taking test does not necessarily correspond to the aim of the test. Some students are forced to take the test, because passing the test is a must to get a diploma in their school. Others take CET-4 to demonstrate their language ability so that they may have an edge in the future job market. Because there is no alternative national English test that is widely recognized at college, all college students take CET-4 repeatedly and blindly. If more language tests with high validity and reliability and designed for various needs are available, students can make a choice in taking the test and the future employers can make use of the different test results according to their expectation of the students language skills. In this way, the misuse and abuse of the results of CET-4 can be effectively limited, and the negative impact can be controlled. Presently, many colleges and universities begin to put the students needs and social demands at the top in reforming College English teaching and learning. A variety of English courses are presented for students, such as, Basic English, English Literature, ESP (English for Special Purpose), etc.. Obviously, the traditional CET-4 cannot meet the demands of English teaching and social needs. It is advisable for test administration to offer a series of College English tests for students to choose. In this way, the test can serve for language teaching and learning and bring about a positive impact. Lessening Administrative Control Over CET-4 The developers of language tests try to improve the test quality and enhance the positive impact of tests, while Educational authorities and politicians can be seen as responsible for the nature of washback, because tests are frequently used to engineer innovation, to steer and guide the curriculum (CHENG, Watanabe, & Curtis, 2004, pp ). The power of language tests is well recognized by many policy makers in the world and the tests are used to impel educational reform and to impose new textbooks and teaching modes. Testing is the darling of the policy-makers (Madaus, 1988, pp ). Although it is helpful to push the reform of language teaching and learning by means of introducing high-stakes language test, its power is quite confined and the result is less endurable. When a new test is introduced into an educational context as a mandate and as an accountability measure, it is likely to produce unintended consequences (CHENG & Couture, 2000, pp ). We can learn a good lesson from the similar practice in Hong Kong, North American, England, and Australia (CHENG et al., 2004, pp ). What is more, this practice is a kind of psychometric imperialism, by which tests become the ferocious master of the educational process (Madaus, 1988, pp ). Without the active participation of teachers and students, the good intention of tests can hardly be realized. CET-4 is no exception. The appearance, development, and reform of CET-4 are an administrative decision to a great extent. The authority should be well aware of the power and limitation of high-stakes language tests and be cautious in making use of them. Conclusions Large-scale and high-stakes language testing is bound to exert great positive and negative impacts on students, teachers, educational institutions, society, and so on, and always the negative impact is more obvious. Therefore, test developers have to take the impact into consideration in every step of test developing. First of all,

6 ON THE CAUSES OF NEGATIVE IMPACT OF CET the objective of test and the limitation of result using should be specified. When the scope of the application of test results expands, the original test objective varies accordingly. So the initial purpose of test has been changed. At this moment, the effort to improve the impact of test by modifying test content or test method is always ineffective. In addition, the test contents should accord to teaching objective and real-life language use. Otherwise, the high-stakes test is subject to being used as a lever for the administrator to control language teaching. In this case, teachers are always not willing to be dominated and the ideal expectation of administrators to reform language teaching through testing cannot be realized. On the contrary, the negative impact of testing will be exaggerated. The development of CET-4 in the past 25 years has proved that it may not always be a sound way to propel college English reforming by employing high-stakes testing. An effective way of reducing the negative impact of testing is to develop various reliable and validating language tests in accordance with the requirements of students and society. Meanwhile, the government, society, and school should be well aware of the complexity of testing impact and make use of language testing as a lever with care and scientific mind. Some of the approaches of promoting positive backwash could be quite costly and would therefore violate the test evaluation criterion of practicality. A case in point is the speaking test of CET-4. But as Hughes (1989) pointed out: Before we decide that we cannot afford to test in ways that will promote beneficial backwash, we have to ask ourselves a question: What will be the cost of not achieving beneficial backwash?. When we compare the cost of the test with the waste of effort and time on the part of teachers and students in activities quite inappropriate to their true learning goals (and in some circumstances, with the potential loss to the national economy of not having more people competent in foreign languages), we are likely to decide that we cannot afford not to introduce a test with a powerful beneficial backwash effect. (p. 47) References Bachman, L. F. (1990). Fundamental consideration in language testing. Oxford: Oxford University Press. Bachman, L. F., & Adrian, S. P. (1996). Language testing in practice. Oxford: Oxford University Press. Buck, G. (1988). Testing listening comprehension in Japanese university entrance examinations. JALT Journal, 10, CAI, J. G. (2007). On the features of college English in China in the transitional period and the countermeasures. Foreign Language Teaching and Research, 1, CHEN, Z. F. (2004). Facts should be valued in academic discussion Discussion with Prof. Liu Run-qing on CET4/6. Foreign Language World, 1, CHENG, L., & Couture, J. C. (2000). Teachers work in the global culture of performance. Alberta Journal of Educational Research, 46(1), CHENG, L., Watanabe, Y., & Curtis, A. (Eds.). (2004). Washback in language testing Research contexts and methods. New Jersey: Lawrence Erlbaum Associates. HAN, B. C., DAI, M. C., & YANG, L. F. (2004). On the problems of college English tests in view of a survey. Foreign Language Teaching and Research, 2, HUANG, Z. L., LIU, L. F., & NI, C. B. (1996). Study and analysis on the new-added testing item of translation in CET-4. Foreign Language Teaching and Research, 3, Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press. JIN, Y. (2006). Promoting test validity and improving its washback. Foreign Language World, 6, LIU, R. Q. (2003). On the reform of college english teaching and learning Conversation by writing No. 3. Foreign Language Teaching and Research, 5, 378. LIU, X. M. (1991). Pondering over the writing test item in CET-4/6. Foreign Language Teaching and Research, 3, Madaus, G. F. (1988). The influence of testing on the curriculum. In L. N. Tanner (Ed.), Critical issues in curriculum: Eighty-seventh yearbook of the national society for the study of education (pp ). Chicago: University of Chicago Press.

7 1372 ON THE CAUSES OF NEGATIVE IMPACT OF CET-4 McNamara, T. (2000). Language testing. Oxford: Oxford University Press. NIU, Q. (2001). The problems of English tests at present colleges and universities. Foreign Language Teaching and Research, 2, Noble, A. J., & Smith, M. L. (1994). Measurement-driven reform: Research on policy, practice, repercussion (Tech, Rep. 381). Tempe, A.Z.: Arizona State University, Center for the Study of Evaluation. Wall, D. (1997). Impact and washback in language testing. In C. Clapham & D. Corson (Eds.), Encyclopedia of language and education, language testing and assessment (Vol. 7, pp ). Dordrecht: Klower Academic. YANG, H. Z. (2000). On Chinese college students practical English ability in view of CET-4/6. Foreign Language World, 1, YANG, H. Z. (2003). Review of CET-4/6 in the past 15 years. Foreign Language World, 3, YANG, H. Z., & Weir, C. (1998). Study on the validity of CET-4/6. Shanghai: Shanghai Foreign Language Press. YE, G. Q., & ZHAO, J. (1999). Conceiving the improvement of test items and system in CET-4/6. Foreign Language Teaching and Research, 4, ZHANG, Y. X. (2003). On the reform of college English for non-english majors. College Teaching in China, 7, 1-4.