~1: 15 /;' J~~~~c...:;.--:.. I. ~ffi ~I J) ':~

~1: 15 /;' J~~~~c...:;.--:.. I ~ffi ~I J) ':~ _

Making CET Writing Sub-test Communicative A Thesis Presented to The College ofenglish Language and Literature Shanghai International Studies University In Partial Fulfillment ofthe Requirement For the Degree of Master of Arts By Yan Zhenhui Under the Supervision of Associate Professor Xu Qiang December 2003

Acknowledgement I would like to thank Associate Prof. Xu Qiang, my supervisor, without whose persistent instructions, encouragements and kindness, this paper would not have been possible. I would also like to thank Prof. Zou Shen, whose profound knowledge and inspiring lectures have aroused my interest in language teaching and testing. And my gratitude will also go to my former college classmate, Miss. Song, whose generous help has endowed me with the opportunity to carry out the relevant research in this dissertation. I also feel obliged to thank all the teachers and students from Hefei Industrial University who have sacrificed their precious time to work on my questionnaires or test papers. Finally, I would like to thank all the teachers that have instructed me. Besides my supervisor Associate Prof. Xu Qiang and Prof. Zou Shen, there are Prof. Mei Deming, Prof. Wang Tongfu, Prof. Chen Jianlin and Prof. LU Guangdan, whose instructions have benefited me a great deal and will benefit my whole life.

Abstract For quite a long time our teaching of writing has not seen much improvement in students' writing performance in the writing sub-test of College English Test (CET). Probably there are many reasons accounting for this, but a fundamental one can be tracked down to the writing sub-test itself. In order to bring about some changes to such a stagnant situation, researchers at home and abroad have proposed suggestions in revising either the way of constructing the task and test rubrics, or the constitution of the marking scale. But until now no significant breakthrough has been made due to a lack of revolutionary changes to the nature of the sub-test. In this paper, I try to update all aspects concerning a writing sub-test and propose a way to make CET writing sub-test communicative. In this dissertation, I first reviewed the characteristics of a communicative writing task and test rubrics, and then by fully considering the situations of Chinese students, I put forward a few example tasks for the future writing sub-test. After that I expounded to great length on how my suggested communicative task was of more advantages to the present one. In the end, I substantiated my conclusion by the data from the questionnaire survey. Furthermore, I think the evaluation system must be accordingly re-devised if a better assessment of candidates' communicative competence is expected. Based on Bachman's communicative language competence model, I designed a new set of marking scale for the proposed writing task. The new scale, in common with the marking scale of IELTS writing sub-test in many aspects, took full account of Chinese characteristics in teaching and learning writing, which resulted in the 1

weighted holistic and analytical scoring in the implementation of the new scale. Subsequently an experiment was conducted by asking a class of sophomores to finish two test papers, CET-6 in June 2000 and an authentic IELTS, with the writing sub-test of the latter substituted by the one I put forward in this paper. The high correlation coefficient between my communicative writing sub-test and the objective sub-tests of IELTS suggested that the newly devised marking scale was more efficient in assessing candidates' communicative competence than the existing one in CET writing sub-test. Finally, the suggested CET writing sub-test in this dissertation was expected to exert good backwash effects on the teaching and learning of writing. For example, it would bring a new perspective to teachers of English writing as to how to teach writing as communication and how to select and use appropriate teaching materials. Hopefully it would lead to the improvement of the students' communicative performance in the writing and make them better equipped for the later real-life communication writing activities. 2

摘 要 长 期 以 来, 写 作 教 学 没 有 能 够 提 高 学 生 在 写 作 测 试 中 的 表 现 导 致 这 种 现 象 的 原 因 可 能 很 多, 但 是 写 作 测 试 的 不 合 理 是 其 中 的 基 本 原 因 为 了 改 变 学 生 写 作 能 力 停 滞 不 前 的 状 况, 国 内 外 许 多 学 者 在 写 作 任 务 考 试 指 令 的 编 写 方 式 方 面, 或 者 评 分 标 准 的 内 容 方 面 提 出 了 改 革 建 议 但 是 至 今 实 践 中 还 无 明 显 突 破 本 文 中, 作 者 试 图 将 以 上 儿 方 面 改 革 结 合 起 来, 把 我 国 的 大 学 英 语 考 试 的 写 作 测 试 变 为 交 际 性 写 作 测 试 文 中 作 者 首 先 评 论 了 交 际 性 写 作 任 务 和 指 令 的 特 点, 然 后 结 合 中 国 学 生 的 实 际, 举 出 了 一 些 可 适 用 于 将 来 大 学 英 语 考 试 写 作 测 试 的 写 作 任 务 实 例 接 着, 详 细 解 释 这 些 交 际 性 写 作 任 务 是 如 何 优 越 于 传 统 写 作 任 务 的 最 后, 作 者 从 问 卷 调 查 中 得 到 的 数 据 证 实 了 她 的 结 论 作 者 认 为 为 了 更 好 地 评 估 学 生 的 交 际 性 写 作 能 力, 我 们 必 须 制 定 相 应 的 交 际 性 评 分 标 准 作 者 在 巴 克 曼 交 际 语 言 能 力 构 想 基 础 上, 借 鉴 了 " 雅 思 " 的 写 作 评 分 标 准, 同 时 在 充 分 考 虑 到 中 国 写 作 教 学 的 特 点 的 情 况 下, 最 终 设 计 了 一 套 以 加 权 算 分 和 整 体 分 析 评 分 相 结 合 为 特 征 的 新 型 交 际 性 评 分 标 准 随 后 作 者 让 大 学 二 年 级 一 个 班 的 学 生 分 别 完 成 2 0 00 年 6 月 的 六 级 试 题 和 一 份 雅 思 全 真 题, 后 者 的 写 作 测 试 部 分 换 成 本 文 中 作 者 建 议 的 交 际 性 写 作 命 题 方 式 考 试 指 令 和 评 分 标 准 交 际 性 写 作 成 绩 和 雅 思 的 客 观 题 成 绩 之 间 的 高 相 关 表 明 作 者 建 议 的 评 分 标 准 能 更 有 效 的 评 估 考 生 在 写 作 测 试 中 表 现 出 来 的 交 际 能 力 大 学 英 语 考 试 的 写 作 测 试 向 交 际 化 方 向 改 革, 必 然 对 写 作 教 学 产 生 积 极 的 反 拨 作 用 比 如 它 将 促 使 我 国 英 语 教 师 从 一 个 新 的 视 角 来 看 待 写 作 教 学 ( 准 确 地 说 是 交 际 性 写 作 教 学 ) 写 作 教 材 的 编 写 和 使 用 等 方 面 的 问 题 它 还 有 望 改 进 考 生 在 改 革 后 的 大 学 英 语 考 试 写 作 测 试 中 体 现 出 的 交 际 能 力, 为 他 们 现 实 生 活 中 的 交 际 活 动 做 好 准 备

Contents Introduction,. 1 1 Current situations of CET writing sub-test and its impact on the teaching and I earmng. f... 4 0 writing. 1.1 Basic information of CET 4 1.2 Requirement of CET writing sub-test 6 1.3 Review of candidates' poor performance in CETwriting sub-test 8 1.4 Impact on the teaching and learning of writing 10 2 Literature review of relevant research 14 2.1 The writing task and test rubrics " 14 2.1.1 Single style of controlled writing '" 14 2.1.2 Length requirement and time limit. 16 2.1.3 Low weight for writing 20 2.1.4 Familiarity with the marking scale 20 2.2 Marking scale 21 2.2.1 Holistic scoring 22 2.2.2 Aspects of consideration for marking 23 2.2.3 Same scale for CET-4 & 6 25 2.3 Problems solved and to be solved 27 2.3.1 Improvements in action 27 2.3.2 Problems still in existence 27 2.3.3 Making CET writing sub-test communicative 28 1

3 Communicative language teaching and testing 30 3.1 Communicative competence 30 3.2 Communicative language teaching 33 3.3 Communicative language teaching of writing 34 3.4 Communicative testing 36 3.5 Communicative testing of writing 38 3.5.1 Situation and context. 39 3.5.2 Style and register. 40 3.5.3 Needs analysis 42 3.5.4 Test rubrics & weight, 46 3.5.5 Marking scale 47 4 Research methodology of the communicative writing task, test rubrics and marking scale 49 4.1 My suggested communicative writing task and test rubrics.49 4.1.1 Examples 49 4.1.2 Characteristics 50 4.2 Questionnaires 51 4.2.1 Goal and expected conclusion 51 4.2.2 Materials, participants, and questionnaires 51 4.2.3 Statistical analysis '" 52 4.2.4 Detailed analysis of example essays 54 4.3 My suggested communicative marking scale and its characteristics.58 4.3.1 Hybrid of analytical and holistic scales 60 4.3.2 Weight determination and its principles 63 4.3.3 Qualitative mode of assessment. 65 2

4.4 Experiment on the validity of the communicative marking scale 65 4.4.1 Theory foundation and expected conclusion 66 4.4.2 Relevant issues in marking 67 4.4.3 Marking example essays 69 4.4.4 Statistical analysis 76 4.5 Practical issues in a large-scale writing sub-test.. 82 4.5.1 Reform of teaching and testing syllabuses 83 4.5.2 Manual marking process 83 4.5.3 E-rater system 85 5 Effects of the communicative writing sub-test on the teaching and learning of writing 87 5.1 Students' attitudes 87 5.2 Teachers' responses 88 5.3 Teaching materials 89 5.4 Teaching methods 91 Conclusion '" 93 Appendix I: Marking scale of CET writing sub-test. 95 II: Marking scales of TWE of TOELF and IELTS writing sub-test. 96 III: Questionnaires '" 100 IV: Score report of my communicative writing task 102 Bibliography I. 103 Bibliography 11.......106 3

Making CET Writing Sub-test Communicative

Introduction Communicative language teaching and testing has become very popular in the past two decades in the field of language teaching and testing. Views on language have changed from being a system of symbols to a tool for communication. This brings about a growing interest in the notional-functional syllabus and the communicative language teaching, as well as some new perspectives on language testing and new considerations on the methods of evaluating the performance of second or foreign language learners. As a result, the communicative language testing came into being. Though inevitable, changes in the teaching syllabus, teaching method and testing format, content and even technique do not take place at the same time and at the same rate. For example, the communicative language teaching has obtained considerable popularity both in China and the world, whereas the communicative language testing has not been so widely accepted and operated. Even the American TOEFL (Test of English as a Foreign Language), which enjoys the world fame, will not be revised towards testing communicative competence until the year of 2005. For Chinese students, worse still, the most famous nation-wide standardized English test, College English Test (CET), has remained its focus on language structures, resistant to the change towards communicative nature. It cannot be denied that, CET, a test that is the largest in its scale and the most influential in its impact in China, has pushed forward English education in China for nearly two decades, and has accordingly brought benefits to the society from both aspects of talents cultivation and economic development. For example, CET listening sub-test has undergone significant changes from structural test items, such 1

as phonemediscrimination, to existing multiple-choices whose contents are more or less based on students' needs in listening in their academic and social life. Such changes in the format and content of listening sub-test have exerted good effects on the listening teaching, and resulted in the improvement of listening performance among college students. However, not all the sub-tests of CET experience similar changes at the same time, especially the writing sub-test, which undergoes little reform over the years and lags far behind the world fashion. With the demand for practical use of English more urgent, in both receptive and productive skills, I think it is high time to make a breakthrough in the writing sub-test as well. On the other hand, the reform will hopefully solve the problems existing in the current teaching and learning of writing, thus improving students' competence of using English in written situations. Therefore, it is suggested to make CET communicative. This is in accordance with the College English Syllabus', which says the aim of English teaching is to foster students' communicative competence. However, when some universities adopt communicative language teaching to meet the target, they are discouraged to see that CET fails to evaluate correspondingly. In order to obtain a high passing rate, some universities even go to the extreme by adopting the test-oriented teaching, in which students' communicative competence is ignored to a great extent'. For a long time, CET writing sub-test has been over-confidently regarded as a IDong Yafen. (eds.) 1991. College English Syllabus: For Students ofartsand Sciences. Shanghai: Shanghai Foreign Language Education Press: 9. ~~,"*$.*~.~.~~.*~W*fim~~OO$~~~~.~~CM~**~*$~$*» 2002/5. 2

communicative test by some teachers or researchers simply because it is a subjective test. In fact, this is a wrong concept for the following two reasons: 1) its writing task is not authentic and 2) its marking scale is not grounded on any communicative language competence model. Furthermore, the defects resulted from the above two points have exerted negative effects on the teaching and learning of writing. If this situation continues, the teaching of writing will remain the weakest area in the teaching of four skills. Therefore, I perceive the possibility of constructing a writing sub-test which will exert positive influence on writing teaching in China, especially in areas ofteaching goals, teaching methods, materials selection, requirement of basic linguistic competence and practice of communication tasks. This is how this dissertation comes into being. I endeavor to put forward a communicative writing sub-test for CET and judge whether it has high validity and reliability, whether it is practical and whether it has expected good backwash to the teaching of writing. By asking students to accomplish my suggested communicative writing task under the new test rubrics and by evaluating their performance based on the newly designed marking scale, I intend to prove that my suggested writing sub-test is suitable and beneficial to the teaching and learning of writing, a fact that is of paramount importance to any revision plan to CET. 3

1 Current situations of CET writing sub-test and its impact on the teaching and learning of writing 1.1 Basic information of CET First initiated in 1987, College English Test (CET) is a nation-wide test designed to measure whether college students have reached the level specified by the College English Syllabus, so as to ensure its implementation. According to the College English Syllabus, CET should be communicative in nature. As the Syllabus stipulates, "Language is a tool for communication, and the ultimate aim of language teaching is to foster students' ability to communicate both orally and through written channels. Language teaching should not only aim at the development of the students' linguistic competence but also at the development of their communicative competence. Importance should be attached both to language training at sentence level and to the gradual improvement of students' communicative competence at discourse level."! Therefore, there is no doubt that CET should be a communicative test to judge whether students have developed their communicative competence. However, in reality it is not. The paper-and-pencil part of CET consists of five sections: "listening comprehension", "reading comprehension", "grammar and vocabulary", JDong Yafen. (eds.) 1991. College English Syllabus: For Students ofarts and Sciences. Shanghai: Shanghai Foreign Language Education Press: 9. «*~~m~~*~» ~iliffm. 1999. «*~~i:g~~*~[f~it*] (~~~t3t*~ffl)). J:i4JJ: J: i4jj9rm~lltl.ll\lif±; ~tj?-: ~~~lltl.ll\lif±: 1. 4

"cloze"(cet-4) or "error correction"(cet-6) and "composition" 1. The Spoken English Test (SET) of CET includes "questions and answers", "monologue" and "discussion"z. Judging from the constitution of the test, I conclude that CET is basically a structural test, which breaks the complexity of language down into "isolated segments in the test and the test depends entirely on the assumption that knowledge of elements of a language is equivalent to knowledge of the language'". For example, CET not only tests listening, speaking, reading and writing separately, but also tests grammar and vocabulary in isolation. Language ability is evaluated without referring to any specific use to which it might be put in real life. Furthermore, many listening and reading multiple-choices do not evaluate directly candidates' language use in real situations, although they improve the reliability ofscoring. In this sense, CET can be categorized mostly as a system-referenced, indirect and discrete-point test, though it later adds SET, which, unfortunately, applies only to a small percentage of the test takers. Therefore, we have to make CET communicative. This does not mean that language accuracy will be sacrificed for language fluency and appropriateness. Instead, all of them will be evaluated in a communicative test. With the expansion of global communication, language use becomes more and more important. Thus, our test should not only evaluate language accuracy but also language fluency and l*~~mll!l, /\~;f:$fle~-ut~ih.!l., 2000, «*~~mv.!l (7\) ~~-ut*~.&ffjm», J:#ij:: J:#ij:;.~mf{ 1nil/tOCU: 2. 2~OO*~~i*ll!l, 7\~~-ut~JJ!l~, 2002, «*~~i'alm, 7\~:t--utlJi*~-utJJj~;9:~-ut~~~», J:#ij:: J: #ij:;'~*f{1h/j /tocu: 2. 3Morrow, K. 1979. 'Communicative language testing: revolution or evolution?' in C. 1. Brumfit and K. Johnson(eds.) The Communicative Approach to Language Teaching. Oxford: Oxford University Press: 143-57. 5

appropriateness. As a communicative test also attempts to obtain four profiles of a learner's communicative performance in English, my concern in this dissertation is how to make CET writing sub-test communicative. The reason why I chose to write on this topic is because CET writing sub-test has long been neglected or over-optimistically regarded as communicative. As I perceive, however, whether a writing sub-test is communicative or not depends heavily on what the writing task is and how it is evaluated. Considering the current situations in the CET writing sub-test, I conclude that the sub-test has a long way to go before it is finally entitled to be communicative. 1.2 Requirement of CET writing sub-test In the writing sub-test of CET, candidates are required to write a short essay of 100-120 words in 30 minutes in CET-4 1 and no less than 120 words in CET-6. From June 2002, the length was increased from 120 to 150 words in CET-6. The sub-test covers 15% of the full mark. In a typical writing sub-test, the topic, occasionally accompanied by a graph, constitutes the whole information about the writing task. Apart from the time and length requirements in the test rubrics there is an outline of the essay, with the topic sentence of each paragraph given in either English or Chinese. la~~ilhm, 7'\~:j:ff-ttE~iitW:iHll.. 2000, «A~~i1tlLY (7'\) ~~iita~&tfr2i», J:itlJ: J:itlJ~'Hft~ ~:±lwh±: 4. 6