Some global trends in assessing language proficiency John Read University of Auckland
Overview of the presentation High stakes proficiency testing Comparing IELTS and TOEFL Ongoing issues in proficiency testing New approaches to assessment Alternatives to standardised tests Outcomes-based assessment frameworks National ESOL standards for New Zealand?
The two dominant tests The Test of English as a Foreign Language (TOEFL) USA 1964 Educational Testing Service The International English Language Testing System (IELTS) UK-Australia 1989 Cambridge ESOL, British Council, IDP Education: IELTS Australia
Criticisms of the old TOEFL Too much focus on grammar and vocabulary. Based around the multiple-choice format. Rewards intensive coaching in test-taking skills. Not a realistic measure of academic language skills. Has no required speaking test.
The corresponding virtues of IELTS A communicative test, involving: the routine testing of all four skills, including direct assessment of speaking. test tasks designed to simulate genuine academic study tasks. the inclusion of longer texts for testing reading. the use of a variety of item types.
A major new test Internet-based TOEFL (ibt) launched in the US in September 2005 currently being rolled out around the world Brisbane April Sydney ] May Melbourne ]
Technological innovation in ibt Fully computer-based test. Delivered through the internet. Digital recording of speaking responses. Online reporting of test scores.
Substantive innovation in ibt A fundamental rethink of the construct of academic language ability An explicit basis for score interpretation (through competency descriptors) Assessment of all four skills, including integrated tasks: listen speak New types of item read listen write
Ongoing issues in proficiency testing WRITING ASSESSMENT handwritten vs. word-processed responses access to a dictionary and other reference sources independent vs. integrated tasks rater training and the prospects for automated assessment
Ongoing issues in proficiency testing SPEAKING ASSESSMENT semi-direct vs. direct tests: advantages and disadvantages the Cambridge ESOL paired format: two examiners and two candidates rater training and automated assessment
Testing and assessment: what s the difference? THREE DEFINITIONS 1. Testing and assessment are essentially interchangeable terms. 2. Assessment is the cover term, with tests as one form of assessment. 3. Testing and assessment as alternative (competing) approaches to the evaluation of achievement and proficiency.
The limits of standard proficiency tests How do we adequately assess the ability of learners in dealing with: the researched essay the seminar presentation seminar or tutorial discussion the multimedia lecture?
Bringing assessment back to the classroom Assessment on the basis of authentic communicative tasks set by teachers, including various forms of alternative assessment: observation checklists and other recordkeeping self-assessment by learners peer assessment conferencing portfolios
The formative-summative dilemma Good classroom assessment provides valuable formative evidence to promote effective learning but then how can that same information be used to meet requirements for external reporting and accountability?
Outcomes-based assessment systems AUSTRALIA NLLIA ESL Bandscales Certificate in Spoken and Written English (CSWE) CANADA Canadian Language Benchmarks EUROPE The Common European Framework of Reference (CEFR) for Languages
Some problems with frameworks Lack of a strong theoretical basis for the framework. Multidimensional language development represented on essentially a one-dimensional scale. Limited consultation with teachers and support from them. Significant workload demands on teachers. Inadequate resourcing for moderation of standards. (based on Brindley, 1998; 2001)
A New Zealand ESOL framework? Establishing National Standards for English Language Levels in Tertiary Institutions An export education levy project funded by Education New Zealand in 2004 and 2005-06 Researchers: John Read and David Hirsh Year One Report available at: www.educationnz.org.nz/eeidf/reports/e4.pdf
Purposes for national standards To assess English proficiency for admission to tertiary academic study. To measure learner progress in multilevel English language programmes. To report the English level of students transferring to a different school. To assess learners for the award of a certificate of achievement. To diagnose continuing language support needs after tertiary admission.
Year One project activities Review of the literature on language proficiency assessment. Interviews with language school staff on assessment needs and reactions to the concept of national standards. Formulation of recommendations on the feasibility of a national framework.
Year One recommendations The development of national standards should take a bottom-up approach. Standards should be designed to complement the strengths of existing assessments and qualifications in the tertiary ESOL sector. A major focus should be the key decision points along the pathways into academic study programmes in tertiary institutions. The assessment procedures should include tests of language knowledge and of academic study skills as well as more open-ended and integrated academic study tasks.
The way forward? THE CURRENT YEAR TWO PROJECT (2005-06) Strand One The feasibility of moderating English standards for university entry among Foundation Studies programmes. Strand Two The development of text-based tests of language knowledge to measure levels and progress
Some broad conclusions The ibt has made some notable advances in technology and test content, without transcending the inherent limitations of a large-scale standardised proficiency test. Outcomes-based assessment frameworks show considerable promise but there are daunting challenges involved in designing, implementing and resourcing them adequately.
John Read ja.read@auckland.ac.nz