Illinois Institute of Technology Chicago, U.S.A.

Transcription

1 Chicago, U.S.A. 18th Annual Colloquium on International Engineering Education Presenters: Dr. Vanita Misquita, Chicago Mr. Philippe Marc ENSEA, France Nov , NYC Graph 1

2 OUTLINE Criteria used by universities when choosing types of English tests for admission decisions Equivalency tables -Is one equivalency table good enough? Criteria to be used when setting cutoff or cut scores

3 Criteria among universities worldwide with regard to English language requirements Schools may focus on section scores AND overall scores of an English language test when making admission decisions across levels or for certain levels/fields Some schools have different score requirements across departments/fields of study and levels Some schools do not provide an overall minimum score for applicants and may require their students to take their own English assessment test Some schools value one test over others due to ease of accessibility and administration, proven reliability perhaps, and cost e.g. strong reliance on the TOEIC among a large majority of Grandes Ecoles ( specialized in Engineering/Technology) in France

4 Common European Framework Reference (CEFR) or (CEF) First published by the Council of Europe in 2001 Considered to be the most influential language framework in the field of 2 nd language learning Designed as a generic reference document as its name applies Descriptors developed during a large research project in Switzerland Descriptors provide a comprehensive description of objectives, learners can hope to achieve at different levels of language proficiency Descriptors are positive and context-free not specific to a language or a context Primarily used in Europe and Turkey not so common in the U.S. & other parts of the world There is no, ONE authorized interpretation of the CEFR A tool for the progression of language proficiency through 6 main levels and where the person stands in relation to that level

5 Difficulty in translating some of the terms for CEFR or CEF ( General branching in 3 broad divisions: A B C Basic User Independent User Proficient User A1 Breakthrough A2 Waystage B1 Threshold B2 Vantage C1 Effective Operational Proficiency C2 Mastery

6 CEFR or CEF levels C2 Mastery e.g. conveys finer shades of meaning with accuracy C1 Effective Operational Proficiency e.g. uses expressions with little search B2 Vantage e.g. maintains conversation with fluency without strain on either party B1 Threshold e.g. follows main points of a discussion around him/her can bring up a new subject A2 Waystage e.g. uses polite forms of greetings, react to news, sort exchanges A1 Breakthrough e.g. answers simple questions

7 CEF or CEFR Descriptors

8 TOEFL ibt & CEFR TOEFL ibt total CEFR 95 C1 72 B2 42 B1 - Found a connection between English proficiency in the TOEFL and skills mentioned in the CEFR - Standard setting method that is subjective based on input of experts ( 23 educators from 16 countries specialized in ESL/EFL) - Based on a survey of top-ranking universities by THE 83-U.S., 13 in the U.K, 14 Canada; 7 -Australia - Should not be used as a one-time tool - Evolves and changes depending on needs - More research on section scores and their relation needed -

9 ETS equivalency Table TOEFL ibt & CEFR Lowering the score to meet the B2 level reducing falsenegative decisions ( denying a qualified student admission) It is important to look at one s minimal requirements to function in an academic setting

10 Descriptions of the band scores IELTS 9 Band scores: 7 Good User - Have an operational command of the language with only occasional unsystematic inaccuracies & inappropriate usage May misunderstand some things in unfamiliar situations 6 Competent User - Effective command despite some inaccuracies, inappropriate usage or misunderstandings Can use fairly complex language in familiar settings 5 - Modest User - Can cope with overall meaning in most situations - Partial command of the language - Can handle basic communication in one s own field

11 IELTS Writing Band Descriptors task achievement; coherence & cohesion; lexical resources: grammatical range & accuracy) Example: Score of 6 - addresses ideas coherently uses an adequate range of vocabulary uses a mix of simple & complex sentences Speaking assessment df (fluency & coherence: lexical resources; grammatical range & accuracy & Pronunciation) Example: Score of 6 - willing to speak at length, may lose coherence due to hesitation generally paraphrases successfully range of pronunciation with mixed control*

12 ETS equivalency Table TOEFL ibt & IELTS Based on analysis of 1,153 individuals who took both the IELTS and the TOEFL ibt Does not endorse using the table to calculate a cutoff or cut score ( threshold) Score comparisons showed that most of the individuals scores in the middle to the high range on both tests ( Sample bias?) Recommend that institutions study the two tests and set their own cutoff or cut score appropriate to meet their needs

13 TOEFL ibt ETS equivalency Table TOEFL ibt & IELTS IELTS

14 Equivalency table used ETS ETS Framework of Reference Table TOEFL ibt & IELTS TOEFL ibt IELTS Section score comparisons also taken into account - Do you agree that a score of 70 in the TOEFL correspods to a 6.0 in the IELTS?

15 IELTS Equivalency Table for IELTS & CEFR ( CEFR C C B B1 N.B. - Based on the threshold of each particular CEFR level - IELTS band scores average score rounded up to the next band, except for a score of.10 Question To what CEFR Level does a score of 6.1 or 6.25 correspond?

16 Score of 80 and 90 on the TOEFL How do they compare? TOEFL 80 & 90 compares with IELTS 6.5 TOEFL 80 & 90 compares with CEFR C1+ TOEFL 72 is the cut for CEFR B2 New decision by ETS to lower the cut from 82 to 72 to correspond to the B2 level after research

17 Clarity English Equivalency Table ( - Provides teaching/study materials as part of a wider course of study - Focusses on four skills grammar, reading & study skills - In existence for the past 20 years - Relies on the preparation of Information and Communication technology ( ICT) packages to help with training in a variety of settings - Has developed Road to IELTS to help students better prepare for the IELTS - Not much information about how they arrived at their equivalency table - Not necessarily restricted to university education

18 Which table is more reliable?

19 3 comparative charts & where they deviate Clarity English B2 = IELTS ; TOEFL ibt C1 = IELTS; TOEFL ibt IELTS & CEF chart B2 = IELTS C1 = IELTS B2 = IELTS; TOEFL ibt; PTE 59-75: Michigan ECCE C1 = IELTS; TOEFL ibt; PTE 76 84:TOEIC 880 C2 = IELTS; PTE 85+; Michigan ECPE

20 CONCLUSION When adapting scores to levels, it is important to analyze carefully the descriptors assigned to the score and the level are they referring to the same outcomes? Some language agencies also look at the estimated number of hours of instruction needed at each level

21 CONCLUSION Test scores are not perfectly reliable Universities with language programs correct for the false positive decision admitting an unqualified student Important for universities to set their own score requirements based on data of previous test takers to the university There is no one reliable fit of comparison of scores due to the various methods used Equivalency tables rely on experts decisions and so are subjective and constantly evolving