Information for universities and policy makers

Transcription

1 Information for universities and policy makers

2 Identifying students with the key academic skills for your courses With increasing financial pressure on universities and rising expectations from students, it is even more important to ensure a match between applicants and the course. Admissions tutors need to identify students with the skills to cope from the start of their course and who will go on to succeed in their studies. About the Thinking Skills Assessment (TSA) TSA tests problem-solving and critical thinking skills that are key for success in university study, but are not assessed separately in school qualifications. The test offers a vital additional selection filter, to help admissions tutors make an informed selection between applicants. It provides a benchmark that all applicants can be measured against, regardless of their background, nationality, gender or qualifications. A proven and established test, used by world-leading universities since Taken in our secure, authorised test centres around the world, or at your institution. Paper or online versions available. Suitable for students with English skills at CEFR Level B2 or above.* Developed, administered and marked by the Admissions Testing Service, part of Cambridge English Language Assessment: a not-for-profit department of the University of Cambridge. Why use TSA? Supporting internationalisation The increasing internationalisation of higher education means a growth in applicants from around the world. TSA provides a common measure for comparing applicants with a wide range of qualifications. Our global test centre network enables your applicants to take the test in their home country. Fair to all applicants TSA assesses students in a fair and objective way, giving all applicants an equal chance to demonstrate their potential. We routinely monitor the test and performance of different groups (by gender, educational background, etc.) to ensure that there is no bias. Showing students potential to succeed Research shows a positive relationship between TSA scores and on-course success. TSA helps ensure individual applicants will be the right fit for your institution, which can increase retention rates. 2 Information for universities and policy makers Underpinned by research Our dedicated research team conducts rigorous investigations to ensure that TSA conforms to the highest quality standards and makes a useful contribution to the university admissions process. We work closely with you to ensure TSA meets your specific requirements through pilots, or research projects focusing on your applicants and students. Supporting inclusion Universities are recommended to consider a range of factors when assessing a student s potential and not to focus solely on school qualifications. TSA measures problem-solving and critical thinking skills, giving applicants the opportunity to show their potential in addition to their school academic record. TSA is designed to be taken with minimal preparation and support materials are freely available from our website, which means that costly preparation courses are unnecessary. * Common European Framework of References for Language (CEFR) the international standard for describing language ability. (For more information, see TSA can be produced in other languages where there is significant demand.

3 What s in TSA? What it tests Problem-solving skills, including numerical and spatial reasoning. Critical thinking skills, including understanding argument and reasoning using everyday language. Question type 50 multiple-choice questions: 25 problem-solving questions 25 critical thinking questions. Duration 90 minutes. Scoring One mark per multiple-choice question. Raw marks are converted to scores on a calibrated scale from Results Results are provided directly to universities in electronic format. We work with new institutions to determine how TSA can best complement their admissions process. Download the TSA specification and past papers at: Information for universities and policy makers 3

4 Sample questions Critical thinking This critical thinking question requires test-takers to identify which of the multiple-choice options best expresses the main conclusion of the argument. Vegetarian food can be healthier than a traditional diet. Research has shown that vegetarians are less likely to suffer from heart disease and obesity than meat eaters. Concern has been expressed that vegetarians do not get enough protein in their diet but it has been demonstrated that, by selecting foods carefully, vegetarians are able to amply meet their needs in this respect. Which of the following best expresses the main conclusion of the above argument? A A vegetarian diet can be better for health than a traditional diet. B Adequate protein is available from a vegetarian diet. C A traditional diet is very high in protein. D A balanced diet is more important for health than any particular food. E Vegetarians are unlikely to suffer from heart disease and obesity. Problem solving This problem-solving question requires students to use the information given to generate a solution. Only a very basic level of arithmetic is required; the challenge is to identify the relevant information, and to devise and apply a method to solve the problem. Of the 100 screws in my work box: 60 are cross-headed and 40 are slot-headed 70 are 3mm diameter, 20 are 4mm diameter and 10 are 5mm diameter 80 are 50 mm long, 5 are 36mm long and 15 are 20mm long. What is the smallest number of cross-headed, 3 mm diameter, 50mm long screws that there could be in the box? A 0 B 10 C 30 D 31 E 60 4 Information for universities and policy makers

5 Who uses TSA? We work with universities around the world. These include: TSA is currently used for entry to a wide range of undergraduate courses, including: Computer Science Land Economy Economics Natural Sciences (Physical and Biological) Economics and Management Philosophy, Politics and Economics (PPE) Engineering Philosophy and Linguistics European Social and Political Studies Psychology and Linguistics Experimental Psychology Psychology and Philosophy Geography Psychological and Behavioural Sciences Human, Social, and Political Sciences Read what institutions have to say about TSA: PPE piloted the TSA in 2007 and have used it for the shortlisting process since College admissions tutors have found that the test provides invaluable information, helping us to assess applicants from diverse educational backgrounds fairly. The Admissions Testing Service has constantly provided efficient administration and reliable support. We particularly value their commitment to ongoing research and development in relation to the test. Dr Alison Hills Philosophy, Politics and Economics (PPE) Admissions Co-ordinator University of Oxford, UK UTP takes pride in nurturing students who are not only technically competent, but who are also able to exercise the ability to think critically and synthesise solutions for real-world problems. TSA keeps pace with what UTP is looking for today, measuring the skills required for success in the 21st century. Datuk Ir (Dr) Abdul Rahim Hashim Vice Chancellor Universiti Teknologi PETRONAS (UTP), Malaysia Read more testimonials on our website: Information for universities and policy makers 5

6 Underpinned by research Our dedicated research team conducts routine operational analyses and a wide range of validation work to ensure TSA is a fair and valid tool in the admissions process. Backed by research Routine operational analyses and a wide range of test validation research is conducted by a dedicated TSA research team to ensure a high quality test that supports the selection and admission of students to challenging courses of study. This continuous programme forms an important part of our test development and review cycle and draws on the expertise of our wider Research and Validation team. Routine statistical analyses Candidate information We collect candidate background information (such as gender, age, and school) during the TSA registration process. Other information provided by universities as part of the application process is also matched to TSA data. This allows us to monitor any changes in the characteristics of TSA candidates, and forms the basis of our research into test fairness. We can make provisions, such as enlarged exam papers or use of a laptop, for candidates who have special requirements. Special consideration requests for candidates whose performance may have been adversely affected by illness, bereavement, or other unforeseen events, can be submitted after the test session and are passed on to the selecting institutions. TSA differentiates between applicants in a valid way, providing a fair and reliable means to compare applicants from a range of backgrounds who have different qualifications and experience. Our research focuses on the relationship between TSA performance and achievement in higher education, as well as the fairness of the test for different applicant groups. Post-test analyses In addition to analyses of item quality, potential bias is investigated by analysing the performance of different candidate groups on each TSA item. This is known as Differential Item Functioning (DIF) analysis and is carried out after each test session. Pre-test item analyses Every question in TSA is trialled in advance with a group of candidates of a similar profile to the typical TSA candidates. This process of pre-testing ensures that the items are of an appropriate difficulty, are free from any inherent ambiguities, and differentiate between lower- and higher-ability candidates. Once an item has been pre-tested, it does not appear in a TSA paper immediately, to minimise the chances of any candidate being advantaged. Information from pre-testing analyses is also fed back to our item-writers, to inform and support the development of new items in the future. 6 Information for universities and policy makers Stephen Bond

7 Validation research Range of validation research We engage in an extensive range of research activities to support the validation of TSA. Our research team s work informs: construct validity (cognitive, context and scoring validity) criterion-related validity (concurrent and predictive validity) consequential validity (e.g. impact on student learning and preparation). Our research also aims to explore key issues for candidates, such as the way they choose to prepare for TSA and the opinions of students from different social and educational backgrounds. Collaborative working We collaborate closely with our stakeholders to: discuss and prioritise key areas for research gather information about their admissions processes track the course performance of admitted applicants. Predictive validity One of the areas of research our stakeholders are particularly interested in is predictive validity. A positive relationship between candidates TSA scores and performance on their academic course provides evidence that TSA scores can usefully differentiate between even those university applicants who present with the highest school attainment. The example plots below show the relationship between overall TSA scores, and i) high and ii) low 1st year examination performance. We also compare the predictive validity of TSA scores for specific groups of candidates for example, males and females. If the same relationship between TSA score and course performance is found for the different groups, this provides strong evidence that TSA measures ability fairly for all candidates. Fig 1: Predicting 1st year examination performance using TSA scores i) ii) 1.0 Probability of achieving a 1 st Probability of achieving a 3 rd or below TSA total score TSA total score Examination data: University of Cambridge. These graphs show data from three cohorts of admitted students. In this study: i) A student with a score of 60 on TSA has around a 0.2 probability of achieving a 1st class exam result, whereas a student with a score of 80 has about a 0.5 probability. ii) Students admitted with a low TSA score have a higher probability of achieving a 3 rd class result or below (3 rd class, ordinary, fail, or left the course). Information for universities and policy makers 7

8 About the Admissions Testing Service The Admissions Testing Service is part of Cambridge English Language Assessment, a not-for-profit department of the University of Cambridge. We offer a range of tests and tailored assessment services to support selection and recruitment for educational institutions, professional organisations and governments around the world. Underpinned by robust and rigorous research, our services include: assessments in thinking skills admissions tests for medicine and healthcare behavioural styles assessment subject-specific admissions tests. Contact us Stakeholder Relations Admissions Testing Service Cambridge English Language Assessment 1 Hills Road Cambridge CB1 2EU Tel: +44 (0) atsinfo@cambridgeenglish.org UCLES 2014 CE/2465/4Y11 All details are correct at the time of going to print in November * *