Self-assessment and placements tests - a worthwhile combination?

Transcription

1 Self-assessment and placements tests - a worthwhile combination? Dr. des. Maike Engelhardt* Dr. des. Joanna Pfingsthorn** *Language Center **Department of English Studies School of Linguistics and Cultural Studies Carl von Ossietzky University Oldenburg

2 Outline General English CvO Uni Language Center Self-assessment grid replacing placement test measuring/comparing placement test performance and selfassessment self-assessment: serving descriptors blocked per CEFR level self-assessment: serving descriptors randomized A worthwhile combination or a waste of time?

3 10 years of CEFR /ELP test takers (1 st yr univ. students) (should have) experienced ca. 8yrs of FL/English instruction in school (+ training in self-assessment, as per curriculum) CvO Language Center: ca. 400 students taking placement tests (English) per year ca. 350 participants in English courses per semester 2 placement test dates prior to semester start

4 Courses available: General English courses from A1 to C1 Business English (English on the Job module) Academic English English courses for researchers and staff 177 participants in current study Placement Test currently: Quick Placement Test (QPT) - paper & pen version

5 e.g. Häcker (2002), Häcker, Dumke, & Schallies (2002), Harada (2001), Jones (1994), Paulson, Paulson & Meyer (1991) Hypotheses Learnerautonomy Selfassessment e.g. Oscarson (1984, 1997), Blanche (1988), LeBlanc & Pinchaud (1985) Standardized assessment

6 Hypotheses 1st hypothesis of self-assessment will lead to the same results as rough placement testing (--> impact of L1, time spent abroad, # of other foreign languages, age, gender) 2nd hypothesis Randomized can-do statements will lead to more accurate self-assessment results

7 Winter semester 2011/ participants QPT with questionnaire and self-assessment grid a) can-do-statements blocked per CEFR level b) can-do-statements randomized QPT tests only Use of English and some reading comprehension, listening and speaking were not tested.

8 Test scores Participants scored 33 pts on average 70% of participants within 24 and 43 pts -> A2 and B2 QPT places them on B1

9 Self-assessment results - Listening Most participants placed themselves on B1 No one thought they are A1 or C2. 0=A1, 1=A2, 2=B1, 3=B2, 4=C1, 5=C2

10 Self-assessment results - Reading Most participants placed themselves on A2 and B1. 0=A1, 1=A2, 2=B1, 3=B2, 4=C1, 5=C2

11 Self-assessment results - Spoken Production Strong tendencies towards A2/B1 (ca. 70%) roughly 20% of participants believe themselves to be B2 0=A1, 1=A2, 2=B1, 3=B2, 4=C1, 5=C2

12 Self-assessment results - Spoken Interaction Most participants expect just basic skills in interaction. 0=A1, 1=A2, 2=B1, 3=B2, 4=C1, 5=C2

13 Self-assessment results - Written Production Tendencies towards A2/B1, few B2. 0=A1, 1=A2, 2=B1, 3=B2, 4=C1, 5=C2

14 Test score and self-assessment - Reading Expected Observed

15 Test score and self-assessment - Writing Expected Observed

16 Test score and self-assessment Multiple regression reveals that: age gender L1 time spent abroad # of other foreign languages cannot be used as predictors of the test score. They also do not predict self-assessment patterns. Self-assessment can serve as a predictor* but to a limited degree: reading -- R=2,73 writing -- R=2,67 listening -- R=1,49 *F(3,125)=19,57; p=,000

17 Comparison of blocked and random descriptors No effects of randomizing on self-assessment of levels Small differences in the degree of conviction

18 Self-assessment can be used as a predictor. Self-assessment is not very accurate. L1, gender, age, # of foreign languages, time spent abroad do not predict patterns of self-assessment or performance. It therefore follows that: Self-assessment is a skill. Once this skill is learned, it may replace a placement test.

19 What we learned: Placement testing cannot be replaced by self-assessment if participants are not instructed how to self-assess first. What we would like to have: A stronger presence of the ELP in FL instruction in schools, leading to skilled self-assessors.

20 We could then have: an online self-assessment tool (leading to) Wishlist an online placement test (with only corresponding test items for self-assessed level of skills plus anchor items from levels above and below) (leading to) an online portal to sign up for corresponding course levels

21 If you are interested in the development of such an online-based assessment tool plus corresponding placement test for English at your language center, please contact : Maike Engelhardt [email protected] Language Center, School of Linguistics and Cultural Studies Carl von Ossietzky Universität Oldenburg

22 Barton, J., & Collins, A. (1993). Portfolios in Teacher Education. Journal of Teacher Education, 44 (3), Behrens, M. (1997). Das Portfolio zwischen formativer und summativer Bewertung. Beiträge zur Lehrerbildung, 15 (2), Behrens, M. (2001). Denkfiguren zum Portfoliosyndrom. journal für lehrerinnen- und lehrerbildung, 1 (4), Elbow, P., & Belanoff, P. (1997). Reflections on an Explosion. In: Kathleen Blake Yancey & I.Weiser (Hrsg.), Situating Portfolios: Four Perspectives. (S.21-33). Logan, Utah: Utah State University Press. Forster, M., & Masters, G. (1996). Portfolios. Assessment Resource Kit (ARK). Camberwell,Melbourne,Victoria: Allanby Press Printers Pty. Ltd. Häcker, T. (2002). Der Portfolioansatz - die Wiederentdeckung des Lernsubjekts? Die Deutsche Schule, 94 (2), Häcker, T., Dumke, J., & Schallies, M. (2002). Weiterentwicklung der Lernkultur: Portfolio als Entwicklungsinstrument für selbstbestimmtes Lernen. Informationsschrift zur Lehrerbildung, Lehrerfortbildung und pädagogischen Weiterbildung, (63), Harada, N. (2001). Leistungsbeurteilung mittels Portfolio. Grundschule, 33 (4), Jones, J. E. (1994). Portfolio Assessment as a Strategy for Self-Direction in Learning. New Directions for Adult and Continuing Education, (64), Oscarsson, M. (1984). Self-Assessment of Foreign Language Skills: A Survey of Research and Development Work. Strasbourg: Council of Europe. Oscarsson, M. (1997) Self-Assessment of Foreign and Second Language Proficiency. In The Encyclopedia of Language and Education, Vol. 7. Kluwer Academic Publishers, pp Paulson, F. L., Paulson, P. R., & Meyer, C. A. (1991). What Makes a Portfolio a Portfolio? Educational Leadership, 48 (5), Sacher, W. (2001). Leistungen entwickeln, überprüfen und beurteilen. Bad Heilbrunn/Obb.: Julius Klinkhardt. Vavrus, L. (1990). Put Portfolios to the Test. Instructor, 100 (1), Weinert, F. E. (1999). Die fünf Irrtümer der Schulreformer. Psychologie Heute, 29 (7), Winter, F. (1996). Schülerselbstbewertung. Die Kommunikation über Leistung verbessern. In: H. Bambach, H.Bartnitzky, C.v.Ilsemann, & G.Otto (Hrsg.), Friedrich Jahresheft: Prüfen und beurteilen. (S.34-37). Seelze: Friedrich Verlag.