Welcome! What do teachers learn from research about assessment? Round table session EAPRIL Luxembourg November 26, 2015

Transcription

1 Welcome! What do teachers learn from research about assessment? Round table session EAPRIL Luxembourg November 26, 2015

2 About us a brief look at the different teachers and schools Lead teacher Greta de Wit-Rademaker Windesheim University Research group Vincent Grit Marleen Heikens Minke Huijgen-Folkerts Remmelt Visscher secondary education secondary education special primary education higher education W i n d e s h e i m z e t k e n n i s i n w e r k i n g

3 We have a question Todaysmeet.com/EAPRILH11 1. What s your most memorable experience in regard to testing? 2. What does this experience tell you about what you think is important in testing? W i n d e s h e i m z e t k e n n i s i n w e r k i n g

4 Assessment literate? DO NOT DISTURB Assessment, a hot item? What can be learned from the assessment? W i n d e s h e i m z e t k e n n i s i n w e r k i n g

5 Collaborative research project Aim: The innovative aim of this research project is to contribute at the participating schools to the quality of student learning by enhancing the formative function of assessment. The research aim is getting insight in the professional development of teachers in their role as assessor of students' learning. Research question: What do teachers learn about meaningful assessment by participating in research? W i n d e s h e i m z e t k e n n i s i n w e r k i n g

6 Analysis of the context + deskresearch Subquestion 1 Intervention Outline of the research project What s about assessment in our school? Listening, observation, analysis of documents Which kind of knowledge, beliefs and conceptions do teachers have about formative assessment? Experimenting with formative assessment together with five colleagues Subquestion 2 Subquestion 3 Analysis, conclusion What does the intervention with formative assessment (for instance: giving tutorand peer feedback) benefit in meaningful learning according students and teachers? What do the experiences with participation in research benefit for the educational (assessment) practice according to the teachers? Analysis at school level and level of the research group. Suggestions to improve the assessment practice

7 The design of a collaborative research project about the start Collaborative learning in research We started with 14 students, who are teachers in primary, secondary, vocational and higher education.

8 Conceptual model at the base of the design Figure 3: Model of professionalisation of Desimone (2009) adapted for research project Meaningful assessment

9 First step: going native! Research question: What is said about assessment by those involved at school? Purpose of the assignment : investigating thoughts and believes about assessment by various stakeholders in school. 1st step: write down without judgment and interpretation what is said about assessment in the next three weeks on your school by colleagues, management, pupils or students. Write down literal wording, by whom, when and where, without asking.

10 Examples of noted quotes about assessment Random quotes from the dataset Who Sector I did my best when taking the assignments. That makes learning the topography test easier Student / pupil The tests of calculus group 4 are made much better than CITO. We have a test and I m nervous Do we have a test? That was not in the electronic learning environment. teacher Student student primary education primary education secondary education secondary education At a project evaluation, they had to divide 17 points over three students, one student would certainly get an insufficient. What exemptions apply? Sometimes a test result is not accompanied by an explanation. That's too bad Go to the exam room at the beginning of an exam and make sure everything runs smoothly. teacher Department head Student Other (exam office) Vocational education Vocational education Higher education higher education

11 Research Going native Steps in the data collection and analysis method: 1. Gather literal quotes in natural settings, without provoking / stimulate. 2. Inductive analyzing of the quotes in a few steps: specify - developing concepts (individual) reduce - defining the key concepts (first in subgroups, then together) integrate - elaborating a 'theory', categories for analysis (collectively) 3. Also let a fellow student analyze quotes => look for similarities and differences. This is important for the reliability of the analysis (triangulation of researchers). 4. Presenting the results. Which quotes may / may not be made based on the data?

12 Categories to analyze Contents of nature of the the quote quote Policy Reflective critical Design Positive appreciative Locus of control Internal Preparation negative Unknown / neutral who Location / context Sector Pupil/ student Classroom Primary education external teacher meeting Secondary education manager hall Vocational education Testtaking informing Counselor Individual talk Higher education Result parent Mail Analysis instructor other Evaluation Other / external This setup was created by comparing, discussing and merging individually chosen categories.

13 What sorting is possible concerning the previous analysis model? Sorting by content: where are the quotes about assessment about? What stands out? Sorting by nature: In what way is discussed about assessment? What stands out? Sorting by locus of control: Is ownership taken in the way people talk about assessment? Next step: combinations are possible, such as for example: Sorting by content & actor: who talks about what aspects of assessment Sorting by nature & actor: who talks about a certain way of assessment? Sorting by nature & sector: In what way is discussed about assessment in primary education compared to secondary education?

14 Display of the results on contents of the quotes Total policy design preparation test taking result analysis evaluation Total Meta-analysis based on 369 quotes from PE, SE, VE and HE

15 Displaying of the results on content per sector PE PE 0 SE SE VE VE 0 HE HE

16 What stands out to the quotes as a whole? 1. Testing = assessment of learning, hardly assessment for learning. What are the most distinctive phrases in the data? What test do we have, when and how and what is the result? 2. Vary by sector Especially in the SE and VE quotes are reduced into the technical quality and the organization of assessment: who, what, when. In a number of datasets from the PE and HE is more variation in the quotes, more critical-reflective comments on the assessment.

17 Subquestion 1: Which kind of knowledge, beliefs and conceptions do teachers have about formative assessment? Things You Do Things You Know Things You Believe

18 Method: position paper (written, individually) and interview (semi-structured one-to-one interview)

19

20 by giving positive feedback I see the students grow, then they dare to do things. Students who do not complete their work and don t get feedback from me drop out. Yes they are also nominated in the weekly task. But for a moment a very simple to specify, next week we are talking about the ordinal numbers. Master the children the ordinal numbers. For the basic level, the learner should they need to be able to name it yet, but may designate. The target level they should be able to name it. You notice in toddlers that they find it difficult and that often it driede house. But there are also plenty who know fine. In this way, it is then checked. On the weekly task, so that parents can also see which learning goal is currently being tested. Children know, we are the little job done, so teacher will ask me things about it. What I find myself yet again important to reflect and evaluate. They judge themselves by using thumbs how it went. You also see here again a difference. Some students, especially early group 2, which always give two big thumbs, I've done everything right. But some students also will critically look after. I would ask, in the piece evaluation by the teacher on the weekly task, then we have a conversation about it. Why do you have only one inch thick, it went quite well today delicious and then I write something. I do that at the end of the week, I look at the five

21 J: De manier om een toets spelenderwijs in te zetten. Ik merk dat daar in de onderbouw veel behoefte aan is. Leerkrachten die lezen de opdrachten, die gaan dat nog heel letterlijk uitvoeren en die dat nog eigenlijk, terwijl dat helemaal niet de bedoeling is, toch nog een leerling roepen, met een leerling individueel zitten. Ik denk dat ik ze wel zou kunnen begeleiden met hoe ze het zouden kunnen inzetten tijdens een werkles. Kinderen hebben het niet door dat je aan het toetsen bent. G: Zijn er ook activiteiten of onderdelen waar je zelf wel ondersteuning bij zou willen hebben? J: Ik denk dat ik momenteel wel aardig bezig ben qua toetsen. Dat dat wel losloopt, dat systeem daarin ook. Eigenlijk zet ik de toetsen weg in de weektaak op de ochtend en dat werkt op dit moment erg goed. Op de middag ga ik met groepjes verrijken en oefenen. Dus zoals het nu loopt ben ik tevreden. Tijdgebrek is dan nog wel iets G: Zijn je ideeën over toetsing veranderd in de afgelopen jaren? J: Ja, toets is nu vaak ook een bevestiging, wat je heus als leerkracht allemaal wel weet en ziet. Op het begin was je nog meer afhankelijk van een toets, dat je er heel erg aan vast zit en dat je nu denkt per leerling hoe doet hij het per leerlijn. Je bent er wel van op de hoogte, het is een bevestiging.

22 Things you just said (1) The test was communication and not an interrogation The assessment for becoming an assessor. The feedback on the assessment is memorable. The test became a group discussion. Assessed a student to formal criteria which focused most on negative aspects. Gave detailed feedback - was unhelpful. W i n d e s h e i m z e t k e n n i s i n w e r k i n g

23 Things you just said (2) The test was a moment of growth instead of measuring. It gives me some insight in to what level I achieved the learning goals before I took the test. Important that assessment should be used to help student learn rather than telling what can and can't do. W i n d e s h e i m z e t k e n n i s i n w e r k i n g

24 Influence of the researcher on the results W i n d e s h e i m z e t k e n n i s i n w e r k i n g

25 Influence of the researcher on the results What would happen if the same research were done by a different researcher? How does the following influence research: Personality of the researcher Relationship between researcher and participants Knowledge and skills of the researcher Support already won by researcher W i n d e s h e i m z e t k e n n i s i n w e r k i n g

26 Influence of the researcher on the results What would happen if the same research were done by, for example you? W i n d e s h e i m z e t k e n n i s i n w e r k i n g

27 pros & cons of a collaborative research project Pro It affects my own daily practices Intersubjectivity Multiple perspectives Learning from each other Collaboration improves quality Stimulating the motivation Con Time consuming (meeting, coanalyzing the data) Interdependancy Less flexible in time schedule, content, etc. W i n d e s h e i m z e t k e n n i s i n w e r k i n g

28 pros & cons of a collaborative research project Think of Time People Money Quality

29 Thanks a lot for feedback! Lead teacher Greta de Wit-Rademaker Windesheim University g.de.wit-rademaker@windesheim.nl Research group Vincent Grit Marleen Heikens Minke Huijgen-Folkerts Remmelt Visscher W i n d e s h e i m z e t k e n n i s i n w e r k i n g

30 Definition of formative assessment Figure 1. The conception of Bloom versus an enlarged conception of Allal en Lopez (2005; p. 245). FA=formative assessment. W i n d e s h e i m z e t k e n n i s i n w e r k i n g

31 Aspects of formative assessment Figuur 2. Aspects of formative assessment (Black & Wiliam, 2009, p8) W i n d e s h e i m z e t k e n n i s i n w e r k i n g