Assessing Students Critical Thinking Skills in Productive Language Tasks

Transcription

1 Assessing Students Critical Thinking Skills in Productive Language Tasks Tara Selenga Kılıçlı and Michael O Neill Doğuş University, Istanbul terdener@dogus.edu.tr moneill@dogus.edu.tr 1

2 Content Introduction to Doğuş University Reflecting on Critical Thinking in Curriculum and Assessment The role of reasoning and valued judgement in students productive tasks DUDFL initiative and implications Questions 2

3 3

4 What type of thinker are you? Accomplished - Intellectual skills and virtues have become second nature in our own lives Advanced - We are committed to lifelong practice and are beginning to internalize intellectual virtues Practicing - We regularly practice and advance accordingly Beginning - We try to improve but without regular practice Challenged Thinker - We are faced with significant problems in our thinking Unreflective Thinker - We are unaware of significant problems with our thinking 4

5 What type of thinkers are our learners? Handout task 1 5

6 Task 2 Follow the instructions for the Stone Fox reading exercise. 6

7 Task 2 Look at the following student response: In my view this is a sentimental story that insults my intelligence. I feel disgusted when I read it and bored silly 7

8 Reasoned and Valued Judgement Importance of task type and matching task type to intellectual standards. 8

9 Issues concerning students productive skills Focus on quality of form at the expense of content Lack of focus on content results in limited range of grammar and lexis Insufficient feedback at draft stage to improve quality of content Use of stock phrases Conceptual mismatch between students perception about the quality of their content (ideas) and the criteria being used? (Learner Training) 9

10 Can write clear detailed texts on a variety of subjects related to his/her field of interest, synthesising and evaluating information and arguments from a number of sources. Can write an essay or report which develops an argument, giving reasons in support of or against a particular point of view and explaining the advantages and disadvantages of various options. Can synthesise information and arguments from a number of sources. Can express views and news effectively in writing and relate to those of others. Can give feedback on and follow up synthesis and inferences and so help the development of discussion.

11 Are we authentically testing higher order performances? How is the construction of meaning achieved and measured in our program? 11

12 Judgement How do we traditionally categorise information that involves some kind of judgement? 12

13 Question types Those with one right answer (factual questions) What is the boiling point of lead? 13

14 Question Types Those with better or worse answers (well reasoned or poorly reasoned answers) How can we best address the most basic and significant economic problems of the world today? 14

15 Question Types Those with as many answers as there are different human preferences (mere opinion) Which would you prefer, a holiday in the mountains or one at the seashore? 15

16 Intellectual standards in curriculum terms Do the curriculum learning objective statements contain references to intellectual standards? If so what is the level of awareness among instructors and students concerning standards? Do materials used make reference to intellectual standards? Do assessment criteria make reference to intellectual standards? 16

17 Project Aim The formation of a checklist of critical thinking skills and concepts that can be used across language skills from both a teaching perspective and a testing perspective, and from a student s perspective and an educator s perspective (CTSPT) 17

18 Imagine you look at the following items during any given day A poem A bank account A personal attack on twitter A pen that doesn t work What thought process are we forced to use in ourselves? 18

19 Reasoning questions to be asked Can we assess reasoning? Is reasoning measurable? Is there good and bad reasoning? If we can assess reasoning, what do we need to be able to assess it.? 19

20 Reasoning next step In order to produce intellectual standards we needed to define the role of reasoning in our program. In order to do this we focussed on what we are actually asking our students to do through the construction of our tasks 20

21 Purposes Questions Points of view Information Inferences Concepts Implications Assumptions Elements of reasoning in student s productive work 21

22 Reasoning questions to be asked Do we have intellectual standards? Who knows about them? Who should know about them? 22

23 Task 3 Intellectual Standards and Curriculum Flaws Look at the list of curriculum flaws and decide which intellectual standard(s) each statement refers to? 23

24 Clarity Precision Accuracy Relevance Consistency Depth/Breadth Coherence Significance Fairness Intellectual Standards 24

25 Intellectual standards Look at Task 4 of the handout. What do all these statements concerning critical thinking abilities contain linguistically? 25

26 Intellectual standards A process - evaluate An object - information A standard relevance 26

27 Entertains challenging ideas: grounds meaning in acute perceptions of textual and cultural complexities How can we determine whether or not a student is entertaining a challenging idea? Must the student just do it or do it well?(conceptual mismatches) 27

28 The Checklist and the Rubric Purpose: What is the purpose of the writer? Is the purpose clearly stated or clearly implied? Is it justifiable, reasonable and significant? Are all parts of the essay relevant to the stated purpose? Questions: Are the questions used well stated? Are the questions clear unambiguous and unbiased? Are related questions articulated and distinguished? Does the expression of the question do justice to the complexity of the matter at hand? Are the question and the purpose of the question directly relevant to each other? Does the proceeding train of thought stay close to the question asked? Does the writer attempt to answer the question asked? 28

29 Purpose Rubric statements The purpose of the writer is clearly stated rather than clearly implied. The purpose of the writer is justifiable, reasonable and significant. All parts of the essay are relevant to the stated purpose. 29

30 Intellectual standards checklist Breadth Do we need to look at this from a different perspective? Do we need to consider another point of view? Do we need to look at this in other ways or what other ways could we look at this? Logic Does all this make sense together? Does your first paragraph fit your last? Does what you say follow from the evidence you have provided? 30

31 Intellectual standards rubric Breadth The writer looks at the subject from different perspectives. The writer considers another/other points of view. The writer looks at the subject in other ways Logic As a whole the piece of writing makes sense together. The first paragraph fits with the last. The conclusions drawn follow the evidence provided 31

32 Teachers guide

33 Matrix of descriptors vertical column = items of reasoning in student production horizontal column relating to intellectual standards Use the matrix to zoom in and out of to produce checklists for writing, checklists and teachers guides for feedback and assessment purposes

34

35 Implications and Conclusions Question types and Faculty needs The need for L1 training in intellectual standards Reliability, Validity and Instructor Training - how does assessment lead teachers in the direction of their practice and how does assessment lead students in terms of their systematic approach to their studies? General mark schemes and task specific mark schemes constructing task specific mark schemes from the general critical thinking skill rubric 35