Rhetoric and Critical Thinking

Transcription

1 Rhetoric and Critical Thinking Annual Meeting Teagle/Spencer Project Duke University June 10th, 2011 Richard Fliegel and John Holland USC Dornsife College

2 Abstract The USC project for the Spencer/Teagle grant sought to create a process for assessing change in critical thinking, demonstrated in undergraduate essays written for regular class assignments; While doing so, we asked what has been done, what we could do, and what really counts as critical thinking in the academic setting. The result was both a process for assessing change in critical thinking over our students years in college and a series of conceptual outcomes.

3 The Context: Limited Assessment At least 45 percent of students in our sample did not demonstrate any statistically significant improvement in Collegiate Learning Assessment performance during the first two years of college. While these students may have developed subject-specific skills that were not tested for by the CLA, in terms of general analytical competencies assessed, large numbers of U.S. college students can be accurately described as academically adrift. -- Richard Arum and Josipa Roksa, Academically Adrift: Limited Learning on College Campuses (2011)

4 Richard J. Shavelson, Measuring College Learning Responsibly (2010) CLA Scoring Criteria

5 USC Project Objectives Create a non-invasive, valid, and reliable method for assessing the change in students critical thinking, demonstrated in essays they write for their teachers; Use that method to provide academic accountability (documenting a learning outcome of our General Education and Writing curricula) And to improve our academic programs (continuing the development of evidence-based pedagogy). To do so, we had to identify the evidence in writing recognized by readers as good critical thinking.

6 Facione Holistic Scoring Rubric for Controlled Study Focuses on assessing evidence of critical thinking, rather than evaluating writing; builds on Peter Facione s earlier work in the assessment of critical thinking through standardized tests.

7 Essay Prompt for Controlled Study Robert Bellah, a professor of sociology at UC Berkeley, has suggested that a good society is one in which attention takes precedence over distraction. For Bellah, attention means the ability to concern ourselves with the larger meaning of things in the longer run, rather than with short-term payoffs. He argues that when we attend to things in terms of long-term goals, we achieve our greatest human potential and deepest happiness. By contrast, distraction, the pursuit of immediate pleasure, pulls our consciousness away from the significant or meaningful. ( ) In your view, does contemporary society emphasize short-term fixes rather than long-terms solutions? - Excerpted from the full prompt

8 Sample Student Essay in Controlled Study Obtained good results using timed, prompted essays; Could we design a rubric assessing critical thinking abilities demonstrated in essays written for regular class assignments?

9 Rubric Redesign, Conversations

10 Six-Point Rubric

11 Four-Point Rubric

12 Results ~ Six-Point Rubric

13 Results ~ Four-Point Rubric

14 Conceptual Outcomes Experience of reading hundreds of student essays, written for different instructors, led to discussions about definitions, assignments, and pedagogy, and raised essential questions: What are we reading for? What should count as critical thinking in the college context?

15 Rhetoric as Critical Reasoning Assessment defines goals and expresses values more clearly than do any number of mission statements. -- Ed White

16 Incoming Students Who are they as writers and critical thinkers? Critical thinking in high school is presented as puzzle-solving and crisis management Taught to write personal narratives, pastiche, and paragraphs (the three p s) Bring little skepticism except about advertising; blogspeak and tweeting as discourse

17 Designing Assignments Our scorers observed that some writing prompts enabled students more readily to demonstrate originality in their critical reasoning: Those that asked them to write about questions of genuine uncertainty, ethical dilemmas, and conflicts without predictable answers. There is a utility for prompts with correct responses, but they are less likely to elicit what general readers consider critical thinking.

18 Using Rhetoric as Critical Reasoning to Extend the Writing Curriculum Writing Process Theory Reading Theory Social Constructionist Theory

19 Effects on the Writing Curriculum Reformation of a Common Assignment for all sections of WRIT 140 Shift from Course Assignments Packets to the Writing Program s Tool Kit Creation of New Goals for Assignments 1 through 5 Integration of assignment development into the WRIT 501 class for new instructors

20 Assignment 5: Public Reasoning on Social Issues Democracy depends significantly upon the ability of the general public to exercise judicious reasoning with regard to important social and political questions. /... / There is therefore an apparent need for public forms of reasoning that can effectively engage broad social issues. Although public reasoning can never be expected to reflect the depth, rigor, or sophistication that expert reasoning achieves through its concentration on exceptionally narrow questions, the public understanding of social issues must attain to a sound reasonableness sufficient to the challenges that such issues impose. In your Social Issues affiliation this semester Writing 140 and your Social Issues lecture and discussion sections you have examined various issues that have social or political import, and that thus fall within the domain of public reasoning. This assignment will provide an opportunity for you to analyze and critique the quality ofpublic reasoning in regard to the particular issues you have studied. Familiarize yourself with the Assignment 5 Overview and Invention Materials and with the Assignment 5 Prewriting Suggestions (see over). Then reflect closely and carefully upon the relationships between public reasoning and the social issues you have studied this semester before responding to the following writing prompt: Evaluate the quality of public reasoning directed toward the social issues you have studied this semester.

21 Preliminary Work with Outside Faculty Found differences in conceptions of critical thinking across disciplines (e.g., Philosophy) Found differences in judgments about critical thinking by the time of reading (as original ideas became more commonplace) Did not engage the natural sciences, where numeracy is a key element of critical thinking

22 Still in Progress Final reading and scoring session Longitudinal study Study of particular data points in the results Questions: What are senior students failing to do in their essays that would have raised their scores? How can we improve instruction in those areas, and enable engagement with uncertainty?

23 Framing Observations All questions are leading questions [they] need no weighting other than the weighting implicit in the choice of topic itself." -- Kenneth Burke Originality does not consist in saying what no one has ever said before, but in saying exactly what you think yourself. -- James Stephens

24 Contact Information Richard Fliegel, Director of General Education and Associate Dean of Undergraduate Programs USC Dornsife College 213/ John Holland, Director of the Writing Program USC Dornsife College 213/