The Five Colleges of Ohio

Transcription

1 The Five Colleges of Ohio Creative and Critical Thinking: Assessing the Foundations of a Liberal Arts Education Report to The Teagle Foundation July 15, 2008

2 The Five Colleges of Ohio Creative and Critical Thinking: Assessing the Foundations of a Liberal Arts Education Funded by the Teagle Foundation The College of Wooster Wooster, Ohio 2

3 3

4 TABLE OF CONTENTS Introduction 1 Key Findings From Year Two 3 Year Two Activities Report 5 Generic Creative and Critical Thinking Rubric 15 Survey Data Analysis 19 Focus Group Data Analysis 57 Denison University Data Analysis 63 Kenyon College Data Analysis 89 Ohio Wesleyan University Data Analysis 119 The College of Wooster Data Analysis 147 Appendix 173 Faculty Creative Thinking Survey 175 Faculty Critical Thinking Survey 183 Student Creative and Critical Thinking Survey 191 Project Personnel 207

5

6 INTRODUCTION In the second year of our project to assess critical and creative thinking on our campuses, 60 faculty across the four campuses developed and administered creative thinking and/or critical thinking rubrics in their classes. Based on the data collected from the application of these rubrics, we successfully created a generic critical and creative thinking rubric that we will test in in longitudinal and cross-sectional studies to be undertaken by 18 faculty members representing the fine arts, humanities, natural sciences, and social sciences on our project campuses. We collected additional data through (1) the administration of critical and creative thinking surveys to faculty, seniors, and first-year students on our campuses, and (2) focus groups with faculty and seniors on our campuses. The surveys will be administered in on all four campuses to first-year students, sophomores, and seniors to begin the process of collected longitudinal data on student perceptions of their critical and creative thinking experiences. The surveys were made available to other colleges and universities via access on our web site. We presented initial findings from Years 1 and 2 of the project at the annual meeting of the American Association of Colleges and Universities in Washington, D.C. in January 2008 and at the annual meeting of the Higher Learning Commission in Chicago, IL, in April Nancy Grace and Sarah Murnen presented at the AAC&U meeting, while Sarah Murnen, Barbara Andereck, and Simon Gray presented at the HLC meeting. Nancy Grace also presented a report on the project to the Fives Colleges of Ohio representatives in January 2008, while Iain Crawford, project initiator, presented the project at a June Mellon Foundation conference in New York City. In April 2008, Nancy Grace also presented key project findings at a Mellon Foundation conference held at Hamilton College in Clinton, New York. In the following pages, readers will find detailed reports of our mid-project results, including the generic rubric that will be tested this coming academic year. Report Prepared by: Nancy Grace, Professor of English, The College of Wooster Sarah Murnen, Professor of Psychology, Kenyon College 1

7 2

8 KEY FINDINGS in YEAR TWO 1. It is possible to construct a valid generic critical and creative thinking rubric that can be used reliably to measure change in critical and creative thinking skills in both longitudinal and cross-sectional testing across the curriculum. 2. Faculty and students value both critical and creative thinking. 3. There seem to be more opportunities for critical than creative thinking in classes. 4. Class projects are evaluated by students as a means to engage in both critical and creative thinking. 5. The event perceived by both first-year students and seniors to involve the most critical thinking was writing papers for classes. The event that was perceived to involve the most creative thinking was engaging in a hobby. 6. Senior students reported more work that involved integration than first-year students. 7. Seniors majoring in the fine arts reported a lower number of activities requiring integration than did seniors majoring in the humanities, natural sciences, and social sciences. Seniors majoring in the natural sciences and social sciences reported a lower number of activities requiring the use of a story, metaphor, or visual element than did those majoring in the fine arts. There was no statistical significant difference between the fine arts majors and the humanities majors. 8. Our campus environments were rated favorably with respect to the potential for creative thinking, although faculty rated idea time and risk taking as somewhat limited in their experience. First-year students were the most optimistic in their ratings of the environment, while seniors were more optimistic than faculty in their ratings of some characteristics such as challenge and supportive environment. 9. Women students reported more experience with both creative and critical thinking and more positive attitudes about creativity compared to men students. 10. Faculty most often consider their own research to be the site of their most creative experiences. 11. In the responses to open-ended questions it was found that classes are highly associated with critical thinking while creative thinking was seen in a variety of activities, but not especially in class. Students might be experiencing the most creativity in their living spaces and in other aspects of student life that perhaps could be integrated more with their academic experiences. 3

9 12. Student issues were frequently noted by students and faculty as barriers to critical thinking, while pedagogy was frequency noted by students as an impediment to creative thinking. 13. Diversity is believed to facilitate both critical and creative thinking. 14. When asked what signifies growth in students critical and creative thinking, critical thinking improvement was distinguished by assessment of arguments and understanding others perspectives, while creative thinking was distinguished by novelty, risk, and curiosity. Synthesis and complexity of thought characterized both critical and creative thinking. 15. Differences in responses between the four schools point out some of the distinctive characteristics of the schools such as the emphasis on senior independent study at Wooster and the opportunities for special interest housing at OWU, which are perceived to facilitate critical and creative thinking. 4

10 YEAR TWO ACTIVITIES REPORT Key Questions of the Grant: Background 1. What tools might we develop to assess two fundamental and related outcomes of a liberal arts education: creative thinking and critical thinking? 2 What are faculty and student perceptions of creative thinking and critical thinking? How do student perceptions of these processes change during their college education? Research Design The research design addresses the two key questions of the grant. The grant activities and the data that were collected to assess each activity are listed by year below. 1. Assessment of creative and critical thinking in the curriculum Faculty develop rubrics using primary trait analysis to measure creative and critical thinking. Year 1: Year 2: Year 3: Core group of faculty learns about use of rubrics, develops pilot rubrics. Volunteer faculty members on all campuses learn to develop rubrics that they apply to their classes. These faculty reported back on their experience, and on student learning as a result of this work. Faculty who created rubrics in either year 1 or 2 asked to help rate terms that might be used for a generic rubric. Selected faculty apply the generic rubric and measure change in student creative critical thinking across time. 2. Assessment of perceptions of creative and critical thinking on campus. A survey was developed to measure perceptions. The focus group method is also employed to gather information. 5

11 Year 1: Year 2: Year 3: Pilot survey conducted and tested in students and faculty. Collected survey data from first-year students, seniors, and faculty on each campus. Held focus group of seniors and faculty members on each campus. Start longitudinal study of first-year students to see how perceptions change across time. Year 2 Data Reports Several types of data were collected in year two of the grant. These data are summarized and included in this report. Report 1: Faculty Response to Developing Rubrics Faculty members on all campuses learned to develop rubrics that they applied in their classes. These faculty were given an on-line survey to respond to concerning their experience. The data are summarized in a report. Report 2: Faculty Measure Change in Student Performance Across Time Although the faculty who developed rubrics in year two were not specifically asked to track changes in student performance across time, a couple of the faculty did do that. Their rubric data on student performance are summarized in a brief report. Report 3: Faculty Help Create a Generic Rubric All of the faculty who created and used rubrics in year 1 and 2 were sent a link to a survey concerning traits that were relevant to their courses that might be included in the generic rubric that will be used in year 3. This report is also included. Report 4: Survey Data A total of 782 individuals (260 first-year students, 375 seniors, and 147 faculty) filled out the survey. A lengthy report of responses is included. In addition to the summary report, each individual school was sent a report with institution-specific data. Report 5: Focus Group Data A summary of responses to the focus groups was developed. In addition, each school was sent specific information about the focus group comments from seniors and faculty at their school. 6

12 Report 6: Characteristics of a Creative Campus Climate at Other Institutions We had the opportunity to administer a brief version of some of the survey questions to faculty and administrators at a presentation we gave at both the Association of American Colleges and Universities (AACU) conference in January 2008 and the Higher Learning Commission (HLC) conference in April The results are presented in a brief report. Data Report 1: Faculty Response to Developing Rubrics In the summer of 2007, workshops on the development of rubrics were held on all four campuses with the intention of recruiting volunteers who would learn to develop rubrics and use them in their classes. Nancy Grace and Sarah Murnen conducted the workshops, and year 1 participants were encouraged to attend and to serve as support persons for faculty on their campus. Fifteen to twenty faculty from each campus were recruited to attend the on-campus workshop. A total of 60 faculty across the four campuses developed and administered rubrics in their classes. Faculty were told that their rubric should deal with some aspect of critical or creative thinking (and ideally, both). It was of interest to determine faculty responses to the workshops and their experience developing and using rubrics. All participating faculty were sent a link to an on-line survey to report on their experience. Fifty-seven participating faculty filled out the survey. Some results are summarized below: What was the focus of the rubric you developed and administered? 78% of the faculty indicated that the rubrics they developed and administered assessed some aspect of creative thinking 89% of the faculty indicated that their rubrics assessed some aspect of critical thinking How difficult was it to develop a rubric? 19.4% indicated that it was not at all difficult 52.8% somewhat difficult 11.1% difficult How many students performed well, according to your rubric responses? Creative Thinking: Critical Thinking: 3.7% All or almost all 6.5% 7

13 40.7% Most 71.0% 40.7% About half 22.6% 11.1% Less than half 0% 3.7% Almost none 0% Faculty were asked the extent to which they agreed with a number of statements using a scale where 1 = strongly disagree 3 = neutral 5 = strongly agree. The responses of the 57 faculty who filled out the survey are reported below. Item Average Response I will continue to use rubrics 4.38 Have discussed the use of rubrics with colleagues 4.29 The use of rubrics Helped me make course expectations clearer to students 4.26 Made me more purposeful about emphasizing or enhancing assignments related to critical thinking 4.00 creative thinking 3.47 Helped me more clearly define my goals for the course 3.80 Helped me keep track of changes in student performance 3.65 Led to results that influenced my subsequent teaching 3.46 Some sample comments from faculty about the effect of using rubrics: after students received rubric results, they were influenced in their interaction with the course. When I saw what was not understood, I realized I needed to slow down. They were particularly helpful in my interactions with students while they were revising their papers. The results more concretely showed me where the students were having problems and thus caused me to talk directly to these issues during class time. 8

14 Report 2: Faculty Measure Change in Student Performance Across Time Year two faculty who attended a workshop on the development of rubrics, developed their own rubric, and administered the rubric in their classes were asked to provide us with the rubric and the data they collected on student performance. A couple of faculty tracked changes in student performance across time. They gave out a similar assignment at the beginning of the course and at the end and tracked changes. These data are shown below: In a biology class a professor tracked student performance on a critical analysis of a research article. The performance on the first analysis was compared to the last analysis, and students improved significantly (p <.05) on the following traits: Novel claim made Rigorous analysis provided Convincing support provided In a women s studies course for an assignment on the analysis of a gendered object student performance on the first (preparatory) paper was compared to their performance on the final paper, and students performed significantly better on the second paper on the following traits: Quality of analysis and evaluation Resourcefulness in locating supporting material Evidence of divergent thinking Note: These data indicated to us that faculty can chart change in student performance across time with respect to creative and critical thinking. This is the model we will be using in the third yearn of the grant. Faculty will administer a generic rubric in their classes and measure change across time. Report 3: Summary of Faculty Responses - Most Important Generic Rubric Traits Twenty-six faculty who had experience in developing and using rubrics responded to a survey concerning traits that might be incorporated into a generic rubric. A number of traits were listed (and defined) and faculty rated the importance of including each trait on the rubric, using a scale where 1 = not at all important and 5 = very important. Only the traits with a mean importance of 4 or greater are shown below. Relevant comments from faculty are included as well. Mean Importance (stdev) Trait 9

15 4.48 (.51) Idea Generation One would have to take into consideration that this aspect of the assignment may not be written / evident IN the assignment. After considering many a student may settle on one idea but getting there is a valuable exercise. Quality rather than quantity of ideas is probably more relevant in science 4.24 (.72) Identifying Question A related trait is developing an interesting claim/argument (i.e. a useful, innovative, unique answer to the question). Sees an opportunity or makes a connection not previously noted 4.12 (.78) Framing a Question Not all assignments require framing questions. Some assignments require learning and synthesizing, but not to produce new work or answer questions, but rather to increase understanding/ownership What is creative to me is being able to see connections and articulate them in a way that invites more exploration 4.36 (.91) Domain and Disciplinary Knowledge Without this there is no critical thinking. Crucial in the sciences. While this is important - not sure it is creative. However, it may be critical for being able to explore things creatively. In addition, for art particularly - vocabulary may be less important that experience with different media - a vocabulary so to speak This would depend on the level at which the rubric is used: if on grad students or senior majors, then sure! 4.21 (.88) Identified Relevant Information Useful for more advanced students, but not as relevant for those new to the discipline 4.92 (.28) Ability to synthesize and make connections I see this as the heart of creativity. This is where the critical and creative thinking take place. Bingo! This is important and may be some aspect of creativity we can readily measure (.70) Engagement with the Material This is the heart of critical thinking. Not relevant for my assignments because my students are not able to read primary literature in physics and because even if they could they would not have enough 10

16 background to judge the author's perspective or assumptions. This item should not be mandatory for instructors to use. Useful for more advanced students, but not as relevant for those new to the discipline 4.50 (.72) Abstraction The rubric may have to have two elements to it - student behavior in the course of the project and the assessment of results of student research (.69) Connect Research/Writing to Larger Themes Here is another element of the critical and creative thinking that we liberal arts colleges claim students gain out of a liberal arts education. Our curricula are successes when students come out able to do this. Again, not so relevant for papers designed to have students gain and express knowledge. Harder for them to do. Useful for more advanced students, but not as relevant for those new to the discipline Not a great indicator as some fields this is not creative but in other it is. I would look for unexpected connections 4.25 (.79) Elements of Argumentation This covers a lot of ground, from writing ability to the explanation of causes, consequences, and implications (which is a lot in itself). Here again are traits, if they are present in student work, of nascent critical and creative thought. Isn't ability to construct an argument that is compelling/convincing what we are trying to assess? Don't we have to assess those traits that make an argument compelling? I'd like to see this split: 1) developing a compelling claim. 2) making the argument well. Some sub-items are relevant. OK if we can choose which we want to include. Not necessarily creative For advanced students 4.00 (.98) Divergent Thinking Here is a trait that would reflect creative and critical thinking to my sense of it. Good for creativity. Perhaps bad for critical thinking or at least gaining background in a scientific field (.69) Coherence, Elegance I also think we should value the ability to present complex ideas and not to oversimplify. These help identify the A papers. Could just be neat and well organized - not necessarily creative 4.50 (.78) Clarity and Completeness 11

17 This can be used as an overall item - taking the entire piece into consideration. Providing an overall perspective. This is related to communication of ideas. It's my feeling that if you cannot communicate your creative or critical thoughts, then it is as good as not being able to think creatively or (especially!) critically, since it's locked away inside you. Report 6: Characteristics of a Creative Campus Climate: Ohio-5 Schools Compared to Others A brief anonymous survey was administered at the AAC&U conference in January 2008 and at the HLC conference in April 2008 to determine the extent to which faculty and administrators present rated their campuses as having various features that promote creative thinking. Responses of conference attendants are shown below using a rating scale where 1 = not at all present on the campus to 7 = extensively. Attribute AACU (N = 32) HLC (N = 80) Freedom 5.26 (1.09) 4.89 (1.32) Idea Support 4.75 (.98) 4.39 (1.2) Trust/Openness 4.55 (1.17) 4.12 (1.24) Risk Taking 4.26 (1.39) 3.71 (1.01) Active models 4.33 (1.12) 4.00 (1.21) Is your campus creative? 4.46 (1.04) 4.44 (1.19) Do you teach creativity? 5.23 (1.33) 4.86 (1.21) Responses to How do you know (your campus is creative)? 23 AACU and 26 HLC respondents wrote comments. The most frequent responses in order: Visibility of various types of student creative projects Existence of active encouragement for new ideas Development of new programs occurs Responses to How do you know (that you are teaching students to think more creatively)? 21 AACU and 26 HLC respondents wrote comments. The most frequent responses in order: Existence of active learning pedagogies such as senior projects, open-ended assignments, etc. Knowledge comes through assessment Responses to If you could change one thing about your campus to encourage more creative thinking, what would it be? 21 AACU and 38 HLC respondents wrote comments. The most frequent responses in order: 12

18 Increased conversations about the topic (and other forms of learning) Increased interdisciplinary opportunities Modeling from the administration Funding or other incentives (and ways to free up time) Create safe zones or other types of support Make creativity a learning goal 13

19 14

20 CREATIVE AND CRITICAL THINKING GENERIC RUBRIC 1. Elements of Argumentation The product exhibits evidence of the following traits. Explanation stating the results of one's reasoning; justifying that reasoning in terms of the evidential, conceptual, methodological, criteriological and contextual considerations upon which the results were based. [Disciplinary/Assignment specific traits: ] Analysis identifying the intended and actual relationships among statements, questions, concepts, descriptions or other forms of representation. Can include defining, cause and effect, as well as comparing and contrasting. [Disciplinary/Assignment specific traits: ] Evaluation assessing the credibility and logical strength of statements or other representations that are accounts or descriptions of a person's perception, experience, situation, judgment, belief, or opinion. [Disciplinary/Assignment specific traits: ] Interpretation comprehending and expressing the meaning or significance of something, such as experiences, situations, data, events, judgments, conventions, beliefs, rules, procedures or criteria. [Disciplinary/Assignment specific traits: ] Logic following the accepted and understood rules for the discipline. [Disciplinary/Assignment specific traits: ]

21 2. Domain and Disciplinary Knowledge drawing upon relevant literature, methods, insights to construct the product. [Disciplinary/Assignment specific traits: ] Synthesis and Connections combining unlike or distinctly different elements in order to tell a coherent story, provide a logical argument or insightful vision, or create a useful object. [Disciplinary/Assignment specific traits: ] Abstract Thinking formulating general concepts by identifying common properties of specific instances; posing overarching theories and seeing the big picture identifying fundamentals, first principles, general structures. [Disciplinary/Assignment specific trait: ] Complexity of Thought using many elements at one or more level, such as questioning assumptions, revealing multiple paths of causation, considering multiple variables, recognizing missing elements, and being tolerant of ambiguity. [Disciplinary/Assignment specific traits: ] Ideas Generated producing alternatives to solving problems, variations on a theme. [Disciplinary/Assignment specific traits: ] a. Fluency the ability to extend an idea (number of) b. Flexibility the ability to cross conceptual boundaries Completeness/Coherence being logically or aesthetically consistent with all separate parts fitting together to form a harmonious or credible whole. [Disciplinary/Assignment specific traits: ] Elegance presenting the product in a refined, understated way. [Disciplinary/Assignment specific traits: ]

22 9. Divergent Thinking going against the grain of the usual or expected in a useful way, not perversely or solipsisticly. [Disciplinary/Assignment specific traits: ] Novelty or Uniqueness (of Idea, Claim, Question, Form) newness in terms of new processes, techniques, materials, concepts used; effects on future creative products. [Disciplinary/Assignment specific traits: ] a. Germinal likely to suggest additional future creative products, etc b. Original unusual or infrequently seen in a universe of products made by people with similar experience and training c. Transformational an existing idea has been transformed via application in a new way or in a new context Engagement degree of attraction to, curiosity about, devotion to, or ownership of the task at hand. [Disciplinary/Assignment specific traits: ] Risk Taking the willingness to undertake a venture that may result in a loss or damage to oneself. [Disciplinary/Assignment specific traits: ] 17

23 Rating Scale Descriptions 6 = Demonstrates trait to an exceptionally high degree 5 = Demonstrates trait to a high degree 4 = Demonstrates trait to an intermediate degree 3 = Demonstrates trait to a sub-par degree 2 = Trait is barely evident 1 = Trait is not evident Definitions Creative Thinking 1. We call the products creative if they represent a transformation or a reconceptualization, have aesthetic coherence and appeal, represent a new configuration or connection of ideas, or serve some functional or explanatory purpose 2. Creative intelligence is involved when skills are used to create, invent, discover, imagine, suppose, or hypothesize. (Sternberg and Grigorenko). 3. Creativity is a novel and useful idea or product the confluence of intrinsic motivation, domain-relevant knowledge and abilities, and creativity-relevant skills ; the latter includes coping with complexities, knowledge of problem-solving heuristics, concentration, ability to set aside problems, and high energy (Sternberg 1999). 4. a confluence of six distinct but interrelated resources: intellectual abilities, knowledge, styles of thinking, personality, motivation, and environment (Sternberg 1999). Critical Thinking 1. Purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. (Facione 1990). 2. The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit. (Facione 1990). 18

24 SURVEY DATA ANALYSIS The major focus of the Ohio-5 grant supported by the Teagle Foundation is to assess creative and critical thinking in liberal arts students. Faculty have been instructed to use primary trait analysis to construct rubrics to measure changes in student creative and critical thinking across time, as indicated by course assignments. For additional background concerning student and faculty perceptions and experiences with critical and creative thinking, surveys were developed and administered. This report focuses on the survey data that were collected from first-year students, senior students, and faculty in Participants: Method On-line surveys were administered to students and faculty at the College of Wooster (COW), Denison University (DU), Kenyon College (KC), and Ohio Wesleyan University (OWU). Surveys were administered to first-year students, seniors, and faculty members. The total sample size in each group by college is shown below: COW DU KC OWU Total Freshmen Seniors Faculty Total In the group of freshmen, 71.6% of the participants were women and 78.5% were Caucasian. In the group of seniors these values were 71.5% women and 88.7% Caucasian; and among faculty, 46.9% female and 85% Caucasian. An attempt was made to balance faculty by discipline by recruiting faculty randomly within each division of the college, and it was hoped that there would be balance of students by discipline of major. (Since some students were double majors, the major they listed first was used to categorize by discipline unless one major was fine arts in which case student major was classified as fine arts due to possible low representation.) These values are indicated below: Fine Arts Humanities Nat Sciences Soc Sciences Freshmen 15.1% 40.1% 23.8% 20.4% Seniors 10.9% 28.5% 24.7% 35.9% Faculty 8.2% 32.2% 26.7% 32.9% 19

25 In terms of status of the faculty, 10.3% were visiting faculty not in a tenure-track position, 26.2% were tenure-track faculty who were not yet tenured, and 63.4% were tenured faculty. Faculty members were selected for participation by a survey coordinator at each school. Faculty were selected randomly within each division of the college and sent an with a link to either the critical thinking or creative thinking questionnaire. All first-year students at three of the schools (not DU); and seniors at all four schools were sent an to request their participation. One-half of students (selected randomly) received an that gave access to the survey with critical thinking questions listed first, and the other half of the students were sent access to a survey with creative questions listed first. All groups were sent reminder s to complete the surveys. Freshmen and faculty (with the exception of those at DU) were sent links starting in January, 2008; and seniors and faculty from DU were sent links starting in March, Survey: Surveys were developed in consultation with small groups of faculty from each of the four schools who were meeting to develop rubrics to assess critical and creative thinking. The survey was pilot tested in The student survey had ten sets of questions, and the two faculty surveys each contained four of the sets of questions. The ten sets of questions were as follows: 1 Words Associated with Critical Thinking Students and faculty received a list of 47 words and were asked to indicate whether or not they associated each word with critical thinking. The words were generated to be representative of creative thinking and/or critical thinking. 2 Words Associated with Creative Thinking Students and faculty received the same 47 words described above, but were asked to indicate whether or not they associated each word with creative thinking. 3 Critical Thinking Involved in Various Student Events Students made ratings of the amount of critical thinking involved in 12 different events such as writing papers for class. Ratings were made on a five-point likert scale with responses of 1 indicating the event did not involve critical thinking, and responses of 5 indicating that the event involved a great deal of critical thinking. 4 Creative Thinking Involved in Various Student Events Ratings of the amount of creative thinking involved in the same 12 events examined in section 3 were made using the same five-point scale. 20

26 5 Critical and Creative Thinking Activities of Students The frequency with which students engaged in 11 different activities related to critical and/or creative thinking such as brainstorming in class was examined. Students responded on a scale where 1 indicated that the student never engaged in the activity, and 6 meant they engaged in the activity frequently. 6 Campus Climate Conditions that Promote Creative Thinking Ratings of the campus climate related to creative thinking such as amount of challenge in the environment were made by both students and faculty. There were 14 different characteristics listed, and respondents used a rating scale where 1 indicated that the characteristic was not present at all, and 7 indicated that it was present to a great extent. 7 Critical Thinking Attitudes Agreement with various attitudes related to critical thinking was examined among both students and faculty. (There were several items that were not applicable to faculty, so did not appear on their questionnaire.) Agreement with each item was assessed with 1 indicating strong disagreement and 7 indicating strong agreement. 8 Creative Thinking Attitudes Agreement with various attitudes related to creative thinking was also examined in both students and faculty. (Several items were not relevant to faculty so were not included on their questionnaire.) Ratings were made on the same 7-point scale used for critical thinking attitudes. 9 Free-Response Questions Critical Thinking Students and faculty were asked a series of open-ended questions about critical thinking including where it occurred, whether there were barriers to it, whether they thought diversity affected it, whether they thought technology affected it, and to explain an experience that involved a great deal of critical thinking. Faculty were asked the same questions except for the last one. Faculty were also asked to indicate what factors indicated growth in students ability to think critically. 10 Free-Response Questions Creative Thinking Students and faculty were asked open-ended questions about creative thinking including where it occurred, whether there were barriers to it, whether they thought diversity affected it, whether they thought technology affected it, and to explain their most creative experience. Faculty were also asked to indicate what factors indicated growth in students ability to think creatively. 21

27 Student groups received all ten sections of the survey with the order of the questions counterbalanced such that half of the students received questions related to critical thinking first, and half of the students received questions about creative thinking first. In an effort to keep the faculty survey short, one half of the faculty received sections 1, 7, and 9; and one half received sections 2, 8 and 10. Both groups of faculty responded to section 6 questions. Additional demographic questions were asked of all participants such as gender, race/ethnicity, student major, faculty discipline, and tenure status of faculty. Appendices A-D contain copies of the four versions of the survey (two student versions with the same questions in different orders, one faculty critical thinking survey, and one faculty creative thinking survey). Procedure: All freshmen participants were recruited with the message that two of them would be chosen at random to receive $200. The faculty were recruited with two chances to win $100, and the seniors were recruited such that there were two chances on each campus to win $100. Participants were told that the address they provided to enter them in to the random sample to win the prize money would not be associated with their responses such that the confidentiality of the data would be maintained. Analyses: Comparing Freshmen Students to Senior Students. Data were compiled across all four schools for the main analyses that were conducted. For questions that were asked of students only, responses of freshmen were compared to those of seniors by each section of the questionnaire. Independent groups t-tests were conducted to compare the value of the means between the two groups. A significance level of p <.005 was set to guard against Type I error. Tables 1, 2 and 6 display data where freshmen were compared to seniors. Comparing All Three Groups. Both students and faculty responded to questions about the creative climate, critical thinking attitudes, and creative thinking attitudes. For these data the responses of freshmen, seniors, and faculty members were compared through one-way analyses of variance. The alpha level was again set at p <.005 to reduce the probability of Type I error. These data are displayed in tables 3, 4, and 5 and summarized below. Additional Analyses of Faculty and Senior Data. The data for seniors were examined for possible differences in responses related to discipline of major and gender. (The data from freshmen were not likely as valid as those from seniors given their limited experience so these data were not examined further.) Similarly, within the faculty data set the possible effects of discipline, gender, and tenure 22

28 status of faculty on the responses were examined. These data are shown in tables 7-11 and summarized below. Qualitative Data: Data yielded from sections 1, 2, 9, and 10 of the survey are qualitative rather than quantitative. The frequencies of various responses were examined and displayed in graphic form in Figures Differences in the frequencies of responses by school were examined through chi-square tests of association using an alpha value of p <.005, and these data are shown in the final figures. School-Specific Analyses. In addition to the main data report, each school was sent a report, which compared the quantitative responses of their seniors and faculty with those of seniors and faculty from the other three schools. Statistically significant differences were highlighted. Quantitative Data: Results The quantitative data are displayed in tables with the average response of each group noted. Highlights of the findings are summarized below. Table 1: Students Rate Critical and Creative Thinking of Various Events. Table 1 shows the extent to which freshmen and seniors perceive various events to be associated with critical and creative thinking. The event perceived by both groups to involve the most critical thinking was writing papers for classes. The event that was perceived to involve the most creative thinking was engaging in a hobby. One academic event that had fairly high ratings for both critical and creative thinking was completing projects for class. There were a few statistically significant differences in the responses of freshmen and seniors. Seniors perceived there to be more critical thinking involved in class interactions, participating in campus cultural events, and surfing the internet than freshmen. Seniors reported less creative thinking associated with communicating with peers over the internet than freshmen. Table 2: Frequency of Critical and Creative Activities of Students. In Table 2 the data concerning the frequency with which students engaged in various activities related to critical and creative thinking are displayed. Some of the more frequent activities included integrating material in various assignments and engaging in abstract thinking. There were a couple of significant differences between seniors and freshmen. Seniors reported a higher frequency of working on papers and projects that required integration, and putting ideas or concepts from different courses together. 23

29 Table 3: Student and Faculty Rate Creative Environment Characteristics. Generally, freshmen, seniors, and faculty rated their campus climate as conducive to creative thinking. In Table 3 various characteristics of a creative environment are listed along with the extent to which students and faculty perceived these to be part of their climate. Some of the characteristics that had the highest ratings included challenge and freedom. Among the faculty, two characteristics were rated as somewhat neutral indicating that they were not perceived to be present to a great extent in the environment, and these were idea time and risk taking. There were many statistically significant differences between the students and faculty in perceptions, mainly due to the fact that freshmen were the most optimistic in their ratings of the environment. In addition, seniors were more optimistic than faculty in their ratings of some characteristics such as challenge and supportive environment. Tables 4 & 5: Critical and Creative Thinking Attitudes of Students and Faculty. The responses to questions concerning attitudes about critical thinking are displayed in table 4. All three groups of participants expressed positive attitudes about critical thinking and its role in the liberal arts environment. Faculty indicated that they highly valued critical thinking, that they believed they could create conditions where critical thinking thrives, and that critical thinking can be assessed. Similar questions were asked about creative thinking and there was slightly lower agreement with these attitudes although values were still high. Some of the interesting differences between the groups were that faculty showed higher agreement than students that creativity should be taught in college courses, but expressed lower agreement that the college provides time and space to be creative. Tables 6 & 7: Critical and Creative Thinking Related to Discipline. In Table 6 we see that both freshmen and seniors rated both critical and creative thinking quite important for each discipline. However, critical thinking was seen as slightly less important in the fine arts than in other disciplines, and creative thinking was seen as slightly less important in the natural and physical sciences compared to other disciplines. Among the faculty, fine arts faculty rated creative thinking as more important to their discipline than did social science faculty. Table 8: Differences in Faculty Responses by Discipline, Gender, and Tenure Status of Faculty Member. The quantitative data from faculty members were examined to see if responses varied by discipline of faculty, gender, and tenure status. Only one significant difference emerged as shown in Table 8. Faculty in the fine arts were more likely to rate debate as characteristic of the school environment than faculty in either the humanities or the natural sciences. Tables 9 & 10: Differences in Senior Student Responses by Discipline of Major. 24

30 The quantitative responses of senior students were examined in relation to the discipline of the students major. Many significant differences emerged. In Table 9 it is revealed that students in the natural and physical sciences rated critical thinking as more important to their major than fine arts students did. Students in the fine arts rated creativity as more important to their major compared to others, especially natural and social science majors. Students reported different experiences and attitudes by discipline as shown in Table 10. Humanities majors were most likely to report working on assignments that required integration of material. Social science students were the least likely to use a story, metaphor, or visual in a class assignment. Perhaps not surprisingly, fine arts students were the most likely to see critical thinking as important in the fine arts. Social science students seemed to see creative thinking as less important to the other disciplines than other students did. Table 11: Differences in Senior Student Responses by Discipline of Major. There were even more significant differences between seniors based on gender as shown in Table 11. These differences reveal that women students were more positive in their attitudes and perceptions about both critical and creative thinking than men students. For example, women perceived a higher level of both critical and creative thinking to be involved in attending cultural events; and more critical thinking in class interactions and in writing papers for classes. Women rated the environment higher in several creative characteristics including challenge and risk taking. Women reported higher agreement that they valued creativity, that the college valued their efforts to be creative, and that the higher education system is conducive to facilitating creative thinking. Women rated critical thinking as more important in fine arts and humanities compared to men; and they rated creative thinking as more important in the humanities. Figures 1-4: Words Associated with Critical and Creative Thinking. Students and faculty indicated whether or not 47 different words were associated with critical and creative thinking. The proportion of individuals who selected each word as representative is shown in figures 1-4. Among the students the words most highly associated with critical thinking were analytical, interpreting, evaluating, and insightful. ( Insightful was also rated as characteristic of creative thinking by a fairly high proportion of individuals.) The words students associated most frequently with creative thinking included imaginative, artistic, visionary, and innovative. Faculty responses were similar. The words with most highly associated with critical thinking for faculty included analytical, evaluating, discerning, and inquisitive; and the words frequently associated with creative thinking were inventive, imaginative, innovative, and original. (For the faculty inquisitive and insightful were associated with both critical and creative thinking with a high degree of frequency.) 25

31 Free-Response Questions: To code responses to the open-ended questions the researcher examined all of the possible responses and developed categories for responses that occurred with a fair degree of frequency (at least.05). All responses were then categorized. It was possible for one person s response to a question to fall into multiple categories, so the categorization is not mutually exclusive. Thus, for responses to any particular question the proportions do not sum to 1. Figure 5: Where Critical Thinking Happens Figure 5 shows that a high proportion of both students (.79 of students) and faculty (.81 of faculty) responded that critical thinking occurred frequently in class. (In contrast, as shown in Figure 6, only.22 of students and.29 of faculty spontaneously associated the class setting with creative thinking.) Some representative quotes from students and faculty that illustrate these data are included below. It certainly thrives in classrooms, though more often in professors offices, where the students feel comfortable expressing themselves and bouncing their ideas off of someone they respect and who can help them learn. Naturally critical thinking often occurs at meals as well as students can test their ideas in a more casual manner. -Student, response classified as class, faculty, and stu life for student life It exists the most inside the classroom. I think a lot of students, including myself, envision a liberal arts college as a place where it also dominates out-of-class settings, but I don't think this vision matches reality in most cases. I do have a small circle of friends who exhibit critical thinking in our every day discussions, but you won't find it happening at most meals, parties, special events, etc. For extra-curricular activities, it all depends. I am in several activist groups and we are always thinking critically about how best to organize campaigns, distribute our message, and motivate people and get them involved with us. That probably isn't true for groups that are more focused on their own experiences and just having fun. -Student, response classified as class, stu life Seminar classes are a great place to find the kinds of discussion and openness necessary for critical thought. Here at the House of Thought we obviously foster that same atmosphere; and we extend that beyond the House in our weekly Socrates' Cafe forums. Some professors provide great opportunities for critical thought in their office hours and in other non-class interactions with students, though many do not. -Student, response classified as class, stu life for student life, faculty, and living since the House of Thought refers to a special interest living space 26

32 I think that our college creates a strong foundation for critical thinking during the initial First Year Seminar process. It sets the stage for "habits of the mind" that can be further developed in future coursework. -Faculty, response coded as class Classrooms and faculty offices. My perception is that it occurs and is valued only in the presence of faculty. -Faculty, response coded as class and faculty Figure 6: Where Creative Thinking Happens Figure 6 displays the categories related to where creative thinking happens on campus. Both students and faculty associated the arts most highly with creative thinking. Students were more likely than faculty to associate their living spaces and student life with creative thinking. Faculty were slightly more likely to associate classes with creative thinking. A few quotations that illustrate the data are included below. Independent projects, like choreographing a dance piece. Also, writing assignments and many homework projects. Living spaces as well, such as the homestead, and I think of my professors offices as creative, brainstorming spaces. -Student, response classified as ind research for the independent project, class for the class assignments, living, for reference to living space, and faculty for reference to the faculty In the classroom, especially when teacher utilized activities to engage students and to encourage them to consider the material and actively participate but also outside the classroom in the variety of co-curriculars offered and student jobs. -Student, coded as class and student life As an art student, obviously I see creativity flourishing in the studios on campus. But I also feel the Small Living Units generate creativity around their specific issues they champion as a house; for instance, the modern foreign language house is a house based on diversity of language and learning about cultural differences that come from a diverse population. I see that process of sharing information as creative. I think most collaborative projects of any kind involve creativity, whether in clubs or class, or just people having to get along with one another; wherever these people meet is a creative place. -Student, response coded as arts, living, class, and student life Creativity flourishes in any place where tolerance and acceptance exist and anxiety of judgment does not. -Faculty, response coded as most places 27