Second Student Pulse Survey Spring 2011 Students Perceptions of Effective Instructional Strategies. Summary of Methods and Findings.

Transcription

1 Second Student Pulse Survey Spring 2011 Students Perceptions of Effective Instructional Strategies Summary of Methods and Findings August, 2011 Submitted by Michele Hansen University College Steven Graunke Information Management and Institutional Research Daniel Trujillo University College Survey Developed with Drew Appleby Psychology Linda Haas Sociology

2 Contents Highlights... 3 Background... 4 Demographic Characteristics of Sample and Respondents... 4 Academic Characteristics of Sample and Respondents... 5 Q1: To what extent do the following teaching methods help you understand course material?... 6 Q2: To what extent to do the following teaching methods help you apply course material you have learned to work, other classroom settings, your major, your career, personal situations, or other academic projects?... 7 Q3: To what extent do you feel that IUPUI teaching methods are providing you with the knowledge, skills, and abilities to be successful in your major or career?... 8 Open Ended Items... 8 Q4: Please describe the teaching methods that help you learn best:... 9 Q5: Please describe the teaching methods that seem least effective in helping you remember, understand, or apply the material:... 9 Q6: What suggestions do you have for improving teaching methods in your classes? Appendix A: Student Pulse Survey: Effective Instructional Strategies Page 2

3 Highlights 353 IUPUI students responded to a web based PULSE survey designed to explore opinions and perceptions of various teaching methods. The adjusted response rate was 11.8%. Interactive Lectures received the #1 highest average rating on a four point Likert scale when students were asked to indicate the extent to which specific teaching methods helped them to Understand (M = 3.30) and Apply (M = 3.10) course material. Active Classroom Discussions received the #2 second highest average rating when students were asked to indicated the extent to which specific teaching methods helped them to Understand (M = 2.94) and Apply (M = 3.03) course material. 28% of student respondents indicated that IUPUI teaching methods Very Much provided the knowledge, skills, and abilities to be successful in a major or career. 48% of participants responded reported that IUPUI teaching methods helped Quite a Bit. 28% of respondents indicated that Interactive, Hands on Instruction was the teaching method that helped them learn the best. This was the #1 most common response provided. Student participants also reported Class Discussions (23%) and Lectures (Traditional Format) (21%) as effective teaching methods. 44% of student respondents indicated Lectures (Traditional Format with No Interaction) as a least effective teaching method in helping to remember, understand, or apply material. This was the #1 most frequent response provided. Student participants also reported Group Activities & Projects (24%) and a Lack of Interactive Learning (15%) as least effective teaching methods. Student participants also provided suggestions for improving teaching methods. More Student Interaction, Participation, & Hands on Learning was the #1 most common suggestion provided (24%). Students also provided the suggestions of Improve Instructor Support (17%), Improve Lectures (No PowerPoint Teaching) (15%), and More Application of Material / Real World Examples (13%). Student Pulse Survey: Effective Instructional Strategies Page 3

4 Background The Student Pulse Survey of Effective Instructional Strategies was designed to explore students opinions and perceptions of various teaching methods. A web based survey of a stratified random sample of 3,000 IUPUI students was conducted during the Spring 2010 semester. The Office of Planning and Institutional Improvement (PAII) funded the web survey which used a questionnaire developed by the Executive Director of Academic Planning and Assessment for University College, and the Assistant Director for Survey Research and Standard Reports. The survey was administered by the Office of Information Management and Institutional Research (IMIR). Of the 3,000 students invited to participate in this survey, 6 declined to participate or could not be reached via e mail, for an adjusted sample size of 2,994 students. Of those, 353 responded to the survey, yielding an adjusted response rate of 11.8%. Demographic Characteristics of Sample and Respondents The following table shows the demographic characteristics of survey respondents. Females were more likely to respond to the survey than males. In addition, 45% of survey respondents were 25 years of age or older compared to 37.9% of the entire sample. Respondents Entire Sample Sex * Percentages Female Male Race American Indian/Alaska Native Asian African American Caucasian International Student Hispanic/Latino Pacific Islander/Native Hawaiian Identifies as two or more racial groups Unknown Age** a Less than 21 years of age years of age years of age or older a As of March 11, 2011 * Statistically significant difference between respondents and entire sample at α <.001. ** Statistically significant difference between respondents and entire sample at α <.05. Student Pulse Survey: Effective Instructional Strategies Page 4

5 Academic Characteristics of Sample and Respondents The following table shows the academic characteristics of survey respondents. Thirty five percent of respondents had a cumulative GPA at or above 3.50 compared to 21.6% of the entire sample. Class Level a Respondents Entire Sample Percentages Associate First year Associate Second year Baccalaureate First year Baccalaureate Sophomore Baccalaureate Junior Baccalaureate Senior Certificate Seeking Other Kelley School of Business School of Dentistry School of Education School of Engineering and Technology Herron School of Art and Design School of Informatics School of Journalism School of Liberal Arts School of Medicine School of Nursing School of Physical Education and Tourism Management School of Science School of Continuing Studies School of Health and Rehabilitation Science School of Public and Environmental Administration School of Social Work University College Cumulative GPA* a 4.00 or higher a As of March 11, 2011 * Statistically significant difference between respondents and entire sample at α <.001. Student Pulse Survey: Effective Instructional Strategies Page 5

6 Q1: To what extent do the following teaching methods help you understand course material? Survey respondents indicated the extent to which specific teaching methods helped them to understand course material. Answers were provided on a 4 point Likert scale with (1) representing very little and (4) indicating very much. Interactive lectures received the highest average rating from respondents (M = 3.30), with approximately 57% of participants reporting that it helped them to understand course material very much. Student respondents also indicated active classroom discussions (M = 2.94) as an effective instructional strategy. Conversely, service learning received the lowest average rating from survey respondents (M = 2.38). These results are consistent with findings from the qualitative, openended survey questions. Lectures The instructor presents information and facts with limited student participation. Interactive Lectures The instructor involves students during lectures by pausing and asking questions, inserting brief demonstrations, or asking for short, ungraded writing exercises followed by class discussion. Small Group Work Working in small groups during class on problem solving activities. Active Classroom Discussions Discussing ideas from your readings or assignments with others during class. Electronic Media Using OnCourse, listservs, chat group, internet, instant messaging, or Facebook to discuss or complete an assignment. Service Learning Participating in a community based project or service learning as part of a regular course. N Very little Quite a Some bit Percentages Very much Mean SD * Scale: 1 = Very Little, 2 = Some, 3 = Quite a bit, 4 = Very much Student Pulse Survey: Effective Instructional Strategies Page 6

7 Q2: To what extent to do the following teaching methods help you apply course material you have learned to work, other classroom settings, your major, your career, personal situations, or other academic projects? Survey respondents indicated the extent to which specific teaching methods helped them to apply course material. Answers were provided on a 4 point Likert scale with (1) representing very little and (4) indicating very much. Interactive lectures received the highest average rating from respondents (M = 3.10). Student participants also reported active classroom discussions (M = 3.03) as an effective teaching method. Conversely, lectures received the lowest average rating from survey respondents (M = 2.32). These results are also consistent with the findings from the qualitative, open ended survey questions. Lectures The instructor presents information and facts with limited student participation. Interactive Lectures The instructor involves students during lectures by pausing and asking questions, inserting brief demonstrations, or asking for short, ungraded writing exercises followed by class discussion. Small Group Work Working in small groups during class on problem solving activities. Active Classroom Discussions Discussing ideas from your readings or assignments with others during class. Electronic Media Using OnCourse, listservs, chat group, internet, instant messaging, or Facebook to discuss or complete an assignment. Service Learning Participating in a community based project or service learning as part of a regular course. N Very little Quite a Some bit Percentages Very much Mean SD * Scale: 1 = Very Little, 2 = Some, 3 = Quite a bit, 4 = Very much Student Pulse Survey: Effective Instructional Strategies Page 7

8 Q3: To what extent do you feel that IUPUI teaching methods are providing you with the knowledge, skills, and abilities to be successful in your major or career? Twenty eight percent of respondents indicated that IUPUI teaching methods very much provided the knowledge, skills, and abilities to be successful in a major or career. Forty eight percent of student participants responded one Likert scale rating lower and reported that IUPUI teaching methods helped quite a bit in this regard. On the other hand, 20.4% of student survey respondents indicated some and 3.6% of students reported very little when asked this same question. 60.0% 50.0% Q3: To what extent do you feel that IUPUI teaching methods are providing you with the knowledge, skills, and abilities to be succesful in your major or career? 48.0% 40.0% 30.0% 28.0% 20.0% 20.4% 10.0% 3.6% 0.0% Percentages Very Much Quite a bit Some Very Little a N = 275, Mean = 3.00, SD = 0.80 Open Ended Items Three (3) open ended questions that gave students an opportunity to provide feedback in their own words. Student participants open ended questionnaire responses were first cleaned for errors and then uploaded into ATLAS.ti, a software program that assists in qualitative data analysis. A coding process was then employed in which student responses were arranged into topical theme categories. Categories were considered to be emerged or notable if 5% or more of student responses fit into that category. In some cases, one comment addressed more than one topic or category. In these instances, the comment was included in multiple analyses and considered in multiple areas of discussion. Many of the comments are concise statements and may not fully reflect the entirety of students opinions Student Pulse Survey: Effective Instructional Strategies Page 8

9 Q4: Please describe the teaching methods that help you learn best: (n = 234) Most Effective Teaching Methods N % Interactive, Hands on Instruction 66 28% Class Discussions 53 23% Lectures (Traditional Format) 49 21% Interactive Lectures 45 19% Small Group Work & Activities 36 15% Use of Visual Learning Aides (PowerPoint, graphics, etc.) 35 15% Application of Material / Real World Examples 32 14% Reading & Writing Activities 28 12% Positive Instructor Support 20 9% Use of Online Learning Tools 17 7% Note: Percentages are rounded to the nearest whole. The remaining responses were so varied that no major themes emerged. Q5: Please describe the teaching methods that seem least effective in helping you remember, understand, or apply the material: (n = 220) Least Effective Teaching Methods N % Lectures (Traditional Format) 97 44% Group Work, Activities, & Projects 53 24% Lack of Interactive Learning 32 15% Online Learning Activities 27 12% Use of Visual Aides ( PowerPoint Teaching ) 18 8% Class Discussions 18 8% Lack of Instructor Support 16 7% Reading & Writing Activities 13 6% Lack of Resources 11 5% Note: Percentages are rounded to the nearest whole. The remaining responses were so varied that no major themes emerged. Student Pulse Survey: Effective Instructional Strategies Page 9

10 Q6: What suggestions do you have for improving teaching methods in your classes? (n = 206) Suggestions for Improvement N % More Student Interaction, Participation, & Hands on Learning 49 24% Improve Instructor Support 34 17% Improve Lectures (No PowerPoint Teaching ) 30 15% More Application of Material / Real World Examples 26 13% General Positive Comment 23 11% Improve Course Structure & Organization 23 11% More Class Discussions 21 10% Develop Online & Multimedia Learning Technology 17 8% Improve Group Projects & Activities 13 6% N/A, None, Nothing 13 6% More Resources 11 5% Question & Answer Sessions 10 5% Note: Percentages are rounded to the nearest whole. The remaining responses were so varied that no major themes emerged. Student Pulse Survey: Effective Instructional Strategies Page 10

11 Appendix A: Pulse Survey Student Engagement in Learning / Perceptions of Effective Instructional Strategies Introduction (e mail text with Pulse Survey). Hello [student s first name], The IUPUI Student Pulse Survey is a program guided by student leaders, which helps the campus administration understand and improve the student experience at IUPUI. Our goal is to take the pulse of the student body, provide survey results to IUPUI students, staff, faculty, and administrators and make tangible improvements to the student experience. Responses to past Student Pulse Surveys caused IUPUI to examine and improve lighting on campus in order to enhance student safety and were used by student leaders to advocate for additional on campus housing and an enhanced Recreation Center. Your feedback really does make a difference! The Pulse survey linked here is designed to obtain your opinions about classroom teaching methods and what methods you find most effective for improving your learning and mastery of course material. The results of this survey will be shared with IUPUI leaders and then used to determine if there is a need to improve classroom teaching methods at IUPUI. These data may be merged with academic records so we can avoid asking questions about gender, class standing (freshman, sophomore, etc.), and other background characteristics. However, all of your answers will be confidential, and we will only be reporting your responses in summary form. Participation is voluntary and your relationship with IUPUI will not be affected by your decision to respond to the questionnaire or not. We encourage you to make your voice heard. Please take advantage of this opportunity to share your thoughts with IUPUI leadership! You may skip items or discontinue the survey at any time if you feel uncomfortable. You can complete this short questionnaire in less than five minutes by clicking this link: [insert link here] For more information, visit the Student Pulse website at or search for us on Facebook. Sincerely, Michele J. Hansen, Ph.D. Director of Assessment, University College mjhansen@iupui.edu Student Pulse Survey: Effective Instructional Strategies Page 11

12 To what extent do the following teaching methods help you understand course material. Please select N/A only if you have not had the opportunity to experience the particular teaching method at IUPUI. To what extent do the following teaching methods help you understand course material. Very little Some Quite a bit Very much N/A Lectures - The instructor presents information and facts with limited student participation. Interactive Lectures The instructor involves students during lectures by pausing and asking questions, inserting brief demonstrations, or asking for short, ungraded writing exercises followed by class discussion. Small Group Work - Working in small groups during class on problem solving activities. Active Classroom Discussions - Discussing ideas from your readings or assignments with others during class Electronic Media - Using OnCourse, listservs, chat group, Internet, instant messaging, or Facebook to discuss or complete an assignments. Service Learning - Participating in a community-based project or service learning as part of a regular course. To what extent to do the following teaching methods help you apply course material you have learned to work, other classroom settings, your major, your career, personal situations, or other academic projects? Please select N/A only if you have not had the opportunity to experience the particular teaching method at IUPUI. To what extent to do the following teaching methods help you apply course material you have learned to work, other classroom settings, your major, your career, personal situations, or other academic projects? Very little Some Quite a bit Very much N/A Lectures - The instructor presents information and facts with limited student participation. Interactive Lectures The instructor involves students during lectures by pausing and asking questions, inserting brief demonstrations, or asking for short, ungraded writing exercises followed by class discussion. Small Group Work - Working in small groups during class on problem solving activities. Active Classroom Discussions - Discussing ideas from your readings or assignments with others during class Electronic Media - Using On Course, listservs, chat group, Internet, instant messaging, or Facebook to discuss or complete an assignments. Service Learning - Participating in a community-based project or service learning as part of a regular course. To what extent do you feel that IUPUI teaching methods are providing you with the knowledge, skills, and abilities to be successful in your major or career? Very little Some Quite a bit Very much Student Pulse Survey: Effective Instructional Strategies Page 12

13 Please help us understand more about how you learn by answering the following questions: Please describe the teaching methods that help you learn best: Please describe the teaching methods that seem least effective in helping you to remember, understand, or apply the material: What suggestions do you have for improving teaching methods in your classes? Student Pulse Survey: Effective Instructional Strategies Page 13