Communication Program Assessment Report

Transcription

1 Communication Program Assessment Report Narrative: The communication faculty at Georgia Perimeter College (GPC) is committed to the mission of the college which includes strengthening student success, fostering a culture of teamwork, and enhancing the economic, social, and cultural vitality of the communities the college serves. Faculty commitment to the mission includes assessing the Communication Program of Study and individual communication courses. The Communication Program of Study includes a survey course in human communication (COMM 1100), a public speaking course (COMM 1201), a business and professional communication course (COMM 2300), an interpersonal communication course (COMM 2105) and an intercultural communication course (COMM 2900). To expand access and enrollment capacity, the communication courses are offered in a variety of course formats, i.e., face-to-face, online, and as a hybrid course. They are also designed for Honors students, learning communities, and with service learning components. All of the communication course offerings at Georgia Perimeter College, regardless of focus and format, are designed to meet the requirements for an Associate of Arts Degree with a concentration in Communication, and for transfer into baccalaureate programs in Speech Communication. Communication Program Goals I. Students will produce well organized communication that exhibits logical thinking and organization, use appropriate style for audience, and meet conventional standards of usage in the following ways: a. They exhibit logical thinking and organization through written assignments on various types of communication, instruction in listening, active listening exercises, and note taking. b. They exhibit use of appropriate style for audience through peer group activities, class discussions, and individual speaking assignments. c. They exhibit their ability to meet conventional standards of language use through comprehension of textbook and supplemental materials, analysis of student and professional reports, and assessment of media. II. Students demonstrate effective problem solving and critical thinking skills through interpreting, presenting, or evaluating ideas in the following ways: a. They interpret ideas through analyzing materials from lectures, research, and interview sources. b. They present ideas through learning a variety of strategies to resolve conflicts, effectively communicate in a verbal and nonverbal manner and perform individual and/or group assignments that reinforce these concepts. c. They evaluate ideas through readings, class discussions, and peer group activities. Note: In the communication classes, the general educational outcomes (I and II above) are assessed through grading a combination of the following learning activities: class participation, homework assignments, papers, individual and group projects, oral presentations, quizzes, and exams.

2 Assessment Goals: To ensure the integrity of the Communication Program of Study, the program is assessed regularly, and all communication courses are assessed regularly. The program was last assessed in 2006, and COMM 1201 was most recently assessed in the fall of COMM 2105 will be assessed in the summer of COMM 1100, COMM 2300, and COMM 2900 will be assessed in the academic year. In the future, all courses will be assessed on a three-year assessment cycle. OIRP s Executive Summary of the Fall 2012 COMM 1201 Assessment The COMM 1201 course assessment consisted of a pre and post assessment. Based on sample size calculations the survey response rates were more than acceptable and statistically reflect the target population. Students self-reported confidence ratings increased for all 30 items with mean scores on the post assessment consistently higher than the mean scores on the pre assessment. The question item with the lowest post-score average was related to nervousness in public speaking. Communicating with others and using communication skills to achieve career goals were the two items with the highest rated averages on the pre-test. Survey Design and Administration COMM 1201 (Communication 1201) Public Speaking is an introductory course in oral communication. The course stresses the fundamental principles of communication with special attention given to critical thinking, global issues, the selection and organization of materials, and the presentation of individual speeches and group deliberations. In Fall Semester 2011, all COMM 1201 class sections participated in an assessment of teaching and learning effectiveness. This survey had previously been administered in the Spring semester of As with this initial survey, students completed a pre and post assessment of the skills or behaviors that they were to develop while enrolled in Communication 1201 and assessed (self-reported) their confidence level on a range of communication skills. The survey instrument consisted of 30 questions asking students to rate their level of confidence on several skills and behaviors relevant to the course outcomes and objectives. The Likert rating scale consisted of 4 response options: no confidence, little confidence, much confidence or complete confidence. For a list of these questions, see Appendix A. These questions were slightly modified from the original survey and unlike the 2005 administration this Spring 2012 survey did not include questions related to campus location or instructor status (full-time vs. part-time).

3 The pre-assessment survey was conducted online using the Class Climate software and was delivered to students GPC addresses on August 26, Students who enrolled in the course after this date but prior to the drop/add date were sent their survey invitations on September 1. Instructors were asked to include information regarding the assessment within their syllabus and during class discussions and to encourage students to participate. In an attempt to further ensure compliance, the pre-survey was also delivered with Certificates of Participation that were automatically generated once a student answered and submitted responses to the question items on the survey instrument. Students were then instructed to send this certificate to their COMM 1201 course instructor. Several issues were experienced during the course of the pre-assessment. Many students indicated they did not receive the certificates even after completing the survey. The survey period was, therefore, extended to accommodate these students. In total, there were 3,264 invitations sent out for the pre course assessment with 2,442 responses. The post-assessment was administered as a paper and pencil survey for the face-to-face courses with strictly online classes continuing with the online administration. The post course assessment online surveys were delivered to students GPC accounts on November 9, The post course assessment hard copy instruments were delivered to each campus on November 8, Instructors were advised to administer the survey between Wednesday, November 9 and Friday, November 18. There were 3,140 students invited to participate in the post course assessment (both online and face to face) and 1,984 responses received. There were 10 courses in which no data was received either because the paper surveys were not received in time or the instructor opted not to participate in the assessment. Finally, as previously mentioned, some students reported issues with receiving the pre-course assessment certificate of completion. In some of these instances, there was no way to determine if the student had actually completed the survey and the results were received. These issues were handled on a case by case basis and some students received a second survey instrument to complete. There is a possibility that some of these students responses were recorded twice; however, based on communication with the Class Climate Survey Software techs, this is very unlikely. In conferring with the techs, their stance is that the system records when a password is used; so, if there was no record of the assigned password being used then the student did not complete the survey. If, perhaps, this is not correct, and there are instances of duplicate responses, the statistical impact of these responses is negligible given the number of instances in which this occurred (less than 100). Pre and post-tests were not matched by course section. For example, if Prof. X conducted the pre-test in her class at the beginning of the semester, but neglected to conduct the post-test at the end, her class was still included in the pre-test sample. This approach is methodologically sound because the project is intended to assess students experience in COMM 1201, aggregated across the entire college community, not to target the teaching effectiveness of specific instructors.

4 Survey Results Comparisons of the pre and post course assessments reveal improvement on every dimension. As with the previous assessment, students self-reported confidence ratings increased for all 30 items. Appendix B provides profile line comparisons of the results. The average response rating is provided for each variable or dimension on both the pre assessment (red profile line) and the post assessment (blue profile line). Amongst the dimensions with the smallest improvement, and the lowest post-score average was the first item related to nervousness in public speaking. These results are similar to the results from the 2005 assessment and are intuitively sound. Shyness or gregariousness is a personality trait and is, therefore, a more stable characteristic of an individual. Many of the other items on the assessment are skills or behaviors that can be taught, learned or modified much easier than any dimension of personality. Communicating with others and using communication skills to achieve career goals were the two items with the highest rated averages on the pre-test and therefore, the smallest gains or improvement on the post test. These smaller gains would be expected since students confidence levels were already fairly high. Additional summaries of pre-post gains are listed below. For the entire sample, the question items showing the largest gain (with total point gain shown in parentheses) were: 1. Q1.14 Describing differences between speaking impromptu and extemporaneously (.95) 2. Q1.15 Knowing what is expected as member of a small (panel) group (.85) 3. Q1.16 Knowing what is expected of a chairperson of panel group (.83) 4. Q1.22 Appropriately citing sources in speeches (.82) 5. Q1.28 Doing oral reports in other classes (.77) The items showing the smallest gain, (with total point gain shown in parentheses), were: 1. Q1.29 Communicating with others (.43) 2. Q1.30 Using my communication skills to achieve my career goals (.49) 3. Q1.27 Recognizing and avoiding plagiarism in a speech (.51) 4. Q1.1 Comfort (lack of nervousness) when speaking in front of a group (.55) 5. Q1.10 Using appropriate gestures, movement, facial expressions when speaking (.57)

5 For the entire sample, the Top 5 ranked factors for self-confidence on the pre-test (with average confidence rating in parentheses) were: 1. Q1.29 Communicating with others (3.13) 2. Q1.30 Using my communication skills to achieve my career goals (3.03) 3. Q1.27 Recognizing and avoiding plagiarism in a speech (2.98) 4. Q1.13 Explaining difference between informative and persuasive speech (2.84) 5. Q1.12 Understanding the importance of being aware of diversity in audiences when speaking (2.82) The Bottom 5 ranked factors for self confidence on the pre-test were: 1. Q14 Describing the difference between speaking impromptu and extemporaneously (2.27) 2. Q16 Knowing what expected of a chairperson in a small (panel) group (2.32) 3. Q21 Avoiding vocalized pauses ( Uhm, Ahh, OK, You Know, etc. (2.34) 4. Q1 Comfort (lack of nervousness) when speaking in front of group (2.373) 5. Q2 Using audience analysis to develop speeches (2.374) For the entire sample, the Top 5 ranked factors for self-confidence on the post-test were: 1. Q13 Explaining difference between informative and persuasive speech (3.59) 2. Q29 Communicating with others (3.56) 3. Q30 Using communication skills to achieve career goals (3.52) 4. Q27 Recognizing and avoiding plagiarism in a speech (3.49) 5. Q18 Describing advantages of rehearsing a speech (3.45) The Bottom 5 ranked factors for self confidence on the post-test were: 1. Q1 Comfort (lack of nervousness) when speaking in front of group (2.93) 2. Q2 Using audience analysis to develop speeches (3.06) 3. Q21 Avoiding vocalized pauses ( Uhm, Ahh, OK, You Know, etc. (3.07) 4. Q10 Using appropriate gestures, movement, facial expressions when speaking (3.125) 5. Q9 Delivering a speech while maintaining good eye contact with the audience (3.13)

6 Implications and Recommendations Results suggest that students in this sample enter COMM 1201 with moderate levels of confidence related to communication skills. These levels of confidence showed fairly consistent gains on the post assessment. As with any self-reported, qualitative assessment, final conclusion should be made in conjunction with other available data including assessment of direct course performance. The communications assessment committee is advised to use these survey results along with the previous data from 2005 and other assessment results to fully ascertain course outcomes. Since no data was directly collected on gender or age, this report does not include a summary of these demographics. However, data is available via our Banner Student Information System that will provide statistics on these variables. This information will be provided in a separate addendum.

7 APPENDIX A

8 APPENDIX B

9

10

11 Action Taken in Response to the COMM 1201 Assessment Findings The assessment findings were discussed with COMM 1201 faculty and representatives from GPC s Office of Institutional Research and Planning. And although everyone was pleased with the findings (Students selfreported confidence ratings increased for all 30 measures), the consensus is that a better assessment of a performance based course like COMM 1201, will be to video tape a set of speeches in all COMM 1201 classes and use a common rubric to grade a representative sample of those speeches to better assess students communication skills. Faculty plan to reassess COMM 1201 in the academic year. COMM 2105 Assessment Plan COMM 2105 will be assessed in the summer of All COMM 2105 students will be asked to apply what they learned in the class in a case study response. A committee of faculty who teach the course chose the case study all students will receive, and all student responses will be graded using a common rubric designed by the same committee of faculty. COMM 1100 Assessment Plan COMM 1100 will be assessed in the academic year. In the spring of 2012 a committee of faculty who teach the course and representatives from OIRP discussed two assessment options: a case study response, like the one planned for COMM 2105, and a common course test. That committee will choose one of these options by the summer of 2012 and design the assessment by the fall of The assessment will be administered by the summer of COMM 2300 Assessment Plan COMM 2300 will be assessed in the academic year. In the spring of 2012 a committee of faculty who teach the course and representatives from OIRP discussed two assessment options: a case study response, like the one planned for COMM 2105, and a common course test. That committee will choose one of these options by the fall of 2012 and design the assessment. The assessment will be administered by the summer of COMM 2900 Assessment Plan COMM 2900 will be assessed in the academic year. In the spring of 2012 a committee of faculty who teach the course and representatives from OIRP discussed two assessment options: a case study response, like the one planned for COMM 2105, and a common course test. That committee will choose one of these options by the fall of 2012 and design the assessment. The assessment will be administered by the summer of 2013.