1 Texas State University Application Running Head: TEXAS STATE UNIVERSITY APPLICATION 2011 Texas State University-San Marcos Application for the Basic Course Division of NCA Program of Excellence Award 2011 Course Directors: Marian L. Houser Kristen LeBlanc Farris
2 Texas State University Application I. Description of the Program With the first course in elocution and oral interpretation beginning in 1904, the Department of Communication Studies at Texas State University-San Marcos has a long tradition of instruction in the communication discipline. This rich history has allowed the development and evolution of the basic communication course as a General Education requirement for all students. The basic communication course must be completed in order to obtain an undergraduate degree for all students at Texas State. As the course provides instruction to many non-majors, the primary function is to offer a holistic presentation of the principles of human communication. The course design, requirements, and student enrollment are all elements that make this basic course unique and important to Texas State University. A. Course Design Entitled Fundamentals of Human Communication or COMM 1310, the basic communication course is a hybrid course that provides instruction in the intercultural, interpersonal, small group, and public speaking contexts. It is designed in a lecture-lab format such that students attend lectures to receive course content while attending lab sessions for experiential learning and skill building. The course is taught by a combination of full-time faculty, adjunct instructors, and graduate teaching assistants. The basic course serves the following functions: 1) to support the university s focus on providing students a multicultural education, 2) to impart the value of communication skills in our changing world, 3) as a means to attract students to become majors in the department, 4) for development of graduate students teaching ability and experience, and 5) provide funding for graduate teaching assistants. B. Course Requirements and Student Resources Students are required to complete the following components to achieve the learning outcomes for the course and receive course credit: three multiple-choice, scenario-based exams assessing course content and two presentations (informative and persuasive). The students are also required to complete an outline for each presentation as well as an audience adaptation plan. The purpose of these assignments is to assess and demonstrate students cognitive, behavioral, and affective success in the course. Student resources for the course include: a textbook, Communication Principles for a Lifetime, (Beebe, Beebe, & Ivy, 2010), a student guidebook with activities, grading rubrics, and tips for presentations (Houser, 2011), and a website of sample informative and persuasive speeches and outlines of former graduate teaching assistants. These resources provide students with necessary content information as well as complementary materials and experiential learning activities that enable them to succeed in the course. C. Course Enrollment Each fall and spring semester, the basic course offers approximately 100 sections and serves over 3000 students. Of these 100 sections, five are lecture-lab format with lecture classes averaging 300 students. These students then attend separate break-out lab classes taught by graduate teaching assistants. Lab classes average 30 students, and lab instructors engage in skill building and real-life application of course material. Additionally, lab instructors aid students in the development of presentations and outlines for the speeches in their lab courses. The remaining 45 sections are taught as independent stand-alone classes with a cap of 30 students. Thus, approximately five lecturers, 25 graduate teaching assistants and 25
3 Texas State University Application adjunct instructors teach the basic communication course each year. Past and current enrollment for the basic communication course is documented in the following chart: Table 1: The Growth of COMM 1310 at Texas State II. Rationale As stated in the description, COMM 1310 serves as a general education requirement on campus, in part, due to the development of multicultural components which meet the needs of all students and in line with the university s status as a Hispanic-Serving Institution. One goal of the university is to have an ethnicity and gender distribution representative of the State of Texas. COMM 1310 supports the university s efforts by teaching students to recognize and appreciate diversity as well as to respond and appropriately adapt to differences in messages from various cultures and genders to enhance understanding. Ultimately the Basic Course supports the 97 th Annual Convention s message of VOICE, as well as NCA s overall mission by promoting competent communication to improve the quality of human life and relationships, and the use of knowledge about communication to solve human problems (National Communication Association, n.d.). To this end, the basic course is organized and focused to meet the needs of its student population. The following information provides support for this rationale:
4 Texas State University Application A. Program Distinctions There are multiple components that help make COMM 1310 distinct among other basic courses across the United States. Size With over 5,000 students enrolled annually, we believe our program is among one of the largest hybrid communication courses in the United States. Once COMM 1310 was assured of its general education status, it became clear that more than 75% of the university s students would be enrolling in the course. For years the course was taught with 4 large lecture sections / breakout labs and adjunct sections. In order to meet the needs of increasing enrollment, a 5 th large lecture with approximately 360 students in each was added in Fall 2011 experienced the highest enrollment numbers totaling In order to ensure course material is delivered in a standardized manner, training is essential. Pre-Semester Instructor Training The basic course director and basic course administrator (graduate teaching assistant) plan and host an intense five-day Teaching and Learning Academy (TLA) one week prior to the start of each semester. New instructional assistants are required to attend each day, with graduate teaching assistants and adjunct instructors attending the final day. Each day of training is organized to meet the following objectives: Learn COMM 1310: principled approach, learning objectives, lecture/lab format, & its importance to the department, college, and university. Learn and administer, fairly and consistently, basic course policies Use TRACS university software to set up grade book, post announcements, etc. Plan, Prepare, Present, and Assess an experiential lab that meets the course s learning objectives outlined in the Student Guidebook. Use communication in the classroom to develop teacher-student relationships that yield appropriate power and influence. Manage student misbehaviors in and out of the classroom. Conduct student labs making a favorable impression, being confident, posed, organized, credible, and appropriately assertive. Each day of the TLA is organized into Instructional Modules: Perceptions, Power, and Learning in the Classroom; First Impressions: Preparing for Your 1 st lab; Dark Side of Being a Lab Instructor; Lab Demonstrations; and Basic Course Administration. Though labs are standardized, it is our hope to help each new instructional assistant locate their personal teaching voice to create course ownership. Instructor Support Materials Aside from the materials utilized for the week-long Teaching and Learning Academy training, each year the basic course director creates, and updates, the Student Guidebook. Each student purchases this 250 page text which directs them through the course by providing chapter activities, grading rubrics, exam study guides, and sample examination questions. This text helps
5 Texas State University Application students prepare for lab sessions and cuts down on lab preparation time for instructional assistants (e.g., preventing duplication of grading rubrics on speech days). The Instructor Resource Manual (IRM) is also created by the basic course director for each instructional assistant, graduate teaching assistant, and adjunct instructor. This 200 page text contains sample syllabi, course rules and regulations, exam procedures, new chapter activities, and advice for educational practices based on research in higher education and instructional communication. The Sample of Student Speeches, created by the Basic Course Director, is offered to each student purchasing a textbook for class. Students are supplied with an online access code that allows them to access six student speeches (three Informative and three Persuasive speeches) and their corresponding speech outlines. It is important for students to know what is expected and to understand the value in being organized and prepared. The speeches are delivered by former graduate teaching assistants in the program. Instructional Assistants use these sample speeches for in-class activities and homework assignments. We find this helps considerably with the communication apprehension students frequently experience and hope to avoid (Ayres & Huett, 2000). The DVD of student speeches is now being used by basic course programs at other universities. Just-In-Time Workshops To avoid information overload for the new instructional assistants, seven preparatory workshops are held at pertinent times throughout the semester. Exam Administration The Exam Administration workshop assists the new instructional assistants in administrating the standardized examinations for the basic course. Multicultural Awareness The Multicultural workshop was developed in fall 2011 and consists of lab activities designed to aid GIAs and GTAs in delivering appropriate multicultural learning tools to assist Texas State students. Approximately 60% of the students enrolled in the basic course indicate they have never traveled outside the state of Texas. Due to this as well as the status of Texas State as a Hispanic Serving Institution, it is important to create greater awareness and, ultimately, acceptance of diversity. Interpersonal & Small Group The Interpersonal/Small Group workshop allows experienced IAs to mentor and share their favorite labs with new GIAs. Presentational Speaking o Informative Presentation o Persuasive Presentation The three Presentational Speaking workshops are geared toward achieving Inter-Rater Reliability in grading the two speech assignments. It is our belief that if the basic course is to remain credible, it must be taught consistently. Therefore, GIAs watch, grade, and compare rubric scores to reach consistency.
6 Texas State University Application Course Wrap-Up The Course Wrap-Up workshop consists of discussions re: grade book, student absences, extra credit, and communicating with students who contest grades. B. The Communication Lab The primary goal of the Communication Lab is to enhance the oral communication competence of students through application of theory to course instruction and individual skill development. An online URL address has been set up to enable students to create appointments for the lab. The Lab gives students opportunities to learn by practicing and recording assignments and experiential activities. Both multicultural activities and speech presentations are videotaped. Students then view themselves and engage in self-critique along with a trained staff member who provides assessment and constructive criticism of student skills. In addition the staff, made up of instructional teaching assistants, work with individual students experiencing communication reticence and communication apprehension. Beginning in fall 2011, the Communication Lab is also used to broaden the multicultural awareness and understanding of Texas State students. The IAs and TAs who staff the lab are trained to lead groups of students in activities designed for the lab: The Diversity Board and Cross-Cultural Perceptions. The Diversity Board activity challenges and encourages students to seriously think about what diversity means and how diversity influences behaviors and communication between people. The Cross-Cultural Perception activity involves student demonstrations and observations of specific cultural behaviors that are indicative of specific cultural differences: individualistic vs. collectivistic, high vs. uncertainty avoidance, high vs. low power distance, masculine vs. feminine, and short-term vs. long-term time orientation. C. Course Objectives: Cognitive, Behavioral, and Affective Cognitive COMM 1310 is organized around five fundamental principles for human communication: Be aware of your communication with yourself and others. Effectively use and interpret verbal messages. Effectively use and interpret nonverbal messages. Listen and respond thoughtfully to others. Appropriately adapt messages to others These five principles are threaded throughout three course units: Unit 1 Understanding and Applying the 5 Principles; Unit 2 Communicating in the Interpersonal and Small Group Contexts; Unit 3 Presentational Speaking. Behavioral This component of the basic course focuses on communication skills we want students to practice and learn to use to become competent communicators. While behavioral practices are applied in student lab sections throughout Unit 1 (e.g., Intercultural Understanding & Adaptation Activity), in Unit 2 students are required to learn and practice Conflict Management skills from a cooperative approach. The PUGSS approach to conflict management (Describe Problem, Achieve Understanding, Identify Goals, Brainstorm Solutions, and Select best Solution) was created specifically for COMM 1310 and is included in the Student Guidebook to assist students
7 Texas State University Application in this process. Unit 3 requires students to deliver an Informative and Problem-Solution presentation. Both assignments require initial and revised outlines (using proper outline format) to assist them in speaking and writing processes. Affective We view COMM 1310 as a front-porch course as it serves to welcome and introduce Texas State students to the department, the major, and the discipline. Each instructor is invaluable in creating student affect toward communication and the class and each chapter is supported by activities to promote learning, application, and liking. These activities are consolidated within the Student Guidebook, IRM, and the Instructor Resources TRACS site (a collaborative online learning environment) created specifically for all 1310 instructors to engage in activitysharing. To support the university s multicultural focus, student attitudes and affect toward communicating in the multicultural context are examined and discussed within each text chapter. III. Departments, Programs, and Personnel The basic communication course is served by the following personnel with specific responsibilities to COMM 1310: Basic Course Director Marian L. Houser The Basic Course Director is responsible for the development of the basic course. Dr. Marian Houser primarily develops course content, creates examinations and grading rubrics, and administers instructor training to adjunct instructors and graduate instructional and teaching assistants through the Teaching and Learning Academy. Assistant Basic Course Director Kristen LeBlanc Farris The Assistant Basic Course Director, Kristen LeBlanc Farris, is primarily responsible for assessing the basic course (item analysis for exams and creation and analysis of assessment data) and providing course materials for adjunct instructors and stand-alone graduate teaching assistants. Basic Course Administrator Patrick Hale Patrick Hale, Basic Course Administrator (reassigned every 3 semesters), is a graduate student who assists the Basic Course Director in training other graduate teaching assistants and administering exams. Department Chair Steven A. Beebe Dean of College of Fine Arts and Communication Timothy P. Mottet Dean Timothy Mottet (beginning his first year fall as Dean of the College of Fine Arts and Communication) and Dr. Steven Beebe serve as administrators dealing with enrollment issues and adherence to university and general education policies. Lecturers Marian L. Houser, Kristen LeBlanc Farris, Richard Cheatham, Lisa Furler, Jeremy Hutchins Graduate Teaching Assistants Adjunct Instructors All five lecturers teach large sections of the basic course and supervise graduate instructional assistants who serve as lab instructors for the course. Some qualified and experienced graduate assistants (18+ graduate hours in communication studies), along with our adjunct instructors, teach stand-alone sections of the basic course.
8 Texas State University Application Systems Support Specialist Bob Hanna Bob Hanna, Systems Support Specialist, aids in the creation of DVDs and CDs which include PowerPoint slides and media clips that correspond with the textbook terminology. These materials are provided to adjunct instructors and stand-alone graduate teaching assistants to aid in the consistency of instruction. Bob Hanna also provides technological services and support to the basic course staff. Graphic Designer Malinda Murray Lastly, Malinda Murray, Graphic Designer, designs and publishes the teaching materials for the course. Program Collaboration As a basic communication course, we collaborate with other departments and organizations in the university system. One such program is Residential College, a program focused on creating an environment in which students integrate their academic and residential lives. As a means to help further the mission of this program, the basic communication course has specific lab sections devoted to students in this program. The students live together as well as have the same classes enabling them to foster a peer-focused learning experience. Additionally, we coordinate with the Partners in Academic Student Success (PASS) and those students who are admitted to the university on a probationary status as well and the Athletics departments by tracking and reporting student success in the basic communication course. This supplies valuable information and insight into the course progress of these students which are vital to their success. IV. Goals of the Nominated Program Communication 1310 Foundations of Human Communication remains an integral part of the undergraduate curriculum within the Department of Communication Studies and in the General Education Program for Texas State University-San Marcos. The goals of the basic course lend support to the mission, vision, and values at every level. A. COMM 1310 and the Department of Communication Studies Communication 1310 remains important to the department for four very important reasons: It provides the opportunity for teaching assistantships in our nationally-ranked MA program. These positions benefit many graduate students in residence and some very talented undergraduates who have an opportunity to determine their desire to pursue a graduate education in communication. The course yields department majors. Few students come to the university as a declared Communication Studies major. After being introduced to the department and major via COMM 1310, many students declare Communication Studies as their major or minor. The basic course facilitates learning about human communication for future majors who will transfer and apply the foundational principles into their upper division classes. Promoting the Department s Mission and Values remains a critical component for COMM 1310:
9 Texas State University Application Departmental Mission: We teach communication principles, research methods, and skills to Texas State students, produce and disseminate co-international constituency, and provide service to the department, the university, the community, and the profession. In support of this mission, COMM 1310 is centralized around the five principles of human communication. Providing a communication foundation for all students, these principles direct the teaching of communication concepts and theories and guide all assignments and activities. As a general education requirement all students are introduced to these concepts and skills. Departmental Vision: We aspire to be nationally and internationally recognized by students and peers for our high quality communication programs and faculty and to be recognized as having the best quality Communication Studies department in Texas. COMM 1310 utilizes the text Communication Principles for a Lifetime whose primary author is Dr. Steven A. Beebe, Chair of the Department of Communication Studies at Texas State. This connection opens the door to providing students first-hand experiences with communication research and scholarship with the hope of cultivating interest in and knowledge of the field. Instructional assistants and adjuncts learn, through the Teaching and Learning Academy training, the value in promoting recognition of faculty scholarship in the department as it pertains to theoretical concepts covered in COMM Departmental Values: We value high quality teaching. We value the importance and centrality of human communication in student education. We value learning principles and skills of human communication. We value openness, collegiality, and a supportive climate and envision a department that is recognized for these values. With these values in mind, the basic course provides a strong and supportive foundation. The Teaching and Learning Academy supports the value of quality teaching by preparing the instructional assistants and adjuncts to enter the classroom prepared to deliver quality classroom experiences to students that promote cognitive, behavioral, and affective objectives of the course. o Cognitive: The examinations in the basic course are application based, promoting knowledge, comprehension, and application to move them up the cognitive domain of Bloom s Taxonomy (Bloom, 1956). o Behavioral and Affective: The lab activities promote the psychomotor and affective domains of Bloom s Taxonomy by encouraging students to become aware, actively participate in, and value communication phenomena and the skills that accompany and enhance effective communication. o Affective: Students in the basic course become aware of the supportive climate created within the program through our consistent teaching and connectedness among instructors. Through our quality teaching and shared values, we reveal our support of the university mission as an institution dedicated to excellence in serving the educational needs of the diverse population of Texas and the world beyond.
10 Texas State University Application Course Objectives and Learning Outcomes Based on the departmental mission and vision, the basic course focuses on learning objectives and outcomes to assist students in understanding, applying, and valuing the communication concepts that will cultivate communication awareness and competency throughout their lives. List, describe, and explain the five principles of human communication and identify how they are integrated into the interpersonal, group/team, and presentational speaking contexts. Analyze and appropriately manage interpersonal conflict by using the five principles of human communication. Identify and describe the appropriate adaptive messages in intercultural communication situations and demonstrate appropriate affective responses to intercultural communication interactions. Develop, organize, and deliver an information and persuasive presentation integrating the five principles within them. B. COMM 1310 and The General Education Core Curriculum The Bachelor of General Studies (BGS) within the University College is an interdisciplinary degree that allows students to enhance personal goals by promoting a curriculum that provides a broad, diverse foundation for University learning. Students challenge themselves to investigate, examine, and synthesize individual subject areas. Goal: Prepare the student in the general abilities of questioning, explaining, and learning that remain universally useful in a rapidly changing world The Basic Course directly supports the goals of the General Education curriculum at Texas State by promoting communication skills, ideas, and concepts that are invaluable to graduates pursuing a career in a multitude of areas that require people skills and the ability to solve problems and relate to others. Specifically we actively embrace the mission and goals of the BGS by: Focusing on the application of skills within personal and professional lives. Advising students of career path connections. Supporting the general education and university s multicultural emphasis. Helping students understand and value the components of ethical communication. Meeting student needs and expectations for quality instruction. Revealing connections between the basic course and other university courses. Understanding the diversity of student experiences and personal and professional goals. Being focused on the student and the communication competencies required for lifelong learning. Contributing to the department, General Education and University retention efforts by creating links between students personal interests, educational opportunities, and affinity for learning. V. Outcomes of the Nominated Program Assessment has been a long established tradition in the basic communication course at Texas State University-San Marcos. In assessing our communication course, we hope to demonstrate:
11 Texas State University Application Student success in achieving course objectives The value of the basic communication course as a general education requirement How we use assessment information to improve course instruction We assess the basic communication course in the following ways: Analysis of student learning of course assignments Pre- and post-test quantitative analysis of student cognitive, behavioral, and affective dimensions of learning Assessment of COMM Lab effectiveness involves pre- and post-test examination of student presentation training Analysis of our instructors and their training in the Teaching and Learning Academy A. Assessment of Course Assignments Students are required to complete three multiple-choice exams, two 5-7 minute presentations, and an outline and audience adaptation plan for each respective presentation. Each course assignment is assessed and revised each semester based on the results of the assessment process. Assessment of Exams For each of the three exams, we conduct item analysis, reliability analysis, and a descriptive statistical analysis. Specifically, we examine the results of each question on the exam to determine whether the question should be counted toward the grade of the students, whether the question should continue to be on the exam, or whether we need to be more effective in teaching the course content in a specific area. We run analyses and find item discriminations to reach our conclusions. For instance, if an item is negatively discriminated, we examine a few possible reasons for the results. We then determine whether the question is ineffective or whether the question is too difficult or too easy. Further analysis of the items provides us with important information for the creation and revision of exams for the basic course. The table below is a partial example of the item analysis performed on an actual exam for our course. Table 2: Item Analysis for Exam 3 Item# A B C D E Blank Difficulty Discrim. Item# * * * * * * * * * * DISTRIBUTION OF ANSWERS ON KEY: A(11) B(15) C(11) D(13) E(0) # OF CASES MEAN STANDARD DEVIATION STANDARD ERROR (79.11%) RELIABILITY:.702
12 Texas State University Application How Assessment Results Improved the Course Based on the results of each exam, we continue to revise items to most accurately assess students cognitive learning of course material. Additionally, during post-examination lab sessions, students analyze their own exam preparation. Lab instructors discuss methods of preparing for the exams and provide students with additional advice about how to perform successfully on exams in the basic course. Assessment of Presentations Students are required to complete two 5-7 minute presentations in the course: one informative and one persuasive. Students are assessed using a standardized grading rubric that focuses on a student s ability to present an effective introduction, body, and conclusion as well as have proficient delivery skills. Students complete their presentations in lab classes, and thus, are assessed by lab instructors. In order to ensure consistent grading of student speeches, lab instructors are trained during the Teaching and Learning Academy using videotaped examples of student speeches. These examples range from unsuccessful to successful attempts. Lab instructors practice grading these speeches until sufficient inter-rater reliability has been achieved. We also examine and report students grades on both the outline and presentation assignments. We report the percentage of students who receive above 80% on their outline and presentation assignments for both the informative and persuasive presentations. The results in the table below were collected in the fall 2010 semester. Table 3: Assessment Results Report of Student Grades on Presentations Assignment N Outline 80% % of Students Grade 80% % of students Informative % % Persuasive % % Based on the results above, the basic course director and department chair concluded that our students needed more thorough example outlines and presentations. Thus, we incorporated more effective example outlines in the Student Guidebook and added a required online access of sample speeches and outlines to the course resources. Additionally, a new assessment instrument was implemented to further examine student achievement of presentational speaking skills. Beginning in the spring of 2011, we began utilizing the Competent Speaker Form to assess student speaking success. Rationale: to offer empirical evidence of the usefulness of the Communication Lab and the training provided to students as well as establish validity of grading rubric by comparing scores on the Competent Speaker public speaking assessment form sanctioned by NCA. Method: Using pre and post-test design-assess two videotaped speeches using the Competent Speaker Form (CSF; Morreale, 1994) o Control Group-no training o Experimental Group-received training over CSF competencies Pilot Test: Inter-rater reliability achieved prior to watching speeches Analysis phase of this project is currently being conducted.
13 Texas State University Application B. Quantitative Analysis of Learning Quantitative methodologies for assessing student cognitive, affective and behavioral learning are utilized in the basic course. A pretest-posttest design is utilized to demonstrate change in student learning. Our assessment process begins with the creation of assessment instruments and pilot testing of those instruments. Following data collection, information from the analysis is used to guide decisions regarding changing and revising course instruction and assessment methods. Tables 4 and 5 (below) are taken from a study of the basic communication course at Texas State University-San Marcos over the course of two full semesters (fall 2010 and spring 2011). A pretest-posttest design was utilized in the assessment process. Instructors administered the pretest at the beginning of the semester before content instruction and the posttest was administered to the same group of students at the close of the semester. Participants in the study consisted of 692 students, representing 25% of the entire student population enrolled during that semester. Students were asked to complete the tests without utilizing textbooks or notes. Instruments were strategically selected for the ability to meet NCA s established criteria for assessment practices (National Communication Association, n.d.) as well as to measure the objectives defined by the General Education Council, the department, and indicators of student learning. Results were as follows: Table 4: Results of Cognitive and Affective Assessment Analysis Cognitive and Affective Pre and Post Test Results Cognitive Results: Mean (pre) Mean (post) t(692)= , p <.001* Affective Results: Mean (pre) 3.86 Mean (post) 4.04 t(692)= -7.57, p <.001* Table 5: Results of Behavioral Assessment Analysis Interpersonal Conflict Competence Pre and Post Test Results Total Conflict Management Results: Mean (pre) Mean (post) t(692)= -9.62, p <.001* Conflict Motivation Results: Mean (pre) Mean (post) t(692)= -1.11, p >.05** Conflict Knowledge Results: Mean (pre) Mean (post) t(692)= , p <.001 Conflict Skill Results: Mean (pre) Mean (post) t(692)= -6.86, p <.001
14 Texas State University Application Cognitive Assessment The instrument utilized in the cognitive assessment was developed by the assistant basic course director. This instrument was revised in 2010 (based on previous assessment results) and has undergone further edits to ensure clarity. Similar to the previous instrument, the revised measure consists of 25 multiple-choice items (each with four response choices) developed to accurately assesses course content surrounding the principles of human communication. Results indicate students cognitive learning is improving from the beginning to the end of the semester. For complete results, see Table 4. We are encouraged by the results of the cognitive assessment, but continue revising the instrument in order to assess the cognitive learning objectives of our course as effectively as possible (see Appendix A). Affective Assessment The second assessment instrument utilized to assess the basic course was the Personal Report of Intercultural Communication Apprehension (PRICA; Neuliep & McCroskey, 1997). This instrument was selected based on the focus of intercultural competence in the course objectives. Additionally, communication apprehension, and in this instance, intercultural communication apprehension, has been identified as an assessment of affect toward communication by previous researchers and educators (Comadena, Hunt, & Simonds, 2007). Thus, this instrument was selected because it effectively measures and demonstrates students affect toward the course, as they willingly utilize the course material to alter their communication outside of the classroom with individuals of varying cultures. The PRICA measures an individual s perceived apprehension when communicating with people from different cultural groups. Results indicated students were less apprehensive when communicating with individuals of different races and/or cultures after completing the course. The outcome of the statistical analysis suggests that students not only developed an awareness of their intercultural fears, but were less apprehensive when considering a communication encounter with individuals of different cultures after taking the basic course. Results of the current assessment can be found in Table 4. Although results of the current assessment indicate instrument reliability, for the fall 2011 semester, the basic course director and assistant director investigated implementation of a new instrument measuring the affective component of learning. The Cultural Sensitivity Scale (Chen & Starosta, 2000) appeared to more accurately assess this learning objective. The scale asks students to indicate on a scale of 1-5 how sensitive they are to people of differing cultures. The director and assistant director continue to find innovative ways to incorporate cultural awareness such as the new multicultural dimension of our Communication Lab. As this is a major component for making it a part of the general education curriculum and aligning with the university vision, it is imperative that instructors continue infusing this content into the course. Behavioral Assessment The third and final assessment instrument is a revised version of Cupach and Spitzberg s (1981) Self-Rated Competence Scale, which measures self-perceived conflict management competence (see Appendix B). One objective of the course focuses on students conflict management skills, thus this scale was selected for its ability to accurately and reliably assess communication competence in the area of conflict management. This instrument was implemented in the assessment process to measure the behavioral dimension of learning for the basic course. The
15 Texas State University Application instrument consists of 18 items, and asks students to indicate how motivated, knowledgeable, and skillful they are at managing conflict in relationships. Results revealed the Mean Score of the students perceived conflict management skills was significantly higher at the end of the semester. This result indicates instruction and experiential skill building provided during the semester likely contributed to the improvement of students behavioral learning of conflict management skills. Although students scores on motivation for managing conflict were not significantly higher at the end of the semester, we understand the reasoning for this. Students enrolled in the class with high mean scores on the motivation component, thus they were already motivated to manage conflict prior to enrolling in the course. They just needed to learn the proper skills and concepts to successfully achieve conflict management, which our data ensures is happening through the basic course. We made considerable revisions to the process of assessing this learning objective as well as to the process of teaching this portion of this unit within the last year. Results from this year s assessment can be found in Table 5. Demographic Information & Assessment The last revision made to the pre-and post-assessment quantitative analysis was the inclusion of a demographic questionnaire this past year. This provided invaluable information regarding the diversity of students who are facing challenges with the course material. Demographic information in the questionnaire include: sex, ethnicity, university classification, and class enrollment information (whether or not the student has taken our course before). Results indicated no significant differences between the groups. This analysis suggests all student groups are able to achieve cognitive, affective, and behavioral learning similarly. C. Instructor Assessment Although assessment in the basic course is primarily related to student learning, we also assess our instructors and their training. As a component of the Teaching and Learning Academy, first time lab instructors are observed while teaching concepts and interacting with students in skill building and experiential learning activities. This allows us to provide feedback to ensure that they are utilizing effective methods of instructional communication. Additionally, this observation and assessment of instructors helps ensure student learning is facilitated. After each instructor training session, a qualitative assessment is completed by instructors (See Table 6).This provides quality information regarding the usefulness of the training seminars. Based on this information, we continue to revise training sessions. Table 6: Just-In-Time Workshop Feedback Form 1. List two to three significant learnings from today s training session. How do you see yourself using this knowledge or insight? 2. What concerns do you have that we can continue to address? 3. What did you like most about today s training session? 4. What did you like least about today s training session? In addition to training feedback, the assistant basic course director created an instrument to use while observing all basic course instructors in the classroom. It is comprised of 26 quantitative questions and asks the observers (Basic Course Lecturers See personnel in Part III) to rate the instructors on a likert-type scale between 1-7, with one indicating Strongly Disagree and seven
16 Texas State University Application indicating Strongly Agree. The instrument also includes three open-ended questions focusing on positive aspects of the teaching session as well as suggestions for improvement (see Appendix C). Two pilot tests have been conducted to determine reliability and validity of the instrument. Pilot Test 1: Three teaching scripts (Highly effective teacher, Somewhat effective teacher, Ineffective teacher) were given to six experienced instructors in the department to determine agreement on High, Medium, and Low teaching effectiveness. Inter-rater reliability scores were as follows: o Ineffective Teaching Script 88.89% agreement o Somewhat Effective Teaching Script 61.1% agreement o Highly Effective Teaching Script 100% agreement Pilot Test 2: Following reliability analysis of the instrument utilizing the teaching scripts, three lecturers observed and assessed GIAs and GTAs using the observation form. o Results and Analysis: Factor analysis and reliability analysis were completed. Correlations were also tested between instructors scores on observation form and two instructor scales (Teacher Communication Concern and Teacher Sense of Self Efficacy). Five Factor Solution was found Factor 1 Student Interaction (α =.89) Factor 2 Presentation (α =.84) Factor 3 Unpacking (α =.83) Factor 4 Instructional Skills (α =.33) Factor 5 Clarification (α =.66) Final Correlations were not significant o In addition, to achieve validity of the observation scale, instructors also completed two instruments: Teacher Communication Concern scale measures concerns about teaching performance and fulfillment of their duties (TCC; Feezel & Myers, 1997) Teachers Sense of Self-Efficacy scale measures instructors self-efficacy in terms of instructional strategies, classroom management, and student engagement (TSES; Tschannen-Moran & Woolfolk Hoy, 2001) o Based upon results of the analysis above, revisions of the observation form have been completed. Second round of observations (with the addition of adjunct faculty) will occur fall VI. Endorsements You will find attached, endorsements from the following individuals: Dr. Steven A. Beebe Chair, Department of Communication Studies; Associate Dean, College of Fine Arts and Communication Dr. Timothy P. Mottet Dean, College of Fine Arts and Communication Dr. Gene Bourgeois Provost and Vice President of Academic Affairs, Texas State University- San Marcos
17 Texas State University Application References Ayres, J., & Huett, B. (2000). An examination of the long-term effect of performance visualization. Communication Research Reports, 17, Beebe, S. A., Beebe, s. J., & Ivy, D. K. (2010). Communication: Principles for a lifetime (4 th ed.). Boston: Allyn and Bacon. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc. Chen, G. M., & Starosta, W. J. (2000). The development and validation of the intercultural sensitivity scale. Human Communication, 3, Comadena, M. F., Hunt, S. K., & Simonds, C. J. (2007). The effects of teacher clarity, nonverbal immediacy, and caring on student motivation, affective and cognitive learning. Communication Research Reports, 24, Cupach, W. R., & Spitzberg, B. H. (1981). Relational competence: Measurement and validation. W.S.C.A. San Jose, CA. Feezel, J. D. & Myers, S. A. (1997). Assessing graduate assistant teacher communication. Communication Quarterly, 45(3), Houser, M. L. (2011). Fundamentals of human communication: COMM 1310 guidebook San Marcos, TX: Minuteman Press. Morreale, S. P. (1994). Public speaking. In W.G. Christ (Ed.), Assessing communication education (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates. National Communication Association (n.d.). NCA s Mission. Retrieved from Neuliep, J. W., & McCroskey, J. C. (1997). The development of intercultural and interethnic
18 Texas State University Application communication apprehension scales. Communication Research Reports, 14, Tschannen-Moran, M & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17,
19 Texas State University Application Appendix A: Cognitive Assessment Instrument 1. When ineffective communicators mindlessly or thoughtlessly say and do things that they may later regret, they are ignoring which fundamental communication principle? 1. Effectively use and interpret verbal messages 2. Effectively use and interpret nonverbal messages 3. Be aware of your communication with yourself and others 4. Appropriately adapt messages to others 2. On a recent Caribbean cruise, Cody commented to a friend, I feel strange talking to the room stewards. They re all from interesting places and their English is excellent, but there s nothing we can talk about. Cody s use of assuming differences prohibited him from achieving which principle of human communication? 1. Be aware of your communication with yourself and others 2. Appropriately adapt messages to others 3. Effectively use and interpret verbal messages 4. Effectively use and interpret nonverbal messages 3. Kristen works at Disney World, and is referred to as a cast member instead of a cashier or employee. Due to this use of language, she feels more proud of her job. This reflects the Sapir-Whorf Hypothesis that helps us better explain the importance of which of the fundamental communication principles below? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret verbal messages 3. Listen and respond thoughtfully to others 4. Appropriately adapt messages to others 4. Marsha prides herself in being other-oriented by being able to tune out internal and external noise when others are talking to her and sharing information. Unfortunately, she realizes that most people she meets are very egocentric and insensitive. Which principle of human communication does Marsha follow and use? 1. Listen and respond thoughtfully to others 2. Appropriately adapt to others 3. Effectively use and interpret nonverbal messages 4. Effectively use and interpret verbal messages 5. Principle 3: Effectively use and interpret nonverbal messages relates directly to which of the following concepts? 1. Self-image 2. Haptics 3. Neologism 4. Information overload 6. Principle 4: Listen and respond thoughtfully to others relates directly to all of the concepts below EXCEPT: 1. Information overload 2. Paraphrasing 3. Attending 4. Reframing 7. Understanding which of the following concepts will allow you to become more effective at Principle 5: Appropriately adapting messages to others? 1. Ethnocentrism 2. Neologism 3. Immediacy 4. Emotional noise 8. Marissa remains an incredibly shy person when she is around other people, especially when meeting them for the first time. She realizes that some people misperceive her shyness as being stuck up, aloof, and/or arrogant. To correct this perception, she constantly reminds herself to talk more with others when meeting them for the first time. This example most accurately represents which principle of human communication? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret verbal messages 3. Effectively use and interpret nonverbal messages 4. Listen and respond thoughtfully to others 9. When effective communicators select appropriate language to communicate with others, they are following which of the fundamental communication principles? 1. Effectively use and interpret verbal messages 2. Effectively use and interpret nonverbal messages 3. Appropriately adapt messages to others 4. Listen and respond thoughtfully to others
20 Texas State University Application Jen and Lisa are tubing down the Guadalupe River. Jen sees a group of good-looking guys floating their way. As they near, Jen and Lisa suck in their stomachs, tense up their muscles, and try not to look at the guys as they are approaching. Their use of courtship readiness cues are examples of which principle of human communication? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret verbal messages 3. Effectively use and interpret nonverbal messages 4. Listen thoughtfully to others 11. Theresa knew that her audience would be skeptical of some of the ideas in her speech, so she decided to present the ideas she figured the audience would agree with first. Which of the principles of communication is Theresa attempting to use when organizing and outlining her presentation? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret verbal messages 3. Appropriately adapt messages to others 4. Listen and respond thoughtfully to others 12. Which of the following two symptoms of groupthink is a result of group members failing to follow Principle 1: Be aware of your communication with yourself and others? 1. Not evaluating ideas and encouraging everyone to participate 2. Feeling apathetic about a task and failing to challenge ideas 3. Prematurely evaluating ideas while including all group members 4. Feeling motivated about the task and challenging ideas as they are presented 13. To comfort his upset friend Hunter, Austin says, I am so sorry that you lost your job this week. I can imagine that you are feeling very upset about it. Is there anything I can do to help you to feel better? Austin s use of empathic language represents which principle of human communication? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret nonverbal messages 3. Effectively use and interpret verbal messages 4. Appropriately adapt messages to others 14. Effective communicators examine others behavior for facial expressions, eye contact, gestures, and posture in order to interpret their messages. These communicators are using which principle of human communication? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret nonverbal messages 3. Effectively use and interpret verbal messages 4. Appropriately adapt messages to others 15. Principle 1: Being aware of your communication with yourself and others is related to all of the following concepts EXCEPT: 1. Self-concept 2. Self-esteem 3. Perceptions 4. Neologism 16. Which of the following concepts will be impacted by Principle 2: effectively use and interpret verbal messages? 1. Bypassing 2. Ethnocentrism 3. Perception checking 4. Territoriality 17. While listening to Savannah talk about her new job, Connor notices that she has animated facial expressions and open posture, is smiling a lot, is making direct eye contact, and is speaking with a raised pitch and tempo. When Savannah is done speaking, Connor says to her, So you re feeling pretty good about this job. It s clear that you re really excited. Connor s use of paraphrasing shows that he is adept in which of the following principles of human communication? 1. Effectively use and interpret nonverbal messages 2. Effectively use and interpret verbal messages 3. Listen and respond thoughtfully to others 4. Appropriately adapt messages to others 18. The dictionary defines apartment as a room or suite of rooms used as a residence. However, Sally feels the word apartment means a comfortable place to relax at the end of the day. This difference between denotative and connotative language reflects which principle of human communication? 1. Be aware of your communication with yourself and others 2. Effectively use and interpret verbal messages 3. Effectively use and interpret nonverbal messages 4. Appropriately adapt messages to others