What Do College Students Think about General Education and Assessment? J. Christine Harmes. James Madison University. B. J. Miller

Transcription

1 What Do College Students Think about General Education and Assessment? J. Christine Harmes James Madison University B. J. Miller Eastern Mennonite University Paper presented at the annual meeting of the Northeastern Educational Research Association, Rocky Hill, CT, October 16-18, 2007.

2 General Education Focus Groups 2 A common goal across many colleges and universities is that of providing a foundational set of courses that all students must complete. This is often referred to as a general education requirement, and many institutions encourage students to complete these requirements before beginning coursework in their major area of study. However, the literature in higher education indicates that the intentions of general education programs may not be reaching fruition (e.g., Arnold, 2006; Ferren & Kinch, 2003; Glynn, Aultman, & Owens, 2005). The purpose of this study was to explore the attitudes and beliefs of university students regarding general education at a midsized, mid-atlantic university. Three other universities recently made good use of the focus group format in researching student attitudes toward general education. At the University of Wisconsin-La Crosse, focus groups composed of students who had completed general education requirements were asked: (1) how they would describe the university s general education program, (2) how they had changed because of it, (3) how general education courses differ from in-major courses, and (4) how the general education program could be improved (University of Wisconsin-La Crosse, 2005). Students revealed that general education courses were presented as an introduction to a major and a perception of the program as a collection of courses without coherence, connection, or meaning (p. 7). Focus group findings were used to provide input for the general education committee when the program was recently restructured. Recommended changes based on the findings included the formal adoption of mission, goal, and outcome statements, as well as the institution of a common first-year foundation (pp. 8-9). At Utah State University, focus groups were used to gain an understanding of the student s experience in general education beyond what was known from satisfaction surveys (Kleinke, 2005). Groups of freshmen, sophomores, and graduating seniors were asked a set of eight questions exploring their perceptions of the general education program in areas such as transparency of the

3 General Education Focus Groups 3 purpose and requirements, difficulties in registration, student learning, and frustrations with the program. Findings were used to identify areas of the program to target for improvement. For example, the students indicated a need for more sources of information as well as more complete information about the purpose and requirements of the general education program. The current study was largely modeled on focus group research conducted at a third university, a sister public institution in Virginia. In 2001, student focus group sessions were held in order to gauge attitudes toward the core curriculum at Virginia Tech (Muffo, 2001). The group discussions were guided by questions regarding the purpose, benefits, and drawbacks of the core curriculum. Students were also asked for suggestions for improving the curriculum. Results from this study suggested that students perceived these courses to be of little value and difficult to schedule. Students were also of the opinion that faculty were unenthusiastic about teaching these courses and failed to connect individual course goals to the larger general education goals. Other findings included the sentiment that the core curriculum helped students to become well-rounded individuals, (p. 1) but that students select their core courses based on a perceived lack of rigor. For the past three decades, focus group research has been increasingly accepted and applied in educational settings (Krueger & Casey, 2000). Conducted properly and rigorously, this interview format provides rich, detailed information and permits valid inferences about a subject of interest. Focus groups have provided useful data for critical educational functions such as planning, decisionmaking, and program improvement (Fabiano & Underhile, 2007). Guidelines for conducting successful focus groups include (Denzin & Lincoln, 2000; Krueger & Casey, 2000; Morgan, 1988): Select a small group of four to twelve participants based on some common interest or characteristic, depending upon the purpose of the study; Employ a skillful moderator who is sensitive to the purpose of the study as well as to group dynamics;

4 General Education Focus Groups 4 Adhere to the goal of collecting information (rather than reaching consensus or making decisions); Prepare open-ended questions in a natural, logical order suited to the purpose of the study. Method Participants A total of 16 students were excused from a testing classroom during a university-wide assessment day to participate in the focus group interviews. Participants were selected by starting at the front of a large lecture hall and randomly choosing students from the ends of the rows until the desired number had been selected. Seven students comprised the focus group in the morning session, and six students comprised the group in the afternoon. The morning group included four females and three males, while the afternoon group included two females and four males. The morning session included additional preliminary questions that led the group in a different direction. Thus, results from the afternoon session were used as the primary data set. Participants represented a variety of majors and academic units, such as history computer science, kinesiology, and psychology. Procedures University sophomores participated in focus group interviews to help us understand the extent of students knowledge about and feelings toward the general education and assessment programs at this university. A semi-structured interview protocol was employed. That is, a set of interview questions was created before the focus group sessions were conducted. However, followup questions and probes were not pre-determined. Discussions were stimulated by the following prompts, but were permitted to diverge from the exact questions: 1. Describe the general education program at our university

5 General Education Focus Groups 5 2. What is the purpose of the general education program? 3. What is lacking in the general education program? 4. What would you remove from the general education program? 5. If you had to provide evidence for the effectiveness of the general education program, how would you do so? Analysis Audio recordings from the focus groups sessions were transcribed and organized by question. Data were then analyzed following the procedures for thematic networks, outlined by Attride-Stirling (2001). This six-step process involves: a) coding, b) identifying themes, c) constructing the networks, d) describing thematic networks, e) summarizing the networks, and f) interpreting the patterns. In an iterative process, the first coder read through the transcripts several times to establish a coding framework. This framework, constructed from key terms and issues that emerged from the transcripts, was used to go back through the data and code individual segments (passages, or meaningful chunks of text). Once the preliminary coding had been completed, the researchers discussed the results and made refinements to the framework in an iterative fashion. As coding and theme development were conducted iteratively and in consultation by both researchers, computation of agreement indices was not appropriate. Any coding disagreements between the researchers were resolved within the iterative theme development process. The data segments were organized by codes, and then re-examined to find basic themes. These basic themes were then refined to ensure that they were specific enough to be discrete and broad enough to encapsulate a set of ideas contained in numerous text segments (Attride-Stirling, 2001, p. 392). Once this set of basic themes had been established, they were further interpreted to find logical groupings. These groupings were characterized by shared issues or underlying concepts, and resulted in the organizing themes. The organizing themes were then examined to find the

6 General Education Focus Groups 6 central idea, or main point, which became the global theme. Thematic networks were created to provide a visual representation, or concept map, to facilitate interpretation. Results In general, participants were actively engaged in the discussion, and appeared to be quite willing to share their views in an open and honest manner. This was especially encouraging, given the potential power issues that may have concerned these undergraduate students by having a doctoral student and a faculty member serve as the group moderators. Following is a discussion of responses to the interview prompts, supplemented with selected quotes. Description of the General Education Program In response to the first prompt, Describe the general education program at our university, students indicated that the program provides an overview of the basics and broad, general knowledge. Several participants said that the program seems somewhat disjointed or lacking in focus. The discussion quickly turned from description to sharing of likes and dislikes regarding the program. Students liked the opportunity to be exposed to new things, and especially liked the courses that had a practical focus, such as the writing course. One student felt that math and science were weighted more heavily than other subject areas. Sources of dissatisfaction were related to a perceived lack of choice, the amount of time and expense needed to complete the requirements, inconsistency across course sections, and student and faculty buy-in. The group expressed frustration that the general education requirements keep them from getting started in their major coursework. They view the general education classes as easy, not real classes, and watered-down versions of classes. Participants shared that they would like for these classes to present more of a challenge, and to be structured in a manner more similar to classes in their major areas. They specifically cited a desire to have more discussion-based sessions. Overall, they expressed a mismatch between the program intent and reality.

7 General Education Focus Groups 7 Purpose for General Education In response to the second question, what is the university s purpose for general education, the following themes emerged: good intent, liberal arts curriculum, not true to intent, too broad, expensive, and constricting. One student shared that some students have changed their majors because of a general education class that piqued their interest. To elicit more detail, a follow-up question was posed to the group: Does the university meet its goals for the clusters? Students were provided with handouts highlighting the goals of the general education program. The first of these was the goal that students become informed citizens. Participants responded that the general education program provides only a minimal contribution toward this goal. In their experiences, current events or contemporary issues weren t discussed. Many students said that they get their information related to current events from the buses and the newspaper. One participant reported that his general education course did challenge him to think about his mindset. When asked how the program might be improved to meet this goal, participants suggested more broad-based ideas such as smaller, discussion-based classes, and setting higher course expectations. They did share that becoming an informed citizen should be a by-product of the college experience, or something that will naturally happen. In discussing the segment of the program related to critical thinking, communication, and technology, participants responded that those courses were worthwhile. They particularly liked the courses with a practical focus, such as writing and public speaking. Some reported that this was the most effective of the general education clusters. One student reported that technology was adequately addressed by this cluster, while several others suggested that a more practical and integrated approach to technology should be incorporated. Students are required to take a series of

8 General Education Focus Groups 8 technology (basic office software) competency tests in their first year. Participants in the focus group expressed dissatisfaction with these tests, and cited that they have very little value or relevance to the skills they need for success. The discussion then moved into various other general education courses and students related experiences. A new theme that emerged was the test-focused nature of several of the general education courses. The group stated that the class is designed around the test and the faculty members and students are test-focused. Additional topics that emerged were a return to the discussion of class size and faculty interaction and frustration with many requirements repeating high school experiences. Suggested Modifications to the General Education Program Participants were then asked what was missing from the general education program and what should be removed. Their responses focused on the following themes: freedom and responsibility, student motivation, need for basic skills, expense, class size and structure, and need for challenge. A final, and quite interesting, topic that emerged was the desire to add courses that focus on life skills that students will need once they enter the real world. Examples cited by the group included financial skills, such as investing, balancing a checking account, buying a house, filing income taxes, etc. They described these as items that their parents had taken care of for them, and they will soon have to be responsible for, with little or no prior training or preparation. Suggestions for Assessing Program Outcomes When asked how they would go about assessing the general education program, students had strong suggestions for modifying the approach. They first addressed dissatisfaction with traditional multiple-choice test formats. Several students suggested implementing performance assessments, and gave examples. One student said, You don t learn from multiple-choice tests. Another suggestion that met with resounding approval from other group members was the idea of

9 General Education Focus Groups 9 asking students at the end of a course whether or not they felt that they learned anything, and whether or not they feel comfortable applying what they learned in the course. Despite a follow-up line of questioning about self-reported opinions serving as meaningful measures of learning, the students maintained that course grades and survey questions are adequate tools for general education assessment. The group also expressed great dissatisfaction with the current Assessment Day approach, with one student saying that someone had offered him money to take the assessments in his place. The discussion time was up at this point, and thus there was no opportunity for additional follow-up on the assessment issue. Thematic Network While participants responded to a series of different prompts, their resulting comments could be organized into an overarching network. The visual thematic network is presented in Figure 1, and is described below. Global Theme The overarching, global theme resulting from the focus group was the mismatch of program intent with reality. Participants stated their understanding of and support for the idea of a broad, foundational general education curriculum. However, their frustrations and suggestions for change can all be tied back to the mismatch of goals to outcomes. Each organizing theme explicates a facet of this mismatch, and is further detailed by its supporting basic themes. Organizing Themes and Basic Themes A strength of the thematic network approach is the ability to display themes in a web-like fashion, as opposed to a linear or hierarchical fashion. This also allows for the depiction of multiple relationships across themes, as can be seen in Figure 1. Within our thematic network, several basic themes supported more than one organizing theme.

10 General Education Focus Groups 10 The organizing themes from this dataset were: value, program structure, class structure, and choice. Under the organizing theme of value, basic themes were commitment/buy-in, expense, practicality, repetition of high school, number of clusters/credits, and delay in starting major. Commitment/buy-in encompasses the level of dedication and motivation on the part of both faculty and students for general education courses. Participants frequently brought up the low level of student motivation for general education courses. Related to faculty motivation, students perceived low levels of engagement and a lack of value on the part of instructors in general education courses. The students suggested that faculty motivation might be increased by specifically hiring or assigning instructors to teach these courses. The expense basic theme includes student concerns that the large number of general education requirements represents a significant investment of time and money to complete. This is related to the next basic theme, repetition of high school. Many participants repeatedly stated that the general education requirements were duplicative of coursework they completed in high school. This elucidates the high level of frustration students expressed when discussing expense. The group suggested that the number of clusters and the number of course requirements should be reduced. A common source of frustration was the fact that completion of general education requirements holds students back from beginning coursework in their major area of study. Supporting the organizing theme of program structure were the following basic themes: cohesion, foundation, assessment approaches, and reiterated themes: practicality, repetition of high school, number of clusters/credits, and delay in starting major. Cohesion refers to the perception that the required courses in the current general education program seem random and lacking in a clear focus or direction; a sense of cohesion is needed. Students did, however, support the potential benefits of a general, foundational segment of coursework to provide a broad perspective and to expose them to subjects they would not otherwise explore. Suggestions for changing assessment

11 General Education Focus Groups 11 approaches centered on moving from multiple-choice tests to performance assessments, and asking students to self-report their levels of confidence in applying material from the course and their perceived learning. Informing the class structure organizing theme were the following basic themes: challenge, lecture format, and test focus. A strong message from focus group participants was the desire for greater challenge and higher expectations in their general education coursework. They suggested that general education courses are selected by students on the basis of which ones may be the least painful in terms of class structure. For example, students mentioned that their classes outside of general education include interesting discussions and activities, as opposed to lecture formats. They also cited a strong test focus in their general education courses. Choice, as an organizing theme, encompasses responsibility, freedom, and flexibility. Focus group participants suggested that as adults, they should be held responsible for choosing appropriate coursework and ensuring adequate foundation and breadth. They want greater freedom to choose which courses to take, and when to take them. Students also expressed a desire for greater flexibility in choosing upper level courses in different subject areas instead of the watered-down general education versions. Conclusions Relating our findings to ideas and results in the literature, revealed convergence. A study described earlier, Muffo (2001), found low perceived value for general education courses, lack of faculty enthusiasm toward teaching these courses, and a failure to connect individual course goals to the larger general education goals. Each of these ideas emerged from our study as well. Our findings also aligned with many of the propositions found in one article related to student motivation in general education and two articles related to general education reform.

12 General Education Focus Groups 12 In their review of motivation literature, Glynn, Aultman, and Owens (2005) suggested ideas for improving motivation in general education such as providing challenges, showing connections between general education courses and students own educational goals, providing students with some control and sense of ownership, and setting high expectations. Many of these suggestions for improving student motivation actually emerged in our data as criticisms that students raised because these elements were missing. Thus, it is not surprising that student motivation related to general education appears to be suffering. Lack of cohesion across the set of courses and the frequent use of large lecture sections were among the common problems with general education as cited by Arnold (2006). Moreover, in their article on general education reform, Ferren and Kinch (2003), presented several ideas that were quite similar to our findings. They proposed that students want to see evidence of faculty enthusiasm and engagement when teaching general education courses. These authors suggest that while students agree that a broad foundational set of courses can be helpful, they also tend to view general education programs as an incoherent set of required courses of little relevance to their career interests, as dreaded requirements and state that they do not work as hard in classes they do not like (p.11). In sum, our findings support, and are supported by the limited body of literature related to university students perceptions of general education. The major finding of a perceived discord between the program s intent and reality comes forth at many levels. Students express willingness to embrace the idea of a foundational curriculum, but are frustrated with several key issues that suppress their ability to complete these requirements in a beneficial way. We were very encouraged to hear students share their desires to take responsibility for the direction of their coursework, to have the license to choose challenging classes, and to express the need for a course on practical, life skills. In turn, we were discouraged to hear that they are disillusioned by limited freedom of choice,

13 General Education Focus Groups 13 low faculty commitment and dedication, and program and class structures that are not conducive to higher-level engagement. Our experience has shown focus groups to be a very effective tool for gathering exploratory data related to student perceptions and knowledge of the general education program. This group interviewing technique allowed for a more efficient process than individual interviews or individual constructed response surveys. Additional benefits included the synergy created within a group atmosphere and the removal of individual pressure or focus that arises when using individualfocused methods. Future Directions As the unit of analysis in focus group research is the group, any approximation to generalizability of findings requires replication across multiple groups (Morgan, 1988). To this end, we plan to conduct additional focus group sessions, following the same procedures, with several more groups of sophomores at the next university-wide assessment day. Our preliminary investigation indicates a clear mismatch between the intent of the general education program and its instantiation. The focus group technique has shown great promise as a data collection tool, and we plan to continue its use. The rich data, while time consuming to transcribe and analyze, has provided deep insights that extremely difficult to gather with other methods. We hope that the qualitative information gathered through this study will be informative to university administrators as they continually refine the general education program. While we cannot expect our results to generalize to other institutions, we hope that our procedures and findings encourage practitioners at other institutions to engage in similar conversations and data-gathering efforts.

14 General Education Focus Groups 14 References Arnold, D. L. (2006, May/June). Moving from general education to liberal education. Change, Attride-Stirling, J. (2001). Thematic networks: An analytical tool for qualitative research. Qualitative Research, 1(3), Denzin, N., & Lincoln, Y. (2000). Handbook of qualitative research (2 nd ed.). Thousand Oaks, CA: Sage Publications. Fabiano, P., & Underhile, R. (2007, April). Focus groups: Top ten misperceptions threatening quality in student affairs assessment. Presentation at the joint meeting of the American College Personnel Association and the National Association of Student Personnel Administrators. Orlando, FL. Ferren, A. S., & Kinch, A. (2003, Summer). The dollars and sense behind general education reform. peerreview, Glynn, S. M., Aultman, L. P., & Owens, A. M. (2005). Motivation to learn in general education programs. The Journal of General Education, 54(2), Kleinke, J. (2005). General education focus group report. Retrieved August 31, 2007 from the Utah State University Web site: Krueger, R., & Casey, M. (2000). Focus groups (3 rd ed.). Thousand Oaks, CA: Sage Publications. Morgan, D. L. (1988). Focus groups as qualitative research. Qualitative Research Methods Volume 16. Newbury Park, CA: Sage. Muffo, J. (2001). Focus group: Student attitudes toward the core curriculum. Retrieved February 7, 2007, from the Virginia Tech Web site: University of Wisconsin-LaCrosse, General Education Design Team. (2005, August). Final report and recommendations. Retrieved August 31, 2007 from the University of Wisconsin-LaCrosse Web site:

15 General Education Focus Groups 15 Practicality Commitment/ buy in Expense Number of clusters/credits Delay in starting major Cohesion Foundation Value Repetition of High School Program structure Assessment approaches Freedom Mismatch of intent with reality Challenge Choice Class structure Responsibility Flexibility Test focus Lecture format Figure 1. Thematic network for focus group data from college students regarding general education