Hillsborough Community College. QEP Baseline Analysis. ACG 2021 Financial Accounting

Transcription

1 Hillsborough Community College QEP Baseline Analysis ACG 2021 Financial Accounting Fall 2007/ Spring 2008

2 Baseline Assessment: Fall 2007 The ACG 2021 pretest was administered to 475 students at the beginning of the Fall 2007 semester. The mean score on the pretest was 28%, with scores ranging from 0% to 65%. At the end of the semester, the posttest was completed by a total of 294 students. The mean score on the posttest was 50% with scores ranging from 0% to 90%. For each student (n = 228) who completed both the pretest and the posttest, a gain score was calculated by subtracting the pretest score from the posttest score. Gain scores indicated improvements of up to 65% from pretest to posttest. However, fifteen students scored lower on the posttest than they did on the pretest, resulting in negative gain scores that ranged from -35% to -5%. This indicates that at least some students did not take the posttest seriously and did not do their best to answer the questions correctly. Negative gain scores have been observed in each of the courses collecting assessment data (either baseline or intervention) for the Gateway Initiative in Fall This calls into question the validity of the pretest/posttest comparisons for assessing learning outcomes. One possible solution to this problem is to imbed the posttest questions in the final exams for the courses. Because the final exam contributes substantially to students course grades, it is more likely that they will be motivated to perform to the best of their ability. This strategy has already been approved by the lead Math faculty and will be implemented for their Spring 2008 posttest. Prepared by the Office of Institutional Research, Hillsborough Community College 1

3 Demographics: Fall 2007 In addition to the pretest and posttest assessments, a number of demographic and descriptive questions were answered by students at the beginning and end of the ACG 2021 course. Responses to these questions are described on the following pages. Student Characteristics Nearly half (47%) of all students indicated on the pretest survey that they worked more than 30 hours per week, and another 24% reported working between 21 and 30 hours per week. The remaining employed students worked between 11 and 20 hours per week (11%) or less than 11 hours per week (3%). Only 15% indicated that they were not employed. Almost half (43%) of all students who completed the pretest indicated that they were not receiving any form of financial aid. Slightly more than two-thirds of the students (69%) indicated that they had not taken any previous accounting courses. Of the students who did report taking a previous accounting course, 14% had taken a course in high school, 10% had taken another accounting course at HCC, and approximately 7% had taken a course at another college or university. Of those students who completed the posttest, 85% indicated that they planned to take another accounting course in the future. Less than one-fourth of all students completing the pretest indicated that they participate in extracurricular activities. As shown in Table 1, approximately 22% reported that they take part in Volunteer or Community Services, while less than 10% reported participating in any other extracurricular activity. Considering the number of students who are working more than 20 hours per week while attending college, this is not particularly surprising. Prepared by the Office of Institutional Research, Hillsborough Community College 2

4 Table 1: Students Reported Extracurricular Activities Extracurricular Activity % Participating Volunteer or Community Services 22% Athletics (as a player on a team) 8% Student Clubs 3% Honors 1% Student Government < 1% Obstacles to Success At the beginning of the semester, students were asked what they thought their major obstacles would be for succeeding in ACG As shown in Figure 1, Time Management was identified by nearly two-thirds (61%) of the students, while each of the other items (math anxiety, family responsibilities, fear of failure, and other ) were identified as an anticipated obstacle by less than 20% each. Figure 1: Anticipated Obstacles to Success in ACG 2021 ANTICIPATED OBSTACLES TO SUCCESS 100% 80% 60% 61% 40% 20% 12% 12% 16% 15% 0% TIME MGT. MATH ANX. FAMILY RESP. FEAR of FAILURE OTHER Prepared by the Office of Institutional Research, Hillsborough Community College 3

5 At the end of the semester, students were asked what obstacles they had actually faced that prevented them from doing their best in ACG Nearly one-fourth (23%) of all responding students indicated that they found accounting difficult to learn, and 20% reported that they experienced test anxiety. Issues relating to time management (work hours, family responsibilities, time for homework, etc.) were identified by between 7% and 16% of the students. The percentages of students who reported facing each of the obstacles listed in this question are shown in Table 2. Table 2: Encountered Obstacles to Success Which... represents obstacles you faced during class that prevented you from doing your best? Accounting is difficult to learn 23% I have test anxiety 20% I worked too many hours 16% The course material was covered too quickly 15% I had no difficulties learning ACG % Family responsibilities prevented me from putting in the necessary time 11% I did not have enough time to do the homework 9% My work hours increased 7% I missed too many classes 6% I felt underprepared 5% Additional Learning Resources At the end of the semester, students were also asked what additional learning resources would have helped them do better in ACG Over 20% of students expressed the belief that better ( improved ) study skills would have helped them, yet less than 5% expressed an interest in a study skills class. Nearly 20% thought that more group work with other students would have been helpful, but only 4% identified formal group study sessions as a potential source of help. This may indicate that students are interested in working together in class, but less Prepared by the Office of Institutional Research, Hillsborough Community College 4

6 interested (or less able) to get together outside of class in a formal study group. Table 3 lists the percentages of students who identified each of the additional learning resources listed in this question. Table 3: Additional Learning Resources Needed What additional learning resources would have helped you do better in ACG-2021? Mark all that apply. Improved study skills 21% More group work with other students 17% More time in class 12% More Accounting tutors available 11% More on-line accounting exercises 11% More one-on-one sessions with my instructor 7% Other 6% Increased hours of operation for the Campus Tutoring Centers 5% A study skills class 5% Formal group study sessions 4% Group study areas on campus 3% Summary Unexpected outcomes, such as the negative gain scores observed in these data, are part of the initial learning process inherent in any action research effort. Accounting faculty are encouraged to consider and identify a viable alternative, such as imbedding posttest questions in the final exam, and to implement this change in time for the Spring 2008 (second semester) posttest, if possible. Information of the greatest value this semester comes from students responses to the survey questions. Students reports of the obstacles they encountered, perceived sources of help and their expressed need for additional learning resources can serve to guide faculty s planning for future implementation strategies. Prepared by the Office of Institutional Research, Hillsborough Community College 5

7 Baseline Assessment: Spring 2008 The ACG 2021 pretest was completed by 394 students at the beginning of the Spring 2008 term. The mean score on the pretest was 30%, with scores ranging from 0% to 65%. At the end of the term, the posttest was completed by a total of 218 students. The mean score on the posttest was 47% with scores ranging from 15% to 80%. For each student (n = 189) who completed the pretest and posttest, a gain score was calculated by subtracting the pretest score from the posttest score. Gain scores indicated improvements of up to 55% from pretest to posttest. However, 26 students (14%) scored lower on the posttest than they did on the pretest, resulting in negative gain scores that ranged from -35% to -5%. This provides evidence that at least some students did not take the posttest seriously and did not do their best to answer the questions correctly. This phenomenon, which also occurred in Fall 2007, emphasizes the need to attach a meaningful consequence to students performance on the posttest if valid data are to be obtained in the future. The validity of the results reported below, as a reflection of students learning outcomes, must be viewed as suspect as evidenced by negative gain scores. Additional analyses are planned to determine if there is a relationship between students posttest scores and their final course grades. When students with negative gain scores were excluded from the analysis, the mean score on the pretest was 29% (n = 368), the mean posttest score was 50% (n = 192), and the mean gain score was 21% (n = 163). However, these figures still cannot be assumed to accurately reflect students learning outcomes. Prepared by the Office of Institutional Research, Hillsborough Community College 6

8 Demographics: Spring 2008 In addition to the pretest and posttest assessments, a number of demographic and descriptive questions were answered by students at the beginning and end of the ACG 2021 course. Responses to these questions are described on the following pages. Student Characteristics At the pretest, 46% of the students reported working more than 30 hours per week, and a quarter (25%) reported working between 21 and 30 hours per week. The remaining employed students worked between 11 and 20 hours per week (14%) or less than 11 hours per week (3%). Thirteen percent were not employed. Approximately 50% (n = 192) of the students in ACG 2021 who could be identified by their student ID carried a full-time course load (12 or more credits or credit-hour equivalents). Over 43% (n = 170) of all students who completed the beginning-of-term questionnaire indicated that they were not receiving any form of financial aid. Of the 394 students who completed the student questionnaire at the beginning of the Spring 2008 term, approximately two-thirds (66%, n = 261) indicated that they had not taken any previous accounting courses. Fifty-three students (13%) reported that they had taken an accounting course in high school, 43 (11%) had taken another accounting course at HCC, and 43 (11%) had taken a course at another college or university; multiple responses were permitted. Of the 217 students who indicated on the end-of-term student questionnaire whether or not they planned to take another accounting course, 90% (n = 195) indicated that they did plan to take another accounting course in the future; 1 student did not provide a response to this item. Approximately one-third (34%, n = 132) of all students completing the student questionnaire at the beginning of the term indicated that they participated in one or more extra-curricular activities. As shown in Table 4, the largest percentage (19%, n = 76) reported that they took part in Volunteer or Community Services. One-fourth (25%, n = 99) of the students participated in one extra-curricular activity; 33 students (8%) indicated that they participated in two or more activities. Considering the number of students who are working more than 20 hours per week Prepared by the Office of Institutional Research, Hillsborough Community College 7

9 while attending college, and the number who are enrolled full-time, this is not particularly surprising. Table 4: Students Reported Extracurricular Activities Do you participate in any of the following [activities]? Mark all that apply. # % Volunteer or Community Services 76 19% Athletics (as a player on a team) 45 11% Students Clubs 25 6% Honors Program 14 4% Student Government 8 2% Student Publications 2 < 1% Obstacles to Success At the beginning of the Spring 2008 term, students were asked what they thought their major obstacles would be for succeeding in ACG Students were instructed to mark all that applied. As shown in Figure 2, Time Management was identified by over half (54%, n = 214) of the students, and Fear of Failure was identified by over one-fifth (22%, n = 85) of the students. Seventy-two students (18%) did not mark any of the obstacles that were listed. Prepared by the Office of Institutional Research, Hillsborough Community College 8

10 Figure 2: Anticipated Obstacles to Success in ACG 2021 ANTICIPATED OBSTACLES TO SUCCESS IN ACG % 80% 60% 54.3% 40% 20% 15.7% 12.2% 15.7% 21.6% 10.7% 0% Time Management Math Anxiety Math Difficult to Learn Family Responsibilities Fear of Failure Other REASONS INDICATED Note: Students were directed to Mark all that apply. At the end of the term, students were asked what obstacle(s) they had actually faced that prevented them from doing their best in ACG Table 5 presents the results for all the obstacles that were listed. The obstacle indicated by the greatest number of students was test anxiety (30%, n = 66). Only one student marked that he/she had added late and fallen behind (< 1%). Prepared by the Office of Institutional Research, Hillsborough Community College 9

11 Table 5: Encountered Obstacles to Success Which represents obstacles(s) you faced during class that prevented you from doing your best? Mark all that apply. I have test anxiety % Accounting is difficult to learn % The course material was covered too quickly % I did not have enough time to do the homework % I worked too many hours % Family responsibilities prevented me from putting in the needed time % My work hours increased % I felt under-prepared for the course. 17 8% I missed too many classes. 10 5% I added this class late and got behind. 1 < 1% Additional Learning Resources At the end of the term, students were also asked what additional learning resources would have helped them do better in ACG Students were instructed to mark all that applied. Table 6 shows the numbers and percentages of students who indicated that each of the resources would have been helpful. Improved study skills was the resource marked by the largest number of students (34%, n = 73). Over 30% (31%, n = 68) of the students indicated that more group work with other students would have helped. Although 15% (n = 33) of the students expressed an interest in having more accounting tutors available, only 7% (n = 15) expressed a need for increased hours of operation for the Campus Tutoring Centers. Prepared by the Office of Institutional Research, Hillsborough Community College 10

12 Table 6: Additional Learning Resources Needed What additional learning resources would have helped you do better in ACG-2021? Mark all that apply. Improved study skills 73 34% More group work with other students 68 31% More on-line accounting exercises 42 19% More time in class 34 16% More accounting tutors available 33 15% More one-on-one sessions with my instructor 31 14% A study skills class 28 13% Formal group study sessions 27 12% Group study areas on campus 27 12% Other 24 11% Increased hours of operation for the Campus Tutoring Centers 15 7% Prepared by the Office of Institutional Research, Hillsborough Community College 11

13 Comparison of Fall 2007 and Spring 2008 Baseline Assessments The mean scores on both the pretest and posttest were very similar in Fall 2007 and Spring The low posttest means and the number of negative gain scores reinforce the need to modify the protocol for administering the posttest to include an incentive for students to do their best on the assessment. Table 7: Mean Pretest and Posttest Scores Fall 2007 and Spring 2008 FALL 2007 SPRING 2008 Difference Mean Pretest Score 31% 30% (1%) Mean Posttest Score 48% 47% (1%) The percentages of students who reported having no previous accounting course were similar for Fall 2007 (62%, n = 118) and Spring 2008 (66%, n = 261), and the majority of students in both Fall (90%, n = 170) and Spring (90%, n = 195) indicated that they planned to take another accounting course in the future. Table 8 also indicates that there are work demands on the students time and lack of financial aid for many. Approximately 70% of the students reported working more than 20 or more hours per week in both Fall and Spring (68%, n = 129; 71%, n = 279, respectively), and the percentages of students receiving no financial aid were 38% (n = 72) in Fall 2007 and 43% (n = 170) in Spring Prepared by the Office of Institutional Research, Hillsborough Community College 12

14 Table 8: Baseline Assessment Summary Fall 2007 and Spring 2008 FALL 2007 SPRING 2008 Difference No previous accounting course 62% 66% 4% Plan to take another accounting course 90% 90% <1% Worked more than 20 hours per week 68% 71% 3% Received no financial aid 38% 43% 5% Students in the Spring term were more likely to report more obstacles to success in ACG In Spring 2008, 30% (n = 66) of the students indicated that they had test anxiety compared to 20% (n = 60) who marked test anxiety as an obstacle in Fall The highest percentage (23%, n = 70) of students marked that Accounting is difficult to learn in the Fall term; an even higher percentage (30%, n = 65) reported this difficulty in Spring However, it is interesting to note that the percentage of students who reported encountering no obstacles was higher in the Spring (23%, n = 51) than in the Fall (15%, n = 44). Table 10 shows that students in the Spring term were also more likely to identify additional resources that they thought would have helped them to be more successful in ACG Prepared by the Office of Institutional Research, Hillsborough Community College 13

15 Table 9: Obstacles Encountered Fall 2007 and Spring 2008 FALL 2007 SPRING 2008 Difference I have test anxiety 20% 30% 10% Accounting is difficult to learn 23% 30% 6% The course material was covered too quickly 15% 19% 4% I did not have enough time to do the homework 10% 18% 8% I worked too many hours 16% 18% 2% Family responsibilities prevented the needed time 11% 17% 6% My work hours increased 8% 11% 3% I felt under-prepared for the course 5% 8% 3% I missed too many classes 6% 5% 2% Table 10: Additional Resources Fall 2007 and Spring 2008 FALL 2007 SPRING 2008 Change Improved study skills 22% 34% 11% More group work with other students 17% 31% 14% More on-line accounting exercises 11% 19% 8% More time in class 11% 16% 4% More accounting tutors available 11% 15% 4% More one-on-one sessions with my instructor 8% 14% 7% A study skills class 5% 13% 8% Formal group study sessions 4% 12% 8% Group study areas on campus 3% 12% 9% Increased hours of operation for the Tutoring Centers 5% 7% 2% Prepared by the Office of Institutional Research, Hillsborough Community College 14

16 Summary The occurrence of negative gain scores, observed in both the first and second terms of baseline assessment, underscores the need to attach a meaningful consequence to the posttest. Strategies for accomplishing this include imbedding posttest questions in the final exam or offering extra credit points based on posttest performance. It is imperative that such a strategy be identified and accepted by faculty. Information of the greatest value this term once again comes from students responses to the survey questions. There was a great deal of consistency across terms in students reported obstacles to success, as well as the additional resources that they thought would have helped them to be more successful. When asked what obstacles they had encountered, Subject Difficulty and Test Anxiety were reported most frequently by students in both Fall and Spring. When asked to identify additional resources that would have helped them to be more successful, students in both terms were most likely to cite their need for Improved Study Skills, followed by more opportunities for Group Work with Other Students, more Online Accounting Exercises, more Time in Class and the availability of more Accounting Tutors. The consistency in students responses suggests that they are responding candidly to these questions and that they tend to share similar challenges and needs for additional learning resources. The insight gleaned from this information can serve to guide curriculum development, plans for future teaching intervention and students services in order to optimize students learning outcomes and success. Alisa Zujovic, Academic Assessment Officer Jeanne Roberts, Academic Assessment Officer Prepared by the Office of Institutional Research, Hillsborough Community College 15