Coni Francis, PhD, RD Nutrition and Dietetics Program University of Northern Colorado

Transcription

1 Coni Francis, PhD, RD Nutrition and Dietetics Program University of Northern Colorado

2 University of Northern Colorado Nutrition and Dietetics Program Alena Clark, PhD, MPH, RD Jamie Erskine, PhD, RD Albion College Department of Psychological Science Andrea P. Francis, PhD

3 Program Assessment Program Assessment Exam Development Reliability and Validity of Program Assessment Exam Comparison of On-Campus and Distance Learning Academic Integrity in Distance Learning Discussion Conclusions

4 Program improvement Student self-assessment Student practice for professional exams Accreditation requirement Institution reporting

5 Purpose Program assessment Student practice Student self-assessment Taken at the end of the senior year

6 Domain development Exam question writing Exam question revision Exam score review 100 point randomized computer administered exam

7 146 Students took the exam During academic years , and all eligible students Both tracks took the exam On- campus (n=48) Online distance (n= 98) Internal consistency Cronbach s alpha =.85

8 23 (15%) students retook the exam 2 months later Test-retest reliability Pearson s correlation Exam Mean SD r p Time Time

9 Pearson s Correlation between Program assessment exam score Program GPA N=146 Mean SD r p Exam GPA >0.0001

10 50 (34%) students released their RD exam scores Pearson s Correlation between Program assessment exam score RD registration exam score N = 50 Mean SD r p Exam RD Exam

11 Online learning is one of the fastest growing trends in educational technology. U.S. Department of Education, 2010 According to a 2010 meta-analysis by the U.S. Department of Education, students participating in online learning perform as well or better than those participating in classroom instruction. However, this had not been tested within dietetics education.

12 To determine if students enrolled in a didactic program in dietetics (DP) delivered online would perform on an end of program assessment exam as well or better than students taking the same course work through classroom instruction.

13 Mean program assessment exam scores from academic years and for both online instruction and classroom instruction were compared using student s t test

14 Track N Mean Exam Score Oncampus SD Distance p

15 Track N Female Male GPA Previous Degree On campus (96%) 2 (4%) % Online (95%) 5 (5%) 3.51* 65% *p = 0.002

16 Group N Age Black Hispanic Other White On campus (100%) Online * 3 (8%) *p < (5%) 2 (5%) 31 (82%)

17 Since core knowledge standards are the same for all accredited DPs, it would be of interest to see if other institutions would find similar results using this or a similar exam. Since pass rate of the of the credentialing exam for registered dietitians (RD) is used as an indicator to determine the quality of accredited didactic programs in dietetics this exam may be a measure that can be used to validate RD pass rate improvement plans.

18 Loss of academic integrity is a concern across many college campuses Perception that distance education has more opportunities for loss of academic integrity Little research for online academic integrity

19 Require practices to prove the identification of the student being assessed U.S. Department of Education Most accreditors Most institutions

20 To determine if Online proctoring changes the assessment outcome for online students Online proctoring is an effective tool to use for distance student testing Students find the online proctoring service to be acceptable and easy to use.

21 On-campus students using the Testing Center Distance students taking the exam without a proctor with a set timer and instructions not to use outside materials Distance students using an online proctoring service (Proctor U).

22 ProctorU is an online proctoring service that allows exam takers to complete their assessment at home while still ensuring the integrity of the exam. Students can take exams from home seven days a week, connected to a live person during their exam. ProctorU is a member of the National College Testing Association. Proctors ensure common sense academic integrity standards by monitoring the student via web cam, monitoring what they are doing by watching the test takers screen in real time, and authenticating the student s identity using a multifactorial process.

23 Track N Female Male On-campus (96%) Online - Proctored Online Not Proctored (94%) (95%) 2 (4%) 3 (6%) 5 (5%)

24 Group N Age Black Hispanic Other White Oncampus Online - Proctored Online Not Proctored (100%) * 4 (9%) * 3 (8%) 3 (6%) 2 (5%) 3 (6%) 2 (5%) 37 (79%) 31 (82%) *p <

25 Sum of Squares df Mean Square F Sig Between Groups Exam Within Groups Total Between Groups GPA Within Groups Total

26 Track N Mean Assessment Exam Score SD On-campus Online Proctored Online Not Proctored a,b a p = b p <

27 Track N GPA SD On-campus Online Proctored Online Not Proctored a b a p = b p = 0.002

28 Online proctoring was an effective tool for distance student testing. All students were able to complete the exam in the specified time period Students found online proctoring to be easy to use and had no concerns about using it for testing. No student complaints Some students indicated they thought it was easy to use and a good idea.

29 Since this exam does not affect student grades, it is concerning to see that proctoring made such a significant difference in assessment exam scores for these online students. We are in the process of selecting a valid proctoring system for all online exams in our program for student-learning outcome measures for which grades or program assessment are dependent. It will be interesting to see if the GPA is affected for online students when all exams in online courses taken at UNC are proctored

30 Application of these academic integrity results could be used by other online programs/classes, as student identification and security of exams could be barriers for others to provide online learning experiences.

31 Proctored students delivered online instruction performed similarly on the end program assessment exam as students taking the same course work through classroom instruction. A valid proctoring system for online exams is crucial for all student learning outcome measures for which grades or program assessment are dependent.

32