20 20 Assessment Plan Report Date Submitted: February 1, 2015 School: Engelstad School of Health Sciences Program: A.S. Dental Hygiene Person(s) responsible for the design and implementation of assessment plan and writing the report: Missy Abel & Marianne Sampson 1. Program Overview and Assessment Goals A. List the program Learning Outcome(s) that the plan is assessing. Outcome 1: Provide treatments that include preventive and therapeutic procedures to promote and maintain oral health and assist the patient in achieving oral health goals. Outcome 4: Analyze issues of dental hygiene practice with principles of ethical reasoning, ethical decision-making and professional responsibility. B. Provide a brief description of the plan and the assessment question(s) being addressed. Outcome 1: (Question) Did the class of 20 provide treatments that included preventive and therapeutic procedures to promote and maintain oral health and assist the patient in achieving oral health goals to the extent indicated by Commission of Dental Accreditation (CODA)? The DH program incorporates Supervised Clinical Practice, course DH 116 to provide an opportunity for students to remediate clinical skill competency through formative evaluation and hands on tutoring provided by supervising faculty. Through active remediation students are brought to a clinical level of proficiency demonstrated in the clinical Mock Board Exam in May 20. Outcome 4: (Question) Did the class of 20 analyze issues of dental hygiene practice with principles of ethical reasoning, ethical decision-making, and professional responsibility to the extent indicated by CODA? Multiple assessment artifacts are utilized in support of achieving this outcome. In 20, faculty incorporated evaluation rubrics to primary assignments indicating the student s critical thinking skill in assignment completion. Typhon EASI rubric evaluation is used to assess clinical skills incorporating these criteria. Students prepare a reflection document on clinical experiences and finally, a culminating portfolio is created online by each student. C. If this report does not correspond to the most recent program assessment plan sent to the school assessment coordinator please submit a copy of the revised plan along with this report. This report does correspond to the most recent program assessment. D. Indicate when the outcome(s) were assessed and what conclusions were reached from the previous assessment, and what closing the loop changes, if any, were made based on those conclusions. Also indicate whether the assessment method used 1
previously is the same as the one described in this report. If not, what is changed? Outcome 1: Outcome is assessed during the Clinical Mock Board given in May of 20 Conclusion: All students qualified for mock board andwere able to pass. Changes: Two levels of supervised practice have been incorporated into the DH 116 course. Level one for students issued remediation needing hands on instructor assistance, and Level 2 allows students to sign in and practice skills they feel need improvement. Outcome 4: Outcome is assessed during exit interviews. Conclusion: Through faculty evaluation and discussion, assessment revealed the need for higher level critical thinking skills. Changes: Faculty In-service sessions have been devoted to workshops on teaching methodology and teaching assessment incorporating a Tiemann- Markle Matrix. The faculty evaluated test and course objective assessment domain levels, and were asked to revise at least one assessment tool into a higher level domain. 2. Assessment Design and Coherence A. Identify the student product(s) used for direct assessment of the Learning Outcome(s) that you listed in 1.A. Explain the context for this product [course name(s) and number(s), place in curriculum, instructor(s), and so forth). NOTE: If your project depends on anonymity, report only contextual information that doesn t need to be anonymous. If the project focused on a single or common assignment, please attach the assignment handout or explanation in the syllabus as an appendix. Outcome 1: Context: DH 116: Supervised Clinical Practice supports DH 220 Clinical Practice. Remediation is offered throughout the 16 week fall and spring semesters. Faculty: Liz Diaz, Shari Peterson, Marianne Sampson, and Michele Rahmig. Outcome 4: Context: Reflection artifacts and Student Portfolios are assessed at Exit Interview May 20 by Shari Peterson. Typhon evaluations are assessed in DH 220 by Cara Kirby in May 20 B. Explain how the student product was scored and by whom [for example, objective scoring by machine; course instructor using a rubric; judging panel using a rubric] If the project used a rubric, please explain who created the rubric and attach it as an appendix. Outcome 1: The DH 220 Mock Board Examination is a clinical examination of student clinical skills scored by three calibrated faculty members. Each student s performance is assessed by examining the patient and records at the end of the examination period. Each faculty conducts a blind assessment and records their findings on a clinical performance rubric. Following the patient assessments, the 2 nd year clinic course director reviews each faculty assessment and the cumulative assessment rating is determined. The rubric was created many years ago and is reviewed for quality assurance and revision every 3 rd year at program review. Outcome 4: The Reflection artifact and Student Portfolio are evaluated by a rubric created by Shari Peterson. The Typhon evaluations are entered by clinical faculty who observe student competency evaluations. It is scored by computer and compared to course DH 220 requirements. The criteria in Typhon evaluation and the DH 220 2
requirements are controlled by Cara Kirby and follow CODA requirements. C. Explain the fit or match between the program Learning Outcome(s) being assessed and the student product used as a direct measure. In other words, how fully does the quality of the product reveal achievement of the learning objective? [Sometimes there may be a one-to-one correspondence between the learning objective and the product. At other times, only some features of the product are relevant to the learning objective. In such cases, the learning objective might be assessed only by one or two rows of a rubric or by a few selected questions on an exam.] Outcome 1: Provide treatments that include preventive and therapeutic procedures to promote and maintain oral health and assist the patient in achieving oral health goals The DH 220 Mock Board Examination rubric assesses the degree to which the treatment rendered by the student include the appropriate preventive and therapeutic procedures determined by patient s individual needs. Outcome 4: Analyze issues of dental hygiene practice with principles of ethical reasoning, ethical decision-making and professional responsibility. The student artifacts and Typhon evaluations assess the student s ability to conduct their performance according the ADHA Code of Ethics, utilize critical thinking and problem solving skills as they conduct all duties and responsibilities of a dental hygienist. D. Explain how program faculty defined achievement terms (example: minimally competent, proficient, aspiring, satisfactory etc.) for the learning objective and how they distinguished between the levels of achievement (criteria). If the project used a rubric, does the rubric clearly indicate these categories and specify the corresponding criteria? If not, explain how rubric scores correspond to these categories. If program faculty haven t yet defined terms and criteria for achievement of the learning objective(s), how and when do you plan to do so? Outcome 1: Achievement is defined by CODA and customized to fit within the framework of the CSN Dental Hygiene Program. Rubrics are based upon evaluation rubrics used to score student competence in the Clinical Board Examination (WREB) which students must pass before gaining DH licensure. Terms and criteria are clearly defined. Outcome 4: Achievement for student critical thinking analysis and ethical standards are defined by CODA and incorporated into evaluation rubrics to assess student artifacts. Rubrics used for evaluation include standards of critical thinking, ethics, and professionalism. E. If your project used a rubric, did program faculty try to establish inter-rater reliability in the use of the rubric? If so, explain how. If not, explain why. Outcome 1: Yes, inter-rater reliability workshops are conducted for clinical skill assessment during Faculty In-service held at least three times each semester. Outcome 4: Yes, inter-rater reliability workshops are conducted in the use of rubrics during Faculty In-service held at least three times each semester. 3
3. Assessmentt Methods: Narrate how the project was conducted. Who did what when? Outcome 1: see 2B Outcome 4: Students compile artifacts throughout their final semester as they reflect upon patient care and outcomes, compile and complete a portfolio documenting their competency in didactic coursework, clinical skill, and community service. Additionally, they reconcile Typhon reports with treatment records. At the end of the semester, the artifacts are presented for evaluation to the faculty named in 2B. 4. Project Results: Report your results as a table or chart showing the number of student products evaluated and the distribution of performances across the quality categories shown in your rubric or across the continuum of objective scores. Particularly highlight the percentage of performances meeting your program s aspirational goals and the percentage failing to meet minimal standards. Outcome 1: # of DH Students 15 10 5 0 Mock Board comparison with WREB Board Exam Mock Board results 10 WREB Board results (first attempt) Axis Title # Students in course # Testing # Passing Outcome 4: Ethical Reasoning & Professional Responsiblity # of DH Students.5.5 12.5 Reflection Portfolio Typhon Axis Title # Students in course # Artifacts completed # Passing Grades 4
5. Discussion of Results: Outcome 1: The program goal is based upon student completion of the program and student preparedness for licensure testing. Student success rate in DH 220 was 92.8%. Faculty reviewed the non-passing student s remediation attempts and determined the student was given reasonable support and opportunities to complete the course if it were within the student s capabilities. WREB results are outside the control of the CSN faculty, however, faculty carefully evaluated obstacles that unsuccessful candidates encountered and developed plans for change. Outcome 4: All students completed required artifacts which were evaluated based on rubrics described in section 2B. All students received passing grades on Reflection artifact and on Student Portfolio. Typhon assessment of ethical reasoning and professionalism in clinical skill showed 92.8% passing rate with the unsuccessful student being described above in Outcome 1. 6. Closing the loop actions: What follow-up actions do program faculty plan to take next as a result of this project? Outcome 1: The follow-up includes additional opportunities for supervised clinical practice and tutoring from faculty. Two levels of supervised practice have been incorporated into the DH 116 course. Level one for students issued remediation needing hands on instructor assistance, and Level 2 allows students to sign in and practice skills they feel need improvement. Outcome 4: Develop teaching methodology and assessment to reinforce student s higher domain cognitive abilities. Faculty In-service sessions have been devoted to workshops on teaching methodology and teaching assessment incorporating a Tiemann-Markle Matrix. Faculty evaluated course objective assessment domain levels and were asked to revise at least one assessment tool into a higher level domain. 7. Assessment ownership: To what extent were all program faculty involved in this project s discussion and analysis? When, how, and by whom were assessment findings discussed and decisions made about appropriate actions to take in relation to these findings? Outcome 1: The Program Director, clinic coordinator, and remediation instructors were involved in discussion on findings and participated in making changes to the Supervised Practice DH 116 course requirements. Outcome 4: All full time faculty participated in three in-service workshops devoted to teaching methodology and incorporating Tiemann-Markle Matrix into course learning objectives. The CSN college assessment coordinator, Shari Peterson presented these workshops and each faculty member brought a course syllabus to evaluate. It was decided that one assessment tool be upgraded to a higher domain level and instructors will continue each semester to upgrade tests and course objectives based on the matrix. 5