NORTH DAKOTA STATE COLLEGE OF SCIENCE OCCUPATIONAL THERAPY ASSISTANT PROGRAM DEPARTMENT ASSESSMENT REPORT

Transcription

1 NORTH DAKOTA STATE COLLEGE OF SCIENCE OCCUPATIONAL THERAPY ASSISTANT PROGRAM DEPARTMENT ASSESSMENT REPORT Program: Occupational Therapy Assistant Contact Person: Beth Schlepp, M.Ed, COTA/L Year/Assessment Cycle: NDSCS Mission and Philosophy Statements The North Dakota State College of Science is a comprehensive, associate degree granting college founded on a tradition of quality and integrity. We deliver learner focused education through a unique and evolving collegiate experience. Using innovative delivery strategies, NDSCS anticipates and responds to statewide and regional needs by providing access to occupational/technical programs, transfer programs, and workforce training. To enrich people s lives through responsive lifelong learning in a dynamic educational and technological environment. OTA Program Mission and Philosophy Statements Program Goals Assessment Methods The mission of the Occupational Therapy Assistant Program at the North Dakota State College of Science (NDSCS) is to prepare students for entry level practice as generalist occupational therapy assistants. The philosophy of the NDSCS OTA program reflects that of AOTA philosophy that defines Occupational Therapy as a profession based on occupations and the belief that engagement in meaningful occupations is essential for well being (American Occupational Therapy Association, 2002,; Christiansen, Backman, Little, & Nguyen, 1999; Law, Polatajko, Baptiste, & Townsend, 1997; Wilcock, 1998). Demonstrate skills and competencies required of a generalist in an entry level occupational therapy assistant position. Direct Assessment Measures National Certification Examination Scores Case Studies Rubrics Indirect Assessment Measures Advisory Board Input Surveys Employers Alumni Student Evaluations (Formative) Placement Report (Summative)

2 Student Learning Outcome #1 A. Program Assessment Report Effectively interact through written, oral, and nonverbal communication with the client, family, significant others, colleagues, other health providers, and the public in a professionally acceptable manner. (Standard B.5.17) 1. OTA 113 Case Study Beth Spring Semester 2. OTA 214 Case Study Missi Fall Semester 3. OTA 102 Case Study Jody Spring Semester 4. OTA 154 Level I FW Evaluations Missi Spring Semester 5. OTA 218 Twin Town Villa Environmental Assessments Beth Fall Semester 6. OTA 214 Backpack Awareness Day All Faculty Fall Semester 7. OTA 121 Puppet Show Facility Evaluation Tool Beth Fall Semester 1 3. Case studies: Faculty use a common grading rubric for these case studies that represent different areas of practice. OTA 102 and 113 case studies were completed during the first year in the program. The case study for OTA 214 was completed during their second year. The faculty noted an improvement in scores from the 1 st to the 2 nd years. 4. Level I FW Evaluations: On the Fieldwork Performance Evaluation Section 6 Professional Behavior is worth 5 points. 12 students went out on Level I FW. Of those 12, 4/12 (33%) got 5/5; 7/12 (58%) received 4/5; and 1/12 (8%) received a 3/5. 5. TTV Environmental Assessments: 2 nd year students as a class researched and choose an environmental/falls prevention assessment to complete on their TTV resident. Students successfully completed this assessment and received full credit for completing it. 6. Backpack Awareness Day: This is an annual event every September. 1 st and 2 nd year students are paired together and compile all the necessary information for the presentation to 3 rd graders at the Wahpeton and Breckenridge elementary schools of the 20 reports (5%) of the Facility Evaluation Tool noted some dissatisfaction with student presentation of a Kids on the Block puppet show. Overall : Faculty feels that students need more practice with written and oral communication skills to feel more confident in this area when on Fieldwork. Overall, students are

3 demonstrating professional behavior during Fieldwork. 5. Because the instructor does not observe the student complete the assessment, their grade is based on only whether or not they complete the assignment. The assessment will be changed next year so that the students are held more accountable and are assessed more objectively. Students will also conduct an informal roundtable discussion among the residents and the staff to review results. Staff will also complete an eval on the students. 6. Backpack Awareness Day will continue to be a required assignment. We would like to use this assignment as a tool to collaborate with OTR students from the University of Mary in Bismarck if possible. Also, a rubric for Elementary Teachers to evaluate NDSCS students will be implemented. Revisions/Changes Student Learning Outcome #2 Next year faculty will monitor this learning outcome by having students do a community presentation to orientate the public to what OT is. This assignment will be in a first year course to give the students additional time to practice their communication skills. Other possible presentations will include OTA 121 puppet shows and presentations during the Allied Health Career Fair. None Demonstrate a knowledge and understanding of the AOTA Code of Ethics, Core Values and Attitudes of OT Practice, and the AOTA Standards of Practice, and use them as a guide to professional presentation and interaction. (Standard B.9.1) 1. OTA 101 Class Assignment Carolita Fall Semester 2. Professional Behavior Assessments All Faculty Spring Semester 3. Fieldwork Level II Evaluations Missi Spring Semester 4. Monthly Student Meeting Beth and Missi Spring Semester 5. Advisory Board Survey Feedback All Faculty Spring Semester 1. Data not collected due to faculty retirement. 2. Professional Behavior Assessments were completed on all 1 st and 2 nd year students at the end of

4 spring semester. Students are ranked from a 1 4 (1 being poor 4 being high). If a student receives a score of 1 in any item in their 2 nd year, they will not be allowed to go on FW Level II. Students complete a PBA on themselves before meeting with the faculty. Meetings to review the PBA are student led. All students (100%) received at least a score of 2 in all items on the PBA. Compared to last years PBA, students have shown overall improvement in professional behavior and presentation. Lowest scores on this assessment are time management and oral/written communication. 3. Fieldwork Level II Evaluations 12 students completed 2 Level II Fieldworks (24 placements and evaluations). 23/24 (96%) student Level II placement evaluations were scored a 3 or better on Professional Behavior (score of 1 4). 4. At the April mandatory monthly student meeting 22/22 (100%) of 1 st year students attended this meeting. Students were provided with copies of these documents to read and review. All students signed a statement of responsibility for this information. 5. Advisory Board feedback state that our students overall are meeting this outcome. Suggestions for improvement include better cell phone management, better time management skills, and adherence to facility/dept. dress code. Overall : Student professional presentation and behavior needs to be continually monitored to maintain or improve these skills. Revisions/Changes Student Learning Outcome #3 2. The Professional Behavior Assessment will be completed each semester for both 1 st and 2 nd year students. This assessment will also be used as a guide for 1 st year students entering Level I Fieldwork. The criteria that are used on this assessment will be reformatted and used as an employer survey. This survey will be completed by facilities that have hired NDSCS OTA graduates. 4. Faculty will continue to have monthly meetings with 1 st and 2 nd year students to cover professional behavior policies and issues. None Use critical thinking and analysis skills to gather and share data for the purpose of screening and evaluation including, but not limited to, specific screening tools; assessments; skilled observations; checklists; histories; consultations with other professionals; and interviews with the client, family,

5 and significant others. (Standard B.4.1) 1. Jody OTA 102 Spring Semester 2. Beth OTA 111: ROM Measurement and MMT (Nine Hole Peg Test, Box and Block Test) Fall 3. OTA 113: Case Study Beth Spring 4. OTA 112 Interview Beth Spring 5. OTA 218 Occupational Profile Beth Fall 6. OTA 214 Peabody Assessments Missi Fall 7. OTA 218: Cognitive Assessments Beth Fall 1 and 3. Overall, students are improving in the area of gathering data for the purpose of evaluation students completed a 10 pt. competency task sheet during a lab class. The class average for this task was 91.9%. It appears that students have demonstrated competency in this entry level OTA skill. This assignment will continue to be a competency based assignment during the 1 st year of the OTA program. Students will also review these assessments in the Skills Lab during their 2 nd year in the program. 4. In OTA 112, students select interview questions and then practice interviewing a health care professional. In this assignment, students struggled with selecting appropriate questions for the purpose of collecting data. Once the questions are established the students scored higher in carrying out the interview. 5. From 1 st year to the 2 nd year, students are feeling more comfortable interacting and are asking more appropriate questions when completing an Occupational Profile on their senior resident. More complete instructions and classroom review have resulted in increased overall scores nd year students complete the Peabody pediatric assessment on selected pediatric clients. The students completed the assessment together in class with instructor giving scoring instructions. Overall, students were more successful in scoring the assessment through this instructor guided activity nd year students completed their choice of 5 cognitive assessments (Allen Cognitive, Cognitive Assessment of Minnesota, Independent Living Skills, Contextual Memory Test and the Mini Mental Screen Tool) on their assigned resident from Twin Town Villa. Students were assigned a cognitive assessment to research and present to the class. After all the presentations, students are to perform

6 their assessment with their resident. Students are given points for meeting with their resident, carrying out the assessment, and for completing required paperwork. Class average of this assignment was 100%. Overall : Student s critical thinking and analysis skills for the purpose of data collection and evaluation show improvement from 1 st year to 2 nd. Revisions/Changes Student Learning Outcome #4 Revisions/Changes 7. Although, this is a good opportunity for students to work with actual clients, it is not graded based on how the student performs the assessment (graded on getting the assessment completed) as instructor does not observe the student performing the assessment. This assignment will be modified so that students will be more accountable for the results of the assessment. The revised assignment will have the clients complete the assessment on each other in the classroom so that faculty can observe and be able to provide better feedback and grade the students more effectively. The students will them complete the assessment on an actual resident. 6 and 7. With the increase in enrollment, more assessments will have to be purchased. The average cost of these assessments is $100 each. Total cost is $1000. Students will demonstrate critical thinking skills as they assist with the development of occupation based intervention plans and strategies by selecting interventions and procedures to enhance safety, wellness, and performance in all areas of occupation. (Standard B.5.2) 1. OTA 112 Case Study Treatment Plans Beth Spring 2. OTA 113 Video Case Studies Beth Spring 3. Employer Surveys Carolita Fall 1 and 2. Students scored lower on the treatment planning and intervention portion using written case studies. Scores average 1 2 points higher using the video case study format. 3. Not completed. Will complete Fall Overall : Students continue to struggle with treatment planning and selecting treatment interventions. Since treatment planning and implementation are one of the lowest scores on the certification exam,

7 data will be collected from 2 nd year students to see whether or not there is improvement from 1 st to 2 nd year. Faculty will try to incorporate actual clients as case studies for students to plan treatment and select interventions, and will enhance goal planning content in selected OTA courses. Student Learning Outcome #5 None Ensure that the fieldwork experience is designed to promote clinical reasoning appropriate to the occupational therapy assistant role, to transmit the values and beliefs that enable ethical practice, and to develop professionalism and competence in career responsibilities. (Standard B.10.14) OTA 254 Fieldwork Level II Evaluation Missi Spring Semester OTA 255 Fieldwork Level II Evaluation Missi Spring Semester Changes / Revisions On a scale of 1 4 (1 being the lowest), students were evaluated in the Evaluation and Screening and the Treatment Intervention categories of this evaluation. Students scored lowest in the treatment intervention area. 8 of 12 (67%) of the students achieved a 3 or greater on the Fieldwork Level II Evaluation Form. The faculty will incorporate more assignments for our core OTA classes (OTA 102, 112, 113, 214, 218) that will focus on treatment intervention and implementation. Rubrics will be used to score these assignments and a sampling will be taken to see if there is improvement in the students treatment intervention and implementation skills. None Program Outcome #1 Program Outcomes Graduates of the OTA program will pass the National Board for Certification in Occupational Therapy (NBCOT) national competency examination at a 70% or higher level for first time test takers. National Board for Certification of Occupational Therapy report

8 Changes/Revisions Program Outcome #2 Program Outcome #3 The pass rate for the year is 75%. We had 12 students sit for the certification exam and 9 of those students passed. This meets the yearly minimum standard of 70%. To help improve our pass rate the dept. has integrated several items: 1. The students are taking a practice exam from NBCOT after their Level II Fieldwork. 2. The students complete a skills lab at the beginning of each semester, and 3. The students complete a one day cert exam review workshop before graduation. A course fee has been added to OTA 256 to cover the cost of the cert exam review workshop. 65% of OTA class of 2009 will be successfully employed by November. 1. NDSCS 2009 Placement Report 2. OTA Department Student Tracking Placement Report shows an 85% placement rate. 2. Departmental Tracking: 15 students graduated in 2009: 11/15 (73%) are employed in the OT field 2/15 (13%) are pursing further education 3/15 ( 20%) unable to contact 1 student was counted twice as she is working as a COTA and attending a weekend OT program. None 75% of students from the class of 2010 will complete the program. 1. Departmental Tracking Beginning in Fall 2008, 22 students entered the program. During the Fall 2008 Semester, 10 students (45%) dropped the program 12 students entered the Spring 2008 Semester. During the Spring Semester, 1 student (4.5%) dropped the program 11 students remained in the program During the Fall 2008 Semester, all 11 students remained in the program During the Spring 2009 Semester, 1 student from a previous class completed Fieldwork (11+1=12)

9 12 Students graduated in May 2009 Changes/Revisions 12/22 (54.5%) students completed the program. Will continue to track and monitor yearly trends. The OTA dept. will be implementing admission standards beginning Fall 2011, and selecting students based on a point system. The goal of this process is better qualified applicants and better program retention. HOBET Test $25 per student (they are responsible for cost). B. Feedback Loop How do we plan to use information and who is responsible for it and time frame. Faculty will use the information in several ways. First, we will evaluate the data and make any necessary changes in presenting course content. The main revision that will occur as a result of this report is that the OTA Program will put Admission Standards in place for Fall All faculty will be involved in this process. OTA faculty will share our assessment report with our advisory committee, get their feedback and make changes based on their recommendations. Faculty will use this assessment plan in preparation for accreditation self study and our onsite evaluation scheduled for March All OTA faculty are involved in the assessment process and are responsible for their own data collection. The OTA assessment plan is reviewed formally 2 times during the academic school year and informally every month at faculty staff meetings. Moving from specific learning outcomes to examining trends in your program assessment plan as a whole, please answer the following questions: Qualities: 1. What one thing have you learned from this report that will most improve student learning in your program? Implementation of OTA Benchmarks including Skills Lab, Volunteer Hours, Cert Exam Preparation Workshop have helped to assist students to scaffold from the classroom to Level II Fieldwork.

10 This is the first year that we offered all 2 nd year students a certification examination preparation workshop. Our pass rate for 2009 was 50%. Our pass rate for 2010 was 75%. It appears that the Cert Exam Prep workshop helped to improve exam scores. 2. What is a strength of student learning in your program identified from this review? Students are demonstrating more professional behaviors out on Fieldwork, as students are scoring higher on the Fieldwork Evaluation Form. On the Fieldwork Evaluation Form, in the Fundamentals of Practice category which measures professional behavior, students scored at least a 3 (on a 1 4 scale) in all areas. Also, on the Professional Behavior Assessment that is used by faculty to evaluate student professional behavior, 100% of the students had at least a 2 (on a 1 4 scale) in all areas; this is an improvement from last year. 3. What overall improvement in your program has resulted from assessment efforts? How have you used assessment information to make improvements? Cert Exam pass rate has improved. We incorporated a cert exam review preparation workshop for the students after they return from their Level II Fieldwork Experiences. 4. Describe how students are involved in assessment activities in your department, how they are made aware of the learning outcomes and given feedback on their progress toward achieving them. Students are made aware of the assessment outcomes during the first day of all classes when reviewing the course syllabus. The student outcomes are compared to the AOTA Standards and these are discussed with the students at monthly student meetings. Students are informed on how the program is meeting the program outcomes as well. A student representative is selected to attend our advisory board meeting, and has the opportunity to discuss assessment activities with board members. 5. You may include data from any surveys that you are tracking or using in your department that demonstrate satisfaction with your program, such as the Student Satisfaction Inventory, Alumni Survey, Employer Survey, Community College Survey of Student Engagement (CCSSE) etc. Fieldwork Supervisor s Survey Student Evaluation of Fieldwork Student Satisfaction of Program Survey Disability Awareness Facility Tool Student Professional Behavior Assessment Rubrics Advisory Board Feedback

11 Improvement: 6. What areas of student learning in your program would you like to improve as a result of your review? Critical thinking skills and the ability to develop treatment planning and interventions. 7. What specific steps will you take to address the areas for improvement? The OTA program will implement Admission Standards beginning Fall Semester The program will participate in an on site accreditation visit in March 2011 and faculty will use this feedback to make program improvements. The long term goal for improvement will include curriculum revisions to better meet the needs of the students. Short term goals include: Implement the cert exam preparation workshop into OTA 256. Continue to integrate assignments that involve more critical thinking skills. Continue to integrate more assignments that integrate treatment planning and interventions. Both of these items will be emphasized more in the Skills Labs as well. 8. In order to make these improvements, will there be a financial impact? If so, explain. Due to increased enrollment, we may have to order more assessment and classroom modalities. We will also be adding course fees to OTA 256 to cover the cost of the cert exam preparation workshop. Change: 8. Your Assessment Plan should be revised/reviewed every 3 5 years. Based on this year s report, what changes or revisions are you making in your Assessment Plan for next year? (Are outcomes being revised, using different ways to measure, etc.) The OTA dept will be writing a new Assessment Plan for the academic year. It is our goal to make our Strategic Plan and Assessment Plan align better and to make better use of the data that is being collected.