How To Write An Iukson Nursing Assessment Plan Portfolio

Transcription

1 2014 Indiana University By Mary P. Bourke PhD [ASSESSMENT REPORT: MSN] 1 The following report contains a summary of assessment data and recommendations.

2 Table of Contents Assessment Plan... 3 Portfolio Criteria... 5 Characteristics of the MSN Graduating Class of December Evaluation Results for December 2013 MSN Graduates... 9 Executive Summary of Student Performance for Each Outcome with Faculty Interpretation Using Assessment for Program Improvement for Appendix A Appendix B

3 Assessment Report: MSN Program 2014 Brief Summary of the Assessment Plan for MSN Assessment Plan Formative Evaluation of Program Learning Outcomes and Program Elements The systematic assessment plan for the IUKSON master s programs emphasizes formative and summative assessment of student learning outcomes, program-learning outcomes, program outcomes and includes Accreditation Commission for Education in Nursing (ACEN) standards. Faculty are required to provide evidence of meeting learning outcomes/competencies through assessment and evaluation of student artifacts mapped to learning objectives and competencies. Evaluation of learning outcomes from each course is based on best practice such as rubrics, portfolios, and self-reflection. Student attainment of course learning outcomes is reported by the faculty member to the assistant dean and the MSN Committee along with suggestions for course improvement. The committee may accept the suggestions or make additional changes to the course. This information is also reported by faculty in their annual report to show growth towards excellence in teaching. This provides a consistent method for faculty to evaluate how they met course-learning outcomes and thereby measure teaching effectiveness. While student evaluations are important, the IUKSON feels it is important for evaluation of teaching effectiveness to include data other than student evaluations. Focus group data are collected and reported to the MSN committee for recommendations, possible intervention, and/or information. For example, the Assistant Dean visits every MSN class on a weekly basis to inquire about student progress and address any problems or concerns in order to evaluate program effectiveness and to solve problems in a timely manner. Summative Evaluation Components The IUKSON MSN Assessment Plan also includes evaluation of student learning outcomes and program outcomes. The evaluation of student learning outcomes includes the following: 1) Portfolio, 2) Preceptor Evaluation, and 3) AONE Nurse Executive Competencies Assessment Tool for nursing administration pre/post and Nurse Educator Assessment Tool pre-/post-. For the evaluation of program outcomes, data will be collected from the following: 1) certification exam pass rates, 2) program completion 3

4 rates, 3) graduate satisfaction survey, 4) employer satisfaction survey and 5) student employment in area of professional practice. See figure 1. for concept map of IUKSON MSN Assessment Plan. Figure 1. Concept Map of IUKSON MSN Assessment Plan Portfolio The portfolio is a mechanism used to validate that students have acquired knowledge and skills; as well as, demonstrate that course- learning outcomes/competencies have been met. Acquisition of knowledge and skills may be through either course related assignments or methods occurring outside the classroom setting such as Quality Improvement projects. Assignments are collected in e-format, and a grading rubric is provided for overall evaluation of the portfolio, which addresses appearance, completeness, and quality of the materials included. Portfolio contents are compared against program learning outcomes. Every portfolio is evaluated relative to how well the student demonstrates that he/she meets the MSN program learning outcomes. After student portfolios are evaluated for their practicum and assigned a grade, a random sample of portfolios is evaluated for summative evaluation of program learning outcomes and data aggregated and reported to the MSN Committee. The criteria for student portfolios are different for each program-see Appendix A for Education and Appendix B for Administration. 4

5 Portfolio Criteria The following are the criteria for evaluating the portfolio, along with the rubric for evaluation. Criteria The benchmark for evaluating the portfolio is three or above. That is, the mean of the ranking scores for each MSN competency is three or above based on all the students. The mean of the ranking score for the aggregate of students in the MSN program is also three or above. For Each Competency, the following rubric is used: 4=Excellence Evident in meeting MSN Competency Evaluator easily finds evidence that is presented professionally and Evidence presented shows that student has mastered the competency in a manner that suggests the best practice/exemplary practice; or Demonstrates high level of integration of knowledge, unusually strong insight. 3=Satisfactory Performance Evident in meeting MSN Competency Evaluator easily finds evidence that is presented professionally; and Evidence presented shows that the student has mastered the competency in a complete manner; or Demonstrates good integration of knowledge, insight. 2=Just Below Satisfactory Performance Evident in meeting MSN Competency Evaluator finds evidence only after searching the portfolio; or Evaluator finds that evidence is not presented in a professional manner; or Evidence presented shows some areas that are lacking with regard to achievement of the competency. 1=Unsatisfactory Performance Evident in Meeting BSN Outcome Evaluator is unable to find evidence pertinent to the competency; or Evidence presented does not show that the student addressed the competency in a relevant/reasonable manner (lacks attention to basic aspects of competency); or Evidence presented is found to be unsafe, unethical, or fraudulent. 4=Excellence Evident 3=Satisfactory Performance 2=Just Below Satisfactory. 1=Unsatisfactory Performance Rater assigns a numeric value to each category: 5

6 2. Preceptor Evaluation Preceptors will evaluate students at the end of their practicum experience using separate instruments mapped to competencies for each track. The preceptor works with the student so that attainment of student learning outcomes is met for T679 Nursing Education and L579 Nursing Administration. 3. AONE Nurse Executive Competencies Assessment Tool pre-/post- for nursing administration and Nurse Educator Competencies Assessment Tool pre-/post- for nursing education. The AONE competency and nurse educator competency assessment tools will be used pre- and post- to measure student perception of their level of competency before beginning the program and at the end of their program. Current benchmarks for the nurse educator competencies assessment tool are set at an achievement mean of 2.0 (competent) or more in all areas by the end of the program. Current benchmarks for the nursing executive competencies assessment tool are set at an achievement mean of 3.0 (competent) or more in all areas by the end of the program. After pre-/post- data is analyzed per student to inform student perception of their competency, data will be aggregated and combined with portfolio aggregated data and the preceptor aggregated data. This is the third data set to form the triangulation that strengthens evaluation results of program learning outcomes. Evaluation of Program Outcomes As stated previously, the evaluation of program outcomes include data collected from the following: 1) certification exam pass rates, 2) program completion rates, 3) graduate satisfaction survey, 4) alumni survey, 5) employer satisfaction survey and 6) student employment in area of professional practice. As a new program data availability is limited. 1. Certification Exam Pass Rates Certification exam pass rates will be collected and data aggregated and reported to stakeholders. The program outcome goal has been set at 80% for the MSN graduates who take a certification exam and pass it on their first attempt. 2. Program Completion Rates Eighty percent of students will graduate from their program within 1½ times the length of the program. 3. Graduate Satisfaction Near the close of the last semester, students will complete a self-report instrument in which they rate their own abilities in each of the student learning outcome areas. This scale will include several items related to MSN Essentials competencies 6

7 and program learning outcomes. This is scaled on a four point Likert scale, with 4=strongly agree, 3=agree, 2=disagree, and 1=strongly disagree. Since an important part of professional development is identifying one s own strengths and weaknesses and since our students engage in self-rating across the curriculum, this is an appropriate measure. However, being self-report, there is an issue of validity. Therefore, when possible, this measure is considered with other measures. Currently our benchmarks are set at achievement of a Mean of 3.0 or more in all areas. 4. Alumni Survey One-year post graduation, MSN alumni will be mailed a survey similar to the graduate survey though somewhat shorter, that includes self-report items for every program goal. A similar Likert rating scale is used. Currently our benchmarks are set achievement of a Mean of 3.0 or more in all areas. 5. Employer Satisfaction IUKSON will survey employers of graduates of the MSN program to collect data on their perceptions of the level of preparation of the graduates for their work setting, overall satisfaction with graduates, and the quality of their performance as employees. Surveys will be completed on a rotating basis. The goal is that 80% or more of employers will express satisfaction with the program. 6. Student Role-related Professional Practice It is our program outcome goal that 80% of graduates seeking employment will be involved in role related professional practice at one year post-graduation. Data will be collected from the Alumni Survey. Program Elements Additional data may be collected to evaluate program elements per Chen s Theory. Various additional surveys and focus group data are collected and reported to the MSN committee for recommendations, possible changes, and implementation. For example, the assistant dean visits every class on a weekly basis to talk with students about problems or concerns in order to evaluate program effectiveness and to solve problems in a timely manner. Another example, One year into the program a survey was given that measured students' perception of the MSN culture. Prior to beginning the MSN program, faculty defined the graduate culture to serve as a guide to enhance the IUKSON program mission. 7

8 Characteristics of the MSN Graduating Class of December 2013 The graduating class of December 2013 was a very productive hard working class with a graduating GPA of Fourteen MSN students graduated. Two additional students will graduate in May. The mean age was 43 years and 1 month. The MSN graduates reported an age range of 28 to 57 years old. All were female and 100% were working RN s. 73.3% were full time, 20% part time and 6.7% on/call. Students listed the following types of institutions for employment: 66.7 worked for a community general hospital, 6.7 long-term care, 13.3% regional medical center and 6.7% other. Specialty areas include the following: one works in pediatrics, 2 Peri- Operative, 1 Mental Health, 2 Rehabilitation, 1 ICU, 2 Obstetrics, and 5 listed other. Six are nurse managers, 5 are Staff RN s and 3 listed as other. Two students are working in a new role due to the MSN degree. The MSN program admitted 20 students in the inaugural cohort. Due to a 1.5 hour commute to campus each way every week for class, four students with high-level administrative positions were unable to continue and sought enrollment in on-line education. One student encountered unforeseen family issues, which necessitated withdrawal from the program. Of the 15 remaining full-time students 14 graduated in December 2013 and two will graduate in May As a result, this cohort has a 75% completion rate and a 100% graduation rate after completing the first semester. 8

9 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 1: Model excellence in nursing leadership to improve nursing practice within the healthcare system. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Evaluation Results for December 2013 MSN Graduates Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. 9 DATA FROM GRADUATING CLASS Graduate Survey Mean=3.53 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 4.0 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 1.69 Post 3.57 Administration Pre/Post Categories to be revised.

10 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 2: Function within an ethicallegal framework. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. 10 DATA FROM GRADUATING CLASS Graduate Survey Mean=3.86 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 3.93 Education = 3.93 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 2.56 Post 4.05 Administration Pre/Post Categories to be revised.

11 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 3: Synthesize knowledge from nursing, biological, behavioral, social, administrative, educational, and communication sciences for application to area of specialty. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. 11 DATA FROM GRADUATING CLASS Graduate Survey Mean=3.73 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 3.78 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 2.43 Post 3.2 Administration Pre/Post Categories to be revised.

12 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 4: Demonstrate scholarly inquiry and reflection that exemplifies clinical reasoning, critical, creative, and systems thinking. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. 12 DATA FROM GRADUATING CLASS Graduate Survey Mean=3.73 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 4.0 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 1.99 Post 3.88 Administration Pre/Post Categories to be revised.

13 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 5: Frame problems, design interventions, specify outcomes, and maintain quality while balancing human, fiscal, and material resources. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. 13 DATA FROM GRADUATING CLASS Graduate Survey Mean=3.4 Graduate Survey Mean of MSN Essentials=3.59 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 3.90 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 2.05 Post 3.2 Administration Pre/Post Categories to be revised.

14 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 6: Use information technology and knowledgebased resources to inform practice. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year postgraduate. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. DATA FROM GRADUATING CLASS Graduate Survey Mean=3.8 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 3.98 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 2.13 Post 3.69 Administration Pre/Post Categories to be revised. 14

15 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 7: Articulate the effects of culture, diversity, values, and globalization in the design, delivery, and evaluation of health services. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Diversity Survey Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. Month prior to grad December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad, Diversity and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. DATA FROM GRADUATING CLASS Graduate Survey Mean=3.6 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 3.97 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 1.69 Post 3.6 Administration Pre/Post Categories to be revised. Diversity Survey:

16 OUTCOME ASSESSMENT AREA (ORGANIZED BY STUDENT LEARNING OUTCOME) Program Outcome 8: Engage in lifelong learning activities that contribute to the professional nursing development and advancement of the nursing profession. PRIMARY TOOLS FOR PROGRAM EVALUATION SR=SELF REPORT OR=OBSERVED REPORT P=PERFORMANCE ON TEST, PAPER, OR SKILL CHECK Grad Survey (SR) Alum Survey (SR): 1 year Portfolio (P) Preceptor Evaluation (OB) Student Self-Assessment (SR) Indiana University Kokomo School of Nursing MSN Program Results WHEN IS THE MEASURE TAKEN? Month prior to grad Mailed at 1-year post grad. December 2013 Reported 2014 BENCHMARKS AND MEASUREMENT COMMENTS FOR EACH TOOL IN PROGRAM EVALUATION: Portfolio: Please note: for evaluating the portfolio: Our benchmark is at 3 or more: the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN program is 3 or more. Professional Competencies Self- Assessment Tool and Preceptor Evaluation Tool: Based on Benner s Novice to Expert. 1= Novice, 2= Advanced Beginner, 3=Competent, 4= Proficient, 5=Expert. Benchmark: Means at 2.00 or more on all measures. Grad and Alum Survey: All Surveys have values as follows: 1=strongly disagree, 2=Disagree, 3=Agree, 4=strongly Agree. Benchmark: means at 3.00 or more on all three measures. DATA FROM GRADUATING CLASS Graduate Survey Mean=3.93 Alumni Survey=N/A Portfolio Average for Program Outcome 1 = Administration= 4.0 Education = 3.75 Preceptor Evaluation=N/A Tool under revision Student Self-Assessment= Education Pre 2.43 Post 3.2 Administration Pre/Post Categories to be revised. 16

17 Part : Executive Summary of Student Performance for Each Outcome with Faculty Interpretation OUTCOMES OVERALL COMPARISON WITH BENCHMARK: MET, SIGNIFICANTLY EXCEEDED, OR AREA FOR IMPROVEMENT SIGNIFICANTLY EXCEEDED 1. Model excellence in nursing leadership to improve nursing practice within the healthcare system. 2. Function within an ethical-legal framework. SIGNIFICANTLY 3. Synthesize knowledge from nursing, biological, behavioral, social, administrative, educational, and communication sciences for application to area of specialty. 4. Demonstrate scholarly inquiry and reflection that exemplifies clinical reasoning, critical, creative, and systems thinking. 5. Frame problems, design interventions, specify outcomes, and maintain quality while balancing human, fiscal, and material resources. 6. Use information technology and knowledge-based resources to inform practice. 7. Articulate the effects of culture, diversity, values, and globalization in the design, delivery, and evaluation of health services. 8. Engage in lifelong learning activities that contribute to the professional nursing development and advancement of the nursing profession. EXCEEDED SIGNIFICANTLY EXCEEDED SIGNIFICANTLY EXCEEDED SIGNIFICANTLY EXCEEDED SIGNIFICANTLY EXCEEDED SIGNIFICANTLY EXCEEDED SIGNIFICANTLY EXCEEDED POTENTIAL AREAS FOR ACTION* Using Assessment for Program Improvement for 2014 As evidenced by the portfolio and various evaluation methods all MSN Program Outcomes Significantly exceeded expectations. The curriculum is effective and will be continued without major revisions at this time. We will continue to evaluate courses per faculty evaluation of meeting course outcomes and make changes as recommended or approved by the committee. It was found that two instruments (Administration Pre/Post Self-Assessment & the Preceptor Evaluation Tool) need to be revised due to data collection errors. Revised instruments will be piloted using December 2014 graduates. In the fall of 2013 ACEN site visitors recommended the maximum accreditation of 5 years the maximum for a new program. 17

18 NOTE: The following data are not available as we are a new program: Certification Exam Pass Rates, Alumni Survey Results, Employer Satisfaction Survey Results, and Student Role-Related Professional Practice data. Dissemination of Results: This report will be distributed to all regular MSN faculty members prior to MSN Committee Meeting for discussion, possible curriculum revisions and subsequent recommendations for changes. A final report will be submitted for review to the IU Kokomo School of Nursing CNF committee. In addition, at an advisory board meeting in spring 2014 we will share a short report of these findings. Also, for dissemination to all stakeholders, we put a short version of the report on the IU Kokomo School of Nursing website. The report will need to be a brief synopsis of our performance because a long version is burdensome for the reader. There are further supporting documents available upon request. 18

19 Appendix A Portfolio Evaluation Criteria and Rubric Please note: for evaluating the portfolio: Our benchmark is at 3 or more That is, the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN Program/Learning is 3 or more. For Each Criteria, we will use the following rubric: 4=Excellence Evident in Meeting MSN Outcome Evaluator easily finds evidence that is presented professionally; and Evidence presented shows that student has mastered the competency in a manner that suggests the best practice/exemplary practice; or Demonstrates high level of integration of knowledge, unusually strong insight. 3=Satisfactory Performance Evident in Meeting MSN Outcome Evaluator easily finds evidence that is presented professionally; and Evidence presented shows that the student has mastered the competency in a complete manner; or Demonstrates good integration of knowledge, insight. 2=Just Below Satisfactory Performance Evident in Meeting MSN Outcome Evaluator finds evidence only after searching the portfolio; or Evaluator finds that evidence is not presented in a professional manner; or Evidence presented shows some areas that are lacking with regard to achievement of the competency. 1=Unsatisfactory Performance Evident in Meeting MSN Outcome Evaluator is unable to find evidence pertinent to the competency; or Evidence presented does not show that the student addressed the competency in a relevant/reasonable manner (lacks attention to basic aspects of competency); or Evidence presented is found to be unsafe, unethical, or fraudulent. 4=Excellence Evident 3=Satisfactory Performance 2=Just Below Satisfactory. 1=Unsatisfactory Performance Rater assigns a numeric value to each category: 19

20 The Portfolio is designed to show how the program outcomes mapped to the Nurse Educator Competencies were met throughout the MSN program. 20

21 MSN Portfolio Rubric for Nursing Education Program/Learning Outcome #1 1. Model excellence in nursing leadership to improve nursing practice within the health care system. T619 Usability Testing for your Oncourse Practice site. Discuss how your classmates, T619 Simulation. T617 project, course learning outcomes, clinical evaluation tool. T670 Forum 5, course objectives & test questions. Module 7 & teaching plan. T679 Philosophy of teaching & plan, teaching plan in one area of classroom, clinical or staff development. Program Outcome 1 Total Maximum = 32 End of Program Outcome 1 Average _ Nurse Educator Competencies 3 & 4 3. Use assessment and evaluation strategies Uses a variety of strategies to assess and evaluate learning in the cognitive, psychomotor, and affective domains. Implements evidence-based assessment and evaluation strategies that are appropriate to the learner and to learning goals. Uses assessment and evaluation data to enhance the teachinglearning process. Provides timely, constructive, and thoughtful feedback to learners. Demonstrates skill in the design and use of tools for assessing clinical practice. 4. Participate in curriculum design and evaluation in program outcomes. Assessment of an education program and T615 Project. T615 Forum 2 & 3. T617Theory & Mock Evaluation project. Demonstrates knowledge of curriculum development including identifying program outcomes, developing competency statements, writing learning objectives, and selecting appropriate learning activities and evaluation strategies. 21

22 Bases curriculum design and implementation decisions on sound educational principles, theory, and research. Revises the curriculum based on assessment of program outcomes, learner needs, and societal and health care trends. Program/Learning Outcome #2 Function within an ethical-legal framework. Peer eval in all courses, discuss your participation in peer evaluation. T670 Forum 1, critique of three evidence based articles to inform teaching practice. T679 Online Journal, professional development plan. Program Outcome 2 Total Maximum =20 End of Program Outcome 2 Average Nurse Educator Competencies 6. Pursue continuous quality improvement in the nurse educator role. Demonstrates a commitment to life-long learning. Recognizes that career enhancement needs and activities change as experience is gained in the role. Participates in professional development opportunities that increase one s effectiveness in the role. Uses feedback gained from self, peer, student, and administrative evaluation to improve role effectiveness. Engages in activities that promote one s socialization to the role. Program/Learning Outcome #3 Synthesize knowledge from nursing, biological, behavioral, social, administrative, educational, and communication sciences for application to area of specialty. T615 Forum 4. T670 Forum 4, evidence based critical reflection activity. Program Outcome 3 Total Maximum=20 End of Program Outcome 3 Average 22

23 Nurse Educator Competencies 8. Function within the educational environment. Uses knowledge of history and current trends and issues in higher education as a basis for making recommendations and decisions on educational issues. Identifies how social, economic, political, and institutional forces influence higher education in general and nursing education in particular. Incorporates the goals of the nursing program and the mission of the parent institution when proposing change or managing issues. Advocates for nursing and nursing education in the political arena. Program/Learning Outcome 4 Demonstrate scholarly inquiry and reflection that exemplifies clinical reasoning, critical, creative, and systems thinking. T615 Project, T619 Forum 2 & 5. T617 Presentation of method. T670 Forum 1, critique 3 evidence based articles to inform practice. T679 Committee participation, student evals of clinical. Program Outcome 4 Total Maximum=24 End of Program Outcome 4 Average Nurse Educator Competencies 7 Engage in Scholarship Draws on extant literature to design evidence-based teaching and evaluation practices. Exhibits a spirit of inquiry about teaching and learning, student development, evaluation methods, and other aspects of the role. Designs and implements scholarly activities in an established area of expertise. 23

24 Disseminates nursing and teaching knowledge to a variety of audiences through various means. Demonstrates skill in proposal writing for initiatives that include, but are not limited to, research, resource acquisition, program development, and policy development. Demonstrates qualities of a scholar: integrity, courage, perseverance, vitality, and creativity. Program/Learning Outcome 5 Frame problems, design interventions, specify outcomes, maintain quality while balancing human, fiscal, and material resources. T615 Project, T619 social media. T617 Critique of literature. T679 Discussion forum. Program Outcome 5 Total Maximum=20 End of Program Outcome 5 Average Nurse Educator Competencies 5. Function as a change agent and leader. Models cultural sensitivity when advocating for change. Integrates a long-term, innovative, and creative perspective into the nurse educator role. Evaluates organizational effectiveness in nursing education. Implements strategies for organizational change. Promotes innovative practices in educational environments. Develops leadership skills to shape and implement change. Program/Learning Outcome 6 Use information technology and knowledge-based resources to inform practice. T619 Forum 1, 4, 5. T619 Project. T617 Project.T670 Module 1, Module 3 & module 7 & forum 4. End of Program Outcome 6 Total Maximum 56 Average 24

25 Nurse Educator Competencies 1.Facilitate learning. Implements a variety of teaching strategies appropriate to learner needs, desired learner outcomes, content, and context. Grounds teaching strategies in educational theory and evidencebased teaching practices. Recognizes multicultural, gender, and experiential influences on teaching and learning. Engages in self-reflection and continued learning to improve teaching practices that facilitates learning. Uses information technologies skillfully to support the teachinglearning process. Practices skilled oral, written, and electronic communication that reflects an awareness of self and others, along with an ability to convey ideas in a variety of contexts. Models critical and reflective thinking. Creates opportunities for learners to develop their critical thinking and critical reasoning skills. Shows enthusiasm for teaching, learning, and nursing that inspires and motivates students. Demonstrates interest in and respect for learners. Uses personal attributes (e.g., caring, confidence, patience, integrity and flexibility) that facilitate learning. Develops collegial working relationships with students, faculty colleagues, and clinical agency personnel to promote positive learning environments. 25

26 Maintains the professional practice knowledge base needed to help learners prepare for contemporary nursing practice. Serves as a role model of professional nursing Program/Learning Outcome 7 Articulate the effects of culture, diversity, values, and globalization in the design, delivery, and evaluation of health services. T615 Concept Map of med/surg Concepts. T619 Learner assessment as part of the project. T617 Project. T670 Module 1, Module 2, Forum posting 4. T679 Journals. Program Outcome 7 Total Maximum=16 Average Nurse Educator Competencies 2 Facilitate Learner Development and Socialization Identifies individual learning styles and unique learning needs of international, adult, multicultural, educationally disadvantaged, physically challenged, at-risk, and second degree learners. Fosters the cognitive, psychomotor, and affective development of learners. Recognizes the influence of teaching styles and interpersonal interactions on learner outcomes. Assists learners to develop the ability to engage in thoughtful and constructive self and peer evaluation. Program/Learning Outcome 8 Engage in lifelong learning activities that contribute to the professional nursing development and advancement of the nursing profession. T615 Forum Lit review on concept curriculum. T670 Diversity Multigenerational posting. Forum posting 4. T679 Professional development plan. Program Outcome 8 Total Maximum= 8 End of Program Outcome 8 Average 26

27 Nurse Educator Competencies 2 Facilitate Learner Development and Socialization Models professional behaviors for learners including, but not limited to, involvement in professional organizations, engagement in lifelong learning activities, dissemination of information through publications and presentations, and advocacy. Engages in effective advisement and counseling strategies that help learners meet their professional goals. 27

28 Appendix B NURS L579 MSN Administrative Track Portfolio Evaluation Criteria and Rubric Please note: for evaluating the portfolio: Our benchmark is at 3 or more That is, the mean of all ratings across MSN students is 3 or more for each individual competency, and that the overall mean (of all the scores) for every individual student (100%) in the MSN Program/Learning is 3 or more. For Each Criteria, we will use the following rubric: 4=Excellence Evident in Meeting MSN Outcome Evaluator easily finds evidence that is presented professionally; and Evidence presented shows that student has mastered the competency in a manner that suggests the best practice/exemplary practice; or Demonstrates high level of integration of knowledge, unusually strong insight. 3=Satisfactory Performance Evident in Meeting MSN Outcome Evaluator easily finds evidence that is presented professionally; and Evidence presented shows that the student has mastered the competency in a complete manner; or Demonstrates good integration of knowledge, insight. 2=Just Below Satisfactory Performance Evident in Meeting MSN Outcome Evaluator finds evidence only after searching the portfolio; or Evaluator finds that evidence is not presented in a professional manner; or Evidence presented shows some areas that are lacking with regard to achievement of the competency. 1=Unsatisfactory Performance Evident in Meeting MSN Outcome Evaluator is unable to find evidence pertinent to the competency; or Evidence presented does not show that the student addressed the competency in a relevant/reasonable manner (lacks attention to basic aspects of competency); or Evidence presented is found to be unsafe, unethical, or fraudulent. 4=Excellence Evident 3=Satisfactory Performance 2=Just Below Satisfactory. 1=Unsatisfactory Performance Rater assigns a numeric value to each category: 28

29 The Portfolio is designed to show how the program outcomes mapped to the AONE Nurse Executive Competencies were met throughout the MSN program. 29

30 MSN Portfolio Rubric for Nursing Administration (Artifacts for portfolio) Program/Learning Outcome #1 Model excellence in nursing leadership to improve nursing practice within the health care system. Program Outcome 1 Total Maximum = 12 End of Program Outcome 1 Average _ I. Communication and Relationship-Building A. Effective Communication Make oral presentations to diverse audiences on nursing, health care, and organizational issues (Power points for patient care delivery system presentation- L574; Evidence-based change project presentation-practicum) Produce cogent and persuasive written materials to address nursing, health care and organizational issues appropriate to the audience (Patient care delivery system executive summary-l574; Nursing study proposal-r590) B. Relationship Management Build trusting, collaborative relationships with: Staff, Peers, Other disciplines and ancillary services, Physicians, Vendors, Community leaders, Legislators Nursing and other educational programs (Forum -post-best-worst bosses/emotional intelligence-l574; Interprofessional/interdisciplinary meeting minutes/ communication diagram-l574; Forum-Top 10 list of excellence in leadership-practicum) 30

31 Program/Learning Outcome #2 Function within an ethical-legal framework. II. Knowledge of the Health Care Environment A. Clinical Practice Knowledge Maintain knowledge of current nursing practice and the roles and functions of patient care team members (Interview of Advanced Practice Nurse-N504; Policies/Procedures Analysis-L530) Ensure that written organization clinical policies and procedures are reviewed and updated in accordance with evidence-based practice (Policies/Procedures Analysis- L530) D. Health Care Policy Participate in the legislative process concerning health care through membership in professional organization and personal contact with public officials (Legislative Project-L530) Interpret impact of state and federal legislation on nursing and health care organizations (Power Point-Law-L530) IV. Professionalism C. Ethics Articulate the application of ethical principles to operations (Forum Societal Expectations/Culture/Fiscal Policy- L671; Forum-Ethical Legal Pitfalls-practicum) Integrate high ethical standards and core values into every day work activities ((Forum Societal Expectations/Culture/Fiscal Policy- L671; IRB Proposal-R590) E. Advocacy Program Outcome 2 Total Maximum = 28 End of Program Outcome 2 Average 31

32 Role model the perspective that patient care is the core of the organization s work (Forum Patient/family plan of care- L574) Program/Learning Outcome #3 Synthesize knowledge from nursing, biological, behavioral, social, administrative, educational, and communication sciences for application to area of specialty. Program Outcome 3 Total Maximum= 24 End of Program Outcome 3 Average II. Knowledge of the Health Care Environment B. Delivery Models/Work Design Determine when new delivery models are appropriate, and then envision and develop them (Patient care delivery system executive summary-l574) F. Evidence-Based Practice/Outcome Measurement Interpret information from research (Evidence-based change project presentation-practicum) Utilize research findings for the establishment of standards, practices and patient care models in the organization (Evidence-based change project presentation-practicum) Disseminate research findings to patient care team members (Nursing study proposal-r590; Evidencebased change project presentation-practicum) G. Patient Safety Support the development and implementation of an organization-wide patient safety program (Leadership for Effective Change in a Complex Organization Paper-N504) 32

33 III. Leadership E. Change Management Utilize change theory to implement organizational change (Forum response on Change-N504; Leadership for Effective Change in a Complex Organization Paper-N504) Program/Learning Outcome 4 Demonstrate scholarly inquiry and reflection that exemplifies clinical reasoning, critical, creative, and systems thinking. II. Knowledge of the Health Care Environment F. Evidence-Based Practice/Outcome Measurement.Interpret information from research (Patient care delivery system executive summary-l574; Nursing study proposal-r590) Utilize research findings for the establishment of standards, practices and patient care models in the organization (Leadership for Effective Change in a Complex Organization Paper-N504; Patient care delivery system executive summary-l574, Nursing study proposal-r590) Disseminate research findings to patient care team members (Nursing study proposal presentation-r590; Evidence-based change project presentation-practicum) IV. Professionalism D. Evidence-Based Clinical and Management Practice Advocate use of documented best practices (Forum-How have you used evidence based management research in your practicum?) Program Outcome 4 Total Maximum= 16 End of Program Outcome 4 Average 33

34 Program/Learning Outcome 5 Frame problems, design interventions, specify outcomes, maintain quality while balancing human, fiscal, and material resources. II. Knowledge of the Health Care Environment C. Healthcare Economics Articulate federal and state payment systems and regulations, as well as private insurance issues which affect organization s finances. (Forums: societal expectations/culture/fiscal policy, implementation of fiscal policy, fiscal planning/cost/quality-l671) Understand /articulate individual organizations payer mix, CMI and benchmark database. (Forums: societal expectations/ culture/fiscal policy, implementation of fiscal policy, fiscal planning/cost/quality-l671) F. Evidence-Based Practice/Outcome Measurement Interpret information from research (Evidence-based change project -practicum) Utilize research findings for the establishment of standards, practices and patient care models in the organization (Evidence based change project-practicum) Disseminate research findings to patient care team members (Evidence based change project presentation-practicum) V. Business Skills A. Financial Management Manage financial resources by developing business plans (Program planning with fiscal considerations, executive summary and forum- fiscal planning/cost/quality-l671) B. Human Resources Program Outcome 5 Total Maximum= 32 End of Program Outcome 5 Average 34

35 Interpret legal and regulatory guidelines (Power Point-Law-L530) C. Strategic Management Analyze the situation and identify strategic direction Conduct SWOT and gap analyses (SWOT analysis L574) Program/Learning Outcome 6 Use information technology and knowledge-based resources to inform practice. End of Program Outcome 6 Total Maximum 16 Average V. Business Skills E. Information Management and Technology Utilize hospital database management, decision support, and expert system programs to access information and analyze data from disparate sources for use in planning for patient care processes and systems (IT forum-practicum) Evaluate and revise patient care processes and systems (Patient care delivery system executive summary-l574) Use computerized management systems to record administrative data (billing data, quality assurance data, workload data, etc.) (Program planning with fiscal considerations, executive summary and forum- fiscal planning/cost/quality-l671) Recognize the relevance of nursing data for improving practice. (Patient care delivery system executive summary-l574; Nursing study proposal-r590; Evidencebased change project -practicum) 35

36 Program/Learning Outcome 7 Articulate the effects of culture, diversity, values, and globalization in the design, delivery, and evaluation of health services. Program Outcome 7 Total Maximum= 8 Average I. Communication and Relationship-Building D. Diversity Create an environment which recognizes and values differences in staff, physicians, patients and communities (Forum on healthcare costs and delivery; healthcare public opinion-l671) Assess current environment and establish indicators of progress toward cultural competency (Forum- How can managers make diversity work- practicum) Program/Learning Outcome 8 Engage in lifelong learning activities that contribute to the professional nursing development and advancement of the nursing profession. II. Leadership B. Personal Journey Disciplines Assess one s personal, professional and career goals and undertake career planning (Interview of an Advanced Practice Nurse-N504; Philosophy of Leadership- N504; Nurse Executive Competency Checklist; Drop Box-Self-Assessment of administrative experience/plan for certification- L574) Program Outcome 8 Total Maximum= 8 End of Program Outcome 8 Average IV. Professionalism 36