University of Virginia. School of Nursing

Transcription

1 University of Virginia School of Nursing Graduate Program ASSESSMENT/EVALUATION MODEL Student Learning Objectives And Program Objectives

2 Philosophy of the UVa School of Nursing The central purpose of the University of Virginia is to enrich the mind by stimulating and sustaining the spirit of free inquiry directed to understanding the nature of the universe and human existence. The philosophy of the School of Nursing is consistent with that of the University as it prepares leaders in health care. Nursing is both a profession and a discipline that is responsive to changing health needs. It is concerned with human experiences and responses to birth, health, illness, and death within the context of individuals, families, groups, and communities (ANA, 1995). Nurses, often in collaboration with other health care professionals, promote the optimal health care and comfort of individuals and groups through the systematic application of knowledge from nursing and related disciplines. The faculty believes that education is based on humanistic approaches that foster critical thinking and promote awareness of social and cultural diversity among individuals. The faculty views each student as a unique person with special talents, abilities, needs, and goals. Cultural diversity, varying life experiences, and changing socioeconomic factors affect each student differently. To this end, faculty endeavor to provide an environment that assists students to realize their full potential. The acquisition of professional knowledge and the development of clinical competence occur through active involvement of the student in the learning process. Students assume primary responsibility for learning, while faculty provide educational opportunities for knowledge acquisition and professional role development. We believe that an atmosphere of shared growth and inquiry offers the maximum potential for development. Baccalaureate education in nursing is the basic preparation necessary for the practice of professional nursing. This education provides the foundation for the development of professional knowledge, critical thinking, ethical decision-making, leadership skills, and the independent and interdisciplinary pursuit of high standards of health care. Master s education prepares the nurse with strong critical thinking and decision-making skills for advanced practice in acute and primary care or for specialty practice in the areas of management and public health leadership. A primary aim of the master s nursing education is to ensure that every student acquires the ability to analyze, synthesize, and utilize knowledge in a specialty area. Doctoral education prepares the nurse scholar to influence health care through leadership in education, policy, practice, research, and knowledge development. Implicit in the practice of professional nursing is accountability for professional growth and practice, demonstration of leadership, and commitment to the development and application of nursing theory and research. Life-long learning leads to the optimal development of both the individual practitioner and the discipline of nursing.

3 Purpose and Objectives of the SON Graduate Program The purpose of the graduate program is to prepare nurses with strong critical thinking and decision-making skills for generalist practice as a clinical nurse leader (CNL), or advanced practice in acute and primary care and specialty practice in the areas of health systems management and public health leadership. Graduates of the program: G-1. Integrate theoretical and research-based knowledge as a generalist leader or in an advanced nursing practice specialty; G-2. Provide care and comfort to individuals, families and groups experiencing complex health care needs; G-3. Provide care that reflects sensitivity to differences among culturally and ethnically diverse populations; G-4. Assume a leadership role in establishing and monitoring standards of practice to improve patient care in collaboration with nursing experts; G-5. Use ethical principles to guide decision-making in nursing practice; G-6. Evaluate clinical practice in relation to professional practice standards and relevant statutes and regulations; G-7. Apply the research process to improve evidence-based clinical practice and contribute to knowledge development; G-8. Engage in self-directed and purposeful activities in seeking necessary knowledge and skills to enhance career goals; G-9. Examine economic, political, and social forces affecting nursing care delivery in complex health systems; G-10. Promote multidisciplinary collaboration to ensure quality, cost effective care; G-11 Contribute to the development of peers, colleagues, and others to improve patient care and foster the growth of professional nursing; and G-12 Act as change agents to create environments that promote effective nursing practice and patient outcomes. These core characteristics are in accordance with national guidelines for the Clinical Nurse Leader and professional standards of advanced practice nursing specialties.

4 Student Learning Outcome Integration & Application of Theory and Research to Practice: Upon graduation, the advanced-practice nursing student should be able to interpret theoretical and research-based knowledge and apply this knowledge to nursing practice as demonstrated by a written dramatization scenario. Assessment Methods: One of the SON Graduate Program objectives is to prepare advanced practice nurses who are able to integrate theoretical and research-based knowledge into their nursing practice. Several courses within the curriculum address this objective. In November 1999, the SON Graduate Program Committee established the following courses as Core Courses within the Masters Nursing Curriculum: Health Promotion; Epidemiology; Health Policy; Nursing Research; and Nursing Theory. Additional Core Courses identified for those students entering into advanced practice nursing roles include Pathophysiology, Pharmacology, and Advanced Health Assessment. Utilizing a core graduate nursing course that is focused on the specialized body of knowledge in the discipline of nursing and its contexts, primarily nursing theory, ethical principles, historical perspectives and aesthetics, provides an assessment strategy that evaluates the performance of all graduate students across the advanced practice tracks for this measure. GNUR 580: Theoretical Foundations of Nursing (core course) requires several assignments that address the interpretation and application of theoretical and research knowledge across all the contexts of intelligent interpretation of a body of knowledge. The assignment, Formal Intervention Paper (written dramatization scenario), will be used to assess the SLO related to interpretation of theoretical and research-based knowledge, use of ethical principles to guide decision-making in nursing practice, application of the research process to improve evidence-based clinical practice and contribute to knowledge development. This assignment allows the student to identify an advanced practice nursing intervention, supported by a specific nursing theory, as it is applied to a clinical nursing problem within their selected population. Standards: Currently, the course professor utilizes an assignment-grading rubric and reviews annual course scores and adjusts teaching methodologies accordingly. The course professor evaluates each student s course performance in assessing the component parts of intelligent interpretation of current literature about the knowledge of the discipline of nursing and its application to practice by utilizing a grading rubric across all context components. For the purpose of teaching/learning assessment, the SON should evaluate such parameters as the student s ability to explain a theory clearly, cite and consider literature critical of the theory, present examples of when the theory works and doesn t work and explore the application of the theory to the specific clinical nursing problem within a selected population. Following the initial assessment of this SLO, a performance standard should be set for this SLO.

5 UVA School of Nursing Graduate Student Learning & Program Objectives

6 Results/ Findings: The course professor s grades for this intervention assignment have been as follows: Spring 03 A+: 1, A: 3, A-: 1, B+: 2, B: 3, B-: 0 Fall 03 A+: 0, A: 8, A-: 5, B+: 9, B: 0, B-: 0 Spring 04 A+: 0, A: 7, A-: 4, B+: 4, B: 6, B-:2 Fall 04 A+: 0, A: 2, A-: 0, B+: 1, B: 3, B-: 0 Spring 05 A+: 0, A: 9, A-: 8, B+:9, B: 0, B-: 1, C+: 1 (on-line) Fall 05 A+: 0, A: 2, A-: 3, B+: 2, B: 5, B-: 0 More than 50% of students have been graded at 90% or above (A- or better) Use of the results: The Graduate Program Committee recommended that in AY 06-07: The formal intervention paper assignment for GNUR 580: Theoretical Foundations of Nursing, be used for the first time as a SLO to assess knowledge and skill in the interpretation and application of nursing theory to evidence-based practice. An assessment rubric be developed to analyze the overall students performance on this SLO in AY Two faculty members assess a representative sampling of students using the assessment rubric. Subsequently a performance standard for the specific SLO should be identified. Action Plan: Beginning in September 07, utilizing a teaching/learning assessment rubric, report the previous AY semester(s) overall scores on the formal intervention paper (GNUR 580) to the Graduate Program Committee, as a standard for assessment of skill and knowledge competency in the integration and application of theory and research to practice. Following two years of reports (9/07 & 9/08), if the overall SLO performance standard (yet to be determined) is achieved for subsequent years, the course professor will not be required to submit assessment scores to the Graduate Program Committee, unless requested.

7 Student Learning Outcome Provision of Care in Complex Cases with Diverse Populations Upon graduation, the advanced nurse practitioner student should be able to evaluate the pathophysiological impact of complex disease processes on diverse populations as demonstrated by performance on scenario multiple choice format examinations. Assessment Methods: Several of the SON Graduate Program objectives are targeted at preparing advanced practice nurses who are able to evaluate the pathophysiological impact of complex disease processes on diverse populations. Several courses within the curriculum address this objective. In November 1999, the SON Graduate Program Committee established the following courses as CORE COURSES within the Masters in Nursing Curriculum: Health Promotion; Epidemiology; Health Policy; Nursing Research; and Nursing Theory. Additional CORE COURSES identified for those students entering into advanced practice nursing roles include Pathophysiology, Pharmacology, and Advanced Health Assessment. Utilizing a core graduate nursing course that is focused on the pathophysiological impact of complex disease processes on diverse populations provides an assessment strategy that evaluates the performance of all graduate students across the advanced practice tracks for this measure. The majority of graduates from the UVa School of Nursing Masters Program take GNUR 584: Pathophysiology. The course, utilizing lecture and case study methodologies to teach content, assesses individual student learning through the administration of 3 multiple choice format examinations (scenario testing), respectively worth 30%, 30%, and 40% of the final course grade. Test construction involves case-based questioning. This format of interpretative exercise efficiently assesses skill in applying both knowledge and skills to new situations; problem solving; and analysis. Post-examination review sessions are offered, as well as on-line reference materials to assist student learning. Historically and anecdotally, students failing in Pathophysiology (GNUR 584) and Pharmacology (GNUR 550) are also doing poorly in other graduate courses. Thus, this course serves as an indicator of successful student performance across advanced nursing practice programs. This may lay a foundation for the development of a program indicator for at-risk students. Standards: The graduate nursing program should prepare advanced practice nurses who are able to demonstrate skill and knowledge competency in: Evaluating disease impact (synthesis through comprehension, analysis and application, physiological and pathophysiological mechanisms) on the provision of care and comfort to individuals experiencing complex health care needs, that Reflects sensitivity to differences among culturally and ethnically diverse populations.

8 Following the initial assessment of this SLO, a performance standard should be set for this SLO. Results/Findings: The average score for GNUR 584 exams for AY 03-04, 04-05, & is 89. Use of the results: Currently, the course professor reviews the exam items analysis report, assuring that at least 75% of the top 10% of students get questions correct, otherwise the question is thrown out. The course professor also reviews test content to identify problem areas. For example, the course professor identified that students had trouble with Immunology content within the course. In response, the course professor held additional review sessions and developed an on-line reference course to assist student learning. To facilitate further learning, the professor encourages students to ask questions via or during review sessions. Action Plan: Beginning in September 07, report the previous AY GNUR 584 examination scores to the Graduate Program Committee, as a performance standard for assessing overall skill and knowledge competency in: Evaluating disease impact (synthesis through comprehension, analysis and application, physiological and pathophysiological mechanisms) on the provision of care and comfort to individuals experiencing complex health care needs, that Reflects sensitivity to differences among culturally and ethnically diverse populations. Following two years of reports (9/07 & 9/08), if the overall performance standard of a minimum of XX% of students achieved > YY% is met for subsequent years, the course professor will not be required to submit scores to the Graduate Program Committee, unless requested. The Graduate Program Committee shall discuss the development of a formal program remediation plan for students identified at-risk of failing graduate core courses.

9 Program Outcome SON Advanced Practice Certification Pass Rates (ANCC-APN, PNCB-PNP, AANP-FNP) The UVa SON Graduate Program will prepare advanced practice nurses who are successful at attaining certification as advanced practice nurses within their specialty areas as demonstrated by published instrument pass rates. Assessment Methods: The American Nurses Credentialing Center (ANCC) develops, delivers and supports national certification programs (published instrument) for advanced practice nurses in the specialty areas of Family Nurse Practitioner (FNP), Acute Care Nurse Practitioner (ACNP), Geriatric Nurse Practitioner (GNP), Adult Psychiatric & Mental Health Nurse Practitioner (AP&MHNP), and Family Practice & Mental Health Nurse Practitioner (FP&MHNP). Annually in March, the ANCC provides a Nurse Practitioner (NP) Programs Aggregate Data Report for each institution that reports the number of candidates tested within the past calendar year, the specific institutional mean-scaled score, and the overall ANCC mean-scaled score. This report reflects first-time pass rates only. The maximum possible scaled score is 500. The passing score is a scaled score of 350. If fewer than three (3) candidates take the examination, the results are not reported in order to maintain the confidentiality of the individual candidates. The Pediatric Nursing Certification Board (PNCB) develops, delivers and supports a national certification program (published instrument) for advanced practice nurses in the specialty area of Pediatric Nurse Practitioner (PNP). The PNCB provides institutions with approved PNP programs with an annual aggregated report (March) that lists the number of candidates, number of attempts, program average, national average, and pass rate. As of 2004, the report also breaks down the scores into five (5) categories (Pathophysiology, Patho-psychology, Development, Physiology, and Psychosocial), benchmarking the national average response within the categories to the individual program s average responses. The American Academy of Nurse Practitioners Certification Board (AANPCB) develops, delivers, and supports a national certification program (published instrument) for advanced practice nurses in the specialty area of Family Nurse Practitioner (FNP). Standard: 90% of the SON master s/post-master s graduates sitting for a certification examination will pass the first time they take the examination. (200X) Findings: SON FNP Program: % (ANCC %) (ANCC-88.06%) % (ANCC-85.07%) % (ANCC-86.92%) % (ANCC-89.7%)

10 SON ACNP Program: % (ANCC-89.63%) % (ANCC-87.63%) % (ANCC-86.28%) % (ANCC-91.18%) % (ANCC-90.75%) SON GNP Program: % (ANCC-77.62%) 2004 *** % (ANCC-54.76%) % (ANCC-75.47%) new program SON AP&MHNP Program: 2005 too few candidates % (ANCC-50.42%) % (ANCC-76.82%) % (ANCC-84.75%) SON FP&MHNP Program: 2005 *** 2004 too few candidates % (ANCC-56.70%) new program Pediatric NP Program: % (Scores: SON-473/National-476) % (Scores: SON-394/National-482) % (Scores: SON-475/National-490) % (Scores: SON-500/National-501) % (Scores: SON-504/National-501) % (Scores: SON-522/National-519) Use of the Results: The SON receives an annual report from the ANCC and PNCB. The SON Associate Dean for Academic Affairs, Nurse Practitioner Program Directors/Coordinators, and the Graduate Program Committee review the findings as reported by the certification boards to determine progress toward meeting the 90% pass rate standard. The Program Directors/Coordinators and the Graduate Program Committee analyze the results and determine program impact and opportunities for improvement. This analysis allows the SON to benchmark our performance against similar programs within the Commonwealth of Virginia. If further analysis is required, a taskforce may be assigned to investigate program components and requirements. The Graduate Program Committee makes recommendations to the SON Faculty Organization for consideration and modification/adoption.

11 Action Plan: This program performance standard is an annual requirement for the SON s Evaluation Plan under the Commission on Collegiate Nursing Education (CCNE) Accreditation. Therefore, the SON Graduate Program will continue to review the certification scores annually and conduct program assessment as indicated for the purposes of ongoing program improvement and accountability.

12 Program Outcome End-of-Program (EOP) evaluation Upon graduation from the SON Graduate Program, the graduate nursing student should express overall satisfaction with the graduate program and specific satisfaction with achievement of each of the eleven program objectives/outcomes as demonstrated by endof-program evaluations. Assessment Methods: In 2001, the SON defined an overall program assessment strategy that included assessments at three points of contact: end-of-program evaluation; alumni survey (one year post-graduation); and with alumni permission, employer assessment. This strategy was applied across the baccalaureate and graduate programs. The doctoral program utilized only end-of-program evaluations and program alumni survey (one year post graduation). The end-of-program evaluations are conducted in the summer (July/August) following graduation for all program graduates in the past 12 months. Within degree programs, separate evaluation formats may be utilized for distinct program tracks (such as for baccalaureate tracks of traditional, second degree, and RN-to-BSN) to capture the specific needs of the program and/or track populations. The same applies to the graduate program tracks for master s program and post-master s specialty tracks. The SON Graduate End-of-Program Evaluation is conducted during the summer following graduation for all graduates in the past 12 months. The evaluation assesses the graduate nursing student s attitude regarding overall graduate program satisfaction and satisfaction with their achievement of the 11 specific graduate program objectives/outcomes. It also collects information such as: Baccalaureate education program Prior graduate program Professional activities and organizations Employment during SON graduate program Program enrollment status Program track Satisfaction with aspects of academic life Suggestions Current position, type of setting, geographic location Attitude toward preparation for new role Potential to recommend program Experiences that stood out Demographics (age, gender, marital status, number of children, required residence changes to attend graduate program and/or miles commuted) Financed education Comments The respondents are asked to rate all 11 graduate program objectives/outcomes on a scale of 1 to 3, in which 1=did not prepare me, 2=somewhat helpful, and 3=very helpful. End-of-program data are reviewed with the Graduate Program Committee in September.

13 Standard: Following graduation, > 50% of graduate nursing students will report overall satisfaction with the SON Graduate Program (200X). Findings: AY & : Summary Report of Master s End-of-Program Evaluation: (26 respondents (n=9 for AY 2001, n=17 for AY2002) Results show a range of responses with a mean of 2.46 to The highest rated program outcome was apply critical thinking and inquiry skills to the exploration and solution of problems in advanced practice nursing, while the lowest was engage in professional consultation and collaboration in planning health care and influencing health policy. AY : Summary Report of Master s/post Master s End-of-Program Evaluations: (n=27) 14 items Results show a range of responses with a mean of 3.91 to The two highest rated program outcome were establish therapeutic relationships with clients and families (1 st ) and acquiring knowledge and skills necessary for advanced professional practice in nursing (2 nd ), while the two lowest were evaluate scholarship and research for application in nursing specialties (13 th ) and identify nursing problems amenable to investigation and participate in research studies (14 th ). Applying critical thinking was 8 th and engage in professional consultation was 3 rd. Upon graduation, student evaluation of preparedness for an advanced practice role mean score was Use of the Results: Historically the return rate on end-of-program evaluations has been fair to poor. The resources required to conduct an in-depth analysis on the collected information were significant. In recent years (2003, 2004, 2005) the SON end-of-program evaluation process was being evaluated at the baccalaureate program level and an external vendor (EBI Exit Survey) was being considered. The SON was considering a three-year assessment cycle using an electronic format and external vendor with national benchmarking capability. In May 2003, the data were determined to be favorable by consensus of the Graduate Program Committee, especially for the assessment of program objectives/outcomes. However, they did determine that the response rate and the length and nature of the current survey questions warranted more attention, and therefore it was identified as a continuing agenda item for the committee. Action Plan: This program performance standard is an annual requirement for the SON s Evaluation Plan under the Commission on Collegiate Nursing Education (CCNE) Accreditation. Therefore, the SON Graduate Program will continue to review the certification scores

14 annually, and conduct program assessment as indicated for the purposes of ongoing program improvement and accountability. In March 2005, the MSN Program Committee determined that the End-of-Program Evaluation be administered midway through graduate students final semester in order to improve response rates. Students final semester will be determined to accommodate mid-ay graduates. Additionally, in April 2005, the Committee shortened the survey to include primarily the objectives/outcomes of the SON Graduate Program. This process will be adopted for AY XXXX XXXX. The evaluation will assess graduate nursing students attitude regarding: Overall graduate program satisfaction, and Satisfaction with their achievement of 11 specific graduate program objectives. The Graduate Program Committee revised the standard for this Program Outcome.