NC STATE UNIVERSITY. STANDARD 11: Assessment & Outcomes COLLEGE OF VETERINARY MEDICINE

Transcription

1 STANDARD 11: Assessment & Outcomes videos to clarify difficult points, adding a number of after hour surgery skill sessions led by the student surgery club and experienced instructors, modification of courses to add review sessions, low stress surgical models or cadaver practice before moving into actual surgery, giving students medical math practice sheets and building anesthesia puzzle tests for the students to reason through and solve. We are in the process of developing a clinical skills lab, which will further increase the opportunity for hands-on practice of technical skills using models. In addition to the use of the clinical competency books, 4 th year rotations have been asked to identify learning outcomes that include an assessment of at least one core skill that will be directly observed and graded with a performance rubric. These skills will be mapped to each of the 9 competencies to assure that all competencies are covered. See the map of current competencies. The direct observation of clinical competencies measures completion of specific skills. In order to assess student competencies in a more global fashion, a standardized rubric that maps to behavioral categories was developed for 360- degree assessment of 4 th year students (See Table B, Appendix 11.1c). Since initiation in 2007, there have been some modifications of the standard rubric for specific courses to allow more precision, but each rubric assessment is mapped to the standard rubric. The number of marginal grades for each category is the most useful measurement to assess if students need more preparation in one area or another (See Table C, Appendix 11.1.c). Overall, for the years , the categories with the most marginal grades were problem solving skills/judgment and knowledge base. One focus of the curriculum change is to develop additional training in these two areas d Employment rates of graduates (within one year of graduation), The CVM evaluates recent graduate employment through senior student surveys conducted in the final six weeks of the senior year, comparison of similar data collected by the AVMA with national rates, and surveys of alumni at one year after graduation (See Appendix 11.1.d, Tables A, B, C). The alumni survey asks if the graduate was employed within 3 months of graduation. The AVMA survey shows a drop in percentage of students that had received a job offer during 2011 and 2012 and a drop in number of job offers starting in 2010 that stabilized at ~1.6/graduate. These numbers increased in 2013, in both North Carolina and nationally. Other parameters, including number of job offers and percent accepting a job that matched preference for employment was similar from , indicating students could find the job of their preference if made an offer. The NC State senior exit survey mirrors the AVMA data, with no students receiving 5 or more job offers after Alumni data shows an approximate 10-12% decrease in employment at 3-months after graduation for the classes most recently surveyed in 2011 and 2012 (83.9 and 82.9% respectively), as compared with % employment in The drop in employment could be explained by the national recession. We will continue to monitor employment to see if the improvement noted in the 2013 survey continues. To address employment issues we are revising curriculum to make students more marketable by improving communication and business training; areas, which scored lower or were cited as key skills in alumni and employer surveys (See Sections 11.1.e and f). We are also increasing support for training students in job related skills and assisting job placement. We are cognizant of debt burden DVM students encounter and therefore monitor the debt of our students. The AVMA data allows a national comparison and shows that our students graduate with considerably less debt than the national average (e.g. in 2013 mean debt for our students was 62.4% of the national mean). See Appendix 11.1, Table D for details e Assessments of graduating seniors; and assessments of alumni at some post-graduation point (for example, three and/or five years post-graduation) assessing educational preparedness and employment satisfaction, Graduating seniors evaluations of their educational experiences are assessed in the senior exit survey (Table A and B in Appendix 11.1e). Overall, 74-80% of the students rate the curriculum in the excellent/very good category. Student satisfaction with the curriculum increased with each year of the curriculum. Students (and alumni) rated the selective experience and TAU highly; confirming the value of these unique programs. Senior year course evaluations were modified to evaluate senior rotation learning outcomes for the Class of 2013, and in the first year they were highly satisfied with learning outcomes and their ability to achieve them (Table C in Appendix 11.1e). Among alumni, % were satisfied or very satisfied with their experience at the College of Veterinary Medicine (Table D, Appendix 11.1.e). Component areas received a mean score of 4.0 (good) or above except for Career Planning and some years for Classroom Facilities, Library Computer Facilities and Laboratory Facilities. Career Planning received the lowest scores (average) and will be addressed by increasing personnel and support in this area. Although overall satisfaction was good, alumni survey data indicated several areas in which they felt poorly prepared, including primary care diseases (i.e. feline 41

2 STANDARD 11: Assessment & Outcomes urinary tract obstruction, anal sac disease, cat abscesses), preventative medicine protocols (vaccination, parasite control, nutrition), routine non-elective surgery (gastrostomy, enterotomy, cystotomy, splenectomy, laceration repair), client communication (especially difficult clients), emergency procedures and surgery, providing options for cost-conscious clients, dentistry, equine technical skills (equine practitioners only), and routine cytology (ear cytology, UA sediments). Additional data gathered in alumni surveys include comments on material that should be removed from or added to the curriculum. Areas that were repeatedly mentioned for removal or reduction were embryology (eliminate or combine with other course), histology (too detailed), immunology and infectious organism courses (reduce, combine, make more clinically relevant), nutrition (not applicable to large animal practitioners), courses focused on large animal (theriogenology, population medicine, food animal or equine courses for small animal focused individuals), epidemiology, behavior (largely from large animal practitioners, since focus was small animal), toxicology (reduce) and evidence based medicine (wrong place in curriculum). Areas repeatedly cited as needing to be added were dentistry, communications, business, clinically relevant behavior, clinically relevant nutrition, hands-on practice with primary care practice techniques and non-elective surgery, repeated exposure to primary care problems and cases, emergency techniques and medicine, species-specific courses for students who plan on entering a specific area of practice, and case-based courses (especially in second and third years). The College is incorporating this input in its planning for the new curriculum f Assessments of employers of graduates to determine satisfaction with the graduates, See Table A, Appendix 11.1.f for the mean rating of employee s performance compared to graduates respondent has hired from other Universities. Employer surveys are mailed each year for graduates at one and five years after graduation. Return rates are low so data was combined for each year. Overall, employers have rated our graduates above average ( /5). Component skills were all at or above 4.0/5 except for surgical skills, business knowledge, and some years for leadership skills, ability to work with employees from all levels of business, and ability to accurately assess when to consult or refer a case. We used this data to guide decisions to address business knowledge and leadership with a new series of communication and professional skills courses to be coordinated by a new communication faculty member. We increased exposure to surgery and primary care in the senior year by expanding our general practice rotation and will add another general practice/surgery faculty member in to further support this activity g Assessments of faculty (and other instructors, for example interns and residents) related to such subjects as adequacy of clinical resources, facilities and equipment, library and information resources, etc.; and preparedness of students entering phases of education, and See Appendix 11.1.g., Tables A, B, C, and D for the survey results. In December 2013, the faculty were surveyed regarding the adequacy of College resources and student preparation before and after finishing the pre-clinical and clinical phases of the curriculum. Surveys were completed by 61 out of the 140 faculty in the College (44% response rate). Of the 61 who responded to the question, 13 (21%) had taught in the College for less than five years, 10 (16%) for 5-10 years, and 38 (62 %) for more than 10 years. Overall, most of the responding faculty felt that the overall preparedness of students was adequate to excellent for each year of the curriculum. Cited concerns centered on basic knowledge gaps and inadequate didactic lecture time (Table A, Appendix 11.1.g). When asked to rate overall facilities and equipment, 91% of the faculty rated them as adequate or excellent; 9% (5/57 respondents) felt that the main lecture theaters were outdated and the teaching laboratory space and equipment was insufficient for a class size of 100 students. Cited strengths of the facility and equipment were the Terry Center (particularly the integrated service rounds rooms), the available IT support, and recent A/V upgrades to the lecture halls, the TAU and the newly renovated HWC. Concerns cited revolved around the CVM aging facilities and space needs, particularly regarding the TAU, the anatomy laboratory, the EFAVC, and the two main large lecture theaters (Appendix 11.1.g, Table B). When asked to rate the overall library and information resources, 95% of the faculty rated them as very good or excellent and they specifically cited the Director of the VML and her staff as being exceptionally knowledgeable and extremely responsive (Appendix 11.1.g, Table C). When asked to rate the adequacy of the clinical resources that they used, 93% of the faculty rated the VHC caseload as being adequate to excellent for teaching veterinary students, however, concerns exist regarding the equine and farm animal caseload (Appendix 11.1g, Table D). When asked if there were other areas of concern 43% (23/53) answered yes. The concerns most commonly expressed were 42

3 STANDARD 11: Assessment & Outcomes regarding the ongoing curricular change, the enrollment increase and expansion of class size, the busy and complex caseload of the Terry Center, the need for an augmented primary care caseload, the increasing cost of veterinary education, and work/life balance issues for faculty and staff h Additional assessment that might assist the college in benchmarking its educational program. Oversight and benchmarking of NC State s education program is provided by ADAA and the FCCCE. Each course taught in the DVM curriculum is holistically reviewed and assessed by the FCCCE every three years. Extramural learning opportunities are monitored through both the student s evaluation of the experience and the mentor s evaluation of the student and the experience by the Student Services Office and the Extramural Course Coordinators. Faculty teaching ability is assessed by reviewing student instructor evaluations and peer evaluation of teaching. The graduate and residency programs are monitored by the CBS Graduate Studies Committee and the Faculty Committee on House Officer Programs (FCHOP), using tracking of the number of applicants for advanced training positions, programmatic reviews and exit interviews. Each individual residency program tracks the certification board pass rate for its residents and the employment opportunities. The Hospital Board tracks the hospital caseload, hospital revenue, client satisfaction and referring veterinarian satisfaction. Maximizing the quality of the hospital experience exposes students to a busy, positive hospital environment. Each hospital rotation is evaluated using online course and instructor evaluations. Since 1997, DVM students have been offered a wide variety of 1-2 week elective courses, called selectives. Selectives provide flexibility and unique learning opportunities for DVM students but require considerable faculty effort and expense. To assess the impact on student learning, we routinely survey our alumni and also polled all enrolled DVM students in The data shows that selectives are considered valuable or extremely valuable by over 90% of alumni (Appendix 11.1.h, Table A) Institutional outcomes a Describe how the college evaluates progress in meeting its mission (for example, benchmarking with other institutions, etc.), As described in the Executive Summary and Objectives of this report, the College has undertaken a comprehensive strategic planning process which will guide College decision-making to achieve our mission, realize our vision and sustain values. Our strategic goals will be tied to university and College wide metrics and targets. The College compares itself to a group of peer institutions with highly successful veterinary programs (i.e. Cornell, University of California Davis, Colorado State University, The Ohio State University, University of Pennsylvania, and University of Wisconsin-Madison) using the AAVMC Comparative Data Report. Data compared and evaluated included faculty numbers, student numbers, entering student GPA and GRE scores, diversity of faculty and students, tuition, total budget, state appropriations, research expenditures, development (endowment and current gifts), patient visits, and mean student debts b Describe the adequacy of resources and organizational structure to meet the educational purposes (Dean should provide), The College is well served in this regard, with an excellent and committed cadre of faculty and staff, a strong and engaged organizational structure, several unique resources (such as the TAU, outstanding facilities and the CCMTR), and an excellent clinical training environment. We are negotiating a complex evolution of our curriculum, seeking to adopt contemporary practices, and rebalance our efforts towards active learning and introduce new concepts that are essential skills for the current workplace. This process also creates challenges as we seek balance in the curriculum. Provided our faculty and staff remain actively engaged in the process we can be confident of success. As we look to the future, our educational facilities need to keep pace with our aspiration to provide the best possible programs. We will continually seek to renovate and modify our existing facilities, but we must plan for the next major facility project, and that is likely to be an educational building c Describe outcomes assessed for college activities that are meaningful for the overall educational process (for example, scholarly activity of the faculty, faculty awards, faculty and staff perception of teaching resources, student satisfaction with the educational program, teaching improvement benchmarks, and others). If your program assesses other outcomes, briefly describe the results, and The CVM has many distinguished faculty and members of the CVM community who are recognized at the University, state, national and international level. College faculty are also well represented in leadership positions at several veterinary specialty organizations. Our faculty members also serve as editors of many scholarly journals and have authored numerous 43

4 STANDARD 11: Assessment & Outcomes textbooks between Numbers and types of intramural and extramural grants and contracts awarded to faculty, numbers and quality of peer-review publications and service on NIH study sections are indicators of faculty success in biomedical research (see Appendix 10.2.a, Table A for notable measures of CVM research activity for the last three years). Among US institutions with veterinary colleges, NC State ranks 6th in citations per publication (4.28) and 5th in total number of citations (3652) to its 853 publications in the subject category of Veterinary Sciences in Web of Science from Our faculty and staff also are sought after for speaking at and participation in regional, national and international continuing education events. Many of our faculty are leaders in their specialty professional organizations. VHC Client Survey Results indicate many satisfied clients and the overall excellence of our clinical operations d Describe how outcomes findings are used by the college to improve the educational program (give examples). The College uses outcomes findings in a number of ways to improve the educational program. An important example is the curricular review process used to revise the Year 1-3 curriculum and described in Section The outcomes of this process are changes designed to: 1) create a better integrated curriculum that provides additional electives and improved training in communication, professional skills and critical thinking; and 2) an improved process for continuous review of the curriculum to facilitate responsive changes. Expected outcomes are greater job satisfaction, performance and employability of our graduates. Survey responses from alumni indicated that some felt unprepared for primary care medicine and surgery. A task force was charged with evaluating this area and outcomes included 1) a requirement that all focus areas include a primary care rotation; 2) the establishment of a general practice within the HWC to address training in small and exotic animal medicine primary care; and 3) the establishment of a primary care rotation in equine medicine that combines CVM and private practice experiences. Employment and alumni survey data showed that students need more help in preparing for employment and job placement. Staff in Student Services increased efforts in this area in 2012 and the College has allocated funds to hire an administrative assistant to enhance these services in. We will use employment and alumni survey data to track the outcome. In fall 2013, the College conducted a systematic review of senior year clinical rotations. An instructional technologist and educationalist evaluated learning outcomes and assessments for each of the major clinical rotations. The office of the ADAA is in the process of evaluating data and creating an implementation plan for more direct observation and assessment of the major competencies. The College is also modifying our student assessment system to provide faster feedback. Moving forward, the College has created an Outcomes Committee comprised of the ADAA, the Chairs of the Curriculum, Academic Performance and Admissions Committees, and selected faculty to review outcomes data on a biannual basis. The College is also in the process of hiring a doctoral level Director of Academic Outcomes and Assessment to provide guidance for conducting continuous programmatic review, and enhance computerized support to track outcomes and assessment. 44

5 Appendix 11.1.c Table A. Completion of Clinical Competencies Checklist. Year Requested/Required Average # unique skills with Comments Completion Rate % Completion Rate % 75% completion 2009 Do your best to test the system Not calculated 45/334 Beta test of the skill list and the book, Book organized by skill, skill list refined /264 Book organized by skill /266 Book organized by skill /306 First year for graduation, book organized by class /313 Book organized by class 90 TBD TBD Book organized by class, goal for future is to put process on line, searchable by skill or class Table B. Clinical Rotation Evaluation Criteria. Information Gathering Skills (AVMA 1, 8) Honors The information gathered by the student is always precise, logical, thorough, and reliable. The student establishes a broad base of information about the patient, including a complete and pertinent history. Excellent ability to elicit information from the client and avoid ambiguities. Good understanding not to focus the history prematurely. Overall the student's information gathering skills are excellent. Pass Generally provides reliable and complete information, with the subtle points usually elicited. The student has a good under-standing of how to elicit information. Overall summation of the history leads to an accurate description of the patient's problems and the client's concerns. Marginal General information gathering techniques are weak. The student appears to lack the focus and/or communication skills necessary to produce a coherent set of observations. History taking needs marked improvement. Fail Student does not understand the process of gathering information. Histories are most often superficial, often done more by rote, and are most often incomplete with major observations not elicited. Client interactions are awkward and ineffective. Conversations with clients fail to elucidate the essential information. Physical Examination - Necropsy Skills (AVMA 1, 4, 5, 6) Honors Examinations are undertaken in a logical and complete sequence with a very high level of skill and a comprehensive evaluation of all organ systems. Significant and even subtle abnormalities almost always elucidated. Overall the student s skills are excellent and result in a comprehensive assessment of the patient. Pass The examination is generally performed with reasonable competence and normally elicits the major set of pertinent findings. Usually subtle findings are obtained. Overall summation normally leads to an accurate description and prioritization of abnormalities observed. Marginal Examinations are not undertaken systematically or in sufficient detail to result in a complete set of essential observations. Clinical understanding of the examination is limited. Subtle issues either almost never recognized or dominate the evaluation. The student does not have a good grasp of examination techniques. Examination skills need marked improvement. Fail Examinations are not undertaken systematically and are quite random in approach. Key organ systems often are not evaluated, key observations are often totally missed, evaluation skills are exceptionally poor. Examination generally fails to elucidate the essential information. Problem Solving Skills and Judgment (AVMA 1, 2, 3, 4, 5, 6, 7) Honors Clearly identifies all of the major and minor problems, puts them into proper perspective and then establishes sensible differential diagnoses. Superior reasoning ability to solve clinical problems, put information into the correct context, and integrate medical facts with clinical data. Weighs alternatives well and is excellent at correlating abnormal findings with altered physiology. Good thought process in ambiguous situations and spends time appropriate to the complexity of the problem. 70

6 Pass Reasonably skilled in accessing the available information and formulating well-constructed differential diagnoses with alternatives appropriately weighed. With prompting, can usually come to reasonable conclusions with ambiguous clinical cases. Able to apply expected knowledge of disease with no large gaps in knowledge. Identifies when it is necessary to search for information and usually does so. Marginal Often fails to discern relationship of medical facts and clinical data. Medical reasoning awkward. Differential diagnoses generally not prioritized as to probability and importance and often incomplete. Indecisive with more ambiguous cases and frequently unable to reach conclusions even with prompting. Tends to evaluate data without full scope of information. Clinical judgment often fragmented. Substantial improvement necessary. Fail Very significant deficiencies in correlating the primary history and examination findings to the key patient/population problems. Limited capacity to put information into the correct disease context and to integrate medical facts with clinical data. Differential diagnosis lists are often incomplete, usually poorly prioritized, and/or frequently miss major potential problems. Large gaps in knowledge of disease and understanding of diagnostic and therapeutic concepts. Deficiencies in clinical judgment of major concern. Case Presentation and Plans for Patient/Population Management (AVMA 1, 2, 3, 4, 5, 6, 7, 8) Honors Outstanding ability to present cases. Presentations are superior in organization, focus, and clarity. Student knows the significant features of the case. Excellent coordination of pertinent facts with knowledge of disease and the presentation of relevant details. Consistently can formulate a well-structured management plan with wise use of diagnostic and therapeutic options. Pass Usually able to present cases in a clear, organized, complete fashion. Student generally knows all the significant features of the case. Demonstrates understanding of the disease(s) identifies relevant details and puts most concepts into context. Most often is able to formulate a management plan with good use of therapeutic and diagnostic options. Marginal Presentations are often incomplete, rambling and poorly organized. Pertinent information and essential detail are not always presented. Student often unable to fully communicate a clinical understanding of the case and prompting to present pertinent information often needed. Management strategies often vague with poor consideration of therapeutic and diagnostic options. Fail Presentations are incomplete, rambling and poorly organized. Student requires constant prompting to present pertinent information and is often confused and unable to clearly communicate relevant clinical information. Management plans and therapeutic and diagnostic strategies are very frequently vague and/or exceptionally poorly conceived. Knowledge Base (AVMA 1, 2, 3, 4, 5, 6, 7, 8, 9) Honors Outstanding breadth of understanding of disease. Extensive, well applied, and extremely well integrated knowledge of pathophysiology and basic science. Exceptional ability to recall and apply this knowledge to the solution of clinical problems, development of a comprehensive diagnostic plan and choosing practical therapeutics. Pass Has a solid fundamental core of basic science and pathophysiological knowledge. In most circumstances can satisfactorily apply this knowledge to clinical problems, diagnostics, and therapies. Marginal General pathophysiological and basic science knowledge is weak. Understanding of disease processes is poor. Integration of knowledge is often lacking and the student has difficulty in the formulation of a diagnostic plan and using appropriate therapeutics. Fail Markedly incomplete, insufficient, fragmented, and poorly organized pathophysiology and basic science knowledge base. Understanding of disease is exceptionally poor and diagnostic planning inaccurate. Frequently does not understand the basis of appropriate therapy. Procedural Skills (AVMA 1, 2, 3, 4, 5, 6, 7) Honors Demonstrated excellent competence with all the procedural skills expected of students in this rotation. Proficiently and efficiently learns procedural skills with the appropriate balance of care, concern, and confidence. Reliable recognition of level of self-proficiency. Seeks help when needed. Minimizes risk and is very proficient in safely restraining animals. Pass By completion of the rotation demonstrated acceptable proficiency with all the procedural skills expected of students in this rotation. Uses reasonable care in doing procedures and is satisfactory in animal restraint. Current skill levels should improve with further experience. 71

7 Marginal Level of competence with clinical procedural skills falls below the expectation of students completing this rotation and is of concern. Awkward and sometimes reckless. Appears not to always recognize self-limitations. Needs to considerably improve level of skills. Animal handling needs improvement. Overall aptitude of some concern. Fail Level of competence with the clinical procedural skills falls markedly below the expectation of students in this rotation. Inept and sometimes quite careless. Appears not to recognize self-limitations and disregards risks. Needs to immensely improve level of skills to be a safe veterinarian. Concern with animal handling. Major concerns with overall aptitude. Medical Records (AVMA 1, 2, 3, 4, 5, 6, 7, 8) Honors Written medical record keeping outstanding. Excellent ability to create both a complete and succinct medical record. Uses accurate and precise medical terminology. Record keeping always timely. Pass Medical records are generally complete and provide details of most of the critical components. Most often uses correct terminology. Record keeping usually timely. Marginal Medical records are inconsistent, at times incomplete, generally not well constructed, and often lack a clear reporting of essential information. Terminology often used inaccurately. Definite improvement is needed. Fail Medical records are usually poor, most often lack essential information, and often are very poorly constructed. Medical terminology use is often extremely poor. Marked improvement is absolutely essential. Organization and Efficiency (AVMA 8) Honors Very dependable, reliable and industrious. Works exceptionally hard. Always exceptionally well prepared for clinical duties. Always willing to extend extra effort and to assist others with clinical responsibilities. Pass Reliable and can be depended on to complete assignments. Overall work l0evel good. Well prepared for clinical responsibilities. Good team member. Marginal Cannot always be relied upon. Follow through to completion with assignments sometimes problematic. Work level at times insufficient and motivation to work at times questionable. Fail Frequently unreliable. Prominent absences. Does not readily accept responsibilities. Work level often insufficient and motivation to work of considerable of concern. Motivation and Initiative (AVMA 8) Honors Character traits and attitudes that epitomize those fundamental for the humane and responsible care of animals. Highly responsive and energetic. Displays initiative and provides leadership. Exceptionally selfmotivated and eager to acquire knowledge and skill. Reads extensively. Always exceptionally well prepared. Pass Compassionate towards animals and good professional demeanor. Generally fulfills duties and shows a good general interest for learning. Self-motivated to acquire knowledge and skills. Usually is well prepared. Marginal On occasion appears irresponsible. Follow through to completion with assignments sometimes problematic. Generally insufficient outside reading. Appears insufficiently motivated to expand knowledge base. Character traits and attitude that raise possible concern with the motivation to be an effective veterinarian. Fail Frequently irresponsible, unprepared and uncommitted. Notable lack of a professional demeanor. Shows little to no interest for learning, responds poorly to constructive criticism. Limited to no outside reading. Character traits and attitudes raise concern with level of motivation essential for optimum care of animals and the acceptance of responsibility. Communication - Interpersonal Skills (AVMA 8) Honors Exceptional ability to establish rapport and communicate with clients, healthcare professional staff, and instructors. Clear respect for the roles of others and a mature approach that elicits trust. Excellent interpersonal demeanor. Communication with clients is clear, accurate, appropriate, and empathetic, with a good understanding of the client's perspectives and needs. Pass Generally good communication skills with clients, healthcare professional, staff, teachers and others. Good demeanor in interacting with others. Communication with clients appears clear and appropriate, with an understanding of their needs. Marginal Questionable communication skills. Not always able to either clearly elicit or provide concise information. Imperceptive of others at times. Can portray at times a lackluster professional demeanor. Clearly needs to work on interpersonal and communication skills. Fail Clumsy in communication with clients, staff, other healthcare professionals, and instructors. Does not listen well. Has difficulty establishing rapport. Demonstrates insensitivity and poor professional judgment. Interpersonal and communication skills of concern. 72

8 Moral Reasoning and Ethics (AVMA 3, 8) Honors Displays exceptional ethical standards and integrity. Demonstrates respect and compassion for animals, even when circumstances would encourage other behaviors. Intellectually honest, with recognition of self-limitations of veterinary knowledge and skills. Pass Ethical values and/or compassion for animals, recognition of own limitations of veterinary knowledge and skills, and/or integrity are acceptable. No major areas of concern observed. Marginal Ethical values and/or compassion for animals, recognition of own limitations of veterinary knowledge and skills, and/or integrity are borderline. (Please detail concern) Fail Moral reasoning and ethical values are unacceptable. Demonstrates behaviors such as being rude or insensitive to client's needs, showing little compassion for an animal, physically abusing an animal, verbally or physically abusing a colleague, or lying. (Please detail concern). Service Specific Skills and Knowledge Honors fill in honors criteria Pass fill in passing criteria Marginal fill in marginal criteria Fail fill in failing criteria 73

9 Table C. Grade Data for Clinical Rotation Evaluation Criteria (see Rubric Table B, Appendix 11.1.c.). Question/Skill Year Honors Pass Marginal Fail Total Honors Pass % Marginal Fail% Counts Counts Counts Counts Counts % % Case Presentation and Plans for % 76% 2.5% Patient/Population Management % 80% 1.1% % 74% 1.5% 0.2% % 67% 1.8% % 73% 0.9% % 73% 2.8% Total/Av. % % 74% 1.7% 0.0% Communication/ Interpersonal % 67% 1.2% 0.2% Skills % 63% 0.9% 0.1% % 59% 1.2% % 57% 0.7% 0.4% % 58% 1.0% 0.1% % 57% 1.1% Total/Av. % % 60% 1.0% 0.1% Information Gathering Skills % 78% 2.2% 0.1% % 81% 0.7% % 79% 0.7% 0.1% % 75% 0.9% % 76% 0.8% % 78% 1.2% Total/Av. % % 78% 1.1% 0.0% Knowledge Base % 85% 2.8% % 85% 1.6% % 84% 1.2% 0.3% % 79% 2.3% % 83% 2.2% % 85% 2.8% 0.1% Total/Av. % % 83% 2.1% 0.1% Medical Records % 76% 1.2% 0.1% % 75% 1.4% % 73% 0.8% 0.2% % 65% 1.3% % 67% 1.2% % 68% 2.2% 0.2% Total/Av. % % 71% 1.3% 0.1% Page 79 74

10 Moral Reasoning and Ethics % 63% 0.3% 0.1% % 66% 0.5% 0.3% % 63% 0.6% % 65% 0.4% % 66% 0.1% % 72% 0.2% Total/Av. % % 66% 0.4% 0.1% Motivation and Initiative % 61% 1.3% 0.1% % 57% 1.7% % 49% 1.7% 0.1% % 47% 1.3% 0.1% % 47% 1.3% 0.1% % 48% 2.2% Total/Av. % % 52% 1.5% 0.1% Organization and Efficiency % 69% 1.7% 0.2% % 67% 0.9% 0.1% % 62% 1.4% 0.1% % 58% 1.1% % 59% 1.1% 0.1% % 62% 1.8% Total/Av. % % 63% 1.3% 0.1% Physical Examination/ Necropsy Skills Problem Solving Skills and Judgment % 87% 0.9% % 85% 0.5% % 83% 0.8% 0.1% % 82% 0.7% % 85% 0.3% % 86% 1.1% Total/Av. % % 85% 0.7% 0.0% % 76% 3.1% 0.4% % 79% 1.5% 0.1% % 77% 2.0% 0.3% % 74% 2.8% 0.1% % 79% 1.5% % 79% 2.7% Total/Av. % % 77% 2.2% 0.1% Page 80 75

11 Procedural Skills % 79% 0.9% % 78% 0.4% 0.2% % 77% 0.7% 0.1% % 76% 0.6% % 79% 0.5% % 79% 0.4% Total/Av. % % 78% 0.6% 0.0% Overall Rotation Evaluation % 77% 0.8% 0.3% % 75% 0.5% 0.2% % 71% 0.7% 0.3% % 69% 0.6% 0.1% % 71% 0.5% % 76% 0.4% Total/Av. % % 73% 0.6% 0.1% 76

12 Table A. Employment rates of graduates at time of senior exit survey. Employment at Time of Senior Exit survey (late Spring Semester 4 th year) # job offers % 24% 45% 32% 44% 43% 2 15% 8% 14% 16% 19% 20% 3 10% 6% 9% 5% 5% 4% 4 4% 8% 1% 1% 0% 1% 5 4% 9% 0% 0% 0% 0% >5 5% 2% 0% 0% 0% 0% Not applicable (House Officer) 34% 45% 31% 34% 32% 32% Accepted employment 82% 70% 61% 72% 69% 79% Have not accepted employment 18% 30% 38% 28% 31% 20% Table B. Employment data from AVMA senior student surveys. Comparison of NC State mean data with National (Nat l) means. Survey Question NC State Nat l NC State Nat l NC State Nat l NC State Nat l NC State Nat l NC State Nat l Sought/seeking employment in veterinary medicine or advanced education. 93.5% 96.5% 92.0% 95.8% 97.4% 96.7% 100.0% 97.5% 100.0% 96.3% 95.8% 96.1% Accepted a job that matched their preference for employment 83.7% 81.2% 79.1% 85.2% 82.7% 84.2% 88.1% 84.2% 84.3% 81.5% Received any employment or advanced education offers 93.1% 89.8% 79.7% 79.5% 66.7% 78.9% 72.7% 74.3% 61.6% 61.5% 84.1% 80.2% Among those who received an offer Number of offers Accepted a position 79.2% 76.5% 89.1% 83.9% 86.0% 84.6% 92.9% 85.5% 93.3% 88.9% 87.9% 90.3% Accepted advanced education 42.1% 39.6% 44.9% 43.5% 51.2% 49.2% 57.7% 52.0% 62.2% 55.2% 48.3% 39.5% Page 82 77

13 Table C. Per cent of graduates employed at 3-months after graduation. Data derived from responses replying to a survey at 1 year after graduation. Class Year of Survey % employment Employment Question Response Rate Overall Survey response rate /75 (60.0%) 56/75 (74.7%) /76 (30.0%) 28/76 (36.8%) /76 (20.3%) 29/76 (38.2%) /75 (42.7%) 36/75 (48.0%) /77 (40.3%) 37/77 (48.1%) /77 (45.5%) 40/77 (52.0%) Table D. Educational debt data from AVMA senior student surveys. Comparison of NC State mean data with National means.* Values are US dollars. Survey Question NC Nat l NC Nat l NC Nat l NC Nat l NC Nat l NC Nat l State State State State State State What amount of educational debt did you 10,631 10,245 10,406 11,140 13,471 10,454 9,985 11,717 5,604 10,980 5,799 11,017 have when you entered veterinary medical college What will be your total educational debt 91, ,312 93, ,145 95, ,329 95, ,713 75, ,359 90, ,221 upon graduation? DVM Educational Debt 80,702 98,657 83, ,107 82, ,976 85, ,506 69, ,567 84, ,335 % of National Mean 81.8% 79.8% 75.4% 73.0% 55.7% 62.4% Appendix 11.1.e Table A. Overall evaluation of educational experience and satisfaction with their experience at the CVM; NC State survey of graduating seniors. Overall Evaluation of Educational Experience Student Evaluation at Time of Graduation Excellent NA NA 35% 38% 56% 41% Very Good NA NA 41% 43% 34% 36% Good NA NA 23% 15% 9% 17% Fair NA NA 1% 3% 1% 4% Poor NA NA 0% 1% 0% 3% Page 83 78

14 Appendix 11.1.f Assessments of employers of graduates to determine satisfaction with the graduates Table A. Mean Rating of Employee s Performance Compared to Graduates Respondent Has Employed from other Universities. (1 to 5 scale with 1 = poor, 2 = fair, 3 =average, 4 = good, 5 = excellent). Year Reported Rating Mean N Mean N Mea n N Mean N Mean N Mean N Overall Rating Employee's Performance Compared to Graduates of other Veterinary Colleges Respondent Has Employed Component Skills Diagnostic Capabilities Rating Medical Knowledge Rating Surgical Skills Rating Leadership Skills Rating Professional Ethics Rating Work Ethic Rating Confidence Rating Ability to Work Independently Rating Ability to Work As Part of a Team Rating Written Communication Skills Rating Oral Communication Skills Rating Empathy with Clients Rating Business Knowledge Rating Ability to Work with Employees from All Levels of Business Rating Ability to Accurately Assess When to Consult or Refer a Case Rating Ability to Justify Costs of Procedures If Challenged by a Client--Rating Page 88

15 TAU Excellent 43 56% 47 64% 54 63% 51 54% 49 60% 36 46% Very Good 19 25% 19 26% 20 23% 25 26% 21 26% 23 29% Good 13 17% 6 8% 10 12% 14 15% 11 13% 15 19% Fair 2 3% 1 1% 2 2% 3 3% 1 1% 5 6% Poor 0 0% 0 0% 0 0% 2 2% 0 0% 0 0% Physical facilities Excellent 10 13% 9 12% 15 17% 20 21% 30 37% 25 32% Very Good 34 44% 34 47% 39 45% 30 32% 35 43% 29 37% Good 26 34% 28 38% 29 34% 36 38% 16 20% 21 27% Fair 7 9% 2 3% 3 3% 8 9% 1 1% 2 3% Poor 0 0% 0 0% 0 0% 0 0% 0 0% 2 3% Electronic Medical Record Excellent 7 9% 6 8% 9 10% 6 6% 8 10% 10 13% Very Good 14 18% 19 26% 17 20% 17 18% 20 25% 18 23% Good 33 43% 22 30% 36 42% 44 47% 36 44% 28 36% Fair 17 22% 17 23% 16 19% 20 21% 15 19% 16 21% Poor 6 8% 9 12% 8 9% 7 7% 2 2% 6 8% Interactions with advisor Excellent 23 30% 23 32% 33 39% 34 36% 29 36% 29 37% Very Good 19 25% 19 26% 18 21% 30 32% 26 33% 26 33% Good 19 25% 21 29% 22 26% 16 17% 20 25% 19 24% Fair 13 17% 9 12% 9 11% 9 10% 5 6% 4 5% Poor 3 4% 1 1% 3 4% 5 5% 0 0% 0 0% Interactions with faculty Excellent 32 42% 28 38% 26 30% 33 35% 35 43% 29 37% Very Good 33 43% 35 48% 44 51% 44 46% 35 43% 27 34% Good 8 10% 8 11% 16 19% 16 17% 10 12% 17 22% Fair 4 5% 2 3% 0 0% 2 2% 2 2% 6 8% Poor 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% Interactions with teaching staff Excellent 34 44% 26 36% 25 29% 30 32% 37 45% 29 37% Very Good 32 42% 38 52% 41 48% 48 51% 35 43% 25 32% Good 6 8% 8 11% 20 23% 15 16% 9 11% 20 25% Fair 5 6% 1 1% 0 0% 1 1% 1 1% 5 6% Poor 0 0% 0 0% 0 0% 1 1% 0 0% 0 0% Interactions with hospital staff Excellent 23 30% 19 26% 17 20% 15 16% 21 26% 15 19% Very Good 29 38% 28 38% 27 31% 27 28% 27 33% 29 37% Good 21 27% 20 27% 36 42% 38 40% 26 32% 17 22% Fair 3 4% 5 7% 6 7% 13 14% 7 9% 13 16% Poor 1 1% 1 1% 0 0% 2 2% 1 1% 5 6% 80

16 Interactions with residents Excellent 32 42% 33 45% 23 27% 34 36% 27 33% 19 24% Very Good 26 34% 27 37% 39 45% 37 39% 32 39% 31 39% Good 14 18% 10 14% 22 26% 20 21% 20 24% 17 22% Fair 3 4% 3 4% 2 2% 3 3% 3 4% 10 13% Poor 2 3% 0 0% 0 0% 1 1% 0 0% 2 3% Interactions with interns Excellent 28 36% 30 41% 23 27% 26 27% 34 41% 19 24% Very Good 27 35% 31 42% 35 41% 31 33% 29 35% 19 24% Good 15 19% 10 14% 23 27% 24 25% 15 18% 19 24% Fair 7 9% 1 1% 5 6% 10 11% 3 4% 15 19% Poor 0 0% 1 1% 0 0% 4 4% 1 1% 6 8% Interactions with classmates Excellent 23 30% 42 58% 39 45% 35 37% 29 35% 31 39% Very Good 32 42% 23 32% 30 35% 41 44% 37 45% 32 41% Good 17 22% 7 10% 16 19% 14 15% 16 20% 13 16% Fair 3 4% 1 1% 1 1% 4 4% 0 0% 3 4% Poor 2 3% 0 0% 0 0% 0 0% 0 0% 0 0% Interactions with students in other years Excellent 14 18% 20 27% 16 19% 19 20% 18 22% 26 33% Very Good 31 40% 25 34% 24 29% 33 35% 32 39% 24 31% Good 20 26% 19 26% 31 37% 31 33% 20 24% 18 23% Fair 7 9% 7 10% 11 13% 10 11% 11 13% 9 12% Poor 5 6% 2 3% 1 1% 1 1% 1 1% 1 1% * The information from 2010 through 2013 may not be directly comparable to data before 2010 due to differences in data collections method. Table C. Senior year course evaluation. Senior Year Course Evaluation Questions: % of total responses Cress Survey Question # Responses Strongly Agree Agree Disagree Strongly Disagree The course learning outcomes (as stated in the syllabus) were clearly stated and valuable for my education. I was able to accomplish the course learning outcomes (as stated in the syllabus) during this rotation. 1, % 27.0% 1.3% 0.2% 1, % 30.6% 2.2% 0.3% Page 86 81

17 Table D. Alumni survey data collected at 1 and 5 years after graduation. Overall Satisfaction with CVM (expressed as a %) Survey Year Years after Graduation CLASS YEAR Number of Responses Very satisfied Satisfied Neither satisfied or dissatisfied Dissatisfied Very dissatisfied Satisfied +Very satisfied Mean Quality Rating of Selected Aspects of CVM (1 to 5 scale with 1 = poor, 2 = fair, 3 = average, 4 = good, 5 = excellent ) Survey Year Years after Graduation Class Year Number of Responses Course Content Quality of Instructors Instructors Attitudes Classroom Facilities Laboratory Facilities Library Teaching Animal Unit Hospital Facilities Computer Facilities Career Planning Page 87 82

18 Appendix 11.1.f Assessments of employers of graduates to determine satisfaction with the graduates Table A. Mean Rating of Employee s Performance Compared to Graduates Respondent Has Employed from other Universities. (1 to 5 scale with 1 = poor, 2 = fair, 3 =average, 4 = good, 5 = excellent). Year Reported Rating Mean N Mean N Mea n N Mean N Mean N Mean N Overall Rating Employee's Performance Compared to Graduates of other Veterinary Colleges Respondent Has Employed Component Skills Diagnostic Capabilities Rating Medical Knowledge Rating Surgical Skills Rating Leadership Skills Rating Professional Ethics Rating Work Ethic Rating Confidence Rating Ability to Work Independently Rating Ability to Work As Part of a Team Rating Written Communication Skills Rating Oral Communication Skills Rating Empathy with Clients Rating Business Knowledge Rating Ability to Work with Employees from All Levels of Business Rating Ability to Accurately Assess When to Consult or Refer a Case Rating Ability to Justify Costs of Procedures If Challenged by a Client--Rating Page 88

19 APPENDIX 11: OUTCOMES Appendix 11.1g. Table A. Faculty Survey (2013) assessment of the overall preparedness of the students entering their class. Year taught Overall preparedness Not adequate Adequate Very Good Excellent Areas of concern cited 1 10% (3) 34% (10) 17% (5) 38% (11) Little animal experience, weak background in molecular biology and biochemistry. 2 6% (2) 42% (14) 12% (4) 39% (13) Basic knowledge in physiology, immunology and microbiology. 3 6% (2) 31% (11) 20% (7) 43% (15) Basic knowledge of material in first two years of the curriculum, poor math skills. 4 7% (3) 43% (20) 11% (5) 39% (18) Off shore student knowledge base concerns, inadequate number of lectures in cardiology Table B. Faculty Survey (2013) assessment of the quality of facility and equipment. Teaching space/equipment Not adequate Adequate Very Good Excellent Lecture theaters (North and South Theaters and D239) 11% (6) 22% (12) 44% (24) 24% (13) Mid-size classroom (A101, D236, R101) 6% (3) 35% (18) 42% (22) 17% (9) Small lecture spaces and group classrooms 9% (5) 32% (18) 40% (23) 7% (4) Multipurpose teaching laboratory (Anatomy, microscope B104 lab, Clinical Skill/Jr. Surgery) 17% (4) 54% (13) 0% (0) 29% (7) Clinical rounds rooms 10% (4) 22% (9) 29% (12) 39% (16) Equipment available for teaching 4% (2) 30% (16) 35% (19) 31% (17) Best aspect of the facilities and teaching equipment: Terry Center facilities overall (especially the rounds rooms and services spaces), IT support and recent upgrades in technology infrastructure in all classrooms, space gained in the HWC for teaching. Areas of concern cited: Two main lecture theaters are outdated, the number and availability of small group and laboratory teaching spaces, A/V equipment management issues, equipment availability for the teaching laboratories. Improvement priorities: TAU remodel and upgrades, anatomy laboratory expansion and upgrade, Equine and Farm Animal Veterinary Center (EFAVC; large animal hospital) remodel and upgrades, expansion of the HWC to provide more basic skills training, limited seating in the major lecture theaters with poor temperature control, increased multipurpose teaching space, and continued improvement of the AV/IT infrastructure and support. 84