Preparing medical students to enter surgery residencies

Transcription

1 The American Journal of Surgery (2010) 199, Association for Surgical Education Preparing medical students to enter surgery residencies Rebekah A. Naylor, M.D., F.A.C.S.*, Lisa A. Hollett, R.N., M.A., Antonio Castellvi, M.D., R. James Valentine, M.D., F.A.C.S., Daniel J. Scott, M.D., F.A.C.S. Department of Surgery, Southwestern Center for Minimally Invasive Surgery, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX , USA KEYWORDS: Skills; Curriculum; Medical students; Surgery internship; Proficiency-based Abstract BACKGROUND: This study was designed to develop and evaluate an integrated cognitive and proficiency-based skills curriculum based on American College of Surgeons Graduate Medical Education Committee (ACGME) competencies to prepare students for surgery internships. METHODS: Course content included cadaver dissections, didactic sessions, team training, and training in clinical and technical skills. Using previously validated skills curricula (12 open and 5 Fundamental Laparoscopic Surgery [FLS] tasks), trainees underwent pretest, self-practice to proficiency, and post-test. Surveys regarding confidence levels and skills were administered. RESULTS: Mean course evaluation score was on a 5-point Likert scale. Trainees (n 9) achieved proficiency on open tasks and FLS tasks 2 5. The mean confidence self-rating on 51 skills increased on a 5-point Likert scale from to (P.001). CONCLUSIONS: This integrated curriculum did improve confidence levels, and skills proficiency can be achieved in an abbreviated time. The anticipated result would be enhanced baseline abilities for internship Elsevier Inc. All rights reserved. The basic objective of the medical school curriculum is to equip students to enter the practice of medicine with a general knowledge of all medical disciplines. There has been no consistent effort to prepare students with the specific skills needed in each discipline. Interactions with residency program directors have suggested that medical schools should consider the directors expectations regarding entering residents in developing new curricula. 1,2 Interns themselves have identified areas of inexperience that could be addressed in the last year of medical school. 3 * Corresponding author. Tel.: ; fax: address: rebekah.naylor@utsouthwestern.edu Manuscript submitted June 29, 2009; revised manuscript August 24, 2009 At the same time, major changes in surgical education have occurred. Skills training has moved from the operating room and patient care arena to the simulation laboratory. This advance has been motivated by patient safety issues, cost containment in operating rooms, and work hour restrictions. Curricula are competency-based and learners are evaluated according to competency in prescribed areas, including technical skills. Demands of the surgical residency programs have made it essential that entering interns bring with them more skills so that they are able more quickly to progress in meeting the requirements of the residency. Much has been written about adult learning. Medical students and residents learn best experientially by hands-on activities. Their motivation to learn is related to the relevance to their daily work. Learning is further facilitated by the learners participation in setting the objectives for the learning activity /$ - see front matter 2010 Elsevier Inc. All rights reserved. doi: /j.amjsurg

2 106 The American Journal of Surgery, Vol 199, No 1, January 2010 In view of these circumstances, several medical schools have developed a fourth-year elective course to prepare students for surgery internships. 4 7 Despite the promising results reported by these institutions, there remains no national standard for the optimum curriculum content and design. The goal of this study was to develop and evaluate the effectiveness of a curriculum consistent with the American College of Surgeons Graduate Medical Education Committee (ACGME) core competencies and integrating cognitive content with proficiency-based skills training. Methods In 2008, at University of Texas Southwestern Medical Center, a new fourth-year medical student elective, Preparation for Surgical Internship, was initiated. An Institutional Review Board approved study was conducted to evaluate the effectiveness of the curriculum. It was anticipated that participants would have less anxiety and increased skills as they began their surgical training. Nine students (7 men and 2 women), all entering general surgery or a surgical subspecialty career, were enrolled for the 4-week course in February The overall objective was to prepare students to enter a surgical internship. The curricula in similar courses in other medical schools, 4 7 the expectations determined by the American College of Surgeons (ACS) for entering postgraduate year 1 (PGY-1) residents, 8 and needs expressed by our own faculty provided the basis for course planning. Objectives and sessions were designed according to the 6 ACGME competencies. The curriculum was comprised of didactic sessions and seminars, intensive technical skills training, experience with various clinical skills, simulation-based team training, cadaver dissections, and independent study of a case-based core curriculum in general surgery (Table 1). A multidisciplinary faculty included physicians from various specialties, nurse educators, and allied health professionals. There were 28 didactic sessions and seminars occupying 41 hours of scheduled class time. Clinical skills were taught and practiced both in simulation laboratories (airway management, ventilator management and vascular Doppler examination) and in real environment (intubation, insertion of intravenous cannulas, and catheterization). Each week a team of students (2 or 3 persons) dissected 1 portion of an embalmed cadaver and then had a 2-hour demonstration session for the entire class facilitated by surgical faculty. Team training required a team of 3 students to respond and manage a case scenario using the METIman, an emergency care simulator model (Medical Education Technologies, Inc., Sarasota, FL) and then to be debriefed by peers and faculty. Both open and laparoscopic technical skills were taught, each in 2-week blocks. For the open curriculum, a previously described proficiency-based basic knot-tying and suturing curriculum, which included 12 tasks, was used. 9,10 Table 1 Outline of curriculum content according to competencies Competency Patient care Medical knowledge Practice-based learning Communications Professionalism Systems-based practice Curriculum Technical skills open and laparoscopic Airway management Vascular laboratory Answer mock pages Simulation-based team training Pulmonary and ventilator management EKG s and arrhythmias Fluids and electrolytes Nutrition Wound management Orders, operative notes, progress notes, discharge summaries IV insertion, catheterization Anatomy dissections Basics of anesthesia Preoperative assessment of patients Medical comorbidities Drugs used by interns X-rays the intern should recognize Surgical complications Infections Case-based core curriculum review Evidence-based medicine Self-directed learning skills Communication styles Delivering bad news Residents as teachers Expectations of residents Balance of personal and professional life Patient safety Legal issues EKG electrocardiograph; IV intravenous. For the laparoscopic curriculum, the Fundamental Laparoscopic Surgery (FLS) proficiency-based skills curriculum was used. 11,12 For both curricula, after video instruction, all students were required to pretest (1 repetition for each task) with scoring based on previously validated metrics including time and errors. The learners then engaged in selfpractice, training until the previously reported proficiency levels were achieved for each task Ongoing feedback was provided as needed by proctors and faculty; however, most feedback was provided by additional viewings of the video tutorials. The number of repetitions and time to reach proficiency were recorded. All participants underwent posttests (1 repetition for each task, no warm-ups) at the end of the training period. Additionally, the FLS cognitive materials were used, with pre- and post-test quizzes (prepared at our institution) administered. Students completed evaluations of the course overall as well as each session with ratings assigned on a Likert scale of1to5(1 poor and 5 excellent). Upon completion of the skills training, a survey including self-rating of skills on a 5-point Likert scale (1 very poor and 5 excellent)

3 R.A. Naylor et al. Preparing medical students for surgery residencies 107 before and after training as well as confidence levels was given. Through a survey tool listing 51 tasks, confidence levels for each task were measured before and after the course using a Likert scale of 1 to 5 (1 low confidence and 5 high confidence). Statistical analysis was done using the paired t test. Values reported are mean SD. A P value.05 was considered significant. Results All students completed the curriculum except for incomplete training for 1 technical skills task by 1 trainee. Course attendance for all scheduled sessions was 96.7%. The mean overall course evaluation score was on a 5-point Likert scale. All participants rated the facilities, the open skills training, the simulation laboratory experience, the cadaver dissection and the mock pages sessions as 5. The mean rating for faculty and for laparoscopic skills training was 4.9 and for course organization 4.8. According to pretest data (Figures 1 and 2, Table 2), trainees showed wide variability at baseline in their levels of performance on the open and laparoscopic technical skills. At pretest, 1 student was proficient on 1 open task and none was proficient on the FLS tasks. All but 1 student completed the entire prescribed self-practice curricular components. One student failed to complete training on 1 of 12 open tasks. All students completed the entire FLS training protocol. No students reached the maximum allowable number of 80 repetitions for the open tasks or for FLS tasks 2 5; however, for FLS task 1 (peg transfer) all students reached 80 repetitions without achieving proficiency. At post-testing, the participants achieved high scores for both curricula, with significant improvement detected in scores following training (P.001 for open and for laparoscopic skills). Of Normalized Score for 12 tasks Open Basic Skills Curriculum Pre-test Post-test Figure 1 Pre- and post-test performance scores for each trainee (each line represents 1 trainee). Scores have been normalized to the proficiency level for each task. Normalized Score for 5 tasks Fundamentals of Laparoscopic Surgery (FLS)LS) Pre-test Passing score for cer fica on (270) Post-test Figure 2 Pre- and post-test performance scores for each trainee (each line represents 1 trainee). Scores have been normalized according to standard FLS testing protocol. note, all participants exceeded the FLS certification criteria (270 passing score) by a wide margin (Figure 2). 13 Mean score on the FLS cognitive pretest quiz was 49% 11%. After completion of the CD-Rom based training, the mean score on the post-test quiz was 81% 8% (7 of 9 above the 75% passing score). This difference is statistically significant (P.001). Two of the course participants remained at UT Southwestern for surgical internships. In July of the intern year, 5 months after the course, these 2 trainees underwent the pretests for the same open and laparoscopic skills. Both were proficient at the pretest in 3 of 12 open tasks with scores of 77 and 87 compared with a class (39 interns) mean score of with none proficient at more than 1 task. Both preparatory class graduates were also proficient on FLS tasks during internship pretesting one on 3 of 5 tasks and the other on 2 of 5 tasks, with scores of 482 and 470, compared with the group mean score of Among the other 37 interns, 1 was proficient on 3 tasks and 12 on 1 task. Prior to the training, only 1 participant felt comfortable with open skills and none was comfortable with laparoscopic skills. Upon completion of training all 9 students stated that they were comfortable with both open and laparoscopic skills. According to Likert scale ratings, self-eval- Table 2 Training data and mean performance scores for open and laparoscopic skills Training repetitions Pretest score Posttest score P value Open Laparoscopic (FLS)

4 108 The American Journal of Surgery, Vol 199, No 1, January 2010 uation of open skills improved from to (P.001) and laparoscopic skills improved from to (P.001). All trainees reported that the models were helpful and that the proficiency levels were appropriate, motivating them to achieve performance goals and providing appropriate feedback. To better analyze the data regarding confidence levels related to surgical concepts, skills and procedures, the 51 survey items were sorted and grouped into 8 broad categories. The anatomy items concerned identification of structures in 4 regions of the body. Medical documentation tasks (orders, notes, summaries, consents, etc) included 9 items. One question asked for confidence level in knowing how to avoid malpractice lawsuits, 1 dealt with balance of personal and professional life, and 1 with confidence in assuming the role of teacher. Six questions related to technical and clinical skills such as airway management, intravenous line placement, insertion of Foley catheters, open and laparoscopic skills, and managing a code. Diagnostic test interpretation queries included interpretation of blood gas analysis, acid base disorders, ventilator modes and settings, electrocardiogram interpretation, and reading common x- rays. Finally there were several questions related to patient care skills including knowing common drugs, managing fluids and electrolytes, nutrition management, treating medical comorbidities, recognizing and treating surgical complications, management of shock, pain management, and wound care. The summary of the results of this survey is seen in Table 3. Confidence levels improved significantly in all areas except in the ability to balance personal and professional life. The overall confidence level improved from to (P.001.) All students stated that the course reduced anxiety and improved comfort level as they anticipated their internships. Comments The ACGME Committee surveyed surgeons to identify learning needs for graduating students and first-year residents. Along with this, graduating medical students matched Table 3 Mean self-rating scores on confidence levels Category Pre-course Post-course P value Anatomy Medical documents Avoiding malpractice Balancing lifestyle Assuming teacher role Technical and clinical skills Diagnostic test Interpretation Patient Care skills Totals to surgical residency programs were asked to assess their competency. Based on the data, the committee agreed on many areas of essential or desirable knowledge and skill that all surgical interns should have. 2 In 2005, the ACS published Essentials for Surgical Residents Entering PGY Other studies have also been done to determine the expectations that program directors have and how often these expectations are met by entering residents. The program directors note that students have such a widely variable set of necessary skills and competencies that intense supplemental training is required at the beginning of the internship. 1 Previously, students and residents learned mainly as apprentices, observing and participating directly in patient care activities. But the changes in the clinical environment as well as advances in educational theory have resulted in a new model of surgical education in which skills are learned on models and simulators. 14 It has been demonstrated that medical students are as able as residents to acquire basic technical skills. 15 This fact is important for curriculum design since skills curricula designed for residents can be instituted in the undergraduate program. Courses designed to prepare students for surgical internship, often termed boot camp, are being initiated in several institutions. 4 7 The content of the curriculum varies widely from place to place. In all, there is emphasis on the acquisition of technical and clinical skills in laboratory or simulated environment. Most incorporate a mock pages session directed toward answering patient-care emergencies in the hospital. Various means of evaluation of student performance have been used in these courses. In all settings, the participants have expressed increased confidence in their abilities to meet the demands made on a surgery intern. The purpose of the course at UT Southwestern was to teach needed skills and decrease anxiety for graduating medical students entering a surgical internship. The specific course objectives were outlined according to the ACGME s 6 competencies: patient care, medical knowledge, practicebased learning, communication, professionalism, and systems-based practice. This design ensured that competencies which are required in the internship would be addressed at student level. It was possible to fit the essentials defined by the ACS into this framework. Moreover, we used validated proficiency-based skills curricula and documented feasibility of this intensive training in a 4-week period without modification of expert-derived benchmarks designed for resident-level training. The positive response of the participants was evident both in their attendance (97%) and in their overall positive evaluation of the course (mean on a 5-point Likert scale). Each session and component of the course was evaluated by the students and this was in turn communicated to the faculty. This feedback allowed some adjustments and modifications of the curriculum, especially regarding time allotted to some topics and activities. Some faculty planned modification of content and more interaction in their ses-

5 R.A. Naylor et al. Preparing medical students for surgery residencies 109 sions. The participants also suggested a few additional topics for future years. Approximately 48 of 160 class hours were allocated for technical skills training. The open skills curriculum, including 12 knot-tying and suturing tasks, is the same curriculum we use for PGY-1 residents at UT Southwestern during the first 3 months of residency. 10 The 5 FLS tasks are those prescribed by Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and ACS to develop fundamental laparoscopic skills and the passing score for certification is well-defined. 13 PGY-1 residents at our institution must complete this same curriculum in the first 6 months of residency. The outstanding success of the participants in achieving proficiency in both open and laparoscopic skills in 2 weeks for each skill set is remarkable. Moreover, the retention of the skills 5 months later was excellent for the 2 participants who matriculated into our residency program, far surpassing their peers. The success of the course in decreasing anxiety levels as students anticipated the internship experience was verified by the increased mean confidence levels related to each of the 51 concepts, tasks, or skills. These increases were significant in all domains except lifestyle. Even in this area, mean confidence level increased from to 3.8.7, although this was not statistically significant. The main cost of the course was the investment of time required for planning and administration of the course, for teaching, and for assisting and proctoring in the skills laboratory. The use of faculty from many departments diffused the required teaching time for each person, but this remained a major commitment for many. Monetary costs were those of the cadaver and the use of the anatomy laboratory, with skills laboratory supplies donated. The main limitation of the study was the lack of more complete follow up into the internship year. This will be a part of a future study to get feedback from former course participants, their residency program directors, and perhaps peers. While it is not certain if the curriculum will be useful in all institutional settings due to variations in resources, this course is expected to be reproducible at most training locations. A further limitation is incomplete evaluation of trainees according to core competencies. The technical skills performance, the FLS cognitive quiz, and the self-assessment of confidence levels supported competence in patient care, medical knowledge, professionalism, and systems-based practice. Practice-based learning and communications were not addressed in evaluation of the trainees, although they were included in the curriculum. In the future, appropriate assessments will be developed to demonstrate improved competence in these areas. Conclusions In conclusion, this study demonstrates that the integrated cognitive and skills curriculum is effective in improving confidence levels before surgical internship. Proficiency in technical skills can be achieved by medical students in an abbreviated amount of time and is expected to significantly enhance baseline abilities during residency. References 1. Langdale LA, Schaad D, Wipf J, et al. Preparing graduates for the first year of residency: are medical schools meeting the need? Acad Med 2003;78: Graduate Medical Education Committee. Prerequisite objectives for graduate surgical education: a study of the graduate medical education committee American College of Surgeons. J Am Coll Surg 1998;1: Nakayama DK, Steiber A. Surgery interns experience with surgical procedures as medical students. Am J Surg 1990;159: Boehler ML, Rogers DA, Schwind CJ, et al. A senior elective designed to prepare medical students for surgical residency. Am J Surg 2004; 187: Esterl RM, Henzi DL, Cohn SM. Senior medical student boot camp : can result in increased self-confidence before starting surgery internships. Curr Surg 2006;63: Brunt LM, Halpin VJ, Klingensmith ME, et al. Accelerated skills preparation and assessment for senior medical students entering surgical internship. J Am Coll Surg 2008;206: Peyre SE, Peyre CG, Sullivan ME, et al. A surgical skills elective can improve student confidence prior to internship. J Surg Res 2006;133: Sachdeva A. Essentials for Surgical Residents Entering PGY-1. American College of Surgeons, Division of Education; facs.org/education/essentials.pdf. 9. Scott DJ, Goova MT, Tesfay ST. A cost-effective proficiency-based knot-tying and suturing curriculum for residency programs. J Surg Res 2007;141: Goova MT, Hollett LA, Tesfay ST, et al. Implementation, construct validity, and benefit of a proficiency-based knot-tying and suturing curriculum. J Surg Educ 2008;65: Ritter EM, Scott DJ. Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery. Surg Innov 2007;14: Scott DJ, Ritter EM, Tesfay ST, et al. Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 2008;22: Fraser SA, Klassen DR, Feldman LS, et al. Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system. Surg Endosc 2003;17: Reznick RK, MacRae H. Teaching surgical skills changes in the wind. N Engl J Med 2006;355: Hamford JM, Hall JC. Acquiring surgical skills. Br J Surg 2000;87: