MD Program Evaluation Plan (August, 2009) The Commonwealth Medical College (TCMC) requires its medical students to develop competency in the areas of patient care, medical knowledge, practice based learning and improvement, interpersonal and communication skills, professionalism, and systems based practice. The ACGME s Core Competencies were used when the curriculum was originally planned in 2007 and have remained in the forefront as we revisit the initial curricular vision to make it a reality for our first cohort of students who arrive in August, 2009. The expected level of competency attained must be sufficient to allow these new physicians to be successful in graduate medical education programs as well as to reflect the vision and mission of The Commonwealth Medical College. The defines the milestones of competency development. Components of self reflection and assessment are important for both faculty and students. Within each program course, assessment and feedback becomes part of the culture of professional behavior and develops proclivity toward lifelong learning. As a new institution, evaluation and assessment are integral parts of curricular development, linking learning outcomes to authentic forms of assessment that will provide indicators of a student s path of accomplishment as they matriculate through their program of study. The course directors and faculty have committed to designing a program of instruction that incorporates active learning strategies along with formative and summative assessment opportunities. TCMC has obtained PDE approval and preliminary accreditation from the LCME to begin offering the medical curriculum to students beginning in August 2009. Key components of the MD curriculum include: The medical school will function with a distributive model of medical education developing three Regional Campuses (Scranton, Wilkes Barre and Williamsport) to provide students experiences in diverse clinical settings. Clinical education starts on day 1 with students assigned to a Learning Community that corresponds to their Regional Campus, a family from that Regional Campus to follow for a continuity experience and a clinical mentor. Additional clinical experiences will provide students with training in an interdisciplinary setting by working with health professional training programs at regional colleges and universities. Particular emphasis has been placed on admitting students with a propensity for community
service. Students will be exposed to community health improvement projects throughout their medical school experience. The curriculum is an integrated competency based system. Competency based implies a developmental curriculum with multiple measures to document the path to mastery. It is not a one sized fits all course model. Each course has identified key measures that will document the competencies contained therein. Narrative evaluations by course directors and clinical faculty are imperative for many of the competencies not measured by traditional examinations. Course based assessment provides for both formative and summative measures. Students begin year one with basic science, move through an organ based system in year two, an integrated clerkship in year three, and subinternships in year four. Years one and two will occur on the Scranton campus with three one week experiences at the student s regional campus (assigned upon entry). Years three and four will occur at the student s regional campus. With the distribution of students across three regional campuses, consistency is provided through the use of the TCMC portal where technology provides a common opportunity for academic activities and evaluation procedures. This is imperative in years three and four when students are at their regional campus. The LCME Database and PDE Application provide roadmaps for curricular development and assessment of competencies. Overall Review Process The MD Program Evaluation Plan covers a four year academic year program delivery cycle with data collection points throughout the four academic years. Programmatic review occurs in May through July of each year with an extensive, systemic review occurring the first four year cycle of the program. This allows for continued development as the first cohort of students matriculates through the program. After the first cohort completes the four year program cycle, the normal cycle of periodic review of the four year cycle begins. In addition, student performance is reviewed on an annual basis (at minimum) or as necessary if the Associate Dean for Academic or the Associate Dean for Student requires review of a student s record. All students are reviewed annually by the on Academic and professional Standards (CAPS). The first four year cycle of the MD program will have full review of each year of the curriculum each year. The first class arrives in August, 2009 resulting in a complete review of year one in May thru July, 2010. In August, 2010 the second cohort arrives and the first cohort moves into the second year of the curriculum. In May, 2011 a full review of the curriculum will be undertaken for both years one and two. The same process holds for the 2011/2012 academic year when the first three years will be reviewed. The final full review occurs in 2012/2013 with the graduation of cohort one and the full review of years one through four. The AAMC Graduation Survey, which is the gold standard for medical school overall program evaluation, is administered to students upon graduation from the college, will give us an overall view of the four year program in the summer of 2013. Each subsequent graduating class will have AAMC Graduation Survey data that provides for overall program review from a four year
perspective in addition to the annual data collected internally. Longitudinal data analysis begins in year five. Five and Ten year TCMC Post Graduation Surveys will be sent to graduates beginning in 2018(Fiveyear) and 2023(Ten year). The annual program review process enables the faculty and administration of TCMC to make corrections to the curriculum long before the first cohort completes the program. The Curriculum Action (CAC) has overall institutional oversight for the curricula of the college. There is an MD of the CAC that oversees the for compliance with LCME and ACGME competencies. There are four subcommittees of the : Year One, Year Two, Clinical Years (3 & 4), and. Members of the Subcommittee of the MD Curriculum are charged with conducting the program review on an annual basis. This is a full curricular review for the first four years of the program (2009 2013). After that a periodic review schedule will be maintained with 25% of the courses in the program undergoing full review each year, the remaining 75% partial review. The entire program will cycle through complete review every four years. Full review requires each course director to do a self study of their course reviewing learning outcomes and competency assessment through the review of student data, evaluation data and course fit within the overall curriculum map. Curriculum maps are reviewed and updated as necessary. Partial review uses student data and evaluation data such as course evaluations to ensure curriculum compatibility and student satisfaction. The on Academic and Professional Standards (CAPS) of TCMC is responsible for the annual review of all student records. The Associate Dean for Academic handles the initial review of student records and preparation of reports for the on Academic and Professional Standards. At any time, the Associate Dean for Academic or the Associate Dean for Student may request that the group convene if either officer determines that a student is in significant academic or behavioral difficulty. The CAPS convenes to review student records and suggest a course of action for remediation or dismissal. MD Annual Program Evaluation Plan The Curriculum Action provides oversight for all curricula at TCMC. It currently has two major curricular committees: the MD and the MBS. (See Figure Two: Curriculum Action Structure at the end of this document).the Sub will meet annually to review the for the. A report with recommendations will be sent to the. That group will review the report and suggest program revisions as needed. The MD Subcommittee is comprised of a rotating group of program course directors. Each year of the program has a sub committee within this structure. All course directors are members of their respective year sub committees. The will send the report completed by the Sub and the subsequent actions taken to the TCMC Curriculum Action (CAC), who has curricular oversight for the institution in June of each academic year. The CAC will review the recommendations/actions of the MD
Curriculum in early July to ensure overall impact with all TCMC academic programs (for the present MD and MBS). The CAC sends their findings to the Associate Dean for Academic. In July, the MD Course Directors and Faculty will make changes to the program recommended by the MD Curriculum and approved by the CAC. In August, the MD Program Curriculum and Progress Reports are reviewed by the Dean s Cabinet (presented by the Associate Dean for Academic ). A report to the TCMC Board of Trustees follows in September. This process occurs annually in order to provide students with the most relevant and up to date program. Figure One: MD/MBS Program Review Cycle provides an overview of the program evaluation process. Later tables detail the program and student evaluation process components on an annual basis. Figure One: MD Program Review Cycle MD Program Review (April/May) by Course Directors and MD Sub committee of Data Collection throughout academic year Curriculum Advisory Review of MD Program Evaluations and Recommendations (July) Evaluation and Reports to TCMC Board of Trustees (September) Leadership Review and Implementation of Curricular Recommendations (July/August) Evaluation and Cycle Begins Again (August) The MD Annual Program Evaluation Plan allows for review of program curriculum and student data. In addition, budget analysis and program unit analysis is completed for all program units who serve the such as Student, Center for Learning Excellence and the Library among others. This is accomplished as part of the TCMC Institutional Evaluation and Plan.
Table One: MD Annual Program Evaluation Plan Activity Dates Collection of Data Product Due Responsible Activity August thru June Admissions Data On going Admissions Projection December Final Admissions Report June Director of Admissions and MD Admissions December and May Course Grades Registrar August thru May Review of narrative course data for each student Formative (December) and Final Grades (May) Student Narrative Reports for each course in MD Program Grade Analysis Report December and May Review of narrative course data by Course Directors at the conclusion of each course using Competency Based Rubrics. Students experiencing difficulty will have their narrative reports reviewed by additional course faculty, the Associate Dean for Student or the Associate Dean for Academic as needed. A final report on each student will be submitted to the Associate Dean for Academic in May by all course directors. Course Directors
Activity Dates December, May and June March/April Course Conclusion Course Conclusion Activity Review of Student Records Request for Program Review Course Evaluations by Students Course Evaluations by Faculty Collection of Data Product Due Responsible Review of grades and student narrative reports by the on Professional and Academic Standards Sub requests a full or partial review of Year Based Programs. First four years of cycle all will be full review, thereafter 25% of all courses in each year will be full review, 75% will be partial review Student Course Evaluations Faculty Course Evaluations Students experiencing difficulty will meet with the Associate Dean for Academic and a remediation plan will be put into place MD Program Report Statistical Analysis and Qualitative Comments Reported to Course Faculty, Course Directors, and Associate Dean of Academic Statistical Analysis and Qualitative comments reported to Program Director and Associate Dean of Academic Associate Dean for Academic, Director, Center for Learning Excellence, and Associate Dean for Student MD Course Directors and Sub Office of Evaluation and Office of Evaluation and
Activity Dates April Activity Advising Evaluations by Students Collection of Data Product Due Responsible Student Advising Evaluations April Advisor Survey Faculty Analysis of Advisees May (prior to May 15) May (after May 15) Program Retreat Curriculum and Progress Review Associate Dean for Academic, MD Course Directors and Course Faculty, others as invited Sub review of all related data and reports Statistical Analysis and Qualitative comments reported Center for Learning Excellence, Advisors, Program Director and Associate Dean of Academic and Associate Dean of Student Statistical Analysis and Qualitative comments reported Center for Learning Excellence, Advisors, Program Director and Associate Dean of Academic MD Program Review Report and Recommendations Compiled and Reported to MD Curriculum Sub Report Office of Evaluation and Office of Evaluation and Associate Dean for Academic Sub
Activity Dates June July July/ August August September Activity of the Curriculum Action (CAC) Curriculum Action (CAC) MD Program Report Presented to Cabinet MD Program Report presented to TCMC Board of Trustees Collection of Data Product Due Responsible Review of MD Program Report from MD Sub and others as needed Review of MD Program Report from the MD Curriculum of the CAC Review of MD Program Report Review of MD Program Report Program is reviewed at the Level and recommendations forwarded to the CAC for institutional approval Changes made to Program as needed after Curriculum Action (CAC) review of the CAC Associate Dean for Academic and MD Curriculum of the CAC Course Directors and Faculty Associate Dean of Academic Associate Dean of Academic Table Two: First Five Year Evaluation Process details the evaluation process over the next 6 years providing details for the first four year curriculum cycle and subsequent annual reviews.
Table Two: First Five Year Evaluation Process August, 2009 First Cohort (Class of 2013) arrives MD Program and Student Evaluation Process Begins July, 2010 TCMC Curriculum Action Review Program Changes made as needed June, 2011 Review May, 2012 Full review of Year 1, 2, and 3 Courses by MD Course Directors and MD Sub September, 2012 April 2013 Data collected September MD Curriculum Report and Student Data Analysis to TCMC Board of Trustees September, 2009 April 2010 Data collected August, 2010 Cohort 2 arrives Cohort 1 enters Year 2 Report and Student Data Analysis to TCMC Leadership July, 2011 TCMC Curriculum Action Review Program Changes made as needed June, 2012 Review May, 2013 Cohort 1 Graduation AAMC Graduation Survey Full review of Year 1, 2, 3 & 4 Courses by MD Course Directors and May, 2010 Full review of Year 1 Courses by MD Course Directors and MD Sub September, 2010 April 2011 Data collected September MD Curriculum Report and Student Data Analysis to TCMC Board of Trustees August, 2011 Cohort 3 arrives Cohort 2 enters Year 2 Cohort 1 enters Year 3 Report and Student Data Analysis to TCMC Leadership July, 2012 TCMC Curriculum Action Review Program Changes made as needed June, 2013 Review June, 2010 Review May, 2011 Full review of Year 1 and 2 Courses by MD Course Directors and MD Sub September, 2011 April 2012 Data collected September MD Curriculum Report and Student Data Analysis to TCMC Board of Trustees August, 2012 Cohort 4 Arrives Cohort 3 enters Year 2 Cohort 2 enters Year 3 Cohort 1 enters Year 4 Report and Student Data Analysis to TCMC Leadership July, 2013 TCMC Curriculum Action Review Program Changes made as needed
MD Sub August, 2013 Cohort 5 Arrives Cohort 4 enters Year 2 Cohort 3 enters Year 3 Cohort 2 enters Year 4 Report and Student Data Analysis to TCMC Leadership July, 2014 TCMC Curriculum Action Review September, 2013 April 2014 Data collected September MD Curriculum Report and Student Data Analysis to TCMC Board of Trustees August, 2015 Cohort 6 arrives Cycle repeats as in 2013/2014 May, 2014 Cohort 2 Graduation AAMC Graduation Survey Regular Annual Review Cycle Begins by MD of the. 25% of courses in the 4 years of the program are given full review, remaining 75% receive partial review. Full review involves all facets of curriculum, compliance with LCME and ACGME competencies, student progress, course evaluations, sample student work review, narrative review, etc. Partial review is course evaluations and student data review. June, 2014 Review Program Changes made as needed MD Student Data Collection The MD Program will collect data for each MD student during the four years they are enrolled in the program. This data is compiled from a variety of sources: 360 Student Evaluation, examination grades, USMLE Scores (after years 2 and 4), exams with USMLE type questions, narrative review by
course directors and faculty, checklist evaluations by clinical faculty, standardized patient and clinical simulations, objective structured clinical examinations (OSCE), case analysis, oral examinations, laboratory practical examinations, presentations, preceptor ratings, NBME Subject Exams, essays, papers, and community research project. 360 Student Evaluation includes advisor and patient surveys, self and peer assessment along with grades and narrative information from courses. Data collection varies by course and year in program. All are detailed in the LCME Database. Data collection begins with pre matriculation surveys completed in the summer before the first year classes and concludes with the AAMC Graduation Survey and 5 Year and 10 Year Post Graduation Surveys. (See LCME Database for details of competencies assessed: medical knowledge, practice based learning and improvement, interpersonal skills and communication, professionalism, patient care, and systems based practice. Questions about the MD Student and Program Evaluation Plan should be directed to Dr. Barry Linger, Associate Dean for Academic or Dr. Susan Perlis, Assistant Dean for Evaluation and. In addition to the MD Program Evaluation Plan which addresses curricular improvement and student matriculation progress based upon student outcomes, components of the MD Program are evaluated through program unit offices that service the program. Student, Admissions, the Center for Learning Excellence, among others, all participate in the TCMC Institutional Evaluation and Plan at the unit level assuring that all components of the organization that touch the MD Program are carefully reviewed for mission integration and service to constituents. Update: August 6, 2009
Figure Two: Curriculum Action Structure Curriculum Action MBS Curriculum Technology Library Year 1 Year 2 Clinical Curriculum