Returns on the GME Investment: Perspectives on the Costs & Benefits of Resident Education Lisa Howley, MEd, PhD AVP of Medical Education & Physician Development, Associate DIO Associate Professor of Medical Education University of North Carolina at Chapel Hill Carolinas HealthCare System Mary N. Hall, MD Senior Vice President Chief Academic Officer & DIO Professor of Family Medicine University of North Carolina at Chapel Hill
Disclosures We have no financial or conflicting disclosures to report $
Acknowledgments Eric Anderson, MEd Director of Graduate Medical Education Renee Miller, MHA Director of Medical Education Operations Scott Furney, MD Senior Academic Chair, Acute Adult Division Chairman, Department of Internal Medicine Medical Director, Faculty Physician Finance Lynn Hawkins, MBA Assistant Vice President, CHS Finance Eric Runge, MBA Assistant Vice President of Medical Education Operations
Medical Education & Research: A Snapshot 434 faculty (physicians + PhD) 295 residents & fellows 13 Medical Residency Programs 20 Fellowship Programs UNC-Chapel Hill School of Medicine Charlotte Campus 75-3YR Medical Students 221-4YR Medical Students 2 Schools of Nursing & Allied Health Over $20 million in grants for research from external, federal & state organizations National Institutes of Health Department of Defense Department of Education Duke Endowment A-O Foundation NC DHHS, Division of Public Health
Carolinas HealthCare System AT- A - GLANCE 41 hospitals & 900+ care locations in North Carolina, South Carolina & Georgia More than 7,800 licensed beds 10.5 million patient encounters in 2012 3,000+ system-employed physicians, 400+ residents & fellows, 14,000+ nurses, & more than 60,000 employees More than 50 disease-specific certifications from The Joint Commission one of the highest totals in the country among comparable systems The region s only Level I trauma center One of five academic medical centers in North Carolina One of the largest HIT & EMR systems in the country
WHERE WE ARE
Agenda I. Welcome & Overview II. Perspectives on GME ROI III. Sample Holistic GME ROI Process IV. Next Steps & Closing Remarks Assessing the financial impact of GME programs is challenging and complex, as their full costs and benefits are rarely accounted for in one cost center or reported on a single financial statement. A. Matthiesen 2009
Objectives Consider changes in healthcare & impact on GME Discuss the challenges in assessing value for GME Critique the classic fourlevel model of evaluation & traditional metrics for GME Generate suggestions towards ensuring that GME & academic divisions reflect what executive leadership desires for holistic ROI Consider a sample holistic ROI process to demonstrate the value of a GME enterprise
Getting To Know Each Other What is your role in GME? A. Program Director B. Program Coordinator C. DIO / Associate DIO D. GME Administration E. C-Suite Executive F. Resident or Fellow G. Other Complete this sentence: GME at my Sponsoring Institution (SI) adds value in the following way:
Winds of Change are Blowing 11
East Carolina chancellor says medical school threatened The Associated Press February 5, 2015
GME Funding: Institutional Perspective Hospital (SI) GME Revenue-Expense Direct Costs: Direct Resident Stipends & Benefits GME Related Physician Compensation Program + Institutional Administration Average cost per resident = ~$143,000* Revenues: Medicaid Medicare Hospital Other State Revenues Average amount paid (federal) to hospital per resident within cap = ~$100,000 Is this a complete picture? *HCRS, September 30, 2012 & Release & Health Policy Brief: Graduate Medical Education," Health Affairs, August 16, 2012 13
Discussion Questions How are the costs & benefits of operating GME valued in the current healthcare landscape? What is the perception of executive leadership (CEO/Dean, CFO, Governing Board) on the value of GME sponsorship?
Twist on a Traditional Approach Application of an Expanded / Holistic Kirkpatrick Model to GME ROI System Outcomes (Holistic ROI) Impact 5 What are the impacts of GME on the System? What is the alignment between the System & its GME priorities? Patient Outcomes (Quality Indicators) Clinical Performance (Competencies, Milestones) Medical Knowledge (MCQs, Board Pass Rates) Results 4 Behavior 3 Learning 2 Are patients provided with high quality care? Are trainees performing competently? Is knowledge or comprehension gained from GME? Faculty, Resident Satisfaction (Faculty, Resident Surveys) Reaction 1 What are the reactions of key stakeholders to GME? Who are we recruiting into the System?
Communications With Governing Bodies & Senior Executive Leadership: Are We Delivering the Right Messages to Demonstrate Holistic Value? Current Requirements Include: The GMEC must demonstrate effective oversight of the Sponsoring Institution s accreditation through an Annual Institutional Review (AIR) GMEC must identify institutional performance indicators, which include: 1. ACGME Resident/Faculty Survey Results 2. Program s Accreditation Status& Self-Study Results 3. Annual Program Evaluations 4. The AIR must include monitoring procedures for action plans resulting from the review 5. The DIO must submit a written annual executive summary of the AIR to the governing body ACGME Institutional Requirements
Twist on a Traditional Approach Application of an Expanded Kirkpatrick Model to GME ROI System Outcomes Impact 5 What are the impacts of GME on the System? What is the alignment between the System & its GME priorities? Patient Outcomes Results 4 Are patients provided with high quality care? Key Questions: 1. Is patient care better at Systems/Institutions due to GME? 2. What is the value gained on the investment to the System?
What are (Holistic) Value Propositions for GME Investments? A complete analysis often demonstrates a very positive benefit of GME to the sponsor s environment, although the direct subsidy may initially appear substantial. P. Pugno 2010 Because learning is inextricably intertwined with the delivery of services through experience based learning, it is difficult to draw the line between the learning activities and the services provided. K. Smith 2010
Return to your triad Small Group Exercise Brainstorm a list of value propositions at levels 4-5 Don t limit yourselves to those that have been demonstrated at your institutions or those you consider tangible We will invite volunteers to share System Outcomes Patient Outcomes Impact 5 Results 4 What are the impacts of GME on the System? What is the alignment between the System & its GME priorities? Are patients provided with high quality care? Ianuzzi MC, Iannuzzi JC, Holtsbery A, Wright S, Knohl SJ. 2015. Comparing Hospitalist-Resident to Hospitalist-Midlevel Practitioner Team Performance on Length of Stay and Direct Patient Care Cost
Tips for Effective Measures: CCISS Credible: Provides information that is valid and credible in the eyes of management Simplicity: Simple to understand from each stakeholders perspective Important: Connects to strategically important business initiatives rather than what is easy to measure Collectible: Can be collected with no more effort than is proportional to the usefulness that results Specific: Is clearly defined so that people quickly understand and relate to the measure Source: Kerr S. On the folly of rewarding A while hoping for B; Academy of Management Journal; 18 (1995): 769-783.
Holistic Value Propositions for GME Investments Better Quality of Care Lower Rates of Adverse Events Increased Downstream Referral Rates Increased Patient Care & Coverage Increased Physician Satisfaction & Retention Opportunity to Care for Otherwise Underserved Populations Broad Education & Professional Development Beyond GME Recruitment Cost Reduction via Retention of Graduates Recruitment of Nationally Recognized Physicians Community Service Projects Grant Funding National Recognition via Research & Publications Increase in Public Opinion, Value & Prestige (Market Differentiation)
Typical Objections To ROI Methodology It costs too much It takes too much time Is this really needed? Who is asking for this? This is not in my job description I do not understand this What if the results are negative? This is too subjective How can you attribute the return to? Buzachero VV, Phillips J, Phillips PP, Phillips ZL. Measuring ROI in Healthcare: Tools and Techniques to Measure Impact and ROI in Healthcare Improvement Projects and Programs
Public Opinion of Teaching Hospitals The education mission of teaching hospitals teaching tomorrow s doctors in a hands-on environment is good The latest cutting-edge technology, research, & treatments available at teaching hospitals Their favorable personal experience at a teaching hospital Perceived better outcomes because of the collaboration & team approach between doctors & students two heads are better than one The idea that teaching hospitals have the best doctors The kind & caring doctors & nurses at teaching hospitals 3% 23% 13% 61% Favorable Neutral Unfavor Not Familiar AAMC. October 2014. What Americans Say About the Nation s Medical Schools and Teaching Hospitals: Key Findings from the AAMC s 2013 Public Opinion Research by Public Opinion Strategies.
Fortune 500 Companies Returns for Learning & Development Employee replacement costs Turnover costs Economic value of employee behaviors Economic benefits of increased job satisfaction, organizational commitment Source: Green & Brainard, 2005. Society for Human Resource Management
Our Working Process for Holistic ROI Assessment When dire financial circumstances suggest that a residency no longer seems sustainable, it may be important to consider the indirect economic benefits and market based replacements costs for services the residency program provides. P.A. Pugno 2010
Holistic ROI Assessment Charge Current State Task Force Members Timeframe Within the context of the Physician integration, a thoughtful & strategic assessment of GME is to be conducted to determine current resident/fellow allotment, educate the newly formed medical group on GME issues, & consider the ROI value of our GME programs within the larger System. Lack of in-depth understanding & assessment of the ROI for GME at Carolinas HealthCare System Increasing pressures & benefits of aligning & integrating CHS operations & GME strategic priorities Threats to financial stability to GME at the federal, state & local levels Mary Hall, MD; Lisa Howley, PhD; Eric Runge, MBA; Eric Anderson, MEd; Scott Furney, MD; Lynn Hawkins, MBA; Renee Miller, MHA GME Assessment: Initial Model Q4 2014; Monitoring of strategies on-going 27
Planning Framework & Process Assessment of GME ROI Thoughtful & strategic assessment of GME to educate across the newly formed medical group on GME issues, & consider the ROI value of our GME programs within the larger System I: Planning Collect Gather data on the current state of GME by department (includes academics, patient care, research, financial, other) Engage multiple stakeholders Standardized process II: Data Collection Report Generate ROI Working Model Determine action steps short, intermediate, & long term Determine metrics for monitoring steps Integrate within annual institutional review (AIR) IV: Action Plan Analyze Summarize data Conduct member checks on data Analyze themes across departments Determine ROI metrics for tangibles Determine ROI metrics for intangibles III: Data Analysis 28
Data Collection GME Programs DEPARTMENT 1 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK DEPARTMENT 2 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK DEPARTMENT 3 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK DEPARTMENT N DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK Holistic GME Assessment 29
Data Collection GME Programs DEPARTMENT 1 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK Data Collection: ACGME WebADS Data Faculty & Resident Surveys Duty Hour Reports Annual Program Evalautions Annual Updates Letter of Notifications Clinical Service Replacement Provider Cost Analysis Indigent Care ER and Inpatient Coverage Moonlighting for Hospital Services (as appropriate) 30
Resident Clinical Service Replacement Cost Resident Replacement Cost Resident Clinical Service Amount Cost of Service by Physicians/ ACPs Service Replacement Cost Assumptions: A simulation exercise only that assumes GME is discontinued One piece of overall holistic ROI No change in clinical services provided to our patients Efficiency based on perceptions of individual department Chair and Program Directors Impact of supervising and teaching on efficiency is highly variable Based on existing models of care delivery Workload variances exist between attending physician, resident, and ACP 31
Resident Clinical Service Replacement Cost 1. If attending physicians are not supervising residents, how much more or less productive can they be? (quantify in shifts, visits, etc.) 2. How many more or less physicians are needed as a result? (based on required skill set/complexity) 3. How many more or less ACPs are needed as a result? (based on required skill set/complexity) 4. If new FTEs required, rate estimated for skill level 5. All physician rate adjusted from AAMC (75%) to MGMA rate (63%)
Data Collection GME Programs DEPARTMENT 1 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK Survey topics included: Program purpose Diversity QI Projects Retention Rates of Graduates into CHS Practices Balance Between Service & Education Manpower Needs Research Faculty & Resident Awards Educational Innovations Department/program Funding Patient Care/Coverage Educational Program Quality 33
Data Collection GME Programs DEPARTMENT 1 DATA COLLECTION, INCL FINANCIAL SURVEY SMALL GROUP DISCUSSION SUMMARIZATION MEMBER CHECK Panel Members: DIO Associate DIO CHS Finance Director of GME Med Ed Operations Department Members: Chair Program Director(s) Executive Director/VP of Operations STANDARD AGENDA I. Welcome & Introductions by DIO: Why are we here? What are next steps? What will be done with this information? II. Data Review: Summary of Programs Academic: Summary statement, Questions for clarification (if any), Strengths, Concerns (if any) Financial: Summary statement, Questions for clarification (if any), Strengths, Concerns (if any) III. Impact of hypothetical decrease in GME positions? 10% / 33% / 50% IV. Closing: What else would you like to share about your program(s) that was not captured in this assessment? What did we miss? 34
Sample Findings Retention of Graduates into CHS Practices In 2013-2014, CHS hired 33 physicians out of CHS training programs Faster integration into practice Increased familiarity with CHS culture, values, & systems Estimated savings of $745K per hire (~$24.5Million)
Sample Findings GME Enhances National Recognition & Grant Funding Notable faculty contributions towards evidence-based medicine Faculty & residents engage in scholarly activities & innovations that lead to best practices in clinical care During the 2013-2014 academic year, CHS faculty & resident national & international presentations totaled 612 In 2013-2014, CHS faculty received $16 million in research funding
Sample Findings Clinical Service Replacement Provider Costs Reduce the burden on experienced physicians *Fails to account for significant turnover: Estimated that 50%+ existing faculty physicians would exit within 1 year & higher ACP to resident turnover rates Provide Needed Care of Un- & Underinsured Patients Significant costs to replace resident clinical service equates to approximately 5x operating loss on GME programs*
More To Come
Earlier we asked, Final Questions Complete this sentence: GME at my Sponsoring Institution (SI) adds value in the following way: Is this value being measured? If not, why?
What key theme or themes will I leave with & apply at my institution? Thank You! 40