Using Predictive Analytics to Maximize HCAHPS Results Justin Pestrue, Director Clinical Analytics and Business Intelligence Akron General Health System
Special Thanks Luke MacAdam, Manager
<User ID> <Movie ID> <Date of Grade> <Grade 1-5>
Survey of Hospital Leaders* Among top 3 priorities for next 2 years Patient Experience/Satisfaction Quality/Patient Safety 63% 70% Cost management/reduction EMRs/Meaningful Use/IT 37% 35% Employee engagement/satisfaction ACO development/implementation Physician recruitment/retention Construction/capital improvements 22% 18% 17% 11% *The Beryl Institute, Catalyst HC Research, The State of Patient Experience, 2013
Survey of Hospital Leaders* Driving Experience Efforts Ranked Importance 1 - Most Important, 6 - Least Important HCAHPS Scores Desire to provide a better experience 2.4 2.5 Provider of choice in community 3.0 Value based payments 3.6 Competitive Advantage Population Health (ACOs, PCMH, etc.) 4.7 4.9 *The Beryl Institute, Catalyst HC Research, The State of Patient Experience, 2013
Percent of Patients Giving Highest Ratings Discharge Information Communication with Drs Communication with Nurses Cleanliness of Hosp Env Recommend the Hosp Pain Management Overall Hosp Rating Responsiveness of Staff Communication re Meds Quietness of Hospital 54 60 58 60 63 63 73 73 68 71 67 71 67 70 67 79 81 79 78 84 2013 2008
The HCAHPS Challenge
Traditional Culture Change Retrospective Story Based Acronym Titled Provider Focused Analytics Driven Patient Focused Anticipatory Measurable Repeatable Specific
Example Hospital Discharged Inpatients/mo Surveyed Respond Top Box @ 70% Top Box @ 85% 2000 1000 300 210 255 45 patients!
Maximizing HCAHPS Performance max dd E( S dd A dd ) E(R p ) S dd = Score for dimension d, patient p A dd = CMS Adjustment for dimension d, patient p R p = Response rate for patient p
Creating Predictive Models Function of patient data known early in stay Expected scores Expected likelihood of response Add outside data sources to registration data Logistic regression, over 150 independent variables 120,000 patient records over 4 years (35,000 responded to survey)
Select Results Descriptor Delta in Probability Respond to Survey* Delta in Probability rating %9 or 10* Local Co. Resident -18% -14% Gender (male diff) -16% +11% Faith +13% +3.8% Not Married -30% +5% Zip Code Household Inc. +1.0%/$2000 Ave HHI -1%/$2000 *Results of model including over 120,000 inpatients since July 2008
Predicted Overall and Predicted Survey Response
Segmenting Results
High Probability Survey Response Service Risk Advocates Low Probability 9 or 10 High Probability 9 or 10 Disaffected Response Risk Low Probability Survey Response
Model Results Adjusted R squared.17 and.19 Model accuracy identifying the 45 patients Identifying likelihood of less than 9 on 10 scale 75% Identifying likelihood of responding to survey - 69% Overall 50% accurate at identifying the 45 (5x increase over random)
Operationalize Results Patient Admitted Score and Rank Protocols Implemented Monitor Results
Protocols Implemented Concierge Model Assign concierge to admitted patients Meet with patient 15 minutes a day Empowered to address needs Log activity with patient in tracking tool Expanding response team to include other teams
Monitor Results Early Results One hospital able to improve it overall HCAHPS score 10% pts Connecting concierge with patient not easy Concierges meet monthly to discuss their efforts Approach is very promising
Keys to success Thoughtful and disciplined analytics Win senior leader support Turn targeting into a positive thing Listen to the numbers but validate with patients
QUESTIONS? Justin.pestrue@Akrongeneral.org
HCAHPS History 2006 Voluntary Participation 2007 Pay-4- Reporting implemented 2% IPPS update at risk 2008 Scores publicized on hospitalcompare 2010 HCAHPS results are part of VBP pay-4- performance
Survey Conventional Wisdom Food, cleanliness are the biggest issues Only complainers respond Poor patients drag our scores down We need to increase response rates for better scores We need to change our culture Longer respondents take to answer the survey the lower the scores
Poor patients low scores Lower income - lower response, higher scores Complainers respond Model estimates respondents give better scores Increase response rate for better scores Increase scores for better response rates
What to Target Unadjusted Results Vendor view Accessible Uncomplicated Filtered benchmarking Trends with Adjusted 1 Patient, 1 survey, 1 vote Adjusted Results Hospital compare view VBP determinant Payer determinant Truer benchmarking Hints at proportional voting Supports targeted approach
Scores over Response Lag