Methodological Issues in Comparing Hospital Performance: Measures, Risk Adjustment, and Public Reporting

Transcription

1 Methodological Issues in Comparing Hospital Performance: Measures, Risk Adjustment, and Public Reporting Harlan M. Krumholz, MD Yale University School of Medicine July 31, National Forum on Pay for Performance

2

3

4 Comparison of health care spending in Taiwan and the World s major countries

5 NHI public satisfaction ratings

6 Quality of Expat Health in Taiwan

7

8 Cheng, 2015

9

10 It would be great if Prof. Krumholz could share the US experiences in developing hospital report cards regarding:

11 1. How to select appropriate indicators/performance metrics to better represent the performance of hospitals.

12

13 Avedis Donabedian ( ) Published Evaluating the Quality of Medical Care in 1966 Separates quality measures into 3 categories: Structure Process Outcome

14 Types of Quality Measures Outcome Process Structure Assumes good medical care will follow

15 Types of Quality Measures Outcome Structure Assumes good medical care will follow Process Assesses whether good medical care has been applied

16 Structure Measures Pros & Cons Pros Measure important foundational concepts Easy to measure and compare

17 Structure Measures Pros & Cons Pros Measure important foundational concepts Easy to measure and compare Cons Relationship to quality is assumed Improving performance may not improve outcomes

18 Structure Measures Define the presence or absence of specific care resources Include: Facilities and equipment Number of beds EHR capabilities Professional and organizational resources Licensure and certification Joint Commission accreditation Policies and procedures Registry participation

19 Process Measures Assess whether or not specific care is performed Include: Clinical and preventive care CMS Core measures

20 Process Measures Examples Applicable to hospitals: Aspirin administration to heart attack patients Timing of antibiotic initiation for pneumonia

21 Process Measures Pros & Cons Pros Evolve from evidence-based practice guidelines Combine consensus-based information with clear instructions for improving care

22 Process Measures Pros & Cons Pros Evolve from evidence-based practice guidelines Combine consensus-based information with clear instructions for improving care Cons Can be burdensome to collect Exclude some patients from measurement May not correlate well with outcomes

23 Outcome Measures Assess what actually happens to the patient Survival Unintended treatment effects Symptom relief Include: Acute clinical events Health status Patient experience of care

24 Outcome Measures Examples Applicable to hospitals: 30-day all-cause mortality and readmission following AMI, heart failure and pneumonia H-CAHPS Survey

25 Outcome Measures Pros & Cons Pros Measure what matters most to patients

26 Outcome Measures Pros & Cons Pros Measure what matters most to patients Cons Must be influenced by care Requires adequate risk adjustment to account for differences in case mix Influenced by many factors More difficult to measure

27 What makes a good outcome measure? Well thought out measure concept Considering potential applications in design Measure specifications adhere to accepted standards

28 A well thought out measure concept Addresses measurement and performance gaps Measurement gap: Not currently measured Performance gap: variation in performance not explained by variation in patient case mix Considers feasibility, usability, reliability and validity

29 Considering potential applications in design Clinical care Quality improvement Accountability

30 Measure specifications Population to be measured (cohort) Result to be measured (outcome) Outcome attribution (who is responsible) (Risk adjustment) To be covered in Why risk adjust?

31 Measure specifications Population to be measured (cohort) Result to be measured (outcome of interest) Outcome attribution (who is responsible) (Risk adjustment) To be covered in Why risk adjust?

32 Measure specifications Population to be measured (cohort) Result to be measured (outcome) Outcome attribution (who is responsible) (Risk adjustment) To be covered in Why risk adjust?

33 Measure specifications Population to be measured (cohort) Result to be measured (outcome) Outcome attribution (who is responsible)

34 Measure Cohort Reliably identifiable Clinically coherent Sample size, depending upon application

35 Measure Outcome Reliably identifiable Standard measurement period

36 Outcome attribution Ownership of care Concept of shared responsibility

37 2. The mechanism to select risk factors for each outcome indicators, especially to get/build consensus from medical professionals.

38 Hospital Care Outcome Patient Status on Presentation Other Effects

39 Patient Status on Presentation Expected Outcome

40 Expected Outcome versus Observed Outcome

41 Purpose of risk adjustment To help define EXPECTED OUTCOME Level the playing field Account for patient-level factors that influence outcome but do not reflect care quality

42 Avoid risk factors that Represent complications of care Represent healthcare system attributes Will potentially mask disparities

43 3. The statistic models used for risk adjustment, and their strength and weakness.

44 Approach to risk adjustment Account for patient case mix Account for clustering of patients at measurement unit (hospital, group practice, etc.) Avoid risk factors that Represent complications of care Represent healthcare system attributes (e.g., discharge disposition) Will potentially mask disparities (e.g., SES or race)

45 Approach to risk adjustment Account for patient case mix Account for clustering of patients at measurement unit (hospital, group practice, etc.) Avoid risk factors that Represent complications of care Represent healthcare system attributes (e.g., discharge disposition) Will potentially mask disparities (e.g., SES or race)

46 4. The issues regarding quality of data, especially when claim data is the major source for measuring performance.

47 5. The policy/approach that help medical professionals and consumers to understand report card, especially the result of adjusted outcomes.

48 Aspects of outcome measures specific to accountability: Higher threshold of scientific reliability and validity Must be fair to measured entities Should assign performance categories with high degree of confidence Must define comparator

49 Aspects of outcome measures specific to accountability: Higher threshold of scientific reliability and validity Must be fair to measured entities Should assign performance categories conservatively Must define comparator

50 Aspects of outcome measures specific to accountability: Higher threshold of scientific reliability and validity Must be fair to measured entities Should assign performance categories conservatively Must define comparator

51 Aspects of outcome measures specific to accountability: Higher threshold of scientific reliability and validity Must be fair to measured entities Should assign performance categories with high degree of confidence Must define comparator

52 Aspects of outcome measures specific to accountability: Higher threshold of scientific reliability and validity Must be fair to measured entities Should assign performance categories with high degree of confidence Must define comparator

53 CMS s hospital AMI 30-day all-cause mortality measure Reported as risk-standardized mortality rate (RSMR) Measures all-cause mortality after AMI admissions Captures deaths within 30 days of admission Publicly reported on HospitalCompare.gov

54 Risk-standardized mortality rate (RSMR) = Hospital s predicted deaths Hospital's expected deaths national unadjusted mortality rate Predicted = number of deaths within 30 days predicted on basis of hospital s performance with its observed case mix Expected = number of deaths expected on basis of the nation s performance with that hospital s case mix

55 Performance categories for public reporting Hospitals categorized as Better, Worse, or No Different than the national rate Use 95% interval estimate (like 95% confidence interval) to define categories Measure of relative performance

56

57 6. Advice to accelerate/promote public reporting of hospital performance.

58

59

60

61

62

63