06 Validation of risk prediction model

Transcription

1 HA Territory-wide PCI Audit Validation of risk prediction model PCI Audit Working Group Central Committee (Cardiac Services) HA Convention 2007

2 Background Participants: All HA hospitals via CCLs Mechanism: Prospective Comparative Audit Audit Period: 1 Jan Dec 2006 Audit Tools: PHCA s audit form Supplemented by Discharge summary epr consultation note Laboratory results (LIS) Death Registry On-site random record check

3 Objectives Compare baseline characteristics Compare standardized mortality and complication ratios among 10 CCLs Identify risk factors related to post-operation operation mortality and complication within 72 hours Validate the risk prediction model for post-pci mortality/complications

4 Baseline Characteristics N= 3,904 N= 4,894 N= 4,784 N= 5,036 Male Sex 2,870 (74%) 3,639 (74%) 3,539 (74%) 3,709 (74%) Hypercholesterolemia 2,815 (72%) 3,726 (76%) 3,016 (63%) 3,354 (67%) Hypertension 2,573 (66%) 3,211 (66%) 3,039 (64%) 3,184 (63%) Smoking History 2,191 (56%) 2,567 (53%) 2,260 (47%) 2,315 (46%) Age 1,442 (37%) 1,882 (39%) 2,069 (43%) 2,197 (44%) Diabetes Mellitus 1,553 (40%) 1,903 (39%) 1,829 (38%) 1,954 (39%) Previous PCI 1,252 (32%) 1,604 (33%) 1,525 (32%) 1,599 (32%) CVA 297 (8%) 412 (8%) 284 (6%) 314 (6%) Renal Failure 180 (5%) 252 (5%) 236 (5%) 241 (5%)

5 ACS other than AMI, 678, 17.4% Primary (for AMI), 152, 3.9% Rescue (for AMI), 92, 2.4% Unplanned redo PCI, 7, 0.2% Case mix ACS other than AMI, 864, 17.7% Primary (for AMI), 145, 3.0% Rescue (for AMI), 44, 0.9% Unplanned redo PCI, 6, 0.1% Elective, 3835, 78.4% (3904 cases) ACS other than AMI, 627, 13.1% Primary (for AMI), 166, 3.5% Elective, 2975, 76.2% Rescue (for AMI), 71, 1.5% (4894 cases) ACS other than AMI, 698, 13.9% Primary (for AMI), 159, 3.2% Rescue (for AMI), 79, 1.6% 2005* (4784 cases) Elective, 3920, 81.9% 2006* (5036 cases) Elective, 4100, 81.4% * Unplanned redo PCI are not categorized in 2005 and 2006 data

6 Severity of disease >2 vessels attempted, 226, 5.8% >2 vessels attempted, 377, 7.7% 2 vessels attempted, 1014, 26.0% 2 vessels attempted, 1349, 27.6% 2003 (3904 cases) 1 vessel attempted, 2664, 68.2% 2004 (4894 cases) 1 vessel attempted, 3168, 64.7% >2 vessels attempted, 294, 6.2% >2 vessels attempted, 413, 8.2% 2 vessels attempted, 1333, 27.9% 2 vessels attempted, 1387, 27.5% 2005 (4772 cases) (12 cases are excluded due to missing and invalid value) 1 vessel attempted, 3145, 65.9% 2006 (5036 cases) 1 vessel attempted, 3236, 64.3%

7 Characteristics of the worst lesion Type A, 481, 12.3% Type A, 440, 9.0% Type C, 1313, 33.6% Type C, 1672, 34.2% Type B1, 806, 20.6% Type B1, 1155, 23. 6% 2003 (3904 cases) Type B2, 1304, 33.4% 2004 (4894 cases) Type B2, 1627, 33.2% Type C, 1467, 30.7% Type A, 239, 5.0% Type B1, 1150, 24.0% Type C, 1804, 35.8% Type A, 336, 6.7% Type B1, 966, 19.2% 2005 (4784 cases) Type B2, 1928, 40.3% 2006 (5036 cases) Type B2, 1930, 38.3%

8 Angiographic result Angiographic Result Successful 3,651 (94%) 4,568 (93%) 4,578 (96%) 4,803 (95%) Partially Successful 115 (3%) 182 (4%) 78 (2%) 100 (2%) Unsuccessful 138 (4%) 144 (3%) 128 (3%) 133 (3%) Total 3,904 (100%) 4,894 (100%) 4,784 (100%) 5,036 (100%)

9 Use of PCI devices/drugs Balloon Stent Rotablator DCA Distal Protection Device Thrombectomy Device IABP/CPS Gp 2b3a Inhibitor Closure Devices (N=3,904) (N=4,894) (N=4,784) (N=5,036) 3,468 (89%) 4,458 (91%) 4,295 (90%) 4,622 (92%) 3,194 (82%) 4,472 (91%) 4,491 (94%) 4,756 (94%) 59 (2%) 55 (1%) 34 (1%) 45 (1%) 6 (0.2%) 143 (3%) 7 (0.2%) 1 (0.02%) 123 (3%) 111 (2%) 83 (2%) 71 (1%) 59 (2%) 64 (1%) 47 (1%) 90 (2%) 65 (2%) 105 (2%) 83 (2%) 89 (2%) 154 (4%) 116 (2%) 162 (3%) 164 (3%) 379 (10%) 478 (10%) 977 (20%) 1,204 (24%)

10 Major Complications within 72 hours on event basis (direct and indirect procedure causes included) No. of cases Death 3, (0.6%) 4, (0.61%) 4, (0.7%) 5, (0.5%) Other major complications 45(1.2%) 89 (1.8%) 76 (1.6%) 82 (1.6%) MI-Non Q 21 (0.5%) 50 (1.0%) 31 (0.7%) 52 (1.0%) MI-Q wave 7 (0.2%) 5 (0.1%) 10 (0.2%) 8 (0.2%) Unplanned CABG 1 (0.03%) 6 (0.1%) 4 (0.1%) 3 (0.1%) Unplanned Re-PCI 10 (0.3%) 6 (0.1%) 8 (0.2%) 8 (0.2%) Stroke 0 (0%) 2 (0.04%) 4 (0.1%) 1 (0.02%) Bleeding with Hb Drop of >=3g/dl 6 (0.2%) 10 (0.2%) 12 (0.3%) 7 (0.1%) Cardiac Tamponade 5 (0.1%) 14 (0.3%) 11 (0.2%) 12 (0.2%) Vascular surgery or repair 2 (0.1%) 2 (0.04%) 1 (0.02%) 2 (0.04%)

11 Complications All Patients,2003~06 Indications Elective ACS other than AMI Primary (for AMI) Rescue (for AMI) Unplanned redo PCI* No. of cases Death 13 (0.1%) 41 (1.4%) 39 (6.3%) 17 (5.9%) 1 (7.7%) Other major complications 202 (1.4%) 60 (2.1%) 21 (3.4%) 9 (3.1%) 0 (0%) MI-Non Q 121 (0.8%) 31 (1.1%) 2 (0.3%) MI-Q wave 21 (0.1%) 4 (0.1%) 4 (0.6%) 1 (0.4%) Unplanned CABG 6 (0.04%) 7 (0.2%) 1 (0.4%) Unplanned Re-PCI 16 (0.1%) 7 (0.2%) 8 (1.3%) 1 (0.4%) Stroke 3 (0.02%) 2 (0.1%) 2 (0.3%) Bleeding with Hb Drop of >=3g/dl 22 (0.2%) 8 (0.3%) 3 (0.5%) 2 (0.7%) Cardiac Tamponade 27 (0.2%) 8 (0.3%) 2 (0.3%) 5 (1.8%) Vascular surgery or repair 5 (0.03%) 2 (0.1%) * "Unplanned redo PCI" is not categorized in 2005 and 2006 data.

12 Mortality & Other major complications Primary PCI (for AMI) done Overall No. of cases Death 8 (5.3%) 11 (7.6%) 12 (7.2%) 8 (5%) 39 (6.3%) Other major complications 2 (1.3%) 6 (4.1%) 6 (3.6%) 7 (4.4%) 21 (3.4%) MI-Non Q 1 (0.7%) 1 (0.6%) 2 (0.3%) MI-Q wave 1 (0.7%) 1 (0.6%) 2 (1.3%) 4 (0.6%) Unplanned CABG 0 (0%) Unplanned Re-PCI 2 (1.3%) 2 (1.4%) 1 (0.6%) 3 (1.9%) 8 (1.3%) Stroke 2 (1.2%) 2 (0.3%) Bleeding with Hb Drop of >=3g/dl 1 (0.7%) 1 (0.6%) 1 (0.6%) 3 (0.5%) Cardiac Tamponade 1 (0.7%) 1 (0.6%) 2 (0.3%) Vascular surgery or repair 0 (0%) Among 622 cases with Primary PCI (for AMI) done, 3 cases have missing information on other major complications, and 2 of the aforesaid cases also have missing information on death.

13 Mortality & Other major complications Intercurrent Cardiogenic Shock No. of cases Overall Death 10 (17.5%) 14 (21.9%) 19 (27.1%) 11 (18%) 54 (21.3%) Other major complications 2 (3.5%) 6 (9.4%) 5 (7.1%) 0 (0.0%) 13 (5.1%) MI-Non Q 1 (1.6%) 1 (1.4%) 2 (0.8%) MI-Q-wave 1 (1.8%) 1 (0.4%) Unplanned CABG 0 (0%) Unplanned Re-PCI 2 (2.9%) 2 (0.8%) Stroke 2 (2.9%) 2 (0.8%) Bleeding with Hb Drop of >=3g/dl 2 (3.1%) 1 (1.4%) 3 (1.2%) Cardiac Tamponade 1 (1.8%) 3 (4.7%) 4 (1.6%) Vascular surgery or repair 0 (0%) Among 253 cases with Intercurrent cardiogenic shock, 3 cases have missing information on death and other major complications.

14 Post-PCI PCI Mortality, 2003~ Overall No. of cases 3,904 4,894 4,784 5,036 18,618 Deaths within 72 hours 23 (0.6%) 30 (0.6%) 33 (0.7%) 25 (0.5%) 111(0.6%) Deaths within 30 days * 50 (1.3%) 66 (1.3%) 79 (1.7%) 69 (1.4%) 264(1.4%) * In cases with incomplete death date, impute month end when death day is missing and impute year end when death month is missing.

15 Univariate & Multivariate Analyses (2003 ~ 2006)

16 Mortality within 72 hours

17 Mortality Univariate analysis (2003 ~ 06) No. of cases = 18,553 Deaths: N=111 (0.6%) A. Pre-Intervention Status Heart Failure Sustained Ventricular Tachycardia or Ventr icular Fibr illation N(%) None/Other HF/Past Hx of HF 18, (0.3%) Intercurrent Cardiogenic Shock (21.6%) None 18, (0.4%) Intercurrent VT/VF (within 24hrs) (13.3%) All other VT/VF excluding 1 o VT/VF (1.2%) Angina Type No Angina 1,191 3 (0.3%) <0.01 Major Native Arteries with >=50% lesions No. of cases Death Stable 11,109 9 (0.1%) Unstable 6, (1.6%) no LM involved 17, (0.5%) LM involved (2.7%) p-value <0.01 <0.01 <0.01

18 Mortality Univariate analysis (2003 ~ 06) No. of cases = 18,553 Deaths: N=111 (0.6%) B. Co-morbidity/Risk Factors No. of cases Death N(%) p-value Renal Failure Yes (2.7%) <0.01 No 17, (0.5%) Cerebrovascular Disease Yes 1, (1.7%) <0.01 No 17, (0.5%) Previous Percutaneous Coronary Yes 5, (0.4%) Intervention No 12, (0.7%) Sex Female 4, (0.8%) Male 13, (0.5%) Peripheral Vascular Disease Yes (1.2%) No 17, (0.6%) Hypertension Yes 11, (0.7%) No 6, (0.4%) Age less than 70 10, (0.5%) and over 7, (0.7%) Previous CABG Yes (0.2%) No 18, (0.6%) Smoking History Diabetes Mellitus Family History of premature CAD No 8, (0.6%) Now or Stop<1 Yr 4, (0.7%) Yes but Stop>1 Yr 4, (0.4%) Yes 7, (0.7%) No 11, (0.5%) Yes (0.4%) No 17, (0.6%)

19 Mortality Univariate analysis (2003 ~ 06) No. of cases = 18,553 Deaths: N=111 (0.6%) C. Procedure Summary No. of cases Death N(%) p-value Indications Primary or Rescue AMI/Unplanned redo PCI (6.2%) <0.01 Characteristic of the worst lesion Elective/ACS other than AMI 17, (0.3%) Type A/B1 5, (0.2%) Type B2/C 13, (0.8%) <0.01

20 Mortality Multivariate analysis (2003 ~ 06) No. of PCI cases = 18,393 (Death = 107, 0.58%) OR 95 % C.I. p-value Angina type Unstable Angina no angina or stable angina Vs 6.35 (3.23, 12.47) <.0001 Heart failure Intercurrent Cardiogenic shock None/Other HF/APO/Past Hx of HF Vs (7.03, 20.99) <.0001 Sustained VT/VF Intercurent VT/VF None or other VT/VF Vs 4.28 (2.42, 7.56) <.0001 Major native arteries LM involved No LM involved Vs 2.90 (1.64, 5.13) Cerebrovascular disease Yes No Vs 3.05 (1.78, 5.22) <.0001 Renal failure Yes No Vs 4.15 (2.42, 7.14) <.0001 Indications Goodness-of-fit Primary or Rescue AMI/Unplanned redo PCI Vs Elective/ACS other than AMI 2.76 (1.60, 4.74) p-value of Hosmer & Lemeshow test 0.24 Area under ROC curve 0.90

21 Standardized Mortality Ratio (SMR) (2003~2006) Excluding cases with missing angina type, heart failure, sustained VT/VF, major native arteries, CVA, renal failure or indications No outliers identified Hospital 0.0 A B C D E F G H I J Expected no. of deaths = sum of predicted probability of death from multiple logistic regression model for each hospital

22 Non-death Complication within 72 hours

23 Complication Univariate analysis (2003 ~ 06) No. of cases = 18,448 Complications: N=292 (1.6%) A. Pre-Intervention Status Heart Failure N(%) None/Other HF/Past Hx of HF 18, (1.5%) Intercurrent Cardiogenic Shock (5.2%) Angina Type No Angina 1, (1.6%) <0.05 Major Native Arteries with >=50% lesions Sustained Ventricular Tachycardia or Ventricular Fibrillation No. of cases Complications Unstable 6, (1.9%) Stable 11, (1.4%) no LM involved 17, (1.5%) LM involved (2.5%) None 17, (1.6%) Intercurrent VT/VF (within 24hours) (2.1%) All other VT/VF excluding 1 o VT/VF (1.8%) p-value <0.01 <

24 Complication Univariate analysis (2003 ~ 06) No. of cases = 18,448 Complications: N=292 (1.6%) B. Co-morbidity/Risk Factors Age No. of Complications cases N(%) 70 and over 7, (2.0%) less than 70 10, (1.3%) Yes 1, (2.6%) No 17, (1.5%) p-value <0.01 Cerebrovascular Disease <0.01 Sex Female 4, (2.0%) <0.05 Male 13, (1.5%) Previous CABG Yes (2.7%) <0.05 No 17, (1.5%) Renal Failure Yes (2.3%) No 17, (1.5%) Smoking History No 8, (1.7%) Now or Stop<1 Yr 4, (1.3%) Yes but Stop>1 Yr 4, (1.6%) Hypertension Yes 11, (1.6%) No 6, (1.5%) Peripheral Vascular Disease Yes (1.9%) No 17, (1.6%) Previous Percutaneous Coronary Yes 5, (1.6%) Intervention No 12, (1.5%) Diabetes Mellitus Yes 7, (1.6%) No 11, (1.6%) Family History of premature CAD Yes (1.5%) 1 No 17, (1.6%)

25 Complication Univariate analysis (2003 ~ 06) No. of cases = 18,448 Complications: N=292 (1.6%) C. Procedure Summary No. of Complications cases N(%) p-value Indications Primary or Rescue AMI/Unplanned redo PCI (3.3%) <0.01 Elective/ACS other than AMI 17, (1.5%) Characteristic of the worst lesion Type A/B1 5, (0.8%) Type B2/C 12, (1.9%) <0.01

26 Complication Multivariate analysis (2003 ~ 06) No. of PCI cases = 18,437 (Complication = 291, 1.58%) OR 95 % C.I. p-value Heart failure Cerebrovascular disease Age Intercurrent Cardiogenic shock None/Other HF/Past Hx of HF Yes No >=70 Vs below 70 Vs Vs 2.14 (1.12, 4.10) (1.09, 2.29) (1.15, 1.83) Indications Charactreristic of worst lesion Goodness-of-fit Primary or Rescue AMI/Unplanned redo PCI Elective/ACS other than AMI Type B2/C Type A/B1 p-value of Hosmer & Lemeshow test 0.23 Area under ROC curve 0.63 Vs Vs 1.67 (1.08, 2.60) (1.57, 2.96) <.0001

27 Standardized Complication Ratio (SCR) (2003~2006) Excluding cases with missing heart failure, CVA, age, indications or Characteristic of worst lesion Heterogenous group of complications (from mild to severe) Hospital Expected no. of complications = sum of predicted probability of complications from multiple logistic regression Amodel for each hospital B C D E F G H I J

28 Development and Validation of a predictive model for Post-PCI mortality within 72 hours Excluding cases with unknown outcome within 72 hours post-pci PCI data for No. of PCI cases = 18,618 Deaths = 111 (0.6%) Training Dataset No. of PCI cases = 13,582 Deaths = 86 (0.6%) Validation Dataset 2006 No. of PCI cases = 5,036 Deaths = 25 (0.5%)

29 Receiver operating characteristic (ROC) curves of multiple logistic regression models in Sheet "Model" for training dataset (Years ) and validation dataset (Year 2006) 1 ROC curve Sensitivity Training dataset (area under curve=0.923) Validation dataset (area under curve=0.831) Specificity

30 Predictors of Mortality OR Intercurrent cardiogenic shock 12.1 Unstable Angina 6.4 Intercurrent VT/VF 4.3 Renal Failure 4.2 Hx of CVA 3.1 Left Main involvement 2.9 Primary/Rescue PCI for AMI 2.8

31 Predictors of non-death complications OR Type B2/C lesions 2.2 Intercurrent cardiogenic shock 2.1 Primary/Rescue PCI for AMI 1.7 Hx of CVA 1.6 Age

32 Limitations No data on 6-month 6 month redo-pci for Target Lesion Revascularization (TLR) Self-reported reported cases 2003 a full-time audit nurse to check data accuracy a part-time time audit nurse 2006 onwards data accuracy ensured by individual institutions

33 Way Forward Individual CCLs responsible for data accuracy A designated medical staff as point of contact Half-yearly outstanding report list to individual CCL for FU Collaborate with HI & IT Team GRR for PCI reports integrating audit capability (All the 10 CCLs are using the GRR PCI reporting system since January 2005) Data Quality cross-check check with data warehouse e.g. death registry, LIS, MOE Generic reports by CDARS Individual hospital vs. all hospital average With validated risk prediction model for PCI procedures in HA hospitals Uphold the standard of PCI procedures performed in HA hospitals Interventional Cardiologists more willing to take up high-risk cases Patients more well-informed of the risks of PCI

34 Acknowledgement Hong Kong Public Hospital Cardiologists Association HA Central Committee (Cardiac Services) HAHO Statistics and Research Section HAHO HI & IT Team HAHO Clinical Effectiveness Unit HAHO Central Audit Committee

35 Thank you!