Kaiser Permanente National Guideline Program: Dynamic Updating Process

Transcription

1 Kaiser Permanente National Guideline Program: Dynamic Updating Process Craig W. Robbins, MD, MPH Medical Director, Evidence Based Practice, KP Care Management Institute Gladys I. Tom, MS Senior Project Manager, Evidence Services Unit, KP Care Management Institute

2 Disclosure of Interests (last 3 years) Craig W Robbins, MD, MPH I certify that, to the best of my knowledge, no aspect of my current personal or professional situation might reasonably be expected to affect significantly my views on the subject on which I am presenting, other than the following: I am employed by the Colorado Permanente Medical Group which works exclusively with the Kaiser Foundation Health Plan in the US. I am not here to specifically promote Kaiser Permanente but merely to share our experience in guideline development.

3 KP Dynamic Updating Process: Lessons Learned Updating clinical questions in a rigorous way is labor intensive regardless of whether external guidelines or external systematic reviews are available to use. Concentrating our evidence analyst resources on the clinical questions most in need of updating will help balance efficiency and rigor in our guideline development processes. Slide 3

4 Kaiser Permanente: Largest Non-Profit Health Care Program in the United States Permanente Medical Groups Kaiser Foundation Hospitals Kaiser Permanente Kaiser Foundation Health Plan Founded in regions in 9 states and District of Columbia 9+ million members 15,000+ physicians 165,000+ employees KP Care Management Institute (CMI) KP National Guideline Program (NGP)

5 KP National Guideline Program The primary purpose of the Kaiser Permanente National Guideline Program is to provide the organization with the best available systematically derived clinical guidance to improve care delivery and optimize the health of Kaiser Permanente members.

6 Kaiser Permanente Evidence Network 17 KP National Guidelines ~200 unique clinical questions across these guidelines Supported by full Systematic Reviews, Evidence Based Methodology 3.5 Staff FTEs dedicated to KP NGP work 1 Senior Project Manager, 2.5 Analysts 1.2 Physician FTEs dedicated to KP NGP work 7 Physician EBM Methodologists External vendor: Doctor Evidence Search & Data Extraction Technology platform Data repository, analysis & documentation

7 KP National Guideline Program Portfolio

8 KP NGP Methodology Update Prior KP guideline methodology developed internally Prior significant emphasis on distinguishing between evidencebased vs. consensus-based recommendations Decision to adopt GRADE methodology Separate assessments of evidence quality and strength of recommendations Decision to use AGREE II and AMSTAR tools to assess external CPGs and SRs for use within KP Overall, we are now more systematic, transparent, & explicit in our guideline development processes 8

9 2012, Care Management Institute, Kaiser Permanente Kaiser Permanente National Guideline Program: Process & Methodology Clinical Questions New New CQ Evidence Scheduled Update Existing CPG Prioritize CQs High Priority CQ Low Priority CQ Key to Abbreviations: CQ Clinical Question CPG Clinical Practice Guideline KP Kaiser Permanente SR Systematic Review GDT Guideline Development Team GQ Guideline Quality Committee NGD KP National Guideline Directors No Adoptable CPG? Yes Existing SR by CQ? Yes No Evidence Search Abstract Article Review Inclusion/ Exclusion Data Extraction Critical Appraisal Yes Additional KP CQs? Appraise SR Qualitative/ Quantitative Synthesis Assess Evidence Quality Systematic Review Rationale Recommendations No No Recommendations Acceptable? Yes Acceptable SR? Yes No Assess Implementability of Recommendations GDT Approval Internal Review GQ NGD Approval New Relevant Studies? Yes National Guideline No Implementation

10 Kaiser Permanente National Guideline Program: Process & Methodology National Guideline Guideline Implementation Regions (EHR, CDS, Operations) Patient Education Tools (KP.ORG, National Patient Instructions Clinician & Staff Education Tools (Clinical Library) Disease Management Accreditation (NCQA) Improved Health of KP Members 2012, Care Management Institute, Kaiser Permanente Key to Abbreviations: EHR Electronic Health Record CDS Clinical Decision Support KP.ORG KP publicly-accessible Internet site NCQA National Committee for Quality Assurance

11 Challenges in Systematic Review (SR) & Clinical Practice Guideline (CPG) Development CPGs not transparent to clinicians Evidence basis not always explicitly linked Difficult to assess rigor of development Very resource-intensive Infinite needs - Finite resources Balance between efficiency and rigor Investment in developing expertise & infrastructure Updating SRs & CPGs Every 2 years? Dynamic updating, based on evidence & impact?

12 KP NGP Dynamic Updating Process: Goal To continuously direct our limited evidence services resources to the clinical questions/recommendations most in need of updating, thereby reaching an effective balance between rigor and efficiency in our guideline development processes. To do so, we re developing processes to monitor the KP NGP portfolio s consistency with current evidence at the level of the individual clinical questions/recommendations. 12

13 KP NGP Dynamic Updating Process: Status Confirming relevance of clinical questions with guideline clinical leaders Prioritizing retained clinical questions for literature monitoring Updating retained clinical questions and their search strategies Monitoring literature searches limited to Top 5 general medical interest journals plus specialty specific journals by topic Using qualitative and quantitative signals to identify the need for updating 13

14 KP NGP Dynamic Updating Process: Status For clinical questions of the following priorities: Low update decision deferred until higher priority items addressed Intermediate monitoring for qualitative signals of the need to update High monitoring for qualitative & quantitative signals of the need to update Prioritizing clinical questions for needed updates Updating prioritized clinical questions and their associated recommendations (using updated external or internal SRs) Every two years, having at least two clinicians on the guideline team document that all included clinical recommendations remain valid 14

15 Clinical lead team assess for clinical importance Not critical/ important Remove from GL Critical/ important Quantitative assessment: Meets ANY of these criteria (critical outcomes): 1. The new CI crosses 1.0 when the old one didn't 2. The new CI includes appreciable harm or benefit while the old CI did not; AND both the new & old CI included The new CI does NOT include appreciable harm or benefit while the old one did; AND both the new & old CI included 1.0 Qualitative assessment: New evidence clearly has the potential to affect clinical decision-making: 1. New studies change interpretation of effectiveness 2. New studies change the balance of benefits/harms 3. New studies introduce a potentially alternative intervention 4. New studies introduce a clinically important expansion of treatment Within GL prioritization (lead team) Yes Create/update MA of critical outcomes (mini databases-dre) Rank order priority across GLs (NGDs) High Priority Lit search update by priority Yes No Document in guideline: Date of update Statement reflecting new evidence does not change recommendation Top 5+ journal screen: New evidence match CQ/PICO? No Update not warranted No Yes Go to slide B Qualitative assessment: New evidence clearly has the potential to affect clinical decision-making: 1. New studies change interpretation of effectiveness 2. New studies change the balance of benefits/harms 3. New studies introduce a potentially alternative intervention 4. New studies introduce a clinically important expansion of treatment Update warranted Yes Update warranted No Update not warranted

16 Literature Monitoring: AHRQ SR Surveillance* Method I Qualitative Signals of Need for Update Opposing findings Substantial harm Superior new treatment Important changes in effectiveness Clinically important expansion of treatment Clinically important caveat Quantitative Signals for Need for Update Change in statistical significance (from non-significant to significant) Change in relative effect size of at least 50 percent *from AHRQ Draft Report Surveillance and Identification of Triggers for Updating Systematic Review: Implementation and Early Experience (2012) 16

17 Literature Monitoring: AHRQ SR Surveillance* Method II Still Valid does not need updating. No new or only confirmatory evidence & experts assessed CER conclusion still valid Possibly out of date may need updating. Some new evidence and/or a minority of experts assessed CER conclusion as having new evidence that might change the conclusion Probably out of date may need updating Substantial new evidence and/or a majority of experts assessed the CER conclusion as having new evidence that might change the conclusion Out of date New evidence that rendered the CER conclusion out of date or no longer applicable *from AHRQ Draft Report Surveillance and Identification of Triggers for Updating Systematic Review: Implementation and Early Experience (2012) 17

18 Virtual Collaborative Workspace IT platform as a strategic enabler of our dynamic updating processes Integrated Components: library management system, study data repository, meta-analysis tool, GRADE-Pro, documentationpublication platform Functions: Clinical question definition & cataloguing Evidence identification Evidence analysis & synthesis Rationale & recommendation development SR & CPG production CDS integration (automated eventually)

19 KP Dynamic Updating Process: Lessons Learned Updating clinical questions in a rigorous way is labor intensive regardless of whether external guidelines or external systematic reviews are available to use. Concentrating our evidence analyst resources on the clinical questions most in need of updating will help balance efficiency and rigor in our guideline development processes. Slide 19

20 Questions? How are your organizations approaching the resource challenges of developing and updating guidelines? 20

21 Contact List Craig Robbins, MD, MPH Gladys Tom, MS