Evidence-based guideline development. Dr. Jako Burgers/dr. H.P.Muller Dutch Institute for Healthcare Improvement CBO, Utrecht, The Netherlands

Transcription

1 Evidence-based guideline development Dr. Jako Burgers/dr. H.P.Muller Dutch Institute for Healthcare Improvement CBO, Utrecht, The Netherlands

2 Outline lecture/workshop 1. Aims and objectives of guidelines 2. Introduction to evidence-based guideline development 3. Implementation of guidelines

3 I. Aims and objectives From theory to practice Systematic reviews (e.g. Cochrane reviews) Health technology assessment reports Evidence-based medicine/reports Do we need clinical practice guidelines?

4 Questions to answer 1. Why guidelines? 2. How would you define guidelines? NB: guidelines, clinical guidelines and clinical practice guidelines are considered as synonyms

5 Aims of guidelines To summarise and synthesise knowledge and innovations in medicine To reduce variation in practice To promote evidence-based clinical practice To improve quality of patient care To satify the need for transparancy and accountability

6 Definition (IOM, 1990) Clinical guidelines are: systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances. Institute of Medicine (1990)

7 History international National Institute of Health (US) consensus development program (1977) Canadian Task Force on the Periodic Health Examination (now CTFPHC) grades of evidence (1979) Agency for Health Care Policy and Research (AHCPR, now AHRQ) evidence-based guideline programme ( ) National guideline programmes in most Western countries (1990s)

8 History 25 years CBO 1979: National Organization for Quality Assurance in Hospitals CBO founded by Dutch Association of Medical Specialists and Dutch Association of Hospital Medical Directors with focus on consensus development and peer review 1998: Dutch Institute for Healthcare Improvement CBO with focus on evidence-based guideline development, quality improvement and management in hospitals

9 Trends in time From - regional guidelines from professional groups To - monodisciplinary - multidisciplinary - limited life-expectancy - living guidelines - paper versions - internet - national guideline programmes - informal consensus - evidence-based - focus on development - focus on implementation - guidelines for clinicians - patient versions and patient involvement

10 Van Everdingen JJE, Burgers JS, Assendelft WJJ, ea. Evidence-based richtlijnontwikkeling. Leidraad voor de praktijk. Houten: BSL, Definition ( Haamstede, 2004 ) A guideline is a document with recommendations and instructions to assist healthcare professionals and patients in clinical decision making, based on research findings and consensus among experts, in order to make effective and efficient clinical practice explicit.

11 Guideline versus protocol Guideline Protocol - What and why - How, when and by whom - Flexible - Directive - Mother document - Tools for application - (Often) national - Adapted to local context (e.g. hospital, cancer centre)

12 II. Introduction to evidence-based guideline development Questions to the audience: Do you have experience in guideline development? What is crucial in the process of developing successful guidelines?

13 Steps in guideline development 1. Preparation 2. Design 3. External review/endorsement 4. Dissemination 5. Evaluation Implementation in all steps!!!

14 Guideline development cycle Problem analysis - Medical - Organizational - Patient issues Implementation feedback Formulation of key questions Define search criteria - databases - time period - PICO Implementation Literature search Evidence Based Guideline Development Define selection criteria Selection of literature Critical appraisal of selected literature Evidence table - diagnostics - therapy Dissemination & publication Finalize guideline + endorsement by scientific medical associations Writing first draft of guideline - Summary statement of the evidence - Further considerations - Recommendations for clinical practice Re-writing draft guideline External review - National open meeting - Scientific medical associations/societies - Website Finalize draft version of guideline Development of indicators Re-writing draft Discussion of draft guideline - subgroup - plenair

15 1. Preparation Topic selection Selection of chair (and vice-chair) Selection of working group members by contacting relevant societies/organisations/stakeholders Patient participation Analyses of barriers in practice Formulation specific clinical questions

16 Criteria for selecting guideline topic High prevalence of morbidity and/or mortality High burden of disease High health care costs Gap between research and practice Controversy about optimal care Possibility of achieving consensus

17 Scoping the topic Patient population: disease/condition age, gender exclusion criteria Intervention: prevention/screening diagnosis therapy Target users: professionals patients other

18 Methods of patient participation Literature research to patient views and preferences Patient surveys Participation patient representative in working group Focus group interviews Involvement of patient organisation in external review Patient organisation as financier of guidelines

19 Analysis of barriers in practice Literature research Analysis of registration data Written or telephone survey Individual or group interviews

20 From barrier to question Example non small cell lung cancer (NSCLC) Barrier: - Practice variation in management of patients with stage IIIA NSCLC. Questions: 1. Which is the best therapy for NSCLC stage III? 2. Which (neo)adjuvant therapy is indicated for NSCLC stage III? 3. Which is the preferred sequence and maximum interval between different treatment options?

21 2. Design Literature search and selection Critical appraisal of literature Summary of evidence, including levels of evidence Translation of evidence into recommendations Cost-effectiveness analysis (if necessary) Drafting concept guideline

22 Literature search: databases National Guideline Clearinghouse ( Guidelines International Network ( Cochrane Library Medline (Pubmed) Embase Cinahl, Psychinfo, Cancerlit NB: Google!!

23 Critical appraisal of literature Checklists are available on (downloads)

24 Summary of evidence Study design Quality of individual studies Number of patients Patient characteristics, in-/exclusion criteria Intervention (and comparison) Clinically relevant outcome measures Results: effect size and confidence interval Follow-up time and loss to follow-up Funding Level of evidence

25 Evidence table: an example

26 CBO Levels of evidence (Prevention/Treatment) A1 Systematic review of at least two independent randomised controlled trials of A2-level with consistent results A2 Double-blind randomised controlled trial of good quality and sufficient power B Randomised controlled trial of modest quality or insufficient power, or other comparative study C Non-comparative study D Expert opinion

27 GRADE Grading system 2004 Quality of evidence Study design Lower if * Higher if * High RCT Study Quality: Intermediate Quasi-RCT 1-Serious flaws 2-Very serious flaws Consistency: 1-Important Low Observational study inconsistency Directness: 1-Some uncertainty Very low Any other evidence 2-Major uncertainty Reporting bias Sparse data * 1 = move up or down one grade (for example from high to intermediate) 2 = move up or down two grades (for example from high to low) Association: 2-Extremely strong 1-Strong and no plausible confounders, consistent, and direct GRADE Working Group. BMJ 2004; 328:

28 From evidence to recommendation Evidence Other considerations Other considerations Recommendation

29 Questions to answer 1. If evidence is lacking or controversial, would you still formulate a recommendation? 2. Which factors would you consider beyond the evidence in formulating recommendations for clinical practice?

30 Considered judgement 1. Clinical impact Magnitude of effect Consistency of evidence Relative advantage Generalisability

31 Considered judgement 2. Safety Side effects Risks of complications on short term Risks or complications on long term

32 Considered judgement 3. Patient perspective Needs and expectations Estimated compliance Estimated satisfaction Accessibility of services

33 Considered judgement 4. Professional perspective Professional advantage or risks Time needed for applying the intervention New knowledge and skills needed Change of attitudes, routines and habits Compatibility with norms and values Financial consequences

34 Considered judgement 5. Availability Health services (local, regional or national) Knowledge and skills of professionals 6. Organisation and health care system Organisational changes needed Existing infrastructure for implementation of changes Strategic and political consequences Reimbursement by health care insurance companies

35 Considered judgement 7. Costs and cost-effectiveness Potential cost implications Impact on contracts between purchasers and providers Impact on (public) health care budget 8. Legal aspects Law and regulations regarding professional Law and regulations regarding specific intervention(s) Legal consequences of applying or not applying the recommendations in the guideline

36 Drafting concept guideline Introduction Clinical questions Method: search strategy, selection criteria, method of formulating recommendations Results: summary of evidence, evidence tables Discussion: considered judgment Final recommendations Is a guideline different from a scientific paper?

37 Steps in guideline development 1. Preparation 2. Design 3. External review/endorsement 4. Dissemination 5. Evaluation Implementation in all steps!!!

38 3. External review and endorsement External peer review Consensus meeting (not obligatory) Contact all relevant stakeholders Pilot test among target users Formulation final guideline Endorsement/authorization

39 Legal aspects Guidelines are not laws Non-compliance is allowed, provided that one has arguments Ownership: commissioning or participating organisations Guidelines can be used in contracts between purchasers and providers Guidelines should be independent from industry (and policy makers?)

40 4. Dissemination Diffusion of guideline to: - target group - all relevant organisations Publication in (peer-reviewed) journal Website, PDA, integration in EPD

41 5. Evaluation Quality of guideline and guideline program (using AGREE criteria) Monitoring use of guidelines (process) Measuring effect on patient care (outcome) Update procedure

42 The AGREE Instrument 23 items 4-point Likert Scale Overall assessment User guide Six domains 1. Scope & purpose (3) 2. Stakeholder involvement (4) 3. Rigour of development (7) 4. Clarity & presentation (4) 5. Applicability (3) 6. Editorial independence (2) or

43 Problem analysis - Medical - Organizational - Patient issues Implementation feedback Periodical update of original literature search Dissemination & publication? Formulation of key questions Re-writing draft guideline External review - National open meeting - Scientific medical associations/societies - Website Kwaliteitsinstituut voor de Gezondheidszorg CBO ? Finalize guideline + endorsement by scientific medical associations 2. Define search criteria - databases - time period - PICO Implementation Finalize draft version of guideline Literature search Evidence Based Guideline Development Concept of living guideline Development of indicators Re-writing draft Define selection criteria Selection of literature 4. Critical appraisal of selected literature Evidence table - diagnostics - therapy Writing first draft of guideline - Summary statement of the evidence 3. - Further considerations - Recommendations for clinical practice Discussion of draft guideline - subgroup - plenair Decision moments: 1. New topic(s)/ issue(s) 2. New evidence (original key questions) 3. New recommendation(s) or original recommendations changed 4. No major changes

44 duration: 3 months (this phase has been finished when guideline group starts) Preparation phase topic selection selection chairman contact associations and composition working group analysis of barriers for implementation duration: 1 year Development draft guideline meetings working group division of tasks literature search literature appraisal evidence summary cost-effectiveness and other considerations patient involvement formulation of recommendations Implementation duration: 3 months Comment phase external expert review national meeting and other variants integrate comments write final guideline indicators / organisatorial aspects / financial aspects/quality improvement projects duration: 3 months (or longer if necessary) Dissemination publication of guideline diffusion of guideline publication of articles duration: maximim of 5 years Evaluation evaluation of guideline development evaluation of implementation monitor guideline use and impact revision and update of guideline

45 III. Implementation Questions to the audience: 1. Are you using guidelines in practice? 2. What are potential barriers for implementing guidelines in practice?

46 Why don t physicians follow clinical practice guidelines? Don t know that guideline exist guideline is too large guideline is too complicated disagree with content Don t want to change routins and habits ( never change a winning.. ) guideline is cookbook medicine guideline threatens professional autonomy no confidence in guideline developing organisation other guideline available Don t do Because of individual patient preferences Lack of time Lack of skills External barriers: availability, organisation and costs Cabana MD, et al. JAMA 1999;282 (1):

47 Implementation of change Early Adopters Early majority Late majority Kiss of Yes Innovators Laggards 2,5% 13,5% 34% 34% 13,5% 2,5% 20%: tipping point Rogers: 1995; Gladwell: 2000

48 Statement Good guidelines are effective guidelines Burgers JS. Quality of clinial practice guidelines (thesis). Nijmegen, 2002.

49 Characteristics of effective guidelines Supported with scientific evidence Easy / no complex decision tree No new skills needed Compatible with norms and values Burgers JS. Quality of clinial practice guidelines (thesis). Nijmegen, 2002.

50 Implementation strategies Professional interventions (e.g. education, audit and feedback, reminders, outreach visits) Organisational interventions (e.g. clinical pathways, staff and resources) Patient mediated interventions (e.g. promotion in public media) Financial interventions (e.g. (dis)incentives)

51 Effective implementation Most interventions are effective in some situations; no intervention is effective in all situations Effect on professional behavior is larger than on patient outcome Combined strategies are likely better than single strategy

52 Burgers JS. Quality of clinical practice guidelines (thesis). Nijmegen, In sum: key features of good guideline programs People involved in guideline development - credible organisation responsible for guideline development - target users involved in guideline development ('ownership') - balanced multidisciplinary guideline development group - patient involvement at any stage of the development process Methodology of guideline development - systematic review of the literature, including existing high-quality guidelines - combining evidence linkage and expert consensus in formulating recommendations - external peer review - formal update procedure - use of quality criteria for guidelines and guideline development Dissemination and implementation strategy - production of different guideline formats, including patient versions, and tools for applications - use of the Internet - multiple implementation strategies

53 Guideline development and implementation is team work!

54 More information EBRO- manual (in Dutch): Mail to Thanks for your interest!!