Using Criteria to Appraise a Meta-analyses

Size: px
Start display at page:

Download "Using Criteria to Appraise a Meta-analyses"

Transcription

1 Using Criteria to Appraise a Meta-analyses Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S., M.R.C.P.I.,.F.R.C.R. Department of Radiology, Division of Cardiothoracic Radiology, University of Michigan, Ann Arbor, Michigan

2 AUR 59th Annual Meeting Westin Boston Waterfront, Boston, Massachusetts April 12-15, 2011 March 23-26, 2010

3 Critical Appraisal of Secondary Research Reviews of the scientific literature Selective summaries Highly subjective Objective summaries Quantitative information syntheses of the best research evidence The challenge is where along this spectrum the review lies

4 Principles of Critical Appraisal Secondary Literature Systematic review and Meta-analysis of diagnostic test accuracy (1) Finding the evidence Framing objectives of the review Identifying the relevant literature (2) Assessment of study quality and applicability to the clinical problem at hand

5 Principles of Critical Appraisal Secondary Literature Systematic review and Meta-analysis of diagnostic test accuracy (3) Summarizing the evidence Qualitatively and if appropriate, quantitatively (meta-analysis) (4) Clinical Interpretation Application of findings and development of recommendations

6 Questions Question 1: What is the PICO of the study and is it close enough to your PICO? Question 2: How well was the study done? Question 3: What do the results mean and could they have been due to chance?

7 Question 1 What is the PICO of the study and is it close enough to your PICO?

8 Finding the Evidence Defining the research question P Population / problem I Intervention / index test C Control / reference test O Outcome

9 Identifying the Evidence Literature search strategy 1. The target condition of interest (all subheadings) 2. The name of the test (all subheadings) and text 3. Combine the results of these searches with Boolean operator AND 4. Further search terms to restrict the search should only be introduced if this approach produces unmanageably large and nonspecific results

10 Question 2 How well was the study done?

11 Biases Studies that are chosen for inclusion Selection bias Published in the first place Publication bias Level of importance attributed to the study results by the secondary researcher Results are summarized

12 QFAS Steps EBP Secondary research Critical appraisal issue Q Formulate a question (PICO) Formulate a question (PICO) Does the research ask a clearly focused question (PICO) and use it to direct the search? F Find the best evidence Find the best evidence Did the search find all the best evidence? A Appraise the included studies Appraise the included studies Have the studies been critically appraised? S Synthesis the results (summary tables and plots) Have the results been synthesized with appropriate summary tables and plots?

13 Question Does the research ask a clearly focused question (PICO) and use it to direct the search? Excluded studies should be recorded with reasons for the exclusions Helps to eliminate selection bias

14 Find Search strategy Did the search find all the best evidence? Use PubMed for MEDLlNE, OLDMEDLINE, EMBASE, OVID, CANCERLIT, and the Cochrane Library databases, etc

15 Find Search strategy Did the search find all the best evidence? Searching of relevant journals Conference proceedings Reference lists of articles in the searches Non-English as well as English papers Overcoming selection bias

16 Find Overcoming publication bias Much primary research missed because of publication bias Reluctant to submit/publish null or negative results Good-quality analysis takes account of unpublished studies

17 Find Overcoming publication bias Discussed in the methods section Check Clinical trials registries Contacting experts Conference proceedings Internet Other sources of unpublished literature

18 Appraise Have the studies been critically appraised (using appropriate quality criteria)? Searching has been well done Validity of the results and conclusions depends on the quality of the individual studies included Good-quality analysis should include a critical appraisal for each of the studies

19 Appraise Have the studies been critically appraised (using appropriate quality criteria)? This ideally should be appraised by two assessors working independently Points of disagreement discussed and a consensus reached Critical assessment can be subjective - avoided a single reviewer

20 Quality Appraisal of Evidence 1. Methods and criteria used for the selection of the study population 2. Selection of the reference standard 3. Execution of index test strategy, comparator and reference standard 4. Interpretation of index test, comparator and reference standard 5. Analysis of the results

21 Quality Appraisal of Evidence QUADAS QUality Assessment Diagnostic Accuracy Studies

22 The QUADAS tool Yes No Unclear 1. Was the spectrum of patients representative of the patients who will receive the test in practice? ( ) ( ) ( ) 2. Were selection criteria clearly described? ( ) ( ) ( ) 3. Is the reference standard likely to correctly classify the target condition? ( ) ( ) ( ) Is the time period between reference standard and index test short enough to be reasonably sure that the target condition did not change between the two tests? Did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis? ( ) ( ) ( ) ( ) ( ) ( ) 6. Did patients receive the same reference standard regardless of the index test result? ( ) ( ) ( ) 7. Was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard)? ( ) ( ) ( ) 8. Was the execution of the index test described in sufficient detail to permit replication of the test? ( ) ( ) ( ) 9. Was the execution of the reference standard described in sufficient detail to permit its replication? ( ) ( ) ( ) 10. Were the index test results interpreted without knowledge of the results of the reference standard? ( ) ( ) ( ) 11. Were the reference standard results interpreted without knowledge of the results of the index test? ( ) ( ) ( ) 12. Were the same clinical data available when test results were interpreted as would be available when the test is used in practice? ( ) ( ) ( ) 13. Were uninterpretable/ intermediate test results reported? ( ) ( ) ( ) 14. Were withdrawals from the study explained? ( ) ( ) ( )

23

24 Synthesis Have the results been synthesized with appropriate summary tables and plots? Summary of the included studies (table) Show the types of studies Tests Number of subjects Results (95% confidence intervals) Graphic presentations forest plots

25 Synthesis Have the results been synthesized with appropriate summary tables and plots? Heterogeneity should be assessed and discussed

26 Question 3 What do the results mean?

27 Results Weighted values to each individual study according to their size Expressed in a standard way Sensitivity, specificity, PPV, NPV, AUROC curve RR, OR, etc Traditionally displayed in a figure called a forest plot

28

29 Forest Plot Individual studies are represented by a black square and a horizontal line Corresponds to point estimate and 95% CI The size (area) of the black square reflects the weight of the study Weight is proportional to the inverse of the study variance

30 Forest Plot The result is not significant at conventional levels (P > 0.05) The diamond at the bottom represents the combined or pooled result of all studies with its 95% CI

31

32 Statistical Methods for Pooling Data SROC Hierarchical summary ROC (HSROC) Bivariate randomeffects model

33 Summary ROC Curve

34 Exploring Heterogeneity Heterogeneity can be assessed using the 'eyeball' test Formally with statistical tests, such as I 2 and the Cochran chi-square (Q) test The amount of heterogeneity is calculated as the I 2 value 0 if none near 1 if a lot

35 Exploring Heterogeneity The statistical significance is assessed using the Cochran Q test If Cochran Q is statistically significant - there is definite heterogeneity If Cochran Q is not statistically significant but the ratio of Cochran Q and the degrees of freedom (Q/df) is greater than 1 - there is possible heterogeneity

36 Exploring Heterogeneity The statistical significance is assessed using the Cochran Q test If Cochran Q is not statistically significant and Q/df is less then 1 - then heterogeneity is very unlikely

37 Exploring Publication Bias Funnel plot +/- linear regression

38 Assessment of Publication Bias Funnel plot Non-parametric linear regression Parametric linear regression formally test for funnel plot asymmetry

39 Clinical Interpretation Fagan plot (Bayes Nomogram) of the Evidence

40 Clinical Interpretation Likelihood ratio scattergraph 1. Left Upper Quadrant, Likelihood Ratio Positive > 10, Likelihood Ratio Negative <0.1: Exclusion & Confirmation 2. Right Upper Quadrant, Likelihood Ratio Positive >10, Likelihood Ratio Negative >0.1: Confirmation Only 3. Left Lower Quadrant, Likelihood Ratio Positive <10, Likelihood Ratio Negative <0.1: Exclusion Only 4. Right Lower Quadrant, Likelihood Ratio Positive <10, Likelihood Ratio Negative >0.1: No Exclusion or Confirmation of the Evidence

41 Clinical Interpretation Predictive Values and Probability Modifying Plot of the Evidence

42 Probability Modifying Plot

43 Section/topic # Preferred Reporting Items for Systematic Reviews and Meta-Analyses Checklist item TITLE Title ABSTRACT Structured summary INTRODUCTION Rationale Objectives METHODS Protocol and registration Eligibility criteria Information sources Search Study selection Data collection process Data items Risk of bias in individual studies Summary measures Synthesis of results 1 Identify the report as a systematic review, meta-analysis, or both. 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. 3 Describe the rationale for the review in the context of what is already known. 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS). 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number. 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated. 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators. 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made. 12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis. 13 State the principal summary measures (e.g., risk ratio, difference in means). 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I 2 ) for each meta-analysis. Reported on page #

44 Section/topic Risk of bias across studies Additional analyses RESULTS Study selection Study characteristics Risk of bias within studies Results of individual studies Synthesis of results Risk of bias across studies Additional analysis DISCUSSION Summary of evidence Limitations Conclusions FUNDING Funding # Checklist item 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies). 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified. 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram. 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, followup period) and provide the citations. 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot. 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. 22 Present results of any assessment of risk of bias across studies (see Item 15). 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers). 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias). 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review. Reported on page #

45 AMSTAR 2

46

47 Summary Question The review is a systematic review based on a clear clinical question and the PICO is similar to ours

48 Summary Find The inclusion criteria and search methods are stated in the methods section Inclusion criteria were based on the clinical question A comprehensive search of the literature was conducted, including MEDLINE (PubMed) and EMBASE

49 Summary Find Contacted the authors of included papers directly and checked the reference lists for further relevant papers They also searched the Cochrane controlled trial register for unpublished clinical trials

50 Summary Appraise The studies were critically appraised using the QUODAS, DELPHI, Jadad quality scores Scores are shown in the paper

51 Summary Synthesise The paper includes a clear summary table of the included studies Forest plots Heterogeneity analysis for the outcomes Publication bias Clinical interpretation of the evidence

52 Using Criteria to Appraise a Meta-analyses Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S., M.R.C.P.I.,.F.R.C.R. Department of Radiology, Division of Cardiothoracic Radiology, University of Michigan, Ann Arbor, Michigan

PCORI Methodology Standards: Academic Curriculum. 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.

PCORI Methodology Standards: Academic Curriculum. 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved. PCORI Methodology Standards: Academic Curriculum 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved. Module 5: Step 3 Search the Literature Category 11: Systematic Reviews Prepared

More information

Systematic Reviews and Meta-analyses

Systematic Reviews and Meta-analyses Systematic Reviews and Meta-analyses Introduction A systematic review (also called an overview) attempts to summarize the scientific evidence related to treatment, causation, diagnosis, or prognosis of

More information

Can I have FAITH in this Review?

Can I have FAITH in this Review? Can I have FAITH in this Review? Find Appraise Include Total Heterogeneity Paul Glasziou Centre for Research in Evidence Based Practice Bond University What do you do? For an acutely ill patient, you do

More information

Methods for Meta-analysis in Medical Research

Methods for Meta-analysis in Medical Research Methods for Meta-analysis in Medical Research Alex J. Sutton University of Leicester, UK Keith R. Abrams University of Leicester, UK David R. Jones University of Leicester, UK Trevor A. Sheldon University

More information

Training Program in Meta-Analysis

Training Program in Meta-Analysis Training Program in Meta-Analysis June 23-25, 2015 The University of Arizona College of Pharmacy Tucson, Arizona A 3-day, hands-on training program for personnel in healthcare decisionmaking, industry

More information

Guidance notes on planning a systematic review

Guidance notes on planning a systematic review James Hardiman Library Guidance notes on planning a systematic review In contrast to the traditional or narrative literature review, systematic literature reviews use a more rigorous and well-defined approach

More information

LEVEL ONE MODULE EXAM PART ONE [Clinical Questions Literature Searching Types of Research Levels of Evidence Appraisal Scales Statistic Terminology]

LEVEL ONE MODULE EXAM PART ONE [Clinical Questions Literature Searching Types of Research Levels of Evidence Appraisal Scales Statistic Terminology] 1. What does the letter I correspond to in the PICO format? A. Interdisciplinary B. Interference C. Intersession D. Intervention 2. Which step of the evidence-based practice process incorporates clinical

More information

Weighing the Evidence: Critical Appraisal and Systematic Review of RCTs

Weighing the Evidence: Critical Appraisal and Systematic Review of RCTs Weighing the Evidence: Critical Appraisal and Systematic Review of RCTs Dr. Rosie Mayston, Centre for Global Mental Health, Institute of Psychiatry, King s College London Overview Today s session aims

More information

Secondary Data &Meta-Analysis

Secondary Data &Meta-Analysis Secondary Data &Meta-Analysis Secondary Data - Overview Secondary data: data that have already been collected for a different purpose, but may be relevant to the research problems at hand. Primary data:

More information

Systematic reviews and meta-analysis

Systematic reviews and meta-analysis Evidence-Based Medicine And Healthcare Singapore Med J 2005 Vol 46(6) : 270 CME Article Systematic reviews and meta-analysis S Green ABSTRACT Systematic reviews form a potential method for overcoming the

More information

Biostat Methods STAT 5820/6910 Handout #6: Intro. to Clinical Trials (Matthews text)

Biostat Methods STAT 5820/6910 Handout #6: Intro. to Clinical Trials (Matthews text) Biostat Methods STAT 5820/6910 Handout #6: Intro. to Clinical Trials (Matthews text) Key features of RCT (randomized controlled trial) One group (treatment) receives its treatment at the same time another

More information

Outline. Publication and other reporting biases; funnel plots and asymmetry tests. The dissemination of evidence...

Outline. Publication and other reporting biases; funnel plots and asymmetry tests. The dissemination of evidence... Cochrane Methodology Annual Training Assessing Risk Of Bias In Cochrane Systematic Reviews Loughborough UK, March 0 Publication and other reporting biases; funnel plots and asymmetry tests Outline Sources

More information

EVIPNet Capacity-Building Workshop Ethiopian Health and Nutrition Research Institute, Addis Ababa, Ethiopia 18 to 22 February 2008

EVIPNet Capacity-Building Workshop Ethiopian Health and Nutrition Research Institute, Addis Ababa, Ethiopia 18 to 22 February 2008 EVIPNet Capacity-Building Workshop Ethiopian Health and Nutrition Research Institute, Addis Ababa, Ethiopia 18 to 22 February 2008 Assessment Criteria for Systematic Reviews (Last updated by John Lavis

More information

Formulating the Review Question & Writing a Protocol

Formulating the Review Question & Writing a Protocol Formulating the Review Question & Writing a Protocol Madhukar Pai, MD, PhD Associate Professor Department of Epidemiology & Biostatistics McGill University, Montreal, Canada Email: madhukar.pai@mcgill.ca

More information

Principles of Systematic Review: Focus on Alcoholism Treatment

Principles of Systematic Review: Focus on Alcoholism Treatment Principles of Systematic Review: Focus on Alcoholism Treatment Manit Srisurapanont, M.D. Professor of Psychiatry Department of Psychiatry, Faculty of Medicine, Chiang Mai University For Symposium 1A: Systematic

More information

Aim to determine whether, and in what circumstances, treatment a is better than treatment b in acute ischaemic stroke

Aim to determine whether, and in what circumstances, treatment a is better than treatment b in acute ischaemic stroke How to write a paper systematic review Note please see Advice on how to write a paper for information on general aspects of manuscript preparation, which should be read before you start to write up your

More information

Interpretation of forest plots Part I

Interpretation of forest plots Part I Interpretation of forest plots Part I 1 At the end of this lecture, you should be able to understand the principles and uses of forest plot. You should also be able to modify the forest plots. 2 What is

More information

Evidence Based Practice Information Sheets for Health Professionals. An Introduction to Systematic Reviews. This Sheet Covers the Following Concepts:

Evidence Based Practice Information Sheets for Health Professionals. An Introduction to Systematic Reviews. This Sheet Covers the Following Concepts: Supplement 1, 2001 ChangingPractice Evidence Based Practice Information Sheets for Health Professionals An Introduction to Systematic Reviews The series Changing Practice has been designed to support health

More information

The systematic review of studies of diagnostic test accuracy

The systematic review of studies of diagnostic test accuracy The Joanna Briggs Institute Reviewers Manual 2015 The systematic review of studies of diagnostic test accuracy www.joannabriggs.org Joanna Briggs Institute Reviewers Manual: 2015 edition/supplement Copyright

More information

DIAGNOSTIC TEST APPRAISAL FORM

DIAGNOSTIC TEST APPRAISAL FORM DIAGNOSTIC TEST APPRAISAL FORM SCREENING AND TEST EVALUATION PROGRAM (http://www.health.usyd.edu.au/step/) SCHOOL OF PUBLIC HEALTH, UNIVERSITY OF SYDNEY COPYRIGHT This appraisal form may be freely used

More information

Journal Article Reporting Standards (JARS)

Journal Article Reporting Standards (JARS) APPENDIX Journal Article Reporting Standards (JARS), Meta-Analysis Reporting Standards (MARS), and Flow of Participants Through Each Stage of an Experiment or Quasi-Experiment 245 Journal Article Reporting

More information

Web appendix: Supplementary material. Appendix 1 (on-line): Medline search strategy

Web appendix: Supplementary material. Appendix 1 (on-line): Medline search strategy Web appendix: Supplementary material Appendix 1 (on-line): Medline search strategy exp Venous Thrombosis/ Deep vein thrombosis.mp. Pulmonary embolism.mp. or exp Pulmonary Embolism/ recurrent venous thromboembolism.mp.

More information

Guidelines for AJO-DO submissions: Randomized Clinical Trials June 2015

Guidelines for AJO-DO submissions: Randomized Clinical Trials June 2015 Guidelines for AJO-DO submissions: Randomized Clinical Trials June 2015 Complete and transparent reporting allows for accurate assessment of the quality of trial and correct interpretation of the trial

More information

ChangingPractice. Appraising Systematic Reviews. Evidence Based Practice Information Sheets for Health Professionals. What Are Systematic Reviews?

ChangingPractice. Appraising Systematic Reviews. Evidence Based Practice Information Sheets for Health Professionals. What Are Systematic Reviews? Supplement 1, 2000 ChangingPractice Evidence Based Practice Information Sheets for Health Professionals Appraising Systematic Reviews The series Changing Practice has been designed to support health professionals

More information

PCORI Methodology Standards: Academic Curriculum. 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.

PCORI Methodology Standards: Academic Curriculum. 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved. PCORI Methodology Standards: Academic Curriculum 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved. Module 9: Putting it All Together: Appraising a Systematic Review Category 11: Systematic

More information

Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics.

Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics. Business Course Text Bowerman, Bruce L., Richard T. O'Connell, J. B. Orris, and Dawn C. Porter. Essentials of Business, 2nd edition, McGraw-Hill/Irwin, 2008, ISBN: 978-0-07-331988-9. Required Computing

More information

A Guide To Producing an Evidence-based Report

A Guide To Producing an Evidence-based Report A Guide To Producing an Evidence-based Report by Dr. James L. Leake Professor and Head, Community Dentistry, Dept. of Biological and Diagnostic Sciences, Faculty of Dentistry University of Toronto Evidence-based

More information

PEER REVIEW HISTORY ARTICLE DETAILS TITLE (PROVISIONAL)

PEER REVIEW HISTORY ARTICLE DETAILS TITLE (PROVISIONAL) PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf)

More information

Characteristics of studies

Characteristics of studies PICO 6 for ALKO DOB FINAL Review information Authors [Empty name] 1 1 [Empty affiliation] Citation example: [Empty name]. PICO 6 for ALKO DOB FINAL. Cochrane Database of Systematic Reviews [Year], Issue

More information

Protocols. Gary Collins. EQUATOR Network, Centre for Statistics in Medicine NDORMS, University of Oxford

Protocols. Gary Collins. EQUATOR Network, Centre for Statistics in Medicine NDORMS, University of Oxford Protocols Gary Collins EQUATOR Network, Centre for Statistics in Medicine NDORMS, University of Oxford EQUATOR OUCAGS training course 4 October 2014 Session outline What is a study protocol? Why is a protocol

More information

18/11/2013. Getting the Searches off to a good start: Scoping the Literature and Devising a Search Strategy

18/11/2013. Getting the Searches off to a good start: Scoping the Literature and Devising a Search Strategy Getting the Searches off to a good start: Scoping the Literature and Devising a Search Strategy Sheila Fisken Liaison Librarian Information Services University of Edinburgh Sheila.Fisken@ed.ac.uk Chaotic

More information

GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES

GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES GUIDELINES FOR REVIEWING QUANTITATIVE DESCRIPTIVE STUDIES These guidelines are intended to promote quality and consistency in CLEAR reviews of selected studies that use statistical techniques and other

More information

Evidence based Dental Practice: Systematic Reviews

Evidence based Dental Practice: Systematic Reviews Evidence based Dental Practice: Systematic Reviews Joana Cunha-Cruz Oral Health Sciences School of Dentistry University of Washington Seattle, March 2013 Outline Background Definition and importance of

More information

Critical Appraisal of a Randomized Trial

Critical Appraisal of a Randomized Trial Critical Appraisal of a Randomized Trial Presented by Brian S. Alper, MD, MSPH, FAAFP Founder of DynaMed; Vice President of Evidence-Based Medicine Research and Development, Quality and Standards Introduction

More information

Predictive Values, Sensitivity and Specificity in Clinical Virology. Alison M Kesson Infectious Diseases and Microbiology 2009

Predictive Values, Sensitivity and Specificity in Clinical Virology. Alison M Kesson Infectious Diseases and Microbiology 2009 Predictive Values, Sensitivity and Specificity in Clinical Virology Alison M Kesson Infectious Diseases and Microbiology 2009 Reality Diagnostic tests are never perfect. False positive and false negative

More information

Systematic Reviews and Clinical Practice Guidelines

Systematic Reviews and Clinical Practice Guidelines CHAPTER 11 Systematic Reviews and Clinical Practice Guidelines Geri LoBiondo-Wood Go to Evolve at http://evolve.elsevier.com/lobiondo/ elsevier for review questions, critiquing exercises, and additional

More information

Critical appraisal of systematic reviews

Critical appraisal of systematic reviews Critical appraisal of systematic reviews Abalos E, Carroli G, Mackey ME, Bergel E Centro Rosarino de Estudios Perinatales, Rosario, Argentina INTRODUCTION In spite of the increasingly efficient ways to

More information

Data Extraction and Quality Assessment

Data Extraction and Quality Assessment Data Extraction and Quality Assessment Karen R Steingart, MD, MPH Francis J. Curry International Tuberculosis Center University of California, San Francisco Berlin, 12 November 2010 karenst@uw.edu The

More information

An Introduction to the Critical Appraisal Section (including example questions)

An Introduction to the Critical Appraisal Section (including example questions) An Introduction to the Critical Appraisal Section (including example questions) Introduction This section is common to the specialties of Dental Public Health, Oral Medicine, Oral Surgery, Orthodontics,

More information

Course Text. Required Computing Software. Course Description. Course Objectives. StraighterLine. Business Statistics

Course Text. Required Computing Software. Course Description. Course Objectives. StraighterLine. Business Statistics Course Text Business Statistics Lind, Douglas A., Marchal, William A. and Samuel A. Wathen. Basic Statistics for Business and Economics, 7th edition, McGraw-Hill/Irwin, 2010, ISBN: 9780077384470 [This

More information

WHAT IS A JOURNAL CLUB?

WHAT IS A JOURNAL CLUB? WHAT IS A JOURNAL CLUB? With its September 2002 issue, the American Journal of Critical Care debuts a new feature, the AJCC Journal Club. Each issue of the journal will now feature an AJCC Journal Club

More information

If several different trials are mentioned in one publication, the data of each should be extracted in a separate data extraction form.

If several different trials are mentioned in one publication, the data of each should be extracted in a separate data extraction form. General Remarks This template of a data extraction form is intended to help you to start developing your own data extraction form, it certainly has to be adapted to your specific question. Delete unnecessary

More information

Checklist for Quasi-Experimental Studies (non-randomized experimental studies)

Checklist for Quasi-Experimental Studies (non-randomized experimental studies) The Joanna Briggs Institute Critical Appraisal tools for use in JBI Systematic Reviews Checklist for Quasi-Experimental Studies (non-randomized experimental studies) http://joannabriggs.org/research/critical-appraisal-tools.html

More information

Guide for Rating of Strength of Recommendations

Guide for Rating of Strength of Recommendations Adjuvant Endocrine Therapy for Women with Hormone Receptor-Positive Breast Cancer: American Society of Clinical Oncology Clinical Practice Guideline Focused Update Table of Contents 1. OVERVIEW: CLINICAL

More information

BIOSTATISTICS QUIZ ANSWERS

BIOSTATISTICS QUIZ ANSWERS BIOSTATISTICS QUIZ ANSWERS 1. When you read scientific literature, do you know whether the statistical tests that were used were appropriate and why they were used? a. Always b. Mostly c. Rarely d. Never

More information

Assessing MOHLTC Research Protocols Secondary Data Studies By: Tommy Tam, MSc

Assessing MOHLTC Research Protocols Secondary Data Studies By: Tommy Tam, MSc Assessing MOHLTC Research Protocols Secondary Data Studies By: Tommy Tam, MSc Definition of secondary data study: Secondary data studies use data source(s) collected by someone other than the user for

More information

Finding and Evaluating Evidence: Systematic Reviews and Evidence-Based Practice

Finding and Evaluating Evidence: Systematic Reviews and Evidence-Based Practice University Press Scholarship Online You are looking at 1-10 of 54 items for: keywords : systematic reviews Systematic Reviews and Meta-Analysis acprof:oso/9780195326543.001.0001 This book aims to make

More information

Understanding, appraising and reporting meta-analyses that use individual participant data

Understanding, appraising and reporting meta-analyses that use individual participant data Understanding, appraising and reporting meta-analyses that use individual participant data Jayne Tierney, Claire Vale, Maroeska Rovers, Lesley Stewart IPD Meta-analysis Methods Group 21 st Annual Cochrane

More information

TITLE: Cannabinoids for the Treatment of Post-Traumatic Stress Disorder: A Review of the Clinical Effectiveness and Guidelines

TITLE: Cannabinoids for the Treatment of Post-Traumatic Stress Disorder: A Review of the Clinical Effectiveness and Guidelines TITLE: Cannabinoids for the Treatment of Post-Traumatic Stress Disorder: A Review of the Clinical Effectiveness and Guidelines DATE: 27 June 2012 CONTEXT AND POLICY ISSUES In 2009, CADTH reviewed the clinical

More information

Tools for a critical appraisal of evidences from diagnostic Test Accuracy studies

Tools for a critical appraisal of evidences from diagnostic Test Accuracy studies Tools for a critical appraisal of evidences from diagnostic Test Accuracy studies evidence the movement was officially launched in 1992 in the Journal of American Medical Association Evidence-Based Medicine

More information

FOCUSSED CLINICAL QUESTION:

FOCUSSED CLINICAL QUESTION: The use of a multifactorial falls risk assessment and management plan reduces the risk of falling and the monthly fall rate of older adults, and is the most effective component of a falls prevention programme

More information

Appendix 5. Systematic Review Proposal Outline

Appendix 5. Systematic Review Proposal Outline Appendix 5 Systematic Review Proposal Outline Developing a Protocol for a Systematic Review Using the Joanna Briggs Institute (JBI) Method The New Jersey Centre for Evidenced Based Healthcare and Patient-Outcomes

More information

Systematic review of a health system intervention:

Systematic review of a health system intervention: Systematic review of a health system intervention: lessons learnt from a review of results-based financing approaches in Africa and Asia. Emma Jolley, Juliet Milgate and Elena Schmidt Format 1. Introduction

More information

Phone: 020 79115334 Email: j.tripney@ioe.ac.uk

Phone: 020 79115334 Email: j.tripney@ioe.ac.uk Title Registration for a Systematic Review: Post-basic technical and vocational education and training (TVET) interventions to improve employability and employment of TVET graduates in low- and middle-income

More information

The SARH (DFID) Systematic Review (SR) Programme for South Asia About systematic reviews

The SARH (DFID) Systematic Review (SR) Programme for South Asia About systematic reviews The SARH (DFID) Systematic Review (SR) Programme for South Asia About systematic reviews What are systematic reviews? A systematic review is a high-level overview of primary research on a particular research

More information

Overview, Strengths, and Limitations of Systematic Reviews and Meta-Analyses

Overview, Strengths, and Limitations of Systematic Reviews and Meta-Analyses Overview, Strengths, and Limitations of Systematic Reviews and Meta-Analyses 2 Alfred A. Bartolucci and William B. Hillegass Core Message It is important and timely to provide some historical context,

More information

ASKING CLINICAL QUESTIONS

ASKING CLINICAL QUESTIONS MODULE 1: ASKING CLINICAL QUESTIONS There are two main types of clinical questions: background questions and foreground questions. Background questions are those which apply to the clinical topic for all

More information

Overview. EBCP Terminology

Overview. EBCP Terminology EBCP Terminology Absolute risk reduction (ARR) The absolute arithmetic difference in rates of outcomes between experimental and control groups in a trial, calculated as EER-CER. Bias A systematic tendency

More information

Low-fat or low-carb diet for long-term weight loss Student EBM presentations

Low-fat or low-carb diet for long-term weight loss Student EBM presentations Low-fat or low-carb diet for long-term weight loss Student EBM presentations Kin Seng Tong & Freya Glover University of Oxford 6 October 2015 The clinical scenario Mr. Smith is a 25-year-old receptionist

More information

Promoting professional behaviour change in healthcare what interventions work, and why? Protocol for a theory-led overview of systematic reviews

Promoting professional behaviour change in healthcare what interventions work, and why? Protocol for a theory-led overview of systematic reviews Promoting professional behaviour change in healthcare what interventions work, and why? Protocol for a theory-led overview of systematic reviews Mark J Johnson, Carl R May Abstract Background: An overview

More information

The Big 50 Revision Guidelines for S1

The Big 50 Revision Guidelines for S1 The Big 50 Revision Guidelines for S1 If you can understand all of these you ll do very well 1. Know what is meant by a statistical model and the Modelling cycle of continuous refinement 2. Understand

More information

Critical Appraisal of Article on Therapy

Critical Appraisal of Article on Therapy Critical Appraisal of Article on Therapy What question did the study ask? Guide Are the results Valid 1. Was the assignment of patients to treatments randomized? And was the randomization list concealed?

More information

Assessing study quality: critical appraisal. MRC/CSO Social and Public Health Sciences Unit

Assessing study quality: critical appraisal. MRC/CSO Social and Public Health Sciences Unit Assessing study quality: critical appraisal MRC/CSO Social and Public Health Sciences Unit Assessing study quality: why? Why critical appraise? Evidence informed decision-making assumes people make better

More information

Epidemiology. Learning Objectives

Epidemiology. Learning Objectives Epidemiology David G. Weismiller, MD, ScM, FAAFP Professor Department of Family Medicine The rody School of Medicine at East Carolina University Greenville, North Carolina Learning Objectives 1. Explain

More information

Confidence Intervals for the Area Under an ROC Curve

Confidence Intervals for the Area Under an ROC Curve Chapter 261 Confidence Intervals for the Area Under an ROC Curve Introduction Receiver operating characteristic (ROC) curves are used to assess the accuracy of a diagnostic test. The technique is used

More information

Inferential Statistics

Inferential Statistics Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

More information

Guidelines for Preparation of Review Protocols

Guidelines for Preparation of Review Protocols Guidelines for Preparation of Review Protocols Type of document: Policy _x_ Guideline Procedure Version: 1.0, January 1 2001 Decision: Steering Group, Date? A Campbell Systematic Review is meant to review

More information

Critical appraisal form

Critical appraisal form Assessing scientific admissibility and merit of published articles Critical appraisal form Section A: Reference of Article Author(s) and Affiliation(s): Title of Article: Journal: Volume and Page Numbers:

More information

Systematic Reviews in JNEB

Systematic Reviews in JNEB Systematic Reviews in JNEB Introduction 3 Associate Editors and 3 Board of Editors members Panel then small workshop tables Evaluation, please Thank you to Elsevier for refreshments! KCN Systematic Reviews

More information

The Joanna Briggs Institute Reviewers Manual 2014. Methodology for JBI Umbrella Reviews

The Joanna Briggs Institute Reviewers Manual 2014. Methodology for JBI Umbrella Reviews The Joanna Briggs Institute Reviewers Manual 2014 Methodology for JBI Umbrella Reviews Joanna Briggs Institute Reviewers Manual: 2014 edition/supplement Copyright The Joanna Briggs Institute 2014 The Joanna

More information

Guidance for the format and content of the protocol of non-interventional post-authorisation safety studies

Guidance for the format and content of the protocol of non-interventional post-authorisation safety studies 26 September 2012 EMA/623947/2012 Patient Health Protection Guidance for the format and content of the protocol of non-interventional post-authorisation Introduction From 10 January 2013, marketing authorisation

More information

When Does it Make Sense to Perform a Meta-Analysis?

When Does it Make Sense to Perform a Meta-Analysis? CHAPTER 40 When Does it Make Sense to Perform a Meta-Analysis? Introduction Are the studies similar enough to combine? Can I combine studies with different designs? How many studies are enough to carry

More information

The Literature Review in the Masters Dissertation. Roberta Sammut

The Literature Review in the Masters Dissertation. Roberta Sammut The Literature Review in the Masters Dissertation Roberta Sammut If I have seen further it is by standing on the shoulders of giants Isaac Newton Why are reviews needed Research information is like small

More information

Protocol registration and outcome reporting bias in randomised controlled trials of

Protocol registration and outcome reporting bias in randomised controlled trials of Title: Protocol registration and outcome reporting bias in randomised controlled trials of eczema treatment Rationale: As one of the measures to reduce the likelihood of selective reporting bias in RCTs,

More information

Evidence-Based Nursing Practice Toolkit

Evidence-Based Nursing Practice Toolkit Evidence-Based Nursing Practice Toolkit Translating Research into Practice Adapted for SRHS by: Lucy Gansauer, MSN, RN, OCN & Sherri Stroud, MSN, RN SRHS Nursing Evidence Based Practice Model Organizational

More information

Introduction to Statistical Methods in Meta-Analysis. Jian Wang October 31, 2013

Introduction to Statistical Methods in Meta-Analysis. Jian Wang October 31, 2013 Introduction to Statistical Methods in Meta-Analysis Jian Wang October 31, 2013 What is meta-analysis? Meta-analysis: the statistical synthesis of information from multiple independent studies. Increase

More information

Title of critically appraised topic: Is probiotic use an effective adjuvant therapy to antibiotics in

Title of critically appraised topic: Is probiotic use an effective adjuvant therapy to antibiotics in Title of critically appraised topic: Is probiotic use an effective adjuvant therapy to antibiotics in treatment of Clostridium difficile-associated diarrhea? Author: (This copy has been stripped of identifiers)

More information

Effective Healthcare and The Cochrane Collaboration: Impact

Effective Healthcare and The Cochrane Collaboration: Impact Effective Healthcare and The Cochrane Collaboration: The Vision, The Output, The Impact Dr. Bruce Walker Senior Lecturer in Evidence based practice and Public health School of Chiropractic and Sports Science

More information

Systematic Reviews: Database Selection, Search Strategies & Reference Management Christopher Stave, MLS

Systematic Reviews: Database Selection, Search Strategies & Reference Management Christopher Stave, MLS Systematic Reviews: Database Selection, Search Strategies & Reference Management Christopher Stave, MLS Instructional Program Coordinator Lane Medical Library & Knowledge Management Center Objectives What

More information

Secondary Research in Health Research

Secondary Research in Health Research Secondary Research in Health Research A/Prof Edwin S-Y Chan, PhD Centre for Quantitative Medicine, Office of Clinical Sciences, Duke-NUS Graduate Medical School Head of Epidemiology, Singapore Clinical

More information

Does Selenium protect against lung cancer? Do Selenium supplements reduce the incidence of lung cancer in healthy individuals?

Does Selenium protect against lung cancer? Do Selenium supplements reduce the incidence of lung cancer in healthy individuals? Does Selenium protect against lung cancer? Do Selenium supplements reduce the incidence of lung cancer in healthy individuals? Alexander Blakes and Eleanor Brunt October 2014 The question Mr Wilson s father

More information

Technology Step-by-Step Using StatCrunch

Technology Step-by-Step Using StatCrunch Technology Step-by-Step Using StatCrunch Section 1.3 Simple Random Sampling 1. Select Data, highlight Simulate Data, then highlight Discrete Uniform. 2. Fill in the following window with the appropriate

More information

NKR 13 Alkoholbehandling. Disulfiram for alcohol dependency

NKR 13 Alkoholbehandling. Disulfiram for alcohol dependency NKR 13 Alkoholbehandling. Disulfiram for alcohol dependency Review information Authors Sundhedsstyrelsen 1 1 [Empty affiliation] Citation example: S. NKR 13 Alkoholbehandling. Disulfiram for alcohol dependency.

More information

Transferability of Economic Evaluations in Clinical Trials

Transferability of Economic Evaluations in Clinical Trials Transferability of Economic Evaluations in Clinical Trials Henry Glick Institutt for helseledelse og helse√łkonomi November 25, 2008 The Problem Multicenter and multinational studies are the norm for the

More information

Treatment of seizures in multiple sclerosis (Review)

Treatment of seizures in multiple sclerosis (Review) Koch MW, Polman SKL, Uyttenboogaart M, De Keyser J This is a reprint of a Cochrane review, prepared and maintained by The Cochrane Collaboration and published in The Cochrane Library 009, Issue 3 http://www.thecochranelibrary.com

More information

HTA. Executive summary. R Churchill 1* V Hunot 1 R Corney 2 M Knapp 1 H McGuire 1 A Tylee 1 S Wessely 1

HTA. Executive summary. R Churchill 1* V Hunot 1 R Corney 2 M Knapp 1 H McGuire 1 A Tylee 1 S Wessely 1 A systematic review of controlled trials of the effectiveness and cost-effectiveness of brief psychological treatments for depression The effectiveness of brief psychological treatments for depression

More information

Review question(s) Searches in the literature/sources:

Review question(s) Searches in the literature/sources: PROTOCOL Effective uses of social media in health care: a review of systematic reviews Dean Giustini, Mustafa Ali, Steve Wheeler, Maged N. Kamel Boulos Review question(s) What are the overall uses, benefits

More information

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence

Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence Survival Analysis Using SPSS By Hui Bian Office for Faculty Excellence Survival analysis What is survival analysis Event history analysis Time series analysis When use survival analysis Research interest

More information

Systematic. step overview. Disclosure. Diagnostics Working Group I consult for the Bill & Melinda Gates Foundation I serve on editorial boards of:

Systematic. step overview. Disclosure. Diagnostics Working Group I consult for the Bill & Melinda Gates Foundation I serve on editorial boards of: Systematic Reviews: step-by- step overview Madhukar Pai, MD, PhD Assistant Professor Department of Epidemiology & Biostatistics McGill University, Montreal, Canada Email: madhukar.pai@mcgill.ca Disclosure

More information

Study Design and Statistical Analysis

Study Design and Statistical Analysis Study Design and Statistical Analysis Anny H Xiang, PhD Department of Preventive Medicine University of Southern California Outline Designing Clinical Research Studies Statistical Data Analysis Designing

More information

New JBI Levels of Evidence

New JBI Levels of Evidence New JBI Levels of Evidence Developed by the Joanna Briggs Institute Levels of Evidence and Grades of Recommendation Working Party October 2013 PLEASE NOTE: These levels are intended to be used alongside

More information

Electronic Health Record-based Interventions for Reducing Inappropriate Imaging in the Clinical Setting: A Systematic Review of the Evidence

Electronic Health Record-based Interventions for Reducing Inappropriate Imaging in the Clinical Setting: A Systematic Review of the Evidence Department of Veterans Affairs Health Services Research & Development Service Electronic Health Record-based Interventions for Reducing Inappropriate Imaging in the Clinical Setting: A Systematic Review

More information

Session 5: How to critically appraise a paper an example

Session 5: How to critically appraise a paper an example Session 5: How to critically appraise a paper an example Appraising research broad questions l What is the research question? l Is the study ethical? l Is the study design valid and appropriate? l What

More information

SAMPLING & INFERENTIAL STATISTICS. Sampling is necessary to make inferences about a population.

SAMPLING & INFERENTIAL STATISTICS. Sampling is necessary to make inferences about a population. SAMPLING & INFERENTIAL STATISTICS Sampling is necessary to make inferences about a population. SAMPLING The group that you observe or collect data from is the sample. The group that you make generalizations

More information

Form B-1. Inclusion form for the effectiveness of different methods of toilet training for bowel and bladder control

Form B-1. Inclusion form for the effectiveness of different methods of toilet training for bowel and bladder control Form B-1. Inclusion form for the effectiveness of different methods of toilet training for bowel and bladder control Form B-2. Assessment of methodology for non-randomized controlled trials for the effectiveness

More information

Article Four Different Types of Evidence / Literature Reviews

Article Four Different Types of Evidence / Literature Reviews Article Four Different Types of Evidence / Literature Reviews The rapid growth in the number of reviews undertaken can partly be explained by the current emphasis on evidence-based practice. Healthcare

More information

Critical appraisal of a meta-analysis study

Critical appraisal of a meta-analysis study Critical appraisal of a meta-analysis study Samir Haffar M.D. Associate Professor of Gastroenterology Al-Mouassat University Hospital Damascus Syria Hierarchy of evidence in quantitative studies McGovern

More information

How to write a systematic literature review: a guide for medical students

How to write a systematic literature review: a guide for medical students How to write a systematic literature review: a guide for medical students Author: Rory J Piper, BMedSci(hons) Secretary, NSAMR, 2013 University of Edinburgh 2 How to write a systematic literature review:

More information

Breast cancer treatment for elderly women: a systematic review

Breast cancer treatment for elderly women: a systematic review Breast cancer treatment for elderly women: a systematic review Gerlinde Pilkington Rumona Dickson Anna Sanniti Funded by the NCEI and POI Elderly people less likely to receive chemotherapy than younger

More information

Fairfield Public Schools

Fairfield Public Schools Mathematics Fairfield Public Schools AP Statistics AP Statistics BOE Approved 04/08/2014 1 AP STATISTICS Critical Areas of Focus AP Statistics is a rigorous course that offers advanced students an opportunity

More information