Tools for a critical appraisal of evidences from diagnostic Test Accuracy studies

Transcription

1 Tools for a critical appraisal of evidences from diagnostic Test Accuracy studies

2 evidence the movement was officially launched in 1992 in the Journal of American Medical Association Evidence-Based Medicine Page 2

3 Page 3

4 Page 4 Da Price Orlando 2005

5 Critical appraisal Critical appraisal is the use of explicit, transparent methods to assess the data in published research, applying the rules of evidence to factors such as internal validity, adherence to reporting standards, conclusions and generalizability. Critical appraisal methods form a central part of the systematic review process. They are used in evidence-based healthcare training to assist clinical decision-making, and are increasingly used in evidence-based social care and education provision. Page 5

6 Evidence, Accuracy and EBM Accuracy Efficacy Evidences Disease? Diagnosis Known disease Treatment Positive outcome Page 6

7 EBM of treatments Randomized controlled Trials RCT Only one index: Relative risk (RR) or Odds Ratio (OR) Page 7

8 Measuring treatment effectiveness Outcome present Outcome absent Risk Treated A B EER=Y=A/(A+B) Not Treated C D CER=X=C/(C+D) Relative Risk RR = Y/X Odds Ratio (p / (1-p)) OR = (A/B)/(C/D) EER: Experimental Event Rate CER: Control Event Rate Page 8

9 Meta Analysis Loke YK et al. Comparative cardiovascular effects of thiazolidinediones: systematic review and meta-analysis of observational studies BMJ 2011; 342:d1309 Objective To determine the comparative effects of the thiazolidinediones (rosiglitazone and pioglitazone) on myocardial infarction, congestive heart failure, and mortality in patients with type 2 diabetes. Page 9

10 Diagnostic Accuracy Statistics Randomized controlled Trials RCT Only one index: Relative risk (RR) or Odds Ratio (OR) Diagnostic Test Accuracy DTA Two or more indexes of efficacy Sensitivity Specificity TPV FPV LR(+) LR(-) Diagnostic Odds Ratio (DOR) ROC Curve Page 10

11 Sensitivity Specificity Specificity 80% SPIN high Specificity and Positive results rule IN the diagnosis con alta specificità, un risultato positivo conferma la diagnosi Quality to recognize negative ones 80% Sensitivity Quality to recognize positive ones SNOUT high Sensitivity and Negative results rule OUT the diagnosis con alta sensibilità un risultato negativo esclude la diagnosi Page 11

12 Predictive Value Proportion of people with a positive test who have the target disorder Prev. 30%; Se = 80%; Sp 80% PPV 24/38 = 63% Prev. 10%; Se = 80%; Sp 80% PPV 8/26 = 31% Page 12

13 Prevalence TN TP FN FP Real conditions differ from the experimental conditions: Frequently an high number of tested subjects are negative" Page 13

14 Likelihood ratio LR(+) LR(-) The likelihood that a given test result would be expected in a patient with the target disorder compared with the likelihood that the same result would be expected in a patient without the target disorder L(+) = Se 1 Se L(-) = 1 - Sp Sp Page 14 LR+ > 10 LR- < < LR+ < <LR- <0.2 2 < LR+ < <LR- <0.5 1 < LR+ < <LR- <1.0 Modificano in modo spesso conclusivo la probabilità di malattia Modificano in modo discreto la probabilità di malattia Modificano modestamente la probabilità di malattia Modificano scarsamente la probabilità di malattia e solo di rado sono importanti.

15 Diagnostic Odd Ratio The DOR expresses how much greater the odds of having the disease are for the people with a positive test result than for the people with a negative test result. It is a single measure of diagnostic test performance that combines both likelihood ratios. DOR = LR + LR - Page 15

16 DIAGNOSTIC TEST PERFORMANCES FALSE POSITIVE Iron deficiency anaemia Test cut-off Not Iron Deficiency FALSE NEGATIVE frequency SIDEROPENIC -=- FERRITIN -=- NO- SIDEROPENIC Positive Negative Page 16

17 DIAGNOSTIC TEST PERFORMANCES FALSE POSITIVE Iron deficiency anaemia Not Iron Deficiency Test cut-off FALSE NEGATIVE frequency SIDEROPENIC -=- FERRITIN -=- NO- SIDEROPENIC Positive Negative Page 17

18 ROC Curve 100 Tru ue Positive rate (Sensitivity) Test cut-off False positive rate (100-Specificity) SIDEROPENIC NO SIDEROPENIC Positive Negative Page 18

19 Page 19

20 Confidence Intervals Medical studies is based on observations on a sample of subjects Results from this sample studies draw inferences about the population of all such subjects from which the sample is drawn The main purpose of confidence intervals is to indicate the (im)precision of the sample study as population values The imprecision is indicated by the width of confidence intervals. The width depends on: Sample size Variability of the characteristic being studied Degree of confidence required Page 20

21 Confidence intervals A Confidence interval produces a move from a single value estimate such as a sample mean, difference between sample means, etc. to a range of values that are considered to be plausible for the population Page 21

22 Example: Sensitivity % Free Plasmatic Metanephrines Urinary Fractionated Metanephrines Urinary Catecholamines Page 22

23 Narrative review Mix between Expert opinion and primary studies. Selection of primary studies is based on bibliographic accessibility Language Agreement with expert opinion A narrative review discusses and summarises the literature on a particular topic, without generating any pooled summary figures This type of review usually gives a comprehensive overview of a topic, rather than addressing a specific question such as how effective a treatment is for a particular condition. Narrative reviews do not often report on how the search for literature was carried out or how it was decided which studies were relevant to include Page ottobre

24 Systematic Review A systematic review is a literature review focused on a research question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question Formulating a problem Locating and selecting studies Critical appraisal of studies Collecting data Analyzing and presenting results Interpreting results Improving and updating reviews Arcibald Cochrane Page 24 META-ANALYSIS Meta-analysis is a statistical technique for combining the findings from independent studies to arrive at one overall measure of the effect

25 Meta-analysis Meta-analysis is a statistical technique for combining the findings from independent studies. Meta-analysis is most often used to assess the clinical effectiveness of healthcare interventions; it does this by combining data from two or more randomised control trials. The validity of the meta-analysis depends on the quality of the systematic review on which it is based. Page 25

26 Locating and selecting studies Page 26

27 Forest Plot Page 27

28 Faecal calprotectin for screening of patients with suspected inflammatory bowel disease: diagnostic meta-analysis BMJ 2010; 341:c3369 doi: /bmj.c3369 (Published 15 July 2010) Page 28

29 The diagnostic value of ultrasonography-derived edema of the temporal artery wall in giant cell arteritis: a second meta-analysis by Arida, Aikaterini; Kyprianou, Miltiades; Kanakis, Meletios; Sfikakis, Petros Journal: BMC Musculoskeletal Disorders Vol. 11 Issue 1 DOI: / Page 29

30 Power of Meta analysis Goodacre S et al. Systematic review and meta-analysis of the diagnostic accuracy of ultrasonography for deep vein thrombosis BMC Med Imaging Oct 3;5:6 Page 30

31 Likelihood Ratio + Systematic review and metaanalysis of strategies for the diagnosis of suspected pulmonary embolism BMJ 331 : 259 doi: /bmj (Published 28 July 2005) Page 31

32 LR(+)/LR(-) Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review BMJ 325 : 301 doi: /bmj (Published 10 August 2002) Page 32

33 Diagnostic Odds Ratio Page 33

34 Heterogeneity Page 34

35 Heterogeneity measurement (Inconsistency) Visually overlapping confidence intervals of the various studies Statistically: Cochran Q test Test I-quadrato Page ottobre

36 The diagnostic value of ultrasonography-derived edema of the temporal artery wall in giant cell arteritis: a second meta-analysis by Arida, Aikaterini; Kyprianou, Miltiades; Kanakis, Meletios; Sfikakis, Petros Journal: BMC Musculoskeletal Disorders Vol. 11 Issue 1 DOI: / Page 36

37 sroc Curve and Cut-off disomogeneity Page 37

38 sroc Curve Page 38

39 sroc CURVE Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review BMJ 325 : 301 doi: /bmj (Published 10 August 2002) Page 39

40 sroc Faecal calprotectin for screening of patients with suspected inflammatory bowel disease: diagnostic meta-analysis BMJ 2010; 341:c3369 doi: /bmj.c3369 (Published 15 July 2010) Page 40

41 Curve SROC Page 41

42 Yerarchy of evidence from assessing method performance to introduction YES/ NO Effective Improve outcome clinic, economic Valid Answers diagnostic question Enables decision to be made Page 42 Reliable Accurate, precise

43 Page 43

44 Do You Have Any Questions? We would be happy to help. Page 44 Page 44

45 Funnel plot Page 45