Mini EBM Lecture Series 2007-2008 Lecture II: The Math of EBM Diagnostic Terms: Gold Standard: This is the accepted standard diagnostic test for a particular illness that serves as the reference to which all other diagnostic tests are compared. Prevalence: The percentage of persons with a particular disease within a given population at a given time. True (TP) False (FN) False (FP) True (TN) TP + FP FN + TN TP + FN FP + TN Sensitivity: Of all the people with the disease as defined by the gold standard, the percentage of them who will test positive with the test in question. Sensitivity is a characteristic of the test. Applied to a single patient, it is the probability of the test being positive in someone who has the disease as defined by the gold standard. Sensitivity = true positives / (true positives + false negatives) Specificity: Of all the people without the disease, the percentage of them who will test negative with the test in question. This too is just a characteristic of the test. Applied to your patient, it is the probability of the test being negative in someone who does NOT have the disease. Specificity = true negatives / (true negatives + false positives) Sensitivity and Specificity are solely characteristics of a test and are not affected by the prevalence of the disease. Predictive Value (PPV): Of all the people who test positive, the percentage of those that actually have the disease. Applied to your patient, it is the probability that your patient actually has the disease if he tests positive for it with your test. predictive value = true positives / (true positives + false positives) Predictive Value (NPV): Of the people who test negative, the percentage of those who truly do NOT have the disease. Applied to your patient, it is the probability that your patient truly does NOT have the disease if he tests negative for it. predictive value = true negatives / (true negatives + false negatives) PPV and NPV are affected by the prevalence of the disease, unlike sensitivity and specificity.
Let s do an example: 80 10 90 20 90 110 100 100 Prevalence = 100/200 = 50% Sn = TP/TP+FN = 80/(80+20) = 80/100 = 80% Sp = TN/TN + FP = 90/(90+10) = 90/100 = 90% PPV = TP/TP+FP = 80/90 = 89% NPV = TN/TN+FN = 90/110 = 82% 80 100 180 20 900 920 100 1000 Prevalence = 100/1100 = 9% Sn = TP/TP+FN = 80/(80+20) = 80/100 = 80% Sp = TN/TN + FP = 900/(900+100) = 900/1000 = 90% PPV = TP/TP+FP = 80/180 = 44%% NPV = TN/TN+FN = 900/920 = 98% You can see from the above example that as the prevalence goes down, the NPV goes up and the PPV goes down. Conversely, as the prevalence goes up, the PPV goes up and the NPV goes down.
Likelihood Ratios (LRs) If you learned about Spin and Snout where you use a test with a high specificity to rule something in and a test a high sensitivity to rule something out, FORGET IT NOW. This is not a good way to remember things and will only confuse you in the long-run. What you do need to remember are likelihood ratios. Likelihood Ratio: The probability of a given test result (positive/negative) in someone with disease divided by the probability of a given test result (positive/negative) in someone without disease. Likelihood Ratio (LR+): The probability of a positive test result in someone with disease divided by the probability of a positive test result in someone without disease. This is the TP rate/fp rate. Likelihood Ratio (LR+) = Sensitivity / (1 - Specificity). LR(+) = [TP / (TP + FN)] / [FP / (FP + TN)] Likelihood Ratio (LR-): The probability of a test result being negative in a person who has the disease, divided by the probability of a negative test result in a person who does not have the disease. This is the FN rate/tn rate. Likelihood Ratio (LR-) = (1- Sensitivity) / Specificity. LR(-) = [FN / (TP + FN)] / [TN / (FP + TN)] Dz (+) (-) (+) (TP) (FP) TP + FP (-) (FN) (TN) FN + TN TP + FN FP + TN How do you use LRs? You take the prevalence of a disease and use that as your pre-test probability. You then convert the pre-test probability to odds and multiply that by the LR (+) or LR (-) depending on if you got a positive or negative test result. This gives you the post-test odd which you then convert to a post-test probability. The post-test probability is also the PPV. Pre-test probability pre-test odds Pre-test odds x LR1 x LR2 x LRn = post-test odds Post-test odds Post-test probability
Why do you have to convert probability to odds before you multiply by the LR? It is because you cannot do math with probabilities. Here is an example. What if the weather man says that on Thursday there is a 60% chance of rain and on Friday it is twice as likely to rain as compared to Thursday. Does that mean that there is a 120% chance of rain on Friday? NO. You must convert it to odds before you multiply by two. How do you convert probability to odds? Probability is heads/(heads + tails). Odds is heads/tails. If you flip a coin ten times and you get 5 heads and 5 tails your probability is 5/10. The odds are 5:5, written as 5/5. Going back to the rain example above, there is a 60% probability of rain on Thursday and this is expressed as 60:40 in odds or 60/40 or 3/2. What is the probability of rain on Friday if it is twice that of Thursday? 3/2 x 2 = 6/2 odds = 6/8 probability = 75% probability. Back to Spin and Snout Now that you know how to calculate a LR, let s use an example to demonstrate how these are better than Spin and Snout. If the prevalence of disease is 50%, which of the following tests is better for ruling out disease. By the Snout method it is test A; but, is it really? A Sn=90% Sp=10% LR(+)=1.0 LR(-)=1.0 B Sn=80% Sp=30% LR(+)=1.1 LR(-)=0.7 Pretest probability of 50% = pretest odds of 1:1 result with test A: 1:1 x 1.0 LR(-) = 1:1 odds posttest probability of 50% (unchanged) result with test B: 1:1 x 0.7 LR(-) = 0.7/1 odds posttest probability of 41% With using likelihood ratios, you see that B is actually better than test A for ruling out this disease.
Therapy terms: Exposure Outcome Event No Event Treated a b a+b = n1 Control c d c+d = n0 a+c b+d Experimental Event Rate (EER) = Event rate in treated group = a/(a+b) Control Event Rate (CER) = Event rate in control group = c/(c+d) Absolute Risk Reduction (ARR) = CER-EER; is the difference in the event rate between the control group (CER) and treatment group (EER). Relative Risk (RR) = EER/CER Event rate in treatment group divided by the event rate in the control group. This is also known as risk ratio. RR is used in randomized trials and cohort studies. (Remember that in cohort studies, we use odds ratios. This is another talk for another day.) Relative Risk Reduction (RRR) = EER-CER /CER Number Needed to Treat (NNT) = 1/ARR. NNT is the number of patients who need to be treated to prevent one bad outcome. Don t forget that when a therapy increases an undesirable outcome, you can also calculate Absolute Risk Increase (ARI), Relative Risk Increase (RRI), and Number Needed to Harm (NNH) using the same equations as above. References: 1) Kizer, J.S. Diagnostic ing: The Exaqmple of Thromboembolism (PE/DVT). In: Runge, MS, Greganti, MA. Netter s Internal Medicine. 1 st ed. Teterboro, NJ: Icon Learning Systems; 2003: 37-40. 2) MUSC Intorduction to clinical Reasoning - Evidence Based Medicine Homepage. http://www.musc.edu/dc/icrebm/index.html Accessed 8/14/07.