Biostatistics: Pre-test Primer. Larry Liang MD University of Texas Southwestern Medical Center

Transcription

1 Biostatistics: Pre-test Primer Larry Liang MD University of Texas Southwestern Medical Center

2 Question 1 Which of the following will increase the sensitivity of a test? A. Decrease Type I Error B. Increase false positive results C. Increase true negative results D. Decrease true negative results E. Decrease Type II Error

3 Question 2 A study is designed to evaluate the effect of diet on weight loss. 50 people are all weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A. Chi squared test B. Kruskal-Wallis test C. Paired t-test D. ANOVA E. Mann-Whitney test

4 What you need to know How to calculate things Specificity Sensitivity Standard error of the mean Etc, etc, etc How to pick the right test T-test Chi-squared Kruskal-Wallis Etc, etc, etc

5 The Basics

6 Sensitivity Measures of ability of a test to correctly identify actual positives Disease (As determined by gold standard) Has Disease No Disease Positive True Positive Negative False Negative (Type II Error) i Sensitivity A highly sensitive test has a low Type II error rate A negative result on a sensitive test "rules out" SNOUT = SeNsitivity, a Negative test rules OUT

7 Specificity Measures ability of a test to correctly determine actual negatives Disease (As determined by gold standard) Has Disease No Disease Test Outcome Positive Negative False Positive (Type I Error) True Negative i Specificity Highly specific tests have a low Type I error rate A positive result on a specific test will "rule in" SPIN = SPecificity, a Positive test rules IN

8 Example: High Specificity Should we cancel a case? Dr. BO's evaluation Yes Cancel Don't Cancel Yes Cancel No, don't cancel Sensitivity Specificity 2 / / Dr. BO will do almost any case he hardly cancels anything Because of this, he will correctly identify almost all the cases that SHOULD be cancelled. So, when he DOES cancel a case, that has real meaning. A positive test "Rules In" Dr. BO is highly specific, so it is overwhelmingly likely that a true negative will test negative

9 Example: High Sensitivity Patient is Complex and Difficult Actually Complex Not Complex NG's Pre- Op Evaluatio n Complex Not Complex Sensitivity Specificity 99 / / In this example, we have anesthesia pre-op faculty evaluating patients. This faculty is very conservative so most patients will be classified as "complex" This faculty will be highly sensitive for identifying truly complex patients So a positive test is not very meaningful. Negative test is. "Rules Out" If this faculty calls a patient an ASA I, easy patient, then you know it will be

10 Positive/Negative Predictive Value Important because it gives the accuracy of a positive or negative result Colon Cancer (By biopsy results) Positive Negative Positive 2 18 a Positive Predictive Value Occult blood test Negativ e a Negative Predictive Value 2 / 3 67% 182 / % PPV = 2 / (2 + 18) = 10% NPV = 182 / ( ) = 99.5%

11 Type I and Type II Error Type I Error (a-error): False positives. Accepting a positive result when the true status is negative Type II Error (b-error): False negatives. Accepting a negative result when the true status is positive.

12 Hypothesis Testing Null Hypothesis: Typically that a condition does not exist (not guilty, no difference between groups, drug does not do anything). Jury Decision Reject H 0 : (Guilty) Accept H 0 : (Not guilty) Null Hypothesis: Suspect is not guilty H 0 False: Guilty Correct Crook gets off (Type II Error) H 0 True: Not guilty Innocent goes to jail (Type I Error) Correct Type I Error: Rejecting a null hypothesis which is actually true Type II Error: Accepting a null hypothesis which is actually false Easier to remember false positive and false negative

13 Hypothesis Testing Cont. Null Hypothesis: Typically that a condition does not exist (not guilty, no difference between groups, drug does not do anything). Test Results Reject H 0 : (Positive) Accept H 0 : (Negative) Null Hypothesis: Patient does not have cancer H 0 False: Has Cancer Correct False Negative (Type II Error) H 0 True: No Cancer False Positive (Type I Error) Correct Type I Error: Rejecting a null hypothesis which is actually true Type II Error: Accepting a null hypothesis which is actually false Easier to remember false positive and false negative

14 Statistical Power The probability that a person who has a condition will test negative The probability that a test will reject a false null hypothesis As power increases, the chance of a type II error decreases Positiv e Condition (As determined by gold standard) Positive Negativ e False Negative (Power) Chance of a Type II error is the b of a test Is the same as the sensitivity of a test Power = 1 - b Same as sensitivity

15 Prevalence vs. Incidence Prevalence: Total number of cases in a population in a given time. Example: Prevalence of obesity in the USA in 2003 is 20.9% Incidence: The number of new cases which develop over a given time. Example: 28 cases per 1000 persons per year

16 Bayes' Theorem Shows the relationship between two conditional probabilities Ex: The probability you have breast cancer given a positive mammogram P (A B) = [P(B A) x P(A)] / P(B) P(A B) = Probability you have breast cancer (A) given you have a positive mammogram (B) P(B A) = Probability that a positive mammogram (B) is truly breast cancer (A) P(A) = Probability you have breast cancer overall P(B) = Probability of a positive mammogram (probability of true positive + false positive) Lets say mammograms are 99% specific AND 99% sensitive Lets say prevalence of breast cancer is 1:200 or 0.5% Probability (Cancer Positive Mammogram) = P(Positive Mammogram Cancer) x P(Cancer) / P(Positive Test) = 0.99 x / (chance of picking up true positive + chance of getting a false positive) = 0.99 x / (0.99 x x 0.995) =

17 Bayes Theorem Cont. Bottom line is: When the prevalence is low You are unlikely to have the disease even if the test is accurate

18 Measures of Central Tendency Mean: Average (1, 2, 3, 4, 5, 6, 7) Mean = 4 Median: Middle value when arranged in order (1, 2, 3, 4, 5, 6, 1000) Median = 4 (1, 2, 3, 4, 5, 6) Median = 3.5 Mode: Most common observation (1, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10) Mode = 2

19 Measures of Dispersion Range: Difference between largest and smallest observations Variance: The sum of the square of the difference between an observation and the mean divided by the total observations. Example: Two tests scores 90 and 100. Mean = 95. Variance = (90-95) 2 + (100-95) 2 / 2 = 25 Standard Deviation: Square root of variance From example above, SD = 5 Standard Error of the Mean: = SD / (square root n) = 5 / 1.41 = 3.5

20 Types of Data Categorical: What was your major? What is your favorite color? Numerical (Discrete): How many cars do you have? How many siblings do you have? Numerical (Continuous): How tall are you? What is your blood pressure?

21 Types of Data Parametric Assumes a normal distribution of continuous numerical data Assumes when comparing two populations, they have the same variance. Examples: scores on a test, height of people, golf handicaps Nonparametric Used when parametric assumptions do not apply Examples: 5 year survival, pass/fail, class rank, results of a dice throw

22 Parametric Tests One sample t-test: Compares one group to a hypothetical value Compare blood pressure of Texans to 110/60 Unpaired t-test: Compare two unpaired groups Compare blood pressure of Texans to Floridians Paired t-test: Compare two paired groups Effect of a drug on an blood pressure of a group of people Measure people, give them a drug, measure again. Paired data points ANOVA: Compare 3 or more groups Compare blood pressure of Texans, Floridians, and Sooners Pearson correlation: Measure of correlation between two variables Does blood pressure correlate with weight? Linear Regression: Measures relationship of two variables if you think one causes the other Amount of sodium in diet vs. blood pressure

23 Parametric Test continued Use T test when sample number is < 100 Use Z test when sample number is > 100 Z is later in the alphabet than T, so Z is "bigger than" T.

24 Nonparametric tests Wilcoxon test: Compare one group to a hypothetical value VAS pain score of a group to 5 Mann-Whitney Test: Compare two unpaired groups VAS pain score of morphine group vs. Tylenol group Wilcoxon test: Compare two paired groups VAS pain score of one group before and after fentanyl Kruskal Wallis test: Compare 3 or more groups VAS pain score of morphine, Tylenol, and fentanyl groups Spearman Correlation: Measures correlation between 2 variables VAS pain score vs. type of drug used Nonparametric regression: Measures relationship of two variables if you think one causes the other Type of surgery vs. VAS pain scores

25 Chi Squared and Fisher's Exact Tests Chi-Squared: Used to compare a categorical variable to a set of known probabilities We know a what the probabilities are for a normal casino die Use chi-squared to test an experimental die vs. known odds Fisher's Exact Test: Used when only 2 groups and small numbers (less than 6)

26 Question 1: Which of the following will increase the sensitivity of a test? A. Decrease Type I Error B. Increase false positive results C. Increase true negative results D. Decrease true negative results E. Decrease Type II Error

27 Which of the following will increase the sensitivity of a test? A. Decrease Type I Error B. Increase false positive results C. Increase true negative results D. Decrease true negative results E. Decrease Type II Error Condition (As determined by gold standard) Positive Negative Test Outcom e Positiv e Negati ve True Positive False Negative (Type II Error) False Positive (Type I Error) True Negative a Positive Predictive Value a Negative Predictive Value i Sensitivity i Specificity Answer: E

28 Question 2 A study is designed to evaluate the effect of diet on weight loss. 50 people are all weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A. Chi squared test B. Kruskal-Wallis test C. Paired t-test D. ANOVA E. Mann-Whitney test

29 A study is designed to evaluate the effect of diet on weight loss. 50 people are all weighed, put on an experimental diet, and weighed again at 6 months. Which is the appropriate test to determine statistical significance? A. Chi squared test B. Kruskal-Wallis test C. Paired t-test D. ANOVA E. Mann-Whitney test Type of data? Parametric or Nonparametric Parametric Sample size < 100 Number of groups 1 group, two data points per person Answer C