Statistical Rules of Thumb

Transcription

1 Statistical Rules of Thumb Second Edition Gerald van Belle University of Washington Department of Biostatistics and Department of Environmental and Occupational Health Sciences Seattle, WA WILEY AJOHN WILEY & SONS, INC., PUBLICATION

2 Contents Preface to the Second Edition Preface to the First Edition Acronyms xiv xvi xix 1 The Basics I 1.1 Four Basic Questions Observation is Selection Replicate to Characterize Variability Variability Occurs at Multiple Levels Invalid Selection is the Primary Threat to Valid Inference There is Variation in Strength of Inference Distinguish Randomized and Observational Studies Beware of Linear Models Keep Models As Simple As Possible, But Not More Simple Understand Omnibus Quantities 14 vii

3 viii CONTENTS 1.11 Do Not Multiply Probabilities More Than Necessary Use Two-sided p-values p-values for Sample Size, Confidence Intervals for Results At Least Twelve Observations for a Confidence Interval Estimate ± Two Standard Errors is Remarkably Robust Know the Unit of the Variable Be Flexible About Scale of Measurement Determining Analysis Be Eclectic and Ecumenical in Inference 25 2 Sample Size Begin with a Basic Formula for Sample Size-Lehr's Equation Calculating Sample Size Using the Coefficient of Variation No Finite Population Correction for Survey Sample Size Standard Deviation and Sample Range Do Not Formulate a Study Solely in Terms of Effect Size Overlapping Confidence Intervals Do Not Imply Nonsignificance Sample Size Calculation for the Poisson Distribution Sample Size for Poisson With Background Rate Sample Size Calculation for the Binomial Distribution When Unequal Sample Sizes Matter; When They Don't Sample Size With Different Costs for the Two Samples The Rule of Threes for 95% Upper Bounds When There Are No Events Sample Size Calculations Are Determined by the Analysis 50

4 CONTENTS ix 3 Observational Studies The Model for an Observational Study is the Sample Survey Large Sample Size Does Not Guarantee Validity Good Observational Studies Are Designed To Establish Cause Effect Requires Longitudinal Data Make Theories Elaborate to Establish Cause and Effect The Hill Guidelines Are a Useful Guide to Show Cause Effect Sensitivity Analyses Assess Model Uncertainty and Missing Data 61 4 Covariation Assessing and Describing Covariation Don't Summarize Regression Sampling Schemes with Correlation Do Not Correlate Rates or Ratios Indiscriminately Determining Sample Size to Estimate a Correlation Pairing Data is not Always Good Go Beyond Correlation in Drawing Conclusions Agreement As Accuracy, Scale Differential, and Precision Assess Test Reliability by Means of Agreement Range of the Predictor Variable and Regression Measuring Change: Width More Important than Numbers 84 5 Environmental Studies Begin with the Lognormal Distribution in Environmental Studies Differences Are More Symmetrical Know the Sample Space for Statements of Risk Beware of Pseudoreplication Think Beyond Simple Random Sampling The Size of the Population and Small Effects 96

5 CONTENTS 5.7 Models of Small Effects Are Sensitive to Assumptions Distinguish Between Variability and Uncertainty Description of the Database is As Important as Its Data Always Assess the Statistical Basis for an Environmental Standard Measurement of a Standard and Policy Parametric Analyses Make Maximum Use of the Data Confidence, Prediction, and Tolerance Intervals Statistics and Risk Assessment Exposure Assessment is the Weak Link in Assessing Health Effects of Pollutants Assess the Errors in Calibration Due to Inverse Regression 111 Epidemiology Start with the Poisson to Model Incidence or Prevalence The Odds Ratio Approximates the Relative Risk Assuming the Disease is Rare The Number of Events is Crucial in Estimating Sample Sizes Use a Logarithmic Formulation to Calculate Sample Size Take No More than Four or Five Controls per Case Obtain at Least Ten Subjects for Every Variable Investigated Begin with the Exponential Distribution to Model Time to Event Begin with Two Exponentials for Comparing Survival Times Be Wary of Surrogates Prevalence Dominates in Screening Rare Diseases Do Not Dichotomize Unless Absolutely Necessary Additive and Multiplicative Models 139

6 CONTENTS XI 7 Evidence-Based Medicine Strength of Evidence Relevance of Information: POEM vs. DOE Begin with Absolute Risk Reduction, then Follow with Relative Risk The Number Needed to Treat (NNT) is Clinically Useful Variability in Response to Treatment Needs to be Considered Safety is the Weak Component of EBM Intent to Treat is the Default Analysis Use Prior Information but not Priors The Four Key Questions for Meta-analysts Design, Conduct, and Analysis Randomization Puts Systematic Effects into the Error Term Blocking is the Key to Reducing Variability Factorial Designs and Joint Effects High-Order Interactions Occur Rarely Balanced Designs Allow Easy Assessment of Joint Effects Analysis Follows Design Independence, Equal Variance, and Normality Plan to Graph the Results of an Analysis Distinguish Between Design Structure and Treatment Structure Make Hierarchical Analyses the Default Analysis Distinguish Between Nested and Crossed Designs- Not Always Easy Plan for Missing Data Address Multiple Comparisons Before Starting the Study Know Properties Preserved When Transforming Units Consider Bootstrapping for Complex Relationships 191

7 X/7 CONTENTS 9 Words, Tables, and Graphs Use Text for a Few Numbers, Tables for Many Numbers, Graphs for Complex Relationships Arrange Information in a Table to Drive Home the Message Always Graph the Data Always Graph Results of An Analysis of Variance Never Use a Pie Chart Bar Graphs Waste Ink; They Don't Illuminate Complex Relationships Stacked Bar Graphs Are Worse Than Bar Graphs Three-Dimensional Bar Graphs Constitute Misdirected Artistry Identify Cross-sectional and Longitudinal Patterns in Longitudinal Data Use Rendering, Manipulation, and Linking in High-Dimensional Data Consulting Session Has Beginning, Middle, and End Ask Questions Make Distinctions Know Yourself, Know the Investigator Tailor Advice to the Level of the Investigator Use Units the Investigator is Comfortable With Agree on Assignment of Responsibilities Any Basic Statistical Computing Package Will Do Ethics Precedes, Guides, and Follows Consultation Be Proactive in Statistical Consulting Use the Web for Reference, Resource, and Education Listen to, and Heed the Advice of Experts in the Field 233 Epilogue 236 References 239

8 CONTENTS xiii Author Index 255 Topic Index 261