Elements of statistics (MATH0487-1)

Transcription

1 Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium December 10, 2012

2 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Outline I 1 Introduction to Statistics Why? What? Probability Statistics Some Examples Making Inferences Inferential Statistics Inductive versus Deductive Reasoning 2 Basic Probability Revisited 3 Sampling Samples and Populations Sampling Schemes Deciding Who to Choose Deciding How to Choose Non-probability Sampling Probability Sampling A Practical Application Study Designs Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

3 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Outline II Classification Qualitative Study Designs Popular Statistics and Their Distributions Resampling Strategies 4 Exploratory Data Analysis - EDA Why? Motivating Example What? Data analysis procedures Outliers and Influential Observations How? One-way Methods Pairwise Methods Assumptions of EDA 5 Estimation Introduction Motivating Example Approaches to Estimation: The Frequentist s Way Estimation by Methods of Moments Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

4 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Outline III Motivation What? How? Examples Properties of an Estimator Recapitulation Point Estimators and their Properties Properties of an MME Estimation by Maximum Likelihood What? How? Examples Profile Likelihoods Properties of an MLE Parameter Transformations 6 Confidence Intervals Importance of the Normal Distribution Interval Estimation What? How? Pivotal quantities Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

5 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Outline IV Examples Interpretation of CIs Recapitulation Pivotal quantities Examples Interpretation of CIs 7 Hypothesis Testing General procedure Hypothesis test for a single mean P-values Hypothesis tests beyond a single mean Errors and Power 8 Table analysis Dichotomous Variables Comparing two proportions using2 2 tables About RRs and ORs and their confidence intervals Chi-square Tests Goodness-of-fit test Independence test Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

6 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Outline V Homogeneity test 9 Introduction to Linear Regression Correlation Analysis Simple Linear Regression Centering Inference on Regression Coefficients Checking Model Assumptions Relation between Experience and Wage 10 Frequently Asked Questions Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

7 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Dichotomous Analysis - EDA Variables Estimation Chi-square Confidence Tests Intervals Hypothesis Testing Tab Testing Hypothesis with Table Data Several Types of χ 2 -tests: The following tests give rise to test statistics that follow a χ 2 -distribution under their appropriate null (hypothesis) Test of Goodness of fit Test of independence Test of homogeneity or (no) association Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

8 Recall: Properties of the χ 2 -distribution

9 A family of χ 2 -distributions

10 Critical values of χ 2

11 The χ 2 -table

12 The χ 2 -table via the R software

13 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Dichotomous Analysis - EDA Variables Estimation Chi-square Confidence Tests Intervals Hypothesis Testing Tab The χ 2 goodness-of-fit test Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

14 The χ 2 goodness-of-fit test

15 Example: Handgun survey

18 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Dichotomous Analysis - EDA Variables Estimation Chi-square Confidence Tests Intervals Hypothesis Testing Tab The χ 2 test of independence Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

19 The χ 2 test of independence

20 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Dichotomous Analysis - EDA Variables Estimation Chi-square Confidence Tests Intervals Hypothesis Testing Tab The χ 2 test of no association (homogeneity) Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

21 Example: treatment response

24 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Correlation Analysis -Analysis EDA Estimation Simple Linear Confidence Regression Intervals Centering Hypothesis Inference Testing on Regre Tab Quanitfying Associations Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

25 Correlation Analysis

26 Correlation Coefficients

29 To Remember

30 To Remember

31 Examples

32 Examples

33 Examples

34 Examples

35 Association and Causality

36 Bradford Hill s Criteria for Causality

37 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Correlation Analysis -Analysis EDA Estimation Simple Linear Confidence Regression Intervals Centering Hypothesis Inference Testing on Regre Tab Simple Linear Regression (SLR) Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

38 Example Boxplot of % low income by education level:

39 Simple Linear Regression: Regression Line

40 Regression Analysis represented by Regression Line Equation

41 Interpretation of Regression Model Components

42 Interpretation of Regression Model Components

43 Why is Linear Regression so popular? Linear regression can refer to simple linear regression (one predictor) or multiple linear regression (more than 1 predictor) Linear regression naturally extends to quadratic, cubic... regression to investigate curvilinear relationships Linear regression is flexible, since it can deal with binary X continuous X categorical X confounders interactions (leading to k-order regression models)

44 Example: Galton s study on height

45 Example: Galton s study on height

46 Regression Formalism I

47 Regression Formalism II

48 Regression Formalism: Model Assumptions The regression formalism naturally leads to four model assumptions: The relationship is linear The errors have the same variance The errors are independent of each other The errors are normally distributed When we satisfy the assumptions, it means that we have used all of the information available from the patterns in the data. When we violate an assumption, it usually means that there is a pattern to the data that we have not included in our model, and we could actually find a model that fits the data better.

49 Regression Formalism: Geometric Interpretation

50 Another example: City education versus income

51 Another example: City education versus income

52 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Correlation Analysis -Analysis EDA Estimation Simple Linear Confidence Regression Intervals Centering Hypothesis Inference Testing on Regre Tab Where is our Intercept? Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

53 Need for Centering As in the City education versus income example:

54 Centering Use c = 12, as education level to center with:

55 Centering - New Regression Equation / Interpretation

56 Using Sample Data to Estimate the Truth So far, we have presented our fitted regression line as ŷ i = β 0 +β 1 x i, without having said anything about how to obtain the best such regression line.

57 Using Sample Data to Estimate the Truth Note: Sometimes linear regression is referred to as least squares regression. This has to do with the fact that a criterium of minimizing squared deviations from the mean is often used to estimate the parameters of the regression model. However, other estimation methods exist (beyond the scope of this course). Hence, since we actually used a sample to estimate the population regression line, a more accurate notation would have been ŷ i = ˆβ 0 + ˆβ 1 x i

58 Drawing Conclusions about Population Associations

59 Hypothesis Testing in Regression Formulation of null hypothesis, alternative hypothesis, derivation of test statistic:

60 Hypothesis Testing in Regression Would you have expected this statistic to follow a t-distribution?

61 Hypothesis Testing in Regression Would you have expected this statistic to follow a t-distribution? Summary table:confidence intervals for difference in means

62 Example: Education

63 Example: Education

64 The use of Confidence Intervals It becomes easy once you have a pivotal quantity identified:

65 The use of Confidence Intervals

66 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Correlation Analysis -Analysis EDA Estimation Simple Linear Confidence Regression Intervals Centering Hypothesis Inference Testing on Regre Tab Statistical Modeling General Approach: Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)

67 Statistical Modeling General Approach (continued):

68 Model Validity Checks

69 Model Validity Checks What do we have to check?

70 Model Validity Checks How do we have to check?

71 Residual Plots

72 LINE: Linear relationship

73 LINE: Linear relationship In applied statistics, a partial regression plot attempts to show the effect of adding an additional variable to the model (given that one or more independent variables are already in the model). Partial regression plots are also referred to as added variable plots or adjusted variable plots. Partial regression plots are formed by: 1 Computing the residuals of regressing the response variable against the independent variables but omitting Xi 2 Computing the residuals from regressing Xi against the remaining independent variables 3 Plotting the residuals from 1. against the residuals from 2.

74 LINE: Independent observations The relevant question here is: Are all the subjects surveyed independent of one another? In order to answer this question, one needs information about how the data were collected... Can the independence assumption be assessed graphically?

75 LINE: Independent observations Can the independence assumption be assessed graphically? If the lag plot (Y i versus Y i 1 ) is without structure, then the randomness assumption holds...

76 LINE

77 LINE

78 Graphical Model Validity Checks Plotting y versus x

79 Graphical Model Validity Checks: Residual Analysis Standardized residuals:

80 Graphical Model Validity Checks: Residual Analysis

81 Example 1: Residual Analysis on Health Status

82 Example 2: Non-linearity

83 Example 3: Outliers

84 Example 4: Non-normality

85 Example 5: Heteroscedasticity

86 Minimal Practice: Fit a regression line and...

87 Check model assumptions For instance, plot residuals versus x:

88 Check model assumptions Compute standardized residuals: Note that for a normal distribution, About 68.27% of the values lie within 1 standard deviation of the mean. Similarly, about 95.45% of the values lie within 2 standard deviations of the mean. Nearly all (99.73%) of the values lie within 3 standard deviations of the mean

89 Check model assumptions Plot standardized residuals versus x: Model fit: Residual pattern for people with very little experience?

90 Caution: Outliers Check

91 Caution: Normality Check

92 Caution: Homescedasticity Check

93 Recapitulation Questions?

94 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis - EDA Estimation Confidence Intervals Hypothesis Testing Tab Acknowledgements Most slides on formal statistical inference, chi-square testing and regression are based on an excellent course series of Sandy Eckel, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, USA. Prof. Dr. Dr. K. Van Steen Elements of statistics (MATH0487-1)