Session 3.1 Basics of testing
Testing of significance When having two groups of, the two sets of obtained results are never identical. They show some difference, even with two representative s drawn from the same universe, due to sampling errors (chance role). are thus submitted to certain statistical procedures, to test for significance. Significance means that the difference is real and not due to chance.
Basic principle A difference between groups is observed Appropriate statistical test is applied Test statistic is obtained Statistical tables are consulted your error if you conclude that the difference is real and not due to chance. Difference Statistical test test statistic significance conclusion
Hypothesis formulation. The very first step with definition of the question.. The Null Hypothesis: No difference between groups (H 0 = H 1 ).The Alternative Hypothesis (research) the 2 groups are different (H 0 H 1 )
Hypothesis formulation A not equal B (A or B could be better) Non-directional A is better than B: Directional [powerful] The statistical test gives the probability of rejection of a true null (false +ve). The size governs the power of the test (ability to detect significance)
Truth population versus the results : 2 Groups are different 2 groups are similar
Truth population versus the results : Truth population 2 Groups are different 2 groups are similar 2 Groups are different 2 groups are similar
Truth population versus the results : Truth population 2 Groups are different 2 groups are similar 2 groups are similar
Truth population versus the results : Truth population 2 Groups are different 2 groups are similar null
Truth population versus the results : Truth population null null
Truth population versus the results : Truth population null Correct null
Truth population versus the results : Truth population null null Correct Correct
Truth population versus the results : Truth population null null Correct Type I Error Correct
Truth population versus the results : Truth population null null Correct Type II Error Type I Error Correct
Truth population versus the results : Association between predictor and outcome No association between predictor and outcome null Correct Type II Error Type I Error Correct
Statistical Errors α Error: rejection of a true Null Hypothesis [type 1 error] Conclusion: presence of a difference that does not exist Could be considered as false +ve Measured by the p-value, the threshold of significance: 5% (p <= 0.05)
Statistical Errors (cont.) β Error: failure to reject a false Null Hypothesis [type 2 error] Conclusion: failure to detect a difference that really exists Could be considered as false -ve Related to the number of subjects, Usually fixed between 5% - 10% Power of test = 1- β
Clinical Significance versus Statistical Significance Statistical significance does not necessarily mean that the difference is significant from the clinician s point of view. With large s, very small differences that have little or no clinical importance may turn out to be statistically significant. With small size a clinically significant difference might not be proved to be statistically significant. The practical implications of any finding must be judged on other than statistical grounds.
Golden Rule A statistically significant difference might not be clinically significant (unimportant). A clinically significant (important) difference might not be statistically significant. Always start by clinical judgment and support it statistically. Sample size determination as an initial step is of utmost importance.