# Effect Sizes. Null Hypothesis Significance Testing (NHST) C8057 (Research Methods 2): Effect Sizes

Save this PDF as:

Size: px
Start display at page:

Download "Effect Sizes. Null Hypothesis Significance Testing (NHST) C8057 (Research Methods 2): Effect Sizes"

## Transcription

1 Effect Sizes Null Hypothesis Significance Testing (NHST) When you read an empirical paper, the first question you should ask is how important is the effect obtained. When carrying out research we collect data, carry out some form of statistical analysis on the data (for example, a t-test or ANOVA) which gives us a value known as a test statistic. This test statistic is then compared to a known distribution of values of that statistic that enables us to work out how likely it is to get the value we have if there were no effect in the population (i.e. if the null hypothesis were true). If it is very unlikely that we would get a test statistic of the magnitude we have (typically, if the probability of getting the observed test statistic is less than.05) then we attribute this unlikely event to an effect in our data (see Field, 005). We say the effect is statistically significant. This is known as Null Hypothesis Significance Testing (NHST for short). NHST is used throughout psychology (and most other sciences) and is what you have been taught for the past 18 months (and for many psychology undergraduates it is all they are ever taught). It may, therefore, surprise you to know that it is a deeply flawed process for many reasons. Here are what some much respected statistics experts have to say about NHST (thanks to Dan Wright for some of these quotes see web/acknowledgements sections of this handout): Schmidt & Hunter (00): Significance testing almost invariably retards the search for knowledge by producing false conclusions about research literature (p. 65). Significance tests are a disastrous method for testing hypotheses (p. 65) Meehl (1978): The almost universal reliance on merely refuting the null hypothesis is a terrible mistake, is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology (p. 817). Cohen (1994): NHST; I resisted the temptation to call it Statistical Hypothesis Inference Testing. (p. 997) Reason 1: NHST is Misunderstood Many social scientists (not just students) misunderstand what the p value in NHST actually represents. If I were to ask you what p actually means which answer would you pick: a) p is the probability that the results are due to chance, the probability that the null hypothesis (H0) is true. b) p is the probability that the results are not due to chance, the probability that the null hypothesis (H0) is false. c) p is the probability of observing results as extreme (or more) as observed, if the null hypothesis (H0) is true. d) p is the probability that the results would be replicated if the experiment was conducted a second time. e) None of these. Someone did actually ask undergraduates at a UK university (not Sussex, but I won t name and shame) this question on a questionnaire and 80% chose (a) although the correct answer is Dr. Andy Field, 005 Page 1

2 (c) you will have known this if you read the first paragraph of this handout Only 5% correctly chose (c) 1. As such, many people who use NHST are not testing what they think they re testing and consequently the conclusions they draw are incorrect (because they are based on erroneous beliefs about what p means). Reason : The Null Hypothesis is Never True Now we know the correct interpretation of p we can think about the consequences of it. As we have seen p is the probability of observing results as extreme (or more) as observed, if the null hypothesis (H0) is true. There is one very important problem with p which is that for social science data the null hypothesis is never true (see Cohen, 1990). As such, p is completely meaningless because it s based on an assumption that can never be true! Remember that the null hypothesis is that there is no effect in the population. Cohen points out that the null hypothesis is never true because we know from sampling distributions (see Field, 005a, section 1.6) that two random samples will have slightly different means, and even though these differences can be very small (e.g. one mean might be 10 and another might be ) they are nevertheless different. In fact, even such a small difference would be deemed as statistically significant if a big enough sample were used (see Reason 3). As such, a non-significant result should never be interpreted (despite the fact it often is) as no difference between means or no relationship between variables. So, significance testing can never tell us that the null hypothesis is true, because it never is! As such the idea of accepting the null hypothesis is just plain wrong. Reason 3: NHST depends upon Sample Size Imagine we were interested in whether listening to Cradle of Filth (CoF) turns people into Granny-murdering devil-worshippers. We could (assuming ethics committees didn t exist and our moral values were very low indeed) expose unborn children to Cradle of Filth (or not) and see how they turn out years later. So, we have two groups (exposed to CoF in the womb vs. unexposed control group) and some kind of outcome variable (number of pentagrams drawn on the wall in blood). We could subject these to a simple t-test. Independent Samples Test assumed not assumed Levene's Test for Equality of Variances F Sig. t df Sig. (-tailed) t-test for Equality of Means Mean Difference 95% Confidence Interval of the Std. Error Difference Difference Lower Upper assumed not assumed Levene's Test for Equality of Variances F Sig. Independent Samples Test t df Sig. (-tailed) t-test for Equality of Means Mean Difference 95% Confidence Interval of the Std. Error Difference Difference Lower Upper SPSS Output 1 1 Thanks to Dan Wright for these data. Dr. Andy Field, 005 Page

3 SPSS Output 1 shows the results of two independent t-tests done on the same scenario. In both cases the difference between means is.1 so these tests are testing the same difference between means. Look at the associated t-values and significance though. In one case t is highly significant (p <.001), but in the bottom table it is not (p =.169). How can this be: in both cases the tests are testing the same mean difference of.1? Before I answer that question, let s look at another scenario (again, with CoF and granny murdering!). SPSS Output shows another t-test, which yields a significant result (p <.05). Nothing strange about that you might think, but have a look at the Mean Difference. The value is 0. Now, the t-test tests the null hypothesis that the difference between means is 0, therefore, if the difference between means really is zero then the resulting t should not be significant! How is it possible that this test is telling us that there is a significant difference between means when we know the difference is 0? Independent Samples Test assumed not assumed Levene's Test for Equality of Variances t-test for Equality of Means 95% Confidence Interval of the Mean Std. Error Difference F Sig. t df Sig. (-tailed) Difference Difference Lower Upper SPSS Output The answer to these two anomalies is the same: NHST depends on the sample size of the observed data. In the first case, the reason why it is possible to have the same mean difference but a difference in the significance of the resulting test is because the two tests were based on different samples: the significant test was based on a sample of 00, whereas the non-significant test was based on a sample of only 10! So, significance depends on your sample size. In the later case, the reason why a difference of 0 is significant is because this test is based on a sample of 1 million data points. The mean difference is very small ( or.00 to decimal places) but given a big enough sample, it is significantly different from zero. So, a trivial difference can be significant (and interpreted as important ) if the sample is big enough, conversely, a big and important difference can be non-significant (and interpreted as trivial ) in a small sample (see Field & Hole, 003). Reason 4: NHST is Illogical NHST is based on probabilistic reasoning, which severely limits what we can conclude. Cohen (1994), points out that formal reasoning relies on an initial statement of fact followed by a statement about the current state of affairs, and an inferred conclusion. This syllogism from Field (005a) illustrates what I mean: If a man has no arms then he can t play guitar. o This man plays guitar. o Therefore, this man has arms. The syllogism starts with a statement of fact that allows the end conclusion to be reached because you can deny the man has no arms by denying that he can t play guitar. However, the null hypothesis is not represented in this way because it is based on probabilities. Instead it should be stated as follows: If the null hypothesis is correct, then this test statistic is highly unlikely. Dr. Andy Field, 005 Page 3

4 o This test statistic has occurred. o Therefore, the null hypothesis is highly unlikely. If we go back to Field s (005a) guitar example a similar statement would be: If a man plays guitar then he probably doesn t play for Fugazi (this is true because there are thousands of people who play guitar but only two who play guitar in the band Fugazi!). o Guy Picciotto plays for Fugazi. o Therefore, Guy Picciotto probably doesn t play guitar. This is illogical: the conclusion is wrong because Guy Picciotto does play guitar. By extension, it should be clear that NHST allows very little to be said about the null hypothesis (see Cohen, 1994; Field, 005a for more detail). Reason 5: p <.05 is completely arbitrary! Why do we use a p <.05 as a criterion for accepting significance? Well, essentially it s because Fisher said so. There is no magic behind.05, it s just a reasonable figure and what Fisher decided was appropriate to be sufficiently confident that a genuine effect exists. To some extent it s arbitrary (for more detail on how Fisher reached this value see Field, 005a or Field & Hole, 003): IF Fisher had woken up in a 10% kind of mood we would all be working with p <.10 as our criterion. As such, it can be tempting in NHST to attribute importance to an effect with p =.04, but assume that an effect with p =.06 is unimportant: in equal sample sizes, these effects are actually likely to be very similar! What is an effect size? There is no magical answer to the problems with NHST (although see Cohen, 1994; Schmidt & Hunter, 00 for some suggestions). However, one thing that can be used in conjunction with NHST (not necessarily as a replacement for) are effect sizes. An effect size is simply an objective and standardized measure of the magnitude of observed effect (see Field, 005a; 005b). The fact that the measure is standardized just means that we can compare effect sizes across different studies that have measured different variables, or have used different scales of measurement. So, an effect size based on the Beck depression inventory could be compared to an effect size based on levels of serotonin in blood. Effect Size Measures Many measures of effect size have been proposed, the most common of which are Cohen s, d, and Pearson s correlation coefficient, r (although there are others such as Hedges g, Glass, odds ratios and risk rates: see Rosenthal, 1991). For the purpose of this handout I m going to stick with one of these: the correlation coefficient. There are three reasons for this: (1) it is probably the most common effect size measure (Field, 001, in press; Rosenthal & DiMatteo, 001); () you will be familiar with it already; and (3) it is incredibly versatile. I ll spare you the details of the correlation coefficient (you can read Field, 005a, chapter 4 if you re that interested). Many of you Isn t r a measure will be familiar with the correlation coefficient as a measure of relationships? of the strength of relationship between two continuous variables; however, it is also a very versatile measure of the strength of an experimental effect. It may be difficult for you to reconcile how the correlation coefficient can also be used in this way; however, this is only because students are often taught about it within the context of nonexperimental research. Although I don t want to get into it now (see Field, 005a if you re interested), trust me that r can be used to express differences between Dr. Andy Field, 005 Page 4

5 means and this is the measure that I prefer because it is constrained to lie between 0 (no effect) and 1 (a perfect effect) and is familiar to almost all students and researchers. Figure 1 OK, so you don t believe me. Well, let s use our Cradle of Filth example again to look at a classic difference between means scenario. The data are in Figure 1. This shows the number of pentagrams drawn by children exposed in the womb to CoF compared to controls. First let s do an independent t-test on these means. Group Statistics No Music Cradle Of Filth Std. Error N Mean Std. Deviation Mean assumed not assumed Independent Samples Test Levene's Test for Equality of Variances t-test for Equality of Means 95% Confidence Interval of the Mean Std. Error Difference F Sig. t df Sig. (-tailed) Difference Difference Lower Upper SPSS Output 3 SPSS Output 3 shows the result of this t-test. We could conclude that listening to cradle of filth makes you less likely to draw pentagrams (the mean for the CoF condition is significantly lower than for controls). We could write that children exposed to CoF in the womb drew significantly fewer pentagrams than those exposed to no music, t(8) =.75, p <.05. Ok, let s now do a simple Pearson correlation on the same data (go, on try it for yourself). I know, it sounds crazy, but let s just see what happens. SPSS Output 4 shows the results of this analysis. Note the significance of the correlation coefficient: it s.05. Now look back at the significance of t from SPSS Output 3: it s.05 also. That s odd isn t it? Well, no it s not, what we ve just done is perfectly legitimate, the correlation expresses the difference between these two groups: in fact, provided you code your two groups with 0s and 1s, you can conduct a Pearson correlation on the data and the end result expresses the relationship between the groups and the outcome variable. In fact, this Dr. Andy Field, 005 Page 5

6 relationship is simply the difference between group means! This is known as a point-biserial correlation (see Field, 005a, chapter 4). Correlations Pearson Correlation Sig. (-tailed) N Pearson Correlation Sig. (-tailed) N *. Correlation is significant at the 0.05 level (-tailed). Music Pentagrams Listened To Drawn * * SPSS Output 4 Hopefully, I ve demonstrated that r can be used to express the difference between two groups, but why did r and t have the same significance value. The answer is that the two statistics are directly related: r can be easily obtained from several common test statistics. For example, when a t-test has been used r is a function of the observed t-value and the degrees of freedom on which it is based: r = t t + df With the data above, we get:.749 r = =.697, which is exactly the same as calculating a Pearson r for the original data (see SPSS Output 4)! The reason why the minus sign has gone is because when we calculate Pearson r for these kind of data, the sign of the correlation entirely depends on which way you code the groups (try running a correlation on the same data but change it so that Cradle of Filth are represented by a code of 1 and No Music by a code of 0: the resulting r will be positive). In fact, r can also be obtained from a variety of other test statistics. When ANOVA has been used and an F-ratio is the test statistic, then when there is 1 degree of freedom for the effect, the following conversion can be used: r = F(1, ) F(1, ) + df R In which F(1,-) is simply the F-ratio for the effect (which must have 1 degree of freedom) and df R is the degrees of freedom for the error term on which the F-ratio is based. The reason the degrees of freedom for the effect need to be 1 is simply because this means that things are being compared. It s difficult to interpret effect sizes for complex effects involving lots of groups because you have no idea which groups contribute to the effect. So, it s best to calculate effect sizes for focussed comparisons such as comparisons of two groups or interactions with only 1 degree of freedom. What about categorical data? I hear you ask. No problem, r can also be used to express relationships in categorical data because it is directly related to the chi-square statistic (again, provided this chi-square statistic has only 1 degree of freedom): Dr. Andy Field, 005 Page 6

7 r = χ N () 1 Finally, r can be calculated from the probability value of a test-statistic. First, you must convert the probability into a z-score using tabulated values of the normal distribution (see Field, 005a), and then simply divide the resulting z by the square root of the total sample size on which it is based: r = Why are effect sizes useful? Effect sizes are useful because they provide an objective measure of the importance of an effect. It doesn t matter what effect you re looking for, what variables have been measured, or how those variables have been measured we know that a correlation coefficient of 0 means there is no effect, and a value of 1 means that there is a perfect effect. Cohen (199, 1988) has made some widely accepted suggestions about what constitutes a large or small effect: r = 0.10 (small effect): in this case, the effect explains 1% of the total variance. r = 0.30 (medium effect): the effect accounts for 9% of the total variance. r = 0.50 (large effect): the effect accounts for 5% of the variance. We can use these guidelines to assess the importance of our effects (regardless of the significance of the test statistic). However, r is not measured on a linear scale so an effect with r = 0.4 isn t twice as big as one with r = 0.. For example, in our earlier Cradle of Filth example, we had three sets of data on which we conducted t-tests. Figure, shows these data and the significance of these tests. You ll remember we had two samples with similar differences between means, but different significances and one data set with a near zero difference between means that was highly significant. If we calculate the effect sizes using the values of t in SPSS outputs 1 and we find that our two samples with similar mean difference yield similar effect sizes (.40 and.47); also our sample with a near zero difference between means produces an effect size of zero. As such the effect sizes better reflect what s going on than the significance of the test statistic! Z N r =.40 r =.47 r = Error Bars show 95.0% Cl of Mean 1.00 Error Bars show 95.0% Cl of Mean Error Bars show 95.0% Cl of Mean 1.00 p <.001 ns p <.05 ] ] ] 8.00 ] ] ] No Music Cradle Of Filth 0.00 No Music Cradle Of Filth 0.00 No Music Cradle Of Filth Figure Dr. Andy Field, 005 Page 7

8 Finally, the effect size in the sample is not actually that interesting: it is the size of the effect in the population that is important. However, because we don t have access to this value, we use the effect size in the sample to estimate the likely size of the effect in the population (see Field, 001). This can be done by assimilating effect sizes from similar studies using a technique called meta-analysis (see Field, 001, 005b, 003; Rosenthal & DiMatteo, 001), but that will have to wait for another time! Further Reading Clark-Carter, D. (003). Effect size: The missing piece of the jigsaw. The Psychologist, 16 (1), Field, A. P. (005). Discovering statistics using SPSS ( nd edition). London: Sage. Wright, D. B. (003). Making friends with your data: improving how statistics are conducted and reported. British Journal of Educational Psychology, 73, References Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, Field, A. P. (in press). Is the meta-analysis of correlation coefficients accurate when population effect sizes vary? Psychological Methods. Field, A. P. (005a). Discovering statistics using SPSS ( nd edition). London: Sage. Field, A. P. (005b). Meta-analysis. In J. Miles & P. Gilbert (eds.) A handbook of research methods in clinical and health psychology (pp ). Oxford: Oxford University Press. Field, A. P. (001). Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6 (), Field, A. P., & Hole, G. (003). How to design and report experiments. London: Sage. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, Rosenthal, R. (1991). Meta-analytic procedures for social research (revised). Newbury Park, CA: Sage. Rosenthal, R., & DiMatteo, M. R. (001). Meta-analysis: Recent developments in quantitative methods and literature reviews. Annual Review of Psychology, 5, 59-8 Schmidt, F., & Hunter, J. (00). Are there benefits from NHST? American Psychologist, 57(1), Web See Dan Wright s pages on NHST (and a very useful list of links under the extra bits ): Acknowledgement Some of the NHST material was directly borrowed from Dan s website (above), for which I m very grateful. More indirectly he has been very influential over the years in encouraging me to read various things about NHST to which I would have remained oblivious had I not known Dr. Andy Field, 005 Page 8

9 him. So, although he cannot be blamed for my obsession with Cradle of Filth, his influence permeates much of this handout. Task Lotze et al (001) did a study to see what areas of the brain were activated during anal stimulation: they inserted balloons (not party ones) into people s rectums and inflated them while the person was in an fmri scanner. Then they sang happy birthday and OK, they didn t, but they really did do the balloon thing (see NeuroImage, 14, if you don t believe me). One of the areas of the brain in which they were interested was the secondary somatosensory cortex (S). Let s imagine we replicated this study, but looked at two levels of balloon inflation: 00ml and 400ml. These inflations were done on the same participants. The data below show the level of S activation when the anal balloon was inflated to 00ml and 400ml. 00ml 400ml Enter the data into SPSS. Conduct a t-test on the data to compare the means of the two conditions. Use the t-statistic to compute an effect size Write a mini-results section reporting the results and effect size in APA format. Dr. Andy Field, 005 Page 9

### Introduction to Regression. Dr. Tom Pierce Radford University

Introduction to Regression Dr. Tom Pierce Radford University In the chapter on correlational techniques we focused on the Pearson R as a tool for learning about the relationship between two variables.

### Chapter 9. Two-Sample Tests. Effect Sizes and Power Paired t Test Calculation

Chapter 9 Two-Sample Tests Paired t Test (Correlated Groups t Test) Effect Sizes and Power Paired t Test Calculation Summary Independent t Test Chapter 9 Homework Power and Two-Sample Tests: Paired Versus

### Chapter 2 Probability Topics SPSS T tests

Chapter 2 Probability Topics SPSS T tests Data file used: gss.sav In the lecture about chapter 2, only the One-Sample T test has been explained. In this handout, we also give the SPSS methods to perform

### SCHOOL OF HEALTH AND HUMAN SCIENCES DON T FORGET TO RECODE YOUR MISSING VALUES

SCHOOL OF HEALTH AND HUMAN SCIENCES Using SPSS Topics addressed today: 1. Differences between groups 2. Graphing Use the s4data.sav file for the first part of this session. DON T FORGET TO RECODE YOUR

### Chapter 6: t test for dependent samples

Chapter 6: t test for dependent samples ****This chapter corresponds to chapter 11 of your book ( t(ea) for Two (Again) ). What it is: The t test for dependent samples is used to determine whether the

There are three kinds of people in the world those who are good at math and those who are not. PSY 511: Advanced Statistics for Psychological and Behavioral Research 1 Positive Views The record of a month

### Independent t- Test (Comparing Two Means)

Independent t- Test (Comparing Two Means) The objectives of this lesson are to learn: the definition/purpose of independent t-test when to use the independent t-test the use of SPSS to complete an independent

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship)

1 Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship) I. Authors should report effect sizes in the manuscript and tables when reporting

### Chapter 7 Section 7.1: Inference for the Mean of a Population

Chapter 7 Section 7.1: Inference for the Mean of a Population Now let s look at a similar situation Take an SRS of size n Normal Population : N(, ). Both and are unknown parameters. Unlike what we used

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### Study Guide for the Final Exam

Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### TRANSCRIPT: In this lecture, we will talk about both theoretical and applied concepts related to hypothesis testing.

This is Dr. Chumney. The focus of this lecture is hypothesis testing both what it is, how hypothesis tests are used, and how to conduct hypothesis tests. 1 In this lecture, we will talk about both theoretical

### Contrasts and Post Hoc Tests for One-Way Independent ANOVA Using SPSS

Contrasts and Post Hoc Tests for One-Way Independent ANOVA Using SPSS Running the Analysis In last week s lecture we came across an example, from Field (2013), about the drug Viagra, which is a sexual

### T-tests. Daniel Boduszek

T-tests Daniel Boduszek d.boduszek@interia.eu danielboduszek.com Presentation Outline Introduction to T-tests Types of t-tests Assumptions Independent samples t-test SPSS procedure Interpretation of SPSS

### CHANCE ENCOUNTERS. Making Sense of Hypothesis Tests. Howard Fincher. Learning Development Tutor. Upgrade Study Advice Service

CHANCE ENCOUNTERS Making Sense of Hypothesis Tests Howard Fincher Learning Development Tutor Upgrade Study Advice Service Oxford Brookes University Howard Fincher 2008 PREFACE This guide has a restricted

### Reporting Statistics in Psychology

This document contains general guidelines for the reporting of statistics in psychology research. The details of statistical reporting vary slightly among different areas of science and also among different

### THE KRUSKAL WALLLIS TEST

THE KRUSKAL WALLLIS TEST TEODORA H. MEHOTCHEVA Wednesday, 23 rd April 08 THE KRUSKAL-WALLIS TEST: The non-parametric alternative to ANOVA: testing for difference between several independent groups 2 NON

### Chapter 13: Answers. Ranks. Species Dog Man Total. Test Statistics b. Dog-Like Behaviour a

Task 1 Chapter 13: Answers A psychologist was interested in the cross species differences between men and dogs. She observed a group of dogs and a group of men in a naturalistic setting (20 of each). She

### Introduction to. Hypothesis Testing CHAPTER LEARNING OBJECTIVES. 1 Identify the four steps of hypothesis testing.

Introduction to Hypothesis Testing CHAPTER 8 LEARNING OBJECTIVES After reading this chapter, you should be able to: 1 Identify the four steps of hypothesis testing. 2 Define null hypothesis, alternative

### SPSS Explore procedure

SPSS Explore procedure One useful function in SPSS is the Explore procedure, which will produce histograms, boxplots, stem-and-leaf plots and extensive descriptive statistics. To run the Explore procedure,

### Chapter 7. One-way ANOVA

Chapter 7 One-way ANOVA One-way ANOVA examines equality of population means for a quantitative outcome and a single categorical explanatory variable with any number of levels. The t-test of Chapter 6 looks

### Lesson 1: Comparison of Population Means Part c: Comparison of Two- Means

Lesson : Comparison of Population Means Part c: Comparison of Two- Means Welcome to lesson c. This third lesson of lesson will discuss hypothesis testing for two independent means. Steps in Hypothesis

### Chapter 5 Analysis of variance SPSS Analysis of variance

Chapter 5 Analysis of variance SPSS Analysis of variance Data file used: gss.sav How to get there: Analyze Compare Means One-way ANOVA To test the null hypothesis that several population means are equal,

### Correlational Research

Correlational Research Chapter Fifteen Correlational Research Chapter Fifteen Bring folder of readings The Nature of Correlational Research Correlational Research is also known as Associational Research.

### Lecture Notes Module 1

Lecture Notes Module 1 Study Populations A study population is a clearly defined collection of people, animals, plants, or objects. In psychological research, a study population usually consists of a specific

### Statistics for Management II-STAT 362-Final Review

Statistics for Management II-STAT 362-Final Review Multiple Choice Identify the letter of the choice that best completes the statement or answers the question. 1. The ability of an interval estimate to

### Week 4: Standard Error and Confidence Intervals

Health Sciences M.Sc. Programme Applied Biostatistics Week 4: Standard Error and Confidence Intervals Sampling Most research data come from subjects we think of as samples drawn from a larger population.

### Null Hypothesis Significance Testing: a short tutorial

Null Hypothesis Significance Testing: a short tutorial Although thoroughly criticized, null hypothesis significance testing is the statistical method of choice in biological, biomedical and social sciences

### Case Study in Data Analysis Does a drug prevent cardiomegaly in heart failure?

Case Study in Data Analysis Does a drug prevent cardiomegaly in heart failure? Harvey Motulsky hmotulsky@graphpad.com This is the first case in what I expect will be a series of case studies. While I mention

### Examining Differences (Comparing Groups) using SPSS Inferential statistics (Part I) Dwayne Devonish

Examining Differences (Comparing Groups) using SPSS Inferential statistics (Part I) Dwayne Devonish Statistics Statistics are quantitative methods of describing, analysing, and drawing inferences (conclusions)

### Chapter 21. More About Tests and Intervals. Copyright 2012, 2008, 2005 Pearson Education, Inc.

Chapter 21 More About Tests and Intervals Copyright 2012, 2008, 2005 Pearson Education, Inc. Zero In on the Null Null hypotheses have special requirements. To perform a hypothesis test, the null must be

### SPSS Guide: Tests of Differences

SPSS Guide: Tests of Differences I put this together to give you a step-by-step guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar

### Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Introduction to Analysis of Variance (ANOVA) The Structural Model, The Summary Table, and the One- Way ANOVA Limitations of the t-test Although the t-test is commonly used, it has limitations Can only

### Confidence Intervals on Effect Size David C. Howell University of Vermont

Confidence Intervals on Effect Size David C. Howell University of Vermont Recent years have seen a large increase in the use of confidence intervals and effect size measures such as Cohen s d in reporting

### The Assumption(s) of Normality

The Assumption(s) of Normality Copyright 2000, 2011, J. Toby Mordkoff This is very complicated, so I ll provide two versions. At a minimum, you should know the short one. It would be great if you knew

### Data Mining Techniques Chapter 5: The Lure of Statistics: Data Mining Using Familiar Tools

Data Mining Techniques Chapter 5: The Lure of Statistics: Data Mining Using Familiar Tools Occam s razor.......................................................... 2 A look at data I.........................................................

### INTERPRETING THE REPEATED-MEASURES ANOVA

INTERPRETING THE REPEATED-MEASURES ANOVA USING THE SPSS GENERAL LINEAR MODEL PROGRAM RM ANOVA In this scenario (based on a RM ANOVA example from Leech, Barrett, and Morgan, 2005) each of 12 participants

### How To Run Statistical Tests in Excel

How To Run Statistical Tests in Excel Microsoft Excel is your best tool for storing and manipulating data, calculating basic descriptive statistics such as means and standard deviations, and conducting

### One-Way Analysis of Variance

One-Way Analysis of Variance Note: Much of the math here is tedious but straightforward. We ll skim over it in class but you should be sure to ask questions if you don t understand it. I. Overview A. We

### Chapter 8: Introduction to Hypothesis Testing

Chapter 8: Introduction to Hypothesis Testing We re now at the point where we can discuss the logic of hypothesis testing. This procedure will underlie the statistical analyses that we ll use for the remainder

### t Tests in Excel The Excel Statistical Master By Mark Harmon Copyright 2011 Mark Harmon

t-tests in Excel By Mark Harmon Copyright 2011 Mark Harmon No part of this publication may be reproduced or distributed without the express permission of the author. mark@excelmasterseries.com www.excelmasterseries.com

### The Basics of a Hypothesis Test

Overview The Basics of a Test Dr Tom Ilvento Department of Food and Resource Economics Alternative way to make inferences from a sample to the Population is via a Test A hypothesis test is based upon A

### ACTM State Exam-Statistics

ACTM State Exam-Statistics For the 25 multiple-choice questions, make your answer choice and record it on the answer sheet provided. Once you have completed that section of the test, proceed to the tie-breaker

### Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

### UNDERSTANDING THE INDEPENDENT-SAMPLES t TEST

UNDERSTANDING The independent-samples t test evaluates the difference between the means of two independent or unrelated groups. That is, we evaluate whether the means for two independent groups are significantly

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

### Introduction to Statistics for Computer Science Projects

Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I

### 13: Additional ANOVA Topics. Post hoc Comparisons

13: Additional ANOVA Topics Post hoc Comparisons ANOVA Assumptions Assessing Group Variances When Distributional Assumptions are Severely Violated Kruskal-Wallis Test Post hoc Comparisons In the prior

### The correlation coefficient

The correlation coefficient Clinical Biostatistics The correlation coefficient Martin Bland Correlation coefficients are used to measure the of the relationship or association between two quantitative

### Simple Predictive Analytics Curtis Seare

Using Excel to Solve Business Problems: Simple Predictive Analytics Curtis Seare Copyright: Vault Analytics July 2010 Contents Section I: Background Information Why use Predictive Analytics? How to use

### Statistical Foundations:

Statistical Foundations: Hypothesis Testing Psychology 790 Lecture #9 9/19/2006 Today sclass Hypothesis Testing. General terms and philosophy. Specific Examples Hypothesis Testing Rules of the NHST Game

### Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

### Selecting Research Participants

C H A P T E R 6 Selecting Research Participants OBJECTIVES After studying this chapter, students should be able to Define the term sampling frame Describe the difference between random sampling and random

### The Philosophy of Hypothesis Testing, Questions and Answers 2006 Samuel L. Baker

HYPOTHESIS TESTING PHILOSOPHY 1 The Philosophy of Hypothesis Testing, Questions and Answers 2006 Samuel L. Baker Question: So I'm hypothesis testing. What's the hypothesis I'm testing? Answer: When you're

### A POPULATION MEAN, CONFIDENCE INTERVALS AND HYPOTHESIS TESTING

CHAPTER 5. A POPULATION MEAN, CONFIDENCE INTERVALS AND HYPOTHESIS TESTING 5.1 Concepts When a number of animals or plots are exposed to a certain treatment, we usually estimate the effect of the treatment

### T-test in SPSS Hypothesis tests of proportions Confidence Intervals (End of chapter 6 material)

T-test in SPSS Hypothesis tests of proportions Confidence Intervals (End of chapter 6 material) Definition of p-value: The probability of getting evidence as strong as you did assuming that the null hypothesis

### PASS Sample Size Software

Chapter 250 Introduction The Chi-square test is often used to test whether sets of frequencies or proportions follow certain patterns. The two most common instances are tests of goodness of fit using multinomial

### Two-sample t-tests. - Independent samples - Pooled standard devation - The equal variance assumption

Two-sample t-tests. - Independent samples - Pooled standard devation - The equal variance assumption Last time, we used the mean of one sample to test against the hypothesis that the true mean was a particular

### Correlation Coefficients: Mean Bias and Confidence Interval Distortions

Journal of Methods and Measurement in the Social Sciences Vol. 1, No. 2,52-65, 2010 Correlation Coefficients: Mean Bias and Confidence Interval Distortions Richard L. Gorsuch Fuller Theological Seminary

### Conducting a Power Analysis to Estimate Sample Size & Optimize Research Proposals Module 9

Conducting a Power Analysis to Estimate Sample Size & Optimize Research Proposals Module 9 David Nussbaum, Ph.D. May 25, 2010 OUTLINE 2 1. What is Power; Why is it Important; How is it used? 2. Power:

### Using Excel for inferential statistics

FACT SHEET Using Excel for inferential statistics Introduction When you collect data, you expect a certain amount of variation, just caused by chance. A wide variety of statistical tests can be applied

### Basic Concepts in Research and Data Analysis

Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the

### UNDERSTANDING THE DEPENDENT-SAMPLES t TEST

UNDERSTANDING THE DEPENDENT-SAMPLES t TEST A dependent-samples t test (a.k.a. matched or paired-samples, matched-pairs, samples, or subjects, simple repeated-measures or within-groups, or correlated groups)

### Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D.

Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D. In biological science, investigators often collect biological

### Reliability Analysis

Measures of Reliability Reliability Analysis Reliability: the fact that a scale should consistently reflect the construct it is measuring. One way to think of reliability is that other things being equal,

### Recall this chart that showed how most of our course would be organized:

Chapter 4 One-Way ANOVA Recall this chart that showed how most of our course would be organized: Explanatory Variable(s) Response Variable Methods Categorical Categorical Contingency Tables Categorical

### Two-sample hypothesis testing, II 9.07 3/16/2004

Two-sample hypothesis testing, II 9.07 3/16/004 Small sample tests for the difference between two independent means For two-sample tests of the difference in mean, things get a little confusing, here,

### Hypothesis Testing Summary

Hypothesis Testing Summary Hypothesis testing begins with the drawing of a sample and calculating its characteristics (aka, statistics ). A statistical test (a specific form of a hypothesis test) is an

### The scatterplot indicates a positive linear relationship between waist size and body fat percentage:

STAT E-150 Statistical Methods Multiple Regression Three percent of a man's body is essential fat, which is necessary for a healthy body. However, too much body fat can be dangerous. For men between the

### Data Analysis Tools. Tools for Summarizing Data

Data Analysis Tools This section of the notes is meant to introduce you to many of the tools that are provided by Excel under the Tools/Data Analysis menu item. If your computer does not have that tool

### Testing Hypotheses using SPSS

Is the mean hourly rate of male workers \$2.00? T-Test One-Sample Statistics Std. Error N Mean Std. Deviation Mean 2997 2.0522 6.6282.2 One-Sample Test Test Value = 2 95% Confidence Interval Mean of the

### Testing: is my coin fair?

Testing: is my coin fair? Formally: we want to make some inference about P(head) Try it: toss coin several times (say 7 times) Assume that it is fair ( P(head)= ), and see if this assumption is compatible

### c. The factor is the type of TV program that was watched. The treatment is the embedded commercials in the TV programs.

STAT E-150 - Statistical Methods Assignment 9 Solutions Exercises 12.8, 12.13, 12.75 For each test: Include appropriate graphs to see that the conditions are met. Use Tukey's Honestly Significant Difference

### Overview of Factor Analysis

Overview of Factor Analysis Jamie DeCoster Department of Psychology University of Alabama 348 Gordon Palmer Hall Box 870348 Tuscaloosa, AL 35487-0348 Phone: (205) 348-4431 Fax: (205) 348-8648 August 1,

### SPSS Guide: Regression Analysis

SPSS Guide: Regression Analysis I put this together to give you a step-by-step guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar

### Chapter 7 Part 2. Hypothesis testing Power

Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship

### Bivariate Analysis. Comparisons of proportions: Chi Square Test (X 2 test) Variable 1. Variable 2 2 LEVELS >2 LEVELS CONTINUOUS

Bivariate Analysis Variable 1 2 LEVELS >2 LEVELS CONTINUOUS Variable 2 2 LEVELS X 2 chi square test >2 LEVELS X 2 chi square test CONTINUOUS t-test X 2 chi square test X 2 chi square test ANOVA (F-test)

### INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA)

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) As with other parametric statistics, we begin the one-way ANOVA with a test of the underlying assumptions. Our first assumption is the assumption of

### Experimental Designs (revisited)

Introduction to ANOVA Copyright 2000, 2011, J. Toby Mordkoff Probably, the best way to start thinking about ANOVA is in terms of factors with levels. (I say this because this is how they are described

### Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

### E205 Final: Version B

Name: Class: Date: E205 Final: Version B Multiple Choice Identify the choice that best completes the statement or answers the question. 1. The owner of a local nightclub has recently surveyed a random

### QUANTITATIVE METHODS BIOLOGY FINAL HONOUR SCHOOL NON-PARAMETRIC TESTS

QUANTITATIVE METHODS BIOLOGY FINAL HONOUR SCHOOL NON-PARAMETRIC TESTS This booklet contains lecture notes for the nonparametric work in the QM course. This booklet may be online at http://users.ox.ac.uk/~grafen/qmnotes/index.html.

### CALCULATIONS & STATISTICS

CALCULATIONS & STATISTICS CALCULATION OF SCORES Conversion of 1-5 scale to 0-100 scores When you look at your report, you will notice that the scores are reported on a 0-100 scale, even though respondents

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Linear Models in STATA and ANOVA

Session 4 Linear Models in STATA and ANOVA Page Strengths of Linear Relationships 4-2 A Note on Non-Linear Relationships 4-4 Multiple Linear Regression 4-5 Removal of Variables 4-8 Independent Samples

### International Statistical Institute, 56th Session, 2007: Phil Everson

Teaching Regression using American Football Scores Everson, Phil Swarthmore College Department of Mathematics and Statistics 5 College Avenue Swarthmore, PA198, USA E-mail: peverso1@swarthmore.edu 1. Introduction

### Advanced Techniques for Mobile Robotics Statistical Testing

Advanced Techniques for Mobile Robotics Statistical Testing Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz Statistical Testing for Evaluating Experiments Deals with the relationship between

### Chapter 7. Comparing Means in SPSS (t-tests) Compare Means analyses. Specifically, we demonstrate procedures for running Dependent-Sample (or

1 Chapter 7 Comparing Means in SPSS (t-tests) This section covers procedures for testing the differences between two means using the SPSS Compare Means analyses. Specifically, we demonstrate procedures

### ANOVA MULTIPLE CHOICE QUESTIONS. In the following multiple-choice questions, select the best answer.

ANOVA MULTIPLE CHOICE QUESTIONS In the following multiple-choice questions, select the best answer. 1. Analysis of variance is a statistical method of comparing the of several populations. a. standard

### Projects Involving Statistics (& SPSS)

Projects Involving Statistics (& SPSS) Academic Skills Advice Starting a project which involves using statistics can feel confusing as there seems to be many different things you can do (charts, graphs,

### The Dummy s Guide to Data Analysis Using SPSS

The Dummy s Guide to Data Analysis Using SPSS Mathematics 57 Scripps College Amy Gamble April, 2001 Amy Gamble 4/30/01 All Rights Rerserved TABLE OF CONTENTS PAGE Helpful Hints for All Tests...1 Tests

### Statistical Modelling in Stata 5: Linear Models

Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Centre for Excellence in Epidemiology University of Manchester 08/11/2016 Structure This Week What is a linear model? How

### A Basic Guide to Analyzing Individual Scores Data with SPSS

A Basic Guide to Analyzing Individual Scores Data with SPSS Step 1. Clean the data file Open the Excel file with your data. You may get the following message: If you get this message, click yes. Delete

### PSYCHOLOGY 320L Problem Set #3: One-Way ANOVA and Analytical Comparisons

PSYCHOLOGY 30L Problem Set #3: One-Way ANOVA and Analytical Comparisons Name: Score:. You and Dr. Exercise have decided to conduct a study on exercise and its effects on mood ratings. Many studies (Babyak

### Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm.

Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm. Political Science 15 Lecture 12: Hypothesis Testing Sampling