# Experimental & Behavioral Economics Lecture 6: Non-parametric tests and selection of sample size

Save this PDF as:

Size: px
Start display at page:

Download "Experimental & Behavioral Economics Lecture 6: Non-parametric tests and selection of sample size"

## Transcription

1 Experimental & Behavioral Economics Lecture 6: Non-parametric tests and selection of sample size Based on Siegel, Sidney, and N. J. Castellan (1988) Nonparametric statistics for the behavioral sciences, McGraw-Hill, New York, and teaching material by John Duffy (University of Pittsburgh) Bernd Rönz (HU Berlin) David Danz Summer term

2 Contents 1. Introduction (recap hypothesis testing) 2. Common (non-parametric) tests in experimental economics 3. Selection of sample size (power analysis) 2

3 Hypothesis testing The research hypothesis is the prediction derived from the theory under test. Null hypothesis (H 0 ) is an hypothesis of no effect (e.g., μ 1 = μ 2 ) usually formulated for the purpose of being rejected If rejected, the alternative hypothesis (H 1 ) is supported (not necessarily true) Alternative hypothesis (H 1 ) is the operational statement of the experimenter's research hypothesis. nature of the research hypothesis determines how H 1 should be stated (e.g., μ 1 μ 2, or μ 1 < μ 2, or μ 1 > μ 2 ) 3

4 Hypothesis testing The region of rejection is a region of the sampling distribution under H0 includes all possible values that a test statistic can take on. consists of a set of possible values which are so extreme that when H0 is true the probability of observing them is very small (α) Distribution of some test statistic under H0 4

5 NON-PARAMETRIC TESTS 5

6 Non-parametric tests + If the sample size is very small, there may be no alternative to using a nonparametric statistical test (unless the nature of the population distribution is known exactly) Make usually fewer assumptions about the data Interpretation of nonparametric statistical tests is often more straightforward than the interpretation of parametric tests (easier to learn and to apply than are parametric tests) If assumptions of a parametric statistical model are met in the data, then parametric statistical tests are usually more efficient (lower power-efficiency with non-parametric tests) parametric statistical tests have been systematized: different tests are simply variations on a central theme (non-parametric tests less systematic) 6

7 Non-parametric tests Two independent samples (e.g., between-subject design: same measure for each subject in two treatments) Fisher s Exact Test / Chi-Square Test of independence Median test Wilcoxon-Mann-Whitney Test / Robust Rank Order Test Kolmogorov-Smirnov Test Two dependent samples (e.g., within-subject design: two measures or repeated measure for each subject) McNemar test Sign test / Wilcoxon Signed Ranks Tests 7

8 Scales 1. Nominal (or categorical) scale numbers or other symbols are used to classify an object, person, or characteristic (i.e., to identify the groups to which various objects belong) Example: Gender 2. Ordinal (or ranking) scale (1) + objects in one category of a scale stand in some kind of relation >R to objects in other categories ( higher, more preferred, more difficult, etc.) Example: Socioeconomic status, grades 3. Interval Scale (2) + distances or differences between any two numbers on the scale have can be interpreted in a meaningful way Example: Temperature 4. Ratio Scale (3) + has a true zero point as its origin, thus the ratio of any two scale points is independent of the unit of measurement Example: Weight, age 8

9 NON-PARAMETRIC TESTS INDEPENDENT SAMPLES 9

10 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test Two independent samples Binary variables 10

11 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test H 0 : No relation between the variables (independence) under H0, the conditional probability of observing success for one variable is independent of the realization of the other variable. i.e., Pr(+ I) = Pr(+ II) = Pr(+) 11

12 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test Hypergeometric distribution describes the probability of k successes in n draws without replacement from a finite population of size N containing exactly K successes. In our contingency table: K = (A+C) N = (A + B + C + D) k = A n = (A+B) 12

13 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test Idea: Regard marginal totals as fixed A finite population of size N has (A+C) elements of group I and (B+D) elements of group II We draw a random sample of size (A+B) without replacement V is a random variable = number of observations sampled from group I In our sample, the realization of V is V = A Under H 0, the probability that V takes on the value A is given by the hypergeometric distribution 13

14 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test With marginal totals being fixed, we can write down all possible contingency tables possible tables will be completely determined by alternative values for A (V) P-value is the probability (under H0) of sampling the observed or a more extreme contingency table Let A be the observed frequency in the cell where the row and column containing the smallest and second smallest marginal frequencies intersect. Observed or more extreme contingency tables (two-sided): D D observed = A/(A+C) B/(B+D) Reject H0, if Pr( D D observed ) < α 14

15 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test Example II = observed: D =

16 Two independent samples Binary variables (nominal or ordinal) Fisher s Exact Test Example II = observed: D = Pr( D D observed ) = 16

17 Two independent samples Nominal or ordinal scaling Chi-square test of independence Two variables, independent observations Generalization of Fisher s exact test to more than two discrete categories Expected frequencies in each discrete category should not be too small expected frequencies of each cell must exceed 1 at most 20% of the cells with expected frequencies less than 5 17

18 Two independent samples Nominal or ordinal scaling Chi-square test of independence H 0 : The variables are statistically independent = no relation = groups are sampled from the same population 18

19 Two independent samples Nominal or ordinal scaling Chi-square test of independence Idea: Test whether the deviations of observed cell proportions (conditional probabilities) from cell proportions expected under H 0 (independence) exceed what we can expect by chance (random deviations) 19

20 Two independent samples Nominal or ordinal scaling Chi-square test of independence Test statistic: n ij = observed number of cases categorized in the ith row of the jth column E ij = number of cases expected in the ith row of the jth column when H 0 is true 20

21 Two independent samples Nominal or ordinal scaling Chi-square test of independence Asymptotically (as N gets large), X 2 follows a chi-square distribution with df = (r 1)(c 1), where r is the number of rows and c is the number of columns in the contingency table 21

22 Two independent samples Nominal or ordinal scaling Chi-square test of independence Example 22

23 Two independent samples Nominal or ordinal scaling Chi-square test of independence Example 23

24 Two independent samples Nominal or ordinal scaling Chi-square test of independence Example df = (r 1)(c 1) = 2 Reject H 0 since value of X 2 is beyond the critical value with df = 2 and α =

25 Two independent samples Nominal or ordinal scaling Chi-square test of independence Remark for 2x2 tables: if N not too large, use Fisher s exact test If N large (say N > 30), use chi-square test, but employ test statistic with continuity correction (Yates): 25

26 Two independent samples At least ordinal scale The median test Two independent groups At least ordinal scale H 0 : Groups do not differ in central tendency = groups have been drawn from populations with the same median 26

27 Two independent samples At least ordinal scale The median test Idea: first determine the median score for the combined group (i.e., the median for all scores in both samples) if both groups are samples from populations whose medians are the same, we would expect about half of each group's scores to be above the combined median and about half to be below 27

28 Two independent samples At least ordinal scale The median test Under H 0, the sampling distribution of the number of the m cases in group I that fall above the combined median (A) and the number of the n cases in group II that fall above the combined median (B) is the hypergeometric distribution: 28

29 Two independent samples At least ordinal scale The median test Remarks When several scores may fall right at the combined median: i. The groups may be dichotomized as those scores that exceed the median and those that do not. ii. If m + n is large, and if only a few cases fall at the combined median, those few cases may be dropped from the analysis. Better do (i) and see whether it makes a difference when analysis based on greater than or equal to or greater than. There may be no alternative to the median test, even for interval-scale data, e.g., with censored data (some observations may be off the scale and therefore measured as the maximum (or minimum) previously assigned to the observations.) 29

30 Two independent samples At least ordinal scale Two independent groups At least ordinal scale Asymptotically equivalent to a t-test. H 0 : X and Y come from the same population, Pr(X>Y)= ½ =Pr(X<Y). the median is the same in both groups (assuming that variances of the distributions in both groups are equal) H 1 (one-tail): Wilcoxon Mann-Whitney Test (a.k.a. Mann Whitney U test, Wilcoxon rank-sum test, or Wilcoxon Mann Whitney test) X is stochastically larger than Y, Pr(X>Y) > ½ the bulk of the elements in X are larger than the bulk of the elements in Y H 1 (two-tail): Pr(X>Y) ½ 30

31 Two independent samples At least ordinal scale Wilcoxon Mann-Whitney Test Idea: m = number of observations in the sample from group X n = number of observations in the sample from group Y combine the observations from both groups and rank them in order of increasing size lowest ranks are assigned to the largest negative values (if any) Note that the sum of the first N = (m+n) integers is N = N(N + 1)/2 W x is the sum of the ranks in group X W y is the sum of the ranks in group Y Thus, W x + W y = N(N + 1)/2 31

32 Two independent samples At least ordinal scale Wilcoxon Mann-Whitney Test Idea: If H0 is true, we would expect the average ranks in each of the two groups to be about equal. If W x is very large (or very small), then we may have reason to suspect that the samples were not drawn from the same population. The sampling distribution of W x (together with m and n) when H0 is true is known Hence, we can determine the probability associated with the occurrence under H0 of any W x as extreme as the observed value. 32

33 Two independent samples At least ordinal scale Wilcoxon Mann-Whitney Test Example W x = = 15 W Y = = 13 Pr(W x 15, n = 4, m = 3 ) =.20 (Pr(W x 15, n = 4, m = 3 ) =.8857) Do not reject H0 33

34 Two independent samples At least ordinal scale Wilcoxon Mann-Whitney Test Remarks Use normal approximation for large samples (m > 10 or n > 10) Then, is asymptotically normally distributed with zero mean and unit variance. Wilcoxon test has greater power than the median test The Wilcoxon test considers the rank value of each observation rather than simply its location with respect to the combined median, and, thus, uses more of the information in the data. 34

35 Two independent samples At least ordinal scale Wilcoxon Mann-Whitney Test Remarks When ties occur each of the tied observations the average of the ranks they would have had if no ties had occurred Correction of test statistic may be necessary (see Siegel & Castellan, 1988) Wilcoxon Mann-Whitney Test may be regarded as a permutation test applied to the ranks of the observations and, thus, constitutes a good approximation to the permutation test. 35

36 Two independent samples At least ordinal scale Robust Rank Order Test In order to interpret Wilcoxon tests as a test for equality of medians, we have to assume equal variances in both groups The robust Rank Order Test relaxes the assumption of the same variances, i.e., the underlying distributions may be different when testing equality of medians As before: Two independent groups, at least ordinal scale m = number of observations in the sample from group X n = number of observations in the sample from group Y combine the observations from both groups and rank them in order of increasing size, were lowest ranks are assigned to the largest negative values (if any) 36

37 Two independent samples At least ordinal scale Robust Rank Order Test Procedure For each observation in X [Y] we count the number of observations of Y [X] with a lower rank ( placement of Xi [Yj] ) =: U(YXi) [=: U(XYj)] Calculate the mean of the placements in X [and Y]: Calculate the index of variability of U(YXi) and U(XYj): Test statistic with known distribution: 37

38 Two independent samples At least ordinal scale Robust Rank Order Test Example U(YX) = 3 U(XY) =.75 V x = 2 V y = 2.75 Ù = 1.13 Pr(Ù > 1.13) > 0.1 do not reject H 0 (same conclusion with Wilcoxon Mann-Whitney test) 38

39 More than two independent samples At least ordinal scale Kruksal-Wallis Test Do k > 2 independent samples (ordinal/ordered data) come from the same or different populations? Extension of Mann-Whitney to three or more samples. Analogue to the F-test used in analysis of variance, but without the assumption that all populations under comparison are normally distributed. H0: All k samples have the same distribution functions. H1: At least two of the samples have different distribution functions. 39

40 Two independent samples At least interval scale Kolmogorov-Smirnov Test Prerequisites Here: Two independent samples/groups At least interval scale H0: samples have been drawn from the same population (i.e., from populations with the same distribution) sensitive to any kind of difference in the distributions from which the two samples were drawn- differences in location (central tendency), in dispersion, in skewness, etc. 40

41 Two independent samples At least interval scale Kolmogorov-Smirnov Test Idea If the two samples have been drawn from the same population distribution, then the cumulative distribution functions (CDF) of both samples are expected to be close to each other If the two sample CDFs are "too far apart" at any point, this suggests that the samples come from different populations large deviations between the two sample CDFs is evidence against H0 41

42 Two independent samples At least interval scale Kolmogorov-Smirnov Test Procedure determine the empirical CDF for each sample by using the same intervals for both distributions for each interval we subtract one step function from the other test focuses on the largest of these observed deviations S m (X) := empirical CDF for sample A (of size m), i.e., S m (X) =K/m, where K is the number of observations equal to or less than X S n (X) := empirical CDF for sample B (of size n) Kolmogorov-Smirnov two-sample test statistic one-sided: D m,n = max[s m (X) S n (X)] two-sided: D m,n = max[ S m (X) S n (X) ] Reject H0 if D m,n is too large (sampling distributions of D m,n are known, depend on nature of H1) 42

43 Two independent samples At least interval scale Kolmogorov-Smirnov Test Example 43

44 Two independent samples At least interval scale Kolmogorov-Smirnov Test Example D m,n = 0.70, m = 9, n = 10 Value of test statistic greater than critical value -> reject H0. 44

45 Two independent samples At least interval scale Kolmogorov-Smirnov Test Remarks Can also be used to test an empirical distribution from one sample against some theoretical distribution (as the corresponding chi-square test); then the theoretical distribution must be continuous 45

46 NON-PARAMETRIC TESTS DEPENDENT SAMPLES 46

47 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Two related (dependent) samples Binary variable Test for the significance of changes in some binary response (e.g., by treatment manipulation) Often used in the context of before and after designs 47

48 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Idea B, C: # individuals who responded the same on each treatment (+ and, respectively) A, D: # individuals whose responses changed between treatments (from + to, and from to +, respectively) 48

49 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Idea Thus, (A + D) is the total number of people whose responses changed. Focus on cells in which changes may occur: Without any treatment effect, the number of changes in each direction would be equally likely. H0: Expected number of observations in each cell is (A + D)/2 49

50 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Remember: Test statistic for the Chi-square test of independence O i = number of cases observed in category i E i = number of cases expected in category i (under H0) Applied to cells counting changes, we yield McNemar s statistic: which (approximately) follows a chi-square distribution with df = 1 50

51 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Remarks Correction for continuity (Yates) gives better approximation (correction is necessary because a continuous distribution (chisquare) is used to approximate a discrete distribution): If the total number of changes (A+D) is less than 10, use the binomial test rather than the McNemar test. 51

52 Two dependent samples Binary variable (nominal or ordinal scale) McNemar test Example Under H 0, Pr(X 2 > 1.25) > 0.05 Do not reject H 0 52

53 Two dependent samples At least ordinal scale Sign test Two related samples Variable under consideration has a continuous distribution Xi: score of subject i in treatment X Yi: score of subject i in treatment Y H0: Pr(Xi > Yi) = Pr(Xi < Yi) = ½ = median difference between X and Y is zero 53

54 Two dependent samples At least ordinal scale Sign test Idea focus on the direction of the difference between every Xi and Yi, noting whether the sign of the difference is positive or negative When H0 is true, we would expect the number of pairs which have (Xi > Yi) to be equal to the number of pairs which have (Xi < Yi). H0 is rejected if too few differences of one sign occur. 54

55 Two dependent samples At least ordinal scale Sign test The probability associated with the occurrence of a particular number of positive (and negative) differences can be determined by the binomial distribution with p = 1/2, N = the number of pairs. If a matched pair shows no difference (i.e., the difference is zero and has no sign), it is dropped from the analysis and N is reduced accordingly. 55

56 Two dependent samples At least ordinal scale Sign test Example 56

57 Two dependent samples At least ordinal scale Sign test Example Probability of observing k of n ranks being negative: k pdf cdf With n = 12, p = ½, the probability of observing 2 or less negative (or positive) signs = 2*Pr(X 2) = 2* = Reject H0 57

58 Two dependent samples At least ordinal scale Sign test Remark For large samples (say, N > 35), normal approximation to the binomial distribution is used 58

59 Two dependent samples At least ordinal scale Wilcoxon Signed-Rank Test (a.k.a. Wilcoxon T test) Sign test uses only information about the direction of the differences within pairs Wilcoxon signed-rank test uses also the relative magnitude gives more weight to a pair which shows a large difference between the two conditions than to a pair which shows a small difference. 59

60 Two dependent samples At least ordinal scale Wilcoxon Signed-Rank Test Idea Calculate the difference di = Xi Yi for each matched pair of observations Rank di's without respect to sign Assign to each rank the sign (+ or ) of the di which it represents. If H0 is true, the sum of ranks having plus signs and summed those ranks having minus signs, are expected to be equal Reject H0 if the sum of the positive ranks is too different from the sum of the negative ranks, (suggesting that treatment X differs from treatment Y) 60

61 Two dependent samples At least ordinal scale Wilcoxon Signed-Rank Test N = number of nonzero di s. T + = sum of the ranks which have a positive sign T = sum of the ranks which have a negative sign Note: the sum of all of the ranks is N(N + 1)/2 = T + + T Distribution of T + under H0 is known (Wilcoxon Signed-Rank Test corresponds to permutation test (for paired observations) based on ranks rather than scores di) 61

62 Two dependent samples At least ordinal scale Wilcoxon Signed-Rank Test Example T + = = 73 2*Pr(T + 73, N=12) = Reject H0 (as with sign test, but note lower p-value here) 62

63 Two dependent samples At least ordinal scale Wilcoxon Signed-Rank Test Remarks Ties pairs with di = 0 are dropped from the analysis and the sample size is reduced accordingly. When two or more d's have the same magnitude, their rank is the average of the ranks which would have been assigned if the d's had differed slightly Large Samples T + is approximately normally distributed with 63

64 SELECTION OF SAMPLE SIZE - POWER ANALYSIS 64

65 Power analysis True state of the world (population) H 0 is true H 1 is true Test result (based on sample) Do not reject H 0 Reject H 0 Correct (1- α) Type I Error α Type II Error β Correct (1- β) power 65

66 Power analysis Type I error: rejecting H0 when it is, in fact, true. Pr(Type I error) =: α In experimental economics, common values of α are.05 and.01 Type II error: failing to reject H0 when, in fact, it is false. Pr(Type II error) =: β Distribution of test statistic under H0 True distribution of the test statistic: 66

67 Power analysis Type I error: rejecting H0 when it is, in fact, true. Pr(Type I error) =: α In experimental economics, common values of α are.05 and.01 Type II error: failing to reject H0 when, in fact, it is false. Pr(Type II error) =: β Power of a test: Probability of correctly concluding a significant effect when it really exist in the population = 1 - Pr(Type II error) = 1 - β Usually desired to be

68 Power analysis Distribution of test statistic under H0: True distribution of the test statistic: 68

69 Power analysis Power depends on Level of significance α (+) True effect size in the population (+) Sample size N (+) Variance in the data ( ) The kind of test (e.g., Sign test versus Wilcoxon signed rank test) The nature of H1 (one-sided > two-sided) and other variables, depending upon the test being done. 69

70 Power analysis Given The kind of test (and nature of H1) Probability of Type-I error Power (1 β) Presumed size of effect/parameter Variance in the data (and further assumptions, depending on the test) we can determine the lowest sample size we need in order to detect the presumed effect (with probability (1 β)) 70

71 Power analysis Ways to determine power If the (approximate) distribution of the test statistic is known, we may calculate the power for given parameters directly If the distribution of the test statistic is not known or if an analytical solution is to tedious (e.g. some parameter of a structural model), we may determine power by simulation Example: Two-sample test of proportions See script 71

### Lecture 7: Binomial Test, Chisquare

Lecture 7: Binomial Test, Chisquare Test, and ANOVA May, 01 GENOME 560, Spring 01 Goals ANOVA Binomial test Chi square test Fisher s exact test Su In Lee, CSE & GS suinlee@uw.edu 1 Whirlwind Tour of One/Two

More information

### Statistical tests for SPSS

Statistical tests for SPSS Paolo Coletti A.Y. 2010/11 Free University of Bolzano Bozen Premise This book is a very quick, rough and fast description of statistical tests and their usage. It is explicitly

More information

### Nonparametric Statistics

Nonparametric Statistics J. Lozano University of Goettingen Department of Genetic Epidemiology Interdisciplinary PhD Program in Applied Statistics & Empirical Methods Graduate Seminar in Applied Statistics

More information

### Data Analysis. Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) SS Analysis of Experiments - Introduction

Data Analysis Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) Prof. Dr. Dr. h.c. Dieter Rombach Dr. Andreas Jedlitschka SS 2014 Analysis of Experiments - Introduction

More information

### Chapter G08 Nonparametric Statistics

G08 Nonparametric Statistics Chapter G08 Nonparametric Statistics Contents 1 Scope of the Chapter 2 2 Background to the Problems 2 2.1 Parametric and Nonparametric Hypothesis Testing......................

More information

### 3. Nonparametric methods

3. Nonparametric methods If the probability distributions of the statistical variables are unknown or are not as required (e.g. normality assumption violated), then we may still apply nonparametric tests

More information

### Module 9: Nonparametric Tests. The Applied Research Center

Module 9: Nonparametric Tests The Applied Research Center Module 9 Overview } Nonparametric Tests } Parametric vs. Nonparametric Tests } Restrictions of Nonparametric Tests } One-Sample Chi-Square Test

More information

### Chapter 3: Nonparametric Tests

B. Weaver (15-Feb-00) Nonparametric Tests... 1 Chapter 3: Nonparametric Tests 3.1 Introduction Nonparametric, or distribution free tests are so-called because the assumptions underlying their use are fewer

More information

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

More information

### Chi Square for Contingency Tables

2 x 2 Case Chi Square for Contingency Tables A test for p 1 = p 2 We have learned a confidence interval for p 1 p 2, the difference in the population proportions. We want a hypothesis testing procedure

More information

### 3.6: General Hypothesis Tests

3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

More information

### NAG C Library Chapter Introduction. g08 Nonparametric Statistics

g08 Nonparametric Statistics Introduction g08 NAG C Library Chapter Introduction g08 Nonparametric Statistics Contents 1 Scope of the Chapter... 2 2 Background to the Problems... 2 2.1 Parametric and Nonparametric

More information

### Chi-Square Tests. In This Chapter BONUS CHAPTER

BONUS CHAPTER Chi-Square Tests In the previous chapters, we explored the wonderful world of hypothesis testing as we compared means and proportions of one, two, three, and more populations, making an educated

More information

### Research Variables. Measurement. Scales of Measurement. Chapter 4: Data & the Nature of Measurement

Chapter 4: Data & the Nature of Graziano, Raulin. Research Methods, a Process of Inquiry Presented by Dustin Adams Research Variables Variable Any characteristic that can take more than one form or value.

More information

### Some Critical Information about SOME Statistical Tests and Measures of Correlation/Association

Some Critical Information about SOME Statistical Tests and Measures of Correlation/Association This information is adapted from and draws heavily on: Sheskin, David J. 2000. Handbook of Parametric and

More information

### Rank-Based Non-Parametric Tests

Rank-Based Non-Parametric Tests Reminder: Student Instructional Rating Surveys You have until May 8 th to fill out the student instructional rating surveys at https://sakai.rutgers.edu/portal/site/sirs

More information

### Variables and Data A variable contains data about anything we measure. For example; age or gender of the participants or their score on a test.

The Analysis of Research Data The design of any project will determine what sort of statistical tests you should perform on your data and how successful the data analysis will be. For example if you decide

More information

### Nonparametric Statistics

1 14.1 Using the Binomial Table Nonparametric Statistics In this chapter, we will survey several methods of inference from Nonparametric Statistics. These methods will introduce us to several new tables

More information

### PASS Sample Size Software

Chapter 250 Introduction The Chi-square test is often used to test whether sets of frequencies or proportions follow certain patterns. The two most common instances are tests of goodness of fit using multinomial

More information

### How to choose a statistical test. Francisco J. Candido dos Reis DGO-FMRP University of São Paulo

How to choose a statistical test Francisco J. Candido dos Reis DGO-FMRP University of São Paulo Choosing the right test One of the most common queries in stats support is Which analysis should I use There

More information

### Bivariate Statistics Session 2: Measuring Associations Chi-Square Test

Bivariate Statistics Session 2: Measuring Associations Chi-Square Test Features Of The Chi-Square Statistic The chi-square test is non-parametric. That is, it makes no assumptions about the distribution

More information

### Permutation Tests for Comparing Two Populations

Permutation Tests for Comparing Two Populations Ferry Butar Butar, Ph.D. Jae-Wan Park Abstract Permutation tests for comparing two populations could be widely used in practice because of flexibility of

More information

### Chi-square test Fisher s Exact test

Lesson 1 Chi-square test Fisher s Exact test McNemar s Test Lesson 1 Overview Lesson 11 covered two inference methods for categorical data from groups Confidence Intervals for the difference of two proportions

More information

### COMPARING DATA ANALYSIS TECHNIQUES FOR EVALUATION DESIGNS WITH NON -NORMAL POFULP_TIOKS Elaine S. Jeffers, University of Maryland, Eastern Shore*

COMPARING DATA ANALYSIS TECHNIQUES FOR EVALUATION DESIGNS WITH NON -NORMAL POFULP_TIOKS Elaine S. Jeffers, University of Maryland, Eastern Shore* The data collection phases for evaluation designs may involve

More information

### Statistics: revision

NST 1B Experimental Psychology Statistics practical 5 Statistics: revision Rudolf Cardinal & Mike Aitken 3 / 4 May 2005 Department of Experimental Psychology University of Cambridge Slides at pobox.com/~rudolf/psychology

More information

### Tests of relationships between variables Chi-square Test Binomial Test Run Test for Randomness One-Sample Kolmogorov-Smirnov Test.

N. Uttam Singh, Aniruddha Roy & A. K. Tripathi ICAR Research Complex for NEH Region, Umiam, Meghalaya uttamba@gmail.com, aniruddhaubkv@gmail.com, aktripathi2020@yahoo.co.in Non Parametric Tests: Hands

More information

### NONPARAMETRIC STATISTICS 1. depend on assumptions about the underlying distribution of the data (or on the Central Limit Theorem)

NONPARAMETRIC STATISTICS 1 PREVIOUSLY parametric statistics in estimation and hypothesis testing... construction of confidence intervals computing of p-values classical significance testing depend on assumptions

More information

### Outline of Topics. Statistical Methods I. Types of Data. Descriptive Statistics

Statistical Methods I Tamekia L. Jones, Ph.D. (tjones@cog.ufl.edu) Research Assistant Professor Children s Oncology Group Statistics & Data Center Department of Biostatistics Colleges of Medicine and Public

More information

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

More information

### Statistics and research

Statistics and research Usaneya Perngparn Chitlada Areesantichai Drug Dependence Research Center (WHOCC for Research and Training in Drug Dependence) College of Public Health Sciences Chulolongkorn University,

More information

### 6.4 Normal Distribution

Contents 6.4 Normal Distribution....................... 381 6.4.1 Characteristics of the Normal Distribution....... 381 6.4.2 The Standardized Normal Distribution......... 385 6.4.3 Meaning of Areas under

More information

### Chi-square and related statistics for 2 2 contingency tables

Statistics Corner Statistics Corner: Chi-square and related statistics for 2 2 contingency tables James Dean Brown University of Hawai i at Mānoa Question: I used to think that there was only one type

More information

### ANOVA Analysis of Variance

ANOVA Analysis of Variance What is ANOVA and why do we use it? Can test hypotheses about mean differences between more than 2 samples. Can also make inferences about the effects of several different IVs,

More information

### Statistical basics for Biology: p s, alphas, and measurement scales.

334 Volume 25: Mini Workshops Statistical basics for Biology: p s, alphas, and measurement scales. Catherine Teare Ketter School of Marine Programs University of Georgia Athens Georgia 30602-3636 (706)

More information

### Sample Size and Power in Clinical Trials

Sample Size and Power in Clinical Trials Version 1.0 May 011 1. Power of a Test. Factors affecting Power 3. Required Sample Size RELATED ISSUES 1. Effect Size. Test Statistics 3. Variation 4. Significance

More information

### Supplement on the Kruskal-Wallis test. So what do you do if you don t meet the assumptions of an ANOVA?

Supplement on the Kruskal-Wallis test So what do you do if you don t meet the assumptions of an ANOVA? {There are other ways of dealing with things like unequal variances and non-normal data, but we won

More information

### Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D.

Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D. In biological science, investigators often collect biological

More information

### NCSS Statistical Software

Chapter 06 Introduction This procedure provides several reports for the comparison of two distributions, including confidence intervals for the difference in means, two-sample t-tests, the z-test, the

More information

### Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm

Mgt 540 Research Methods Data Analysis 1 Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm http://web.utk.edu/~dap/random/order/start.htm

More information

### We know from STAT.1030 that the relevant test statistic for equality of proportions is:

2. Chi 2 -tests for equality of proportions Introduction: Two Samples Consider comparing the sample proportions p 1 and p 2 in independent random samples of size n 1 and n 2 out of two populations which

More information

### Using Excel for inferential statistics

FACT SHEET Using Excel for inferential statistics Introduction When you collect data, you expect a certain amount of variation, just caused by chance. A wide variety of statistical tests can be applied

More information

### CHAPTER 11 CHI-SQUARE: NON-PARAMETRIC COMPARISONS OF FREQUENCY

CHAPTER 11 CHI-SQUARE: NON-PARAMETRIC COMPARISONS OF FREQUENCY The hypothesis testing statistics detailed thus far in this text have all been designed to allow comparison of the means of two or more samples

More information

### CHAPTER 14 ORDINAL MEASURES OF CORRELATION: SPEARMAN'S RHO AND GAMMA

CHAPTER 14 ORDINAL MEASURES OF CORRELATION: SPEARMAN'S RHO AND GAMMA Chapter 13 introduced the concept of correlation statistics and explained the use of Pearson's Correlation Coefficient when working

More information

### Choosing the correct statistical test made easy

Classroom Choosing the correct statistical test made easy N Gunawardana Senior Lecturer in Community Medicine, Faculty of Medicine, University of Colombo Gone are the days where researchers had to perform

More information

### QUANTITATIVE METHODS BIOLOGY FINAL HONOUR SCHOOL NON-PARAMETRIC TESTS

QUANTITATIVE METHODS BIOLOGY FINAL HONOUR SCHOOL NON-PARAMETRIC TESTS This booklet contains lecture notes for the nonparametric work in the QM course. This booklet may be online at http://users.ox.ac.uk/~grafen/qmnotes/index.html.

More information

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

More information

### 4) The goodness of fit test is always a one tail test with the rejection region in the upper tail. Answer: TRUE

Business Statistics, 9e (Groebner/Shannon/Fry) Chapter 13 Goodness of Fit Tests and Contingency Analysis 1) A goodness of fit test can be used to determine whether a set of sample data comes from a specific

More information

### II. DISTRIBUTIONS distribution normal distribution. standard scores

Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,

More information

### Nonparametric Two-Sample Tests. Nonparametric Tests. Sign Test

Nonparametric Two-Sample Tests Sign test Mann-Whitney U-test (a.k.a. Wilcoxon two-sample test) Kolmogorov-Smirnov Test Wilcoxon Signed-Rank Test Tukey-Duckworth Test 1 Nonparametric Tests Recall, nonparametric

More information

### Testing Group Differences using T-tests, ANOVA, and Nonparametric Measures

Testing Group Differences using T-tests, ANOVA, and Nonparametric Measures Jamie DeCoster Department of Psychology University of Alabama 348 Gordon Palmer Hall Box 870348 Tuscaloosa, AL 35487-0348 Phone:

More information

### UNDERSTANDING THE TWO-WAY ANOVA

UNDERSTANDING THE e have seen how the one-way ANOVA can be used to compare two or more sample means in studies involving a single independent variable. This can be extended to two independent variables

More information

### Introduction to Statistics for Computer Science Projects

Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I

More information

### Non-Parametric Tests (I)

Lecture 5: Non-Parametric Tests (I) KimHuat LIM lim@stats.ox.ac.uk http://www.stats.ox.ac.uk/~lim/teaching.html Slide 1 5.1 Outline (i) Overview of Distribution-Free Tests (ii) Median Test for Two Independent

More information

### CONTINGENCY TABLES ARE NOT ALL THE SAME David C. Howell University of Vermont

CONTINGENCY TABLES ARE NOT ALL THE SAME David C. Howell University of Vermont To most people studying statistics a contingency table is a contingency table. We tend to forget, if we ever knew, that contingency

More information

### CA200 Quantitative Analysis for Business Decisions. File name: CA200_Section_04A_StatisticsIntroduction

CA200 Quantitative Analysis for Business Decisions File name: CA200_Section_04A_StatisticsIntroduction Table of Contents 4. Introduction to Statistics... 1 4.1 Overview... 3 4.2 Discrete or continuous

More information

### 12.8 Wilcoxon Signed Ranks Test: Nonparametric Analysis for

M12_BERE8380_12_SE_C12.8.qxd 2/21/11 3:5 PM Page 1 12.8 Wilcoxon Signed Ranks Test: Nonparametric Analysis for Two Related Populations 1 12.8 Wilcoxon Signed Ranks Test: Nonparametric Analysis for Two

More information

### First-year Statistics for Psychology Students Through Worked Examples

First-year Statistics for Psychology Students Through Worked Examples 1. THE CHI-SQUARE TEST A test of association between categorical variables by Charles McCreery, D.Phil Formerly Lecturer in Experimental

More information

### Non-Inferiority Tests for Two Proportions

Chapter 0 Non-Inferiority Tests for Two Proportions Introduction This module provides power analysis and sample size calculation for non-inferiority and superiority tests in twosample designs in which

More information

### 1. Chi-Squared Tests

1. Chi-Squared Tests We'll now look at how to test statistical hypotheses concerning nominal data, and specifically when nominal data are summarized as tables of frequencies. The tests we will considered

More information

### Comparing two groups (t tests...)

Page 1 of 33 Comparing two groups (t tests...) You've measured a variable in two groups, and the means (and medians) are distinct. Is that due to chance? Or does it tell you the two groups are really different?

More information

### Hypothesis Testing Level I Quantitative Methods. IFT Notes for the CFA exam

Hypothesis Testing 2014 Level I Quantitative Methods IFT Notes for the CFA exam Contents 1. Introduction... 3 2. Hypothesis Testing... 3 3. Hypothesis Tests Concerning the Mean... 10 4. Hypothesis Tests

More information

### UNIVERSITY OF NAIROBI

UNIVERSITY OF NAIROBI MASTERS IN PROJECT PLANNING AND MANAGEMENT NAME: SARU CAROLYNN ELIZABETH REGISTRATION NO: L50/61646/2013 COURSE CODE: LDP 603 COURSE TITLE: RESEARCH METHODS LECTURER: GAKUU CHRISTOPHER

More information

### Answer keys for Assignment 10: Measurement of study variables

Answer keys for Assignment 10: Measurement of study variables (The correct answer is underlined in bold text) 1. In a study, participants are asked to indicate the type of pet they have at home (ex: dog,

More information

### Chi Square Tests. Chapter 10. 10.1 Introduction

Contents 10 Chi Square Tests 703 10.1 Introduction............................ 703 10.2 The Chi Square Distribution.................. 704 10.3 Goodness of Fit Test....................... 709 10.4 Chi Square

More information

### LAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING

LAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING In this lab you will explore the concept of a confidence interval and hypothesis testing through a simulation problem in engineering setting.

More information

### Chi-Square Test. Contingency Tables. Contingency Tables. Chi-Square Test for Independence. Chi-Square Tests for Goodnessof-Fit

Chi-Square Tests 15 Chapter Chi-Square Test for Independence Chi-Square Tests for Goodness Uniform Goodness- Poisson Goodness- Goodness Test ECDF Tests (Optional) McGraw-Hill/Irwin Copyright 2009 by The

More information

### Study Guide for the Final Exam

Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make

More information

### Nonparametric Test Procedures

Nonparametric Test Procedures 1 Introduction to Nonparametrics Nonparametric tests do not require that samples come from populations with normal distributions or any other specific distribution. Hence

More information

### Terminating Sequential Delphi Survey Data Collection

A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Permission is granted to

More information

### Median of the p-value Under the Alternative Hypothesis

Median of the p-value Under the Alternative Hypothesis Bhaskar Bhattacharya Department of Mathematics, Southern Illinois University, Carbondale, IL, USA Desale Habtzghi Department of Statistics, University

More information

### Permutation tests are similar to rank tests, except that we use the observations directly without replacing them by ranks.

Chapter 2 Permutation Tests Permutation tests are similar to rank tests, except that we use the observations directly without replacing them by ranks. 2.1 The two-sample location problem Assumptions: x

More information

### Parametric and non-parametric statistical methods for the life sciences - Session I

Why nonparametric methods What test to use? Rank Tests Parametric and non-parametric statistical methods for the life sciences - Session I Liesbeth Bruckers Geert Molenberghs Interuniversity Institute

More information

### Chi Square Analysis. When do we use chi square?

Chi Square Analysis When do we use chi square? More often than not in psychological research, we find ourselves collecting scores from participants. These data are usually continuous measures, and might

More information

### Non-Parametric Two-Sample Analysis: The Mann-Whitney U Test

Non-Parametric Two-Sample Analysis: The Mann-Whitney U Test When samples do not meet the assumption of normality parametric tests should not be used. To overcome this problem, non-parametric tests can

More information

### Box plots & t-tests. Example

Box plots & t-tests Box Plots Box plots are a graphical representation of your sample (easy to visualize descriptive statistics); they are also known as box-and-whisker diagrams. Any data that you can

More information

### On Small Sample Properties of Permutation Tests: A Significance Test for Regression Models

On Small Sample Properties of Permutation Tests: A Significance Test for Regression Models Hisashi Tanizaki Graduate School of Economics Kobe University (tanizaki@kobe-u.ac.p) ABSTRACT In this paper we

More information

### Exact Nonparametric Tests for Comparing Means - A Personal Summary

Exact Nonparametric Tests for Comparing Means - A Personal Summary Karl H. Schlag European University Institute 1 December 14, 2006 1 Economics Department, European University Institute. Via della Piazzuola

More information

### 1. Why the hell do we need statistics?

1. Why the hell do we need statistics? There are three kind of lies: lies, damned lies, and statistics, British Prime Minister Benjamin Disraeli (as credited by Mark Twain): It is easy to lie with statistics,

More information

### Types of Data, Descriptive Statistics, and Statistical Tests for Nominal Data. Patrick F. Smith, Pharm.D. University at Buffalo Buffalo, New York

Types of Data, Descriptive Statistics, and Statistical Tests for Nominal Data Patrick F. Smith, Pharm.D. University at Buffalo Buffalo, New York . NONPARAMETRIC STATISTICS I. DEFINITIONS A. Parametric

More information

### Chapter 4. Probability and Probability Distributions

Chapter 4. robability and robability Distributions Importance of Knowing robability To know whether a sample is not identical to the population from which it was selected, it is necessary to assess the

More information

### tests whether there is an association between the outcome variable and a predictor variable. In the Assistant, you can perform a Chi-Square Test for

This paper explains the research conducted by Minitab statisticians to develop the methods and data checks used in the Assistant in Minitab 17 Statistical Software. In practice, quality professionals sometimes

More information

### Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Readings: Ha and Ha Textbook - Chapters 1 8 Appendix D & E (online) Plous - Chapters 10, 11, 12 and 14 Chapter 10: The Representativeness Heuristic Chapter 11: The Availability Heuristic Chapter 12: Probability

More information

### Quantitative Data Analysis: Choosing a statistical test Prepared by the Office of Planning, Assessment, Research and Quality

Quantitative Data Analysis: Choosing a statistical test Prepared by the Office of Planning, Assessment, Research and Quality 1 To help choose which type of quantitative data analysis to use either before

More information

### Chi Squared and Fisher's Exact Tests. Observed vs Expected Distributions

BMS 617 Statistical Techniques for the Biomedical Sciences Lecture 11: Chi-Squared and Fisher's Exact Tests Chi Squared and Fisher's Exact Tests This lecture presents two similarly structured tests, Chi-squared

More information

### Hypothesis Testing. Dr. Bob Gee Dean Scott Bonney Professor William G. Journigan American Meridian University

Hypothesis Testing Dr. Bob Gee Dean Scott Bonney Professor William G. Journigan American Meridian University 1 AMU / Bon-Tech, LLC, Journi-Tech Corporation Copyright 2015 Learning Objectives Upon successful

More information

### Part 3. Comparing Groups. Chapter 7 Comparing Paired Groups 189. Chapter 8 Comparing Two Independent Groups 217

Part 3 Comparing Groups Chapter 7 Comparing Paired Groups 189 Chapter 8 Comparing Two Independent Groups 217 Chapter 9 Comparing More Than Two Groups 257 188 Elementary Statistics Using SAS Chapter 7 Comparing

More information

### Crosstabulation & Chi Square

Crosstabulation & Chi Square Robert S Michael Chi-square as an Index of Association After examining the distribution of each of the variables, the researcher s next task is to look for relationships among

More information

### CHI-Squared Test of Independence

CHI-Squared Test of Independence Minhaz Fahim Zibran Department of Computer Science University of Calgary, Alberta, Canada. Email: mfzibran@ucalgary.ca Abstract Chi-square (X 2 ) test is a nonparametric

More information

### Data Analysis and Uncertainty Part 3: Hypothesis Testing/Sampling

Data Analysis and Uncertainty Part 3: Hypothesis Testing/Sampling Instructor: Sargur N. University at Buffalo The State University of New York srihari@cedar.buffalo.edu Topics 1. Hypothesis Testing 1.

More information

### 1 Nonparametric Statistics

1 Nonparametric Statistics When finding confidence intervals or conducting tests so far, we always described the population with a model, which includes a set of parameters. Then we could make decisions

More information

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

More information

### Tutorial 5: Hypothesis Testing

Tutorial 5: Hypothesis Testing Rob Nicholls nicholls@mrc-lmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction................................ 1 2 Testing distributional assumptions....................

More information

### Chapter 3: Central Tendency

Chapter 3: Central Tendency Central Tendency In general terms, central tendency is a statistical measure that determines a single value that accurately describes the center of the distribution and represents

More information

### Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY

Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY 1. Introduction Besides arriving at an appropriate expression of an average or consensus value for observations of a population, it is important to

More information

### Analysis of numerical data S4

Basic medical statistics for clinical and experimental research Analysis of numerical data S4 Katarzyna Jóźwiak k.jozwiak@nki.nl 3rd November 2015 1/42 Hypothesis tests: numerical and ordinal data 1 group:

More information

### Foundation of Quantitative Data Analysis

Foundation of Quantitative Data Analysis Part 1: Data manipulation and descriptive statistics with SPSS/Excel HSRS #10 - October 17, 2013 Reference : A. Aczel, Complete Business Statistics. Chapters 1

More information

### Chi Square (χ 2 ) Statistical Instructions EXP 3082L Jay Gould s Elaboration on Christensen and Evans (1980)

Chi Square (χ 2 ) Statistical Instructions EXP 3082L Jay Gould s Elaboration on Christensen and Evans (1980) For the Driver Behavior Study, the Chi Square Analysis II is the appropriate analysis below.

More information

### Come scegliere un test statistico

Come scegliere un test statistico Estratto dal Capitolo 37 of Intuitive Biostatistics (ISBN 0-19-508607-4) by Harvey Motulsky. Copyright 1995 by Oxfd University Press Inc. (disponibile in Iinternet) Table

More information

### Lecture 1: t tests and CLT

Lecture 1: t tests and CLT http://www.stats.ox.ac.uk/ winkel/phs.html Dr Matthias Winkel 1 Outline I. z test for unknown population mean - review II. Limitations of the z test III. t test for unknown population

More information

### SCHOOL OF HEALTH AND HUMAN SCIENCES DON T FORGET TO RECODE YOUR MISSING VALUES

SCHOOL OF HEALTH AND HUMAN SCIENCES Using SPSS Topics addressed today: 1. Differences between groups 2. Graphing Use the s4data.sav file for the first part of this session. DON T FORGET TO RECODE YOUR

More information