Statistical foundations of machine learning


 Loren Farmer
 2 years ago
 Views:
Transcription
1 Machine learning p. 1/45 Statistical foundations of machine learning INFOF422 Gianluca Bontempi Département d Informatique Boulevard de Triomphe  CP 212
2 Machine learning p. 2/45 Testing hypothesis Hypothesis testing is the second major area of statistical inference. A statistical hypothesis is an assertion or conjecture about the distribution of one or more random variables. A test of a statistical hypothesis is a rule or procedure for deciding whether to reject the assertion on the basis of the observed data. The basic idea is formulate some statistical hypothesis and look to see if the data provides any evidence to reject the hypothesis.
3 Machine learning p. 3/45 An hypothesis testing problem Consider the model of the traffic in the boulevard. Suppose that the measures of the interarrival times are D N = {10, 11, 1, 21, 2,... } seconds. Can we say that the mean interarrival time θ is different from 10? Consider the grades of two different school sections. Section A had {15, 10, 12, 19, 5, 7}. Section B had {14, 11, 11, 12, 6, 7}. Can we say that Section A had better grades than Section B? Consider two protein coding genes and their expression levels in a cell. Are the two genes differentially expressed? A statistical test is a procedure that aims to answer such questions.
4 Machine learning p. 4/45 Types of hypothesis We start by declaring the working (basic, null) hypothesis H to be tested, in the form θ = θ 0 or θ ω Θ, where θ 0 or ω are given. The hypothesis can be Simple. It fully specifies the distribution of z. Composite. It partially specifies the distribution of z. Example: if D N constitutes a random sample of size N from N(µ, σ 2 ) the hypothesis H : µ = µ 0, σ = σ 0, (with µ 0 and σ 0 known values) is simple while the hypothesis H : µ = µ 0 is composite since it leaves open the value of σ in (0, ).
5 Machine learning p. 5/45 Types of statistical test Suppose we have collected N samples D N = {z 1,...,z N } from a distribution F z and we have declared a null hypothesis H about F. Three are the most common types of statistical test: Pure significance test: data D N are used to assess the inferential evidence against H. Significance test: the inferential evidence against H is used to judge whether H is inappropriate. In other words it is a rule for rejecting H. Hypothesis test: data D N are used to assess the hypothesis H against a specific alternative hypothesis H. In other words this is a rule for rejecting H in favour of H.
6 Machine learning p. 6/45 Pure significance test Suppose that the null hypothesis H is simple. Let t(d N ) be a statistic such that the larger its value the more it casts doubt on H. The quantity t(d N ) is called test statistic or discrepancy measure. Let t N = t(d N ) the value of t calculated on the basis of the sample data D N. Let us consider the pvalue quantity p = Prob {t(d N ) > t N H} If p is small the sample data D N are highly inconsistent with H and p (significance probability or significance level ) is the measure of such inconsistency.
7 Machine learning p. 7/45 Some considerations p is the proportion of situations under the hypothesis H where we would observe a degree of inconsistency at least to the extent represented by t N. t N is the observed value of the statistic for a given D N. Different D N yield different values of p (0, 1). it is essential that the distribution of t(d N ) under H is known. We cannot say that p is the probability that H is true but better that p is the probability that the dataset D N is observed given that H is true Open issues 1. What if H is composite? 2. how to choose t(d N ).
8 Machine learning p. 8/45 Tests of significance Suppose that the value p is known. If p is small either a rare event has occured or perhaps H is not true. Idea: if p is less than some stated value α, we reject H. We choose a critical level α, we observe D N and we reject H at level α if P {t(d N ) > t N H) α This is equivalent to choose some critical value t α and we reject H if t N > t α. We obtain two regions in the space of sample data: critical region S 0 where if D N S 0 we reject H. noncritical region S 1 where the sample data D N gives us noreason to reject H on the basis of the levelα test.
9 Machine learning p. 9/45 Some considerations The principle is that we will accept H unless we witness some event that has sufficiently small probability of arising when H is true. If H were true we could still obtain data in S 0 and consequently wrongly reject H with probability Prob {D N S 0 H} = Prob {t(d N ) > t α H} = α The significance level α provides an upper bound to the maximum probability of incorrectly rejecting H. The pvalue is the probability that the test statistic is more extreme than its observed value. The pvalue changes with the observed data (i.e. it is a random variable) while α is a level fixed by the user.
10 Machine learning p. 10/45 Standard normal distribution 1 Normal distribution function (µ=0, σ=1) 0.4 Normal density function (µ=0, σ=1) Remember that z This means that, if z N(0, 1), then Prob {z z 0.05 } = 0.05 and also that For a generic z N(µ, σ 2 ) Prob { z z 0.05 } = = 0.1 Prob { z µ σ z 0.05 } = = 0.1
11 Machine learning p. 11/45 TP: example Let D N consist of N independent observations of x N(µ, σ 2 ), with known variance σ 2. We want to test the hypothesis H : µ = µ 0 with µ 0 known. Consider as test statistic t(d N ), the quantity ˆµ µ 0 where ˆµ is the sample average estimator. If H is true we know that ˆµ N(µ 0, σ 2 /N). Let us calculate the value t(d N ) = ˆµ µ 0 and assume that the rejection region is S 0 = { ˆµ µ 0 ˆµ µ 0 > t α }. Let us put a significance level α = 10% = 0.1. This means that t α should satisfy Prob {t(d N ) > t α H} = Prob { ˆµ µ 0 > t α H} = Prob {(ˆµ µ 0 > t α ) (OR) (ˆµ µ 0 < t α ) H} = 0.1
12 Machine learning p. 12/45 TP: example (II) For a normal variable x N(µ, σ 2 ) Prob {x µ > 1.645σ} = 1 F x (1.645σ) = 0.05 and consequently Prob {x µ > 1.645σ (OR) x µ < 1.645σ} = = 0.1 It follows that being ˆµ N(µ 0, σ 2 /N) (i.e. ˆµ µ 0 σ/ N N(0, 1)) once we put t α = 1.645σ/ N we have Prob { ˆµ µ 0 > t α H} = 0.1 and that the critical region is S 0 = { D N : ˆµ µ 0 > 1.645σ/ } N
13 Machine learning p. 13/45 TP: example (III) Suppose that σ = 0.1 and that we want to test if µ = µ 0 = 10 with a significance level 10%. After N = 6 observations we have D N = {10, 11, 12, 13, 14, 15}. On the basis of the dataset we compute ˆµ = = 12.5 and t(d N ) = ˆµ µ 0 = 2.5 Since t α = / 6 = , and t(d N ) > t α, the observations D N are in the critical region. The hypothesis is rejected.
14 Machine learning p. 14/45 Hypothesis testing: types of error So far we considered a single hypothesis. Let us now consider two alternative hypothesis: H and H. Type I error. It is the error we make when we reject H if it is true. Significance level represents the probability of making the type I error. Type II error. It is the error we make when we accept H if it is false. In order to define this error, we are forced to declare an alternative hypothesis H as a formal definition of what is meant by H being false. The probability of type II error is the probability that the test leads to acceptance of H when in fact H prevails. When the alternative hypothesis is composite, there is no unique Type II error.
15 Machine learning p. 15/45 An analogy Consider the analogy with a murder trial, where we have as suspect Mr. Bean. The null hypothesis H is Mr. Bean is innocent. The dataset is the amount of evidence collected by the police against Mr. Bean. The Type I error is the error that we make if, being Mr. Bean innocent, we send him to penalty death. The Type II error is the error that we make if, being Mr. Bean guilty, we acquit him.
16 Machine learning p. 16/45 Hypothesis testing Suppose we have some data {z 1,...,z N } F from a distribution F. H and H represent two hypotheses about F. On the basis of the data, one is accepted and one is rejected. Note that the two hypotheses have different philosophical status (asymmetry). H is a conservative hypothesis, not to be rejected unless evidence is clear. This means that a type I error is more serious than a type II error (benefit of the doubt). It is often assumed that F belongs to a parametric family F(z, θ). The test on F becomes a test on θ. A particular example of hypothesis test is the goodness of fit test where we test H : F = F 0 against H : F F 0.
17 Machine learning p. 17/45 The five steps of hypothesis testing 1. Declare the null (e.g. H: honest student) and the alternative hypothesis ( H: cheat student) 2. Choose the numeric value of the type I error (e.g. the risk I want to run). 3. Choose a procedure to obtain test statistic (e.g. number of similar lines). 4. Determine the critical value of the test statistic (e.g. 4 identical lines) that leads to a rejection of H. This is done in order to ensure the Type I error defined in Step Obtain the data and determine whether the observed value of the test statistic leads to an acceptation or rejection of H.
18 Machine learning p. 18/45 Quality of the test Suppose that N students took part to the exam, N N did not copy, N P copied, ˆNN were considered not guilty and passed the exam ˆNP were considered guilty and rejected F P honest students were refused F N cheat students passed.
19 Machine learning p. 19/45 Confusion matrix Then we have Not refused Refused H: Not guilty student () T N F P N N H: Guilty student (+) F N T P N P ˆN N ˆNP N F P is the number of False Positives and the ratio F P /N N represents the type I error. F N is the number of False Negatives and the ratio F N /N P represents the type II error.
20 Machine learning p. 20/45 Specificity and sensitivity Specificity: the ratio (to be maximized) SP = T N F P + T N = T N N N = N N F P N N = 1 F P N N, 0 SP 1 It increases by reducing the number of false positive. Sensitivity: the ratio (to be maximized) SE = T P T P + F N = T P N P = N P F N N P = 1 F N N P, 0 SE 1 It increases by reducing the number of false negatives and corresponds to the power of the test (i.e. it estimates the quantity 1Type II error).
21 Machine learning p. 21/45 Specificity and sensitivity (II) There exists a tradeoff between these two quantities. In the case of a test who return always H (e.g. very kind professor) we have ˆN P = 0, ˆN N = N, F P = 0, T N = N N and SP = 1 but SE = 0. In the case of a test who return always H (e.g. very suspicious professor) we have ˆN P = N, ˆN N = 0, F N = 0, T P = N P and SE = 1 but SP = 0.
22 Machine learning p. 22/45 False Positive and False Negative Rate False Positive Rate: FPR = 1 SP = 1 T N F P + T N = F P F P + T N = F P N N, 0 FPR 1 It decreases by reducing the number of false positive and estimates the Type I error. False Negative Rate FNR = 1 SE = 1 T P T P + F N = F N T P + F N = F N N P 0 FPR 1 It decreases by reducing the number of false negative.
23 Machine learning p. 23/45 Predictive value Positive Predictive value: the ratio(to be maximized) PPV = T P T P + F P = T P ˆN P, 0 PPV 1 Negative Predictive value: the ratio (to be maximized) PNV = T N T N + F N = T N ˆN N, 0 PNV 1 False Discovery Rate: the ratio (to be minimized) FDR = F P T P + F P = F P ˆN P = 1 PPV, 0 FDR 1
24 Machine learning p. 24/45 Receiver Operating Characteristic curve The Receiver Operating Characteristic (also known as ROC curve) is a plot of the true positive rate (i.e. sensitivity or power) against the false positive rate (Type I error) for the different possible decision thresholds of a test. Consider an example where t + N(1, 1) and t N( 1, 1). Suppose that the examples are classed as positive if t > THR and negative if t < THR, where THR is a threshold. If THR =, all the examples are classed as positive: TN = FN = 0 which implies SE = T P N P = 1 and FPR = F P F P +T N = 1. If THR =, all the examples are classed as negative: TP = FP = 0 which implies SE = 0 and FPR = 0.
25 Machine learning p. 25/45 ROC curve SE FPR R script roc.r
26 Machine learning p. 26/45 Choice of test The choice of test and consequently the choice of the partition {S 0, S 1 } is based on two steps 1. Define a significance level α, that is the probability of type I error Prob {reject H H} = Prob {D N S 0 H} α that is the probability of incorrectly rejecting H 2. Among the set of tests {S 0, S 1 } of level α, choose the test that minimizes the probability of type II error Prob { accept H H } = Prob { D N S 1 H } that is the probability of incorrectly accepting H. This is equivalent to look for maximizing the power of the test Prob { reject H H } = Prob { D N S 0 H } = 1 Prob { D N S 1 H } which is the probability of correctly rejecting H. The higher the power, the better!
27 Machine learning p. 27/45 TP example Consider a r.v. z N(µ, σ 2 ), where σ is known and a set of N iid observations are given. We want to test the null hypothesis µ = µ 0 = 0, with α = 0.1 Consider the 3 critical regions S 0 1. ˆµ µ 0 > 1.645σ/ N 2. ˆµ µ 0 > 1.282σ/ N 3. ˆµ µ 0 < 0.126σ/ N For all these tests Prob {D N S 0 H} α, hence the significance level is the same. However if H : µ = 10 the type II error of the three tests is significantly different. What is the best one?
28 Machine learning p. 28/45 µ: H S TP example (II) µ: H On the left: distribution of the test statistic ˆµ if H : µ 0 = 0 is true. On the right: distribution of the test statistic ˆµ if H : µ 1 = 10 is true. The interval marked by S 1 denotes the set of observed ˆµ values for which H is accepted (noncritical region). The interval marked by S 0 denotes the set of observed ˆµ values for which H is rejected (critical region). The area of the black pattern region on the right equals Prob {D N S 0 H}, i.e. the probability of rejecting H when H is true (Type I error). The area of the grey shaded region on the left equals the probability of accepting H when H is false (Type II error). S
29 Machine learning p. 29/45 TP example (III) S 1 µ: H 0 10 S 0 S 1 µ: H On the left: distribution of the test statistic ˆµ if H : µ 0 = 0 is true. On the right: distribution of the test statistic ˆµ if H : µ 1 = 10 is true. The two intervals marked by S 1 denote the set of observed ˆµ values for which H is accepted (noncritical region). The interval marked by S 0 denotes the set of observed ˆµ values for which H is rejected (critical region). The area of the pattern region equals Prob {D N S 0 H}, i.e. the probability of rejecting H when H is true (Type I error). Which area corresponds to the probability of the Type II error?
30 Machine learning p. 30/45 Type of parametric tests Consider random variables with a parametric distribution F(, θ). Onesample vs. twosample: in the onesample test we consider a single r.v. and we formulate hypothesis about its distribution. In the twosamples test we consider 2 r.v. z 1 and z 2 and we formulate hypothesis about their differences/similarities. Simple vs composite: the test is simple if H describes completely the distributions of the involved r.v. otherwise it is composite. Singlesided (or onetailed) vs Twosided (or twotailed): in the singlesided test the region of rejection concerns only one tail of the distribution of the null distribution. This means that H indicates the predicted direction of the difference (e.g. H : θ > θ 0 ). In the twosided test, the region of rejection concern both tails of the null distribution. This means that H does not indicate the predicted direction of the difference (e.g. H : θ θ 0 ).
31 Machine learning p. 31/45 Example of parametric test Consider a parametric test on the distribution of a gaussian r.v., and suppose that the null hypothesis is H : θ = θ 0 where θ 0 is given and represents the mean. The test is onesample and composite. In order to know whether it is one or twosided we have to define the alternative configuration: if H : θ < θ 0 the test is onesided down, if H : θ > θ 0 the test is onesided up, if H : θ θ 0 the test is doublesided.
32 Machine learning p. 32/45 ztest (onesample and onesided) Consider a random sample D N x N(µ, σ 2 ) with µ unknown et σ 2 known. STEP 1: Consider the null hypothesis and the alternative (composite and onesided) H : µ = µ 0 ; H : µ > µ0 STEP 2: fix the value α of the type I error. STEP 3: choose a test statistic: If H is true then the distribution of ˆµ is N(µ 0, σ 2 /N). This means that the variable z is z = (ˆµ µ 0) N N(0, 1) σ It is convenient to rephrase the test in terms of the test statistic z.
33 Machine learning p. 33/45 ztest (onesample and onesided) (II) STEP 4: determine the critical value for z. We reject the hypothesis H is rejected if z N > z α where z α is such that Prob {N(0, 1) > z α } = α. Ex: for α = 0.05 we would take z α = since 5% of the standard normal distribution lies to the right of R command: z α =qnorm(alpha,lower.tail=false) STEP 5: Once the dataset D N is measured, the value of the test statistic is z N = (ˆµ µ 0) N σ
34 Machine learning p. 34/45 TP: example ztest Consider a r.v. z N(µ, 1). We want to test H : µ = 5 against H : µ > 5 with significance level Supose that the data is D N = {5.1, 5.5, 4.9, 5.3}. Then ˆµ = 5.2 and z N = (5.2 5) 2/1 = 0.4. Since this is less than z α = 1.645, we do not reject the null hypothesis.
35 Machine learning p. 35/45 Twosided parametric tests Assumption: all the variables are normal! Name one/two sample known H H ztest one σ 2 µ = µ 0 µ µ 0 ztest two σ1 2 = σ2 2 µ 1 = µ 2 µ 1 µ 2 ttest one µ = µ 0 µ µ 0 ttest two µ 1 = µ 2 µ 1 µ 2 χ 2 test one µ σ 2 = σ0 2 σ 2 σ0 2 χ 2 test one σ 2 = σ0 2 σ 2 σ0 2 Ftest two σ1 2 = σ2 2 σ1 2 σ2 2
36 Machine learning p. 36/45 Student s tdistribution If x N(0, 1) and y χ 2 N are independent then the Student s tdistribution with N degrees of freedom is the distribution of the r.v. z = x y/n We denote this with z t N. If z 1,...,z N are i.i.d. N(µ, σ 2 ) then N(ˆµ µ) ŜS/(N 1) = N(ˆµ µ) ˆσ t N 1
37 Machine learning p. 37/45 ttest: onesample and twosided Consider a random sample from N(µ, σ 2 ) with σ 2 unknown. Let H : µ = µ 0 ; H : µ µ0 Let t(d N ) = T = 1 N 1 N(ˆµ µ0 ) = (ˆµ µ 0) N i=1 (z i ˆµ) 2 ˆσ 2 N a statistic computed using the data set D N.
38 Machine learning p. 38/45 ttest: onesample and twosided (II) It can be shown that if the hypothesis H holds, T T N 1 is a r.v. with a Student distribution with N 1 degrees of freedom. The size α ttest consists in rejecting H if T > k = t α/2,n 1 where t α/2,n 1 is the upper α point of a T distribution on N 1 degrees of freedom, i.e. Prob { t N 1 > t α/2,n 1 } = α/2. where t N 1 T N 1. In other terms H is rejected when T is large. R command: t α/2,n 1 =qt(alpha/2,n1,lower.tail=true)
39 Machine learning p. 39/45 TP example Does jogging lead to a reduction in pulse rate? Eight non jogging volunteers engaged in a onemonth jogging programme. Their pulses were taken before and after the programme pulse rate before pulse rate after decrease Suppose that the decreases are samples from N(µ, σ 2 ) for some unknown σ 2. We want to test H : µ = µ 0 = 0 against H : µ 0 with a significance α = We have N = 8, ˆµ = 2.75, T = 1.263, t α/2,n 1 = Since T t α/2,n 1, the data is not sufficient to reject the hypothesis H. In other terms we have not enough evidence to show that there is a reduction in pulse rate.
40 Machine learning p. 40/45 The chisquared distribution For a N positive integer, a r.v. z has a χ 2 N distribution if z = x x 2 N where x 1,x 2,...,x N are i.i.d. random variables N(0, 1). The probability distribution is a gamma distribution with parameters ( 1 2 N, 1 2 ). E[z] = N and Var[z] = 2N. The distribution is called a chisquared distribution with N degrees of freedom.
41 Machine learning p. 41/45 χ 2 test: onesample and twosided Consider a random sample from N(µ, σ 2 ) with µ known. Let H : σ 2 = σ0; 2 H : σ2 σ0 2 Let ŜS = i (z i µ) 2. It can be shown that if H is true then ŜS/σ2 0 χ 2 N The size α χ 2 test rejects H if ŜS/σ2 0 < a 1 or ŜS/σ2 0 > a 2 where Prob {ŜS σ 2 0 < a 1 } + Prob {ŜS σ 2 0 > a 2 } = α If µ is unknown, you must 1. replace µ with ˆµ in the quantity ŜS 2. use a χ 2 N 1 distribution.
42 Machine learning p. 42/45 ttest: twosamples, twosided Consider two r.v.s x N(µ 1, σ 2 ) and y N(µ 2, σ 2 ) with the same variance. Let DN x and Dy M two independent sets of samples. We want to test H : µ 1 = µ 2 against H : µ 1 µ 2. Let ˆµ x = N i=1 x i N, SS x = N (x i ˆµ x ) 2, ˆµ y = i=1 M i=1 y i M, SS y = M (y i ˆµ y ) 2 i=1 Once defined the statistic T = ( 1 M + 1 N ˆµ x ˆµ y ) ( SS x +SS y M+N 2 ) T M+N 2 it can be shown that a test of size α rejects H if T > t α/2,m+n 2
43 Machine learning p. 43/45 Fdistribution Let x χ 2 M and y χ2 N be two independent r.v.. A r.v. z has a Fdistribution F m,n with M and N degrees of freedom if If z F M,N then 1/z F N,M. If z T N then z 2 F 1,N. z = x/m y/n
44 Machine learning p. 44/45 Fdistribution 0.9 F M,N density: M=20 N=10 1 F M,N cumulative distribution: M=20 N= R script s_f.r.
45 Machine learning p. 45/45 Ftest: twosamples, twosided Consider a random sample x 1,...,x M from N(µ 1, σ 2 1) and a random sample y 1,...,y N from N(µ 2, σ 2 2) with µ 1 and µ 2 unknown. Suppose we want to test Let us consider the statistic H : σ 2 1 = σ 2 2; H : σ 2 1 σ 2 2 f = ˆσ2 1 ˆσ 2 2 = ŜS 1/(M 1) ŜS 2 /(N 1) σ2 1 χ2 M 1 /(M 1) σ2 2χ2 N 1 /(N 1) = σ2 1 σ2 2 F M 1,N 1 It can be shown that if H is true, the ratio f has a Fdistribution F M 1,N 1 We reject H if the ratio f is large, i.e. f > F α,m 1,N 1 where if z F M 1,N 1. Prob {z > F α,m 1,N 1 } = α
Hypothesis Testing Level I Quantitative Methods. IFT Notes for the CFA exam
Hypothesis Testing 2014 Level I Quantitative Methods IFT Notes for the CFA exam Contents 1. Introduction... 3 2. Hypothesis Testing... 3 3. Hypothesis Tests Concerning the Mean... 10 4. Hypothesis Tests
More informationNull Hypothesis Significance Testing Signifcance Level, Power, ttests Spring 2014 Jeremy Orloff and Jonathan Bloom
Null Hypothesis Significance Testing Signifcance Level, Power, ttests 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom Simple and composite hypotheses Simple hypothesis: the sampling distribution is
More information93.4 Likelihood ratio test. NeymanPearson lemma
93.4 Likelihood ratio test NeymanPearson lemma 91 Hypothesis Testing 91.1 Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental
More informationQuantitative Biology Lecture 5 (Hypothesis Testing)
15 th Oct 2015 Quantitative Biology Lecture 5 (Hypothesis Testing) Gurinder Singh Mickey Atwal Center for Quantitative Biology Summary Classification Errors Statistical significance Ttests Qvalues (Traditional)
More informationChapter 8 Introduction to Hypothesis Testing
Chapter 8 Student Lecture Notes 81 Chapter 8 Introduction to Hypothesis Testing Fall 26 Fundamentals of Business Statistics 1 Chapter Goals After completing this chapter, you should be able to: Formulate
More informationHypothesis Testing or How to Decide to Decide Edpsy 580
Hypothesis Testing or How to Decide to Decide Edpsy 580 Carolyn J. Anderson Department of Educational Psychology University of Illinois at UrbanaChampaign Hypothesis Testing or How to Decide to Decide
More information22. HYPOTHESIS TESTING
22. HYPOTHESIS TESTING Often, we need to make decisions based on incomplete information. Do the data support some belief ( hypothesis ) about the value of a population parameter? Is OJ Simpson guilty?
More informationChapter 8 Hypothesis Tests. Chapter Table of Contents
Chapter 8 Hypothesis Tests Chapter Table of Contents Introduction...157 OneSample ttest...158 Paired ttest...164 TwoSample Test for Proportions...169 TwoSample Test for Variances...172 Discussion
More informationHYPOTHESIS TESTING: POWER OF THE TEST
HYPOTHESIS TESTING: POWER OF THE TEST The first 6 steps of the 9step test of hypothesis are called "the test". These steps are not dependent on the observed data values. When planning a research project,
More informationChapter 8. Hypothesis Testing
Chapter 8 Hypothesis Testing Hypothesis In statistics, a hypothesis is a claim or statement about a property of a population. A hypothesis test (or test of significance) is a standard procedure for testing
More informationStatistical Inference and ttests
1 Statistical Inference and ttests Objectives Evaluate the difference between a sample mean and a target value using a onesample ttest. Evaluate the difference between a sample mean and a target value
More informationTutorial 5: Hypothesis Testing
Tutorial 5: Hypothesis Testing Rob Nicholls nicholls@mrclmb.cam.ac.uk MRC LMB Statistics Course 2014 Contents 1 Introduction................................ 1 2 Testing distributional assumptions....................
More informationInferential Statistics
Inferential Statistics Sampling and the normal distribution Zscores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are
More informationStatistiek I. ttests. John Nerbonne. CLCG, Rijksuniversiteit Groningen. John Nerbonne 1/35
Statistiek I ttests John Nerbonne CLCG, Rijksuniversiteit Groningen http://wwwletrugnl/nerbonne/teach/statistieki/ John Nerbonne 1/35 ttests To test an average or pair of averages when σ is known, we
More informationSection 7.1. Introduction to Hypothesis Testing. Schrodinger s cat quantum mechanics thought experiment (1935)
Section 7.1 Introduction to Hypothesis Testing Schrodinger s cat quantum mechanics thought experiment (1935) Statistical Hypotheses A statistical hypothesis is a claim about a population. Null hypothesis
More informationHypothesis testing S2
Basic medical statistics for clinical and experimental research Hypothesis testing S2 Katarzyna Jóźwiak k.jozwiak@nki.nl 2nd November 2015 1/43 Introduction Point estimation: use a sample statistic to
More informationTwoSample TTests Allowing Unequal Variance (Enter Difference)
Chapter 45 TwoSample TTests Allowing Unequal Variance (Enter Difference) Introduction This procedure provides sample size and power calculations for one or twosided twosample ttests when no assumption
More informationModule 7: Hypothesis Testing I Statistics (OA3102)
Module 7: Hypothesis Testing I Statistics (OA3102) Professor Ron Fricker Naval Postgraduate School Monterey, California Reading assignment: WM&S chapter 10.110.5 Revision: 212 1 Goals for this Module
More informationTwoSample TTests Assuming Equal Variance (Enter Means)
Chapter 4 TwoSample TTests Assuming Equal Variance (Enter Means) Introduction This procedure provides sample size and power calculations for one or twosided twosample ttests when the variances of
More informationHypothesis Testing. Bluman Chapter 8
CHAPTER 8 Learning Objectives C H A P T E R E I G H T Hypothesis Testing 1 Outline 81 Steps in Traditional Method 82 z Test for a Mean 83 t Test for a Mean 84 z Test for a Proportion 85 2 Test for
More informationTHE FIRST SET OF EXAMPLES USE SUMMARY DATA... EXAMPLE 7.2, PAGE 227 DESCRIBES A PROBLEM AND A HYPOTHESIS TEST IS PERFORMED IN EXAMPLE 7.
THERE ARE TWO WAYS TO DO HYPOTHESIS TESTING WITH STATCRUNCH: WITH SUMMARY DATA (AS IN EXAMPLE 7.17, PAGE 236, IN ROSNER); WITH THE ORIGINAL DATA (AS IN EXAMPLE 8.5, PAGE 301 IN ROSNER THAT USES DATA FROM
More informationHow to Conduct a Hypothesis Test
How to Conduct a Hypothesis Test The idea of hypothesis testing is relatively straightforward. In various studies we observe certain events. We must ask, is the event due to chance alone, or is there some
More informationProbability and Statistics Lecture 9: 1 and 2Sample Estimation
Probability and Statistics Lecture 9: 1 and Sample Estimation to accompany Probability and Statistics for Engineers and Scientists Fatih Cavdur Introduction A statistic θ is said to be an unbiased estimator
More informationChapter 2. Hypothesis testing in one population
Chapter 2. Hypothesis testing in one population Contents Introduction, the null and alternative hypotheses Hypothesis testing process Type I and Type II errors, power Test statistic, level of significance
More information3.4 Statistical inference for 2 populations based on two samples
3.4 Statistical inference for 2 populations based on two samples Tests for a difference between two population means The first sample will be denoted as X 1, X 2,..., X m. The second sample will be denoted
More informationStatistiek (WISB361)
Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17
More informationInferences About Differences Between Means Edpsy 580
Inferences About Differences Between Means Edpsy 580 Carolyn J. Anderson Department of Educational Psychology University of Illinois at UrbanaChampaign Inferences About Differences Between Means Slide
More informationChapter 1 Hypothesis Testing
Chapter 1 Hypothesis Testing Principles of Hypothesis Testing tests for one sample case 1 Statistical Hypotheses They are defined as assertion or conjecture about the parameter or parameters of a population,
More informationData Analysis. Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) SS Analysis of Experiments  Introduction
Data Analysis Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) Prof. Dr. Dr. h.c. Dieter Rombach Dr. Andreas Jedlitschka SS 2014 Analysis of Experiments  Introduction
More informationChapter 7. Section Introduction to Hypothesis Testing
Section 7.1  Introduction to Hypothesis Testing Chapter 7 Objectives: State a null hypothesis and an alternative hypothesis Identify type I and type II errors and interpret the level of significance Determine
More informationTwosample hypothesis testing, I 9.07 3/09/2004
Twosample hypothesis testing, I 9.07 3/09/2004 But first, from last time More on the tradeoff between Type I and Type II errors The null and the alternative: Sampling distribution of the mean, m, given
More informationGeneral Procedure for Hypothesis Test. Five types of statistical analysis. 1. Formulate H 1 and H 0. General Procedure for Hypothesis Test
Five types of statistical analysis General Procedure for Hypothesis Test Descriptive Inferential Differences Associative Predictive What are the characteristics of the respondents? What are the characteristics
More informationSingle sample hypothesis testing, II 9.07 3/02/2004
Single sample hypothesis testing, II 9.07 3/02/2004 Outline Very brief review Onetailed vs. twotailed tests Small sample testing Significance & multiple tests II: Data snooping What do our results mean?
More informationMATH 10: Elementary Statistics and Probability Chapter 9: Hypothesis Testing with One Sample
MATH 10: Elementary Statistics and Probability Chapter 9: Hypothesis Testing with One Sample Tony Pourmohamad Department of Mathematics De Anza College Spring 2015 Objectives By the end of this set of
More informationLAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING
LAB 4 INSTRUCTIONS CONFIDENCE INTERVALS AND HYPOTHESIS TESTING In this lab you will explore the concept of a confidence interval and hypothesis testing through a simulation problem in engineering setting.
More informationChapter 9: Hypothesis Testing Sections
Chapter 9: Hypothesis Testing Sections Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 TwoSided Alternatives 9.5 The t Test 9.6 Comparing the Means of Two Normal
More informationIntroduction to Hypothesis Testing
I. Terms, Concepts. Introduction to Hypothesis Testing A. In general, we do not know the true value of population parameters  they must be estimated. However, we do have hypotheses about what the true
More informationPrinciples of Hypothesis Testing for Public Health
Principles of Hypothesis Testing for Public Health Laura Lee Johnson, Ph.D. Statistician National Center for Complementary and Alternative Medicine johnslau@mail.nih.gov Fall 2011 Answers to Questions
More informationLesson 1: Comparison of Population Means Part c: Comparison of Two Means
Lesson : Comparison of Population Means Part c: Comparison of Two Means Welcome to lesson c. This third lesson of lesson will discuss hypothesis testing for two independent means. Steps in Hypothesis
More informationIntroduction to Hypothesis Testing. Point estimation and confidence intervals are useful statistical inference procedures.
Introduction to Hypothesis Testing Point estimation and confidence intervals are useful statistical inference procedures. Another type of inference is used frequently used concerns tests of hypotheses.
More informationExample Hypotheses. Chapter 82: Basics of Hypothesis Testing. A newspaper headline makes the claim: Most workers get their jobs through networking
Chapter 82: Basics of Hypothesis Testing Two main activities in statistical inference are using sample data to: 1. estimate a population parameter forming confidence intervals 2. test a hypothesis or
More informationSampling and Hypothesis Testing
Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus
More informationHypothesis Testing (unknown σ)
Hypothesis Testing (unknown σ) Business Statistics Recall: Plan for Today Null and Alternative Hypotheses Types of errors: type I, type II Types of correct decisions: type A, type B Level of Significance
More informationName: Date: Use the following to answer questions 34:
Name: Date: 1. Determine whether each of the following statements is true or false. A) The margin of error for a 95% confidence interval for the mean increases as the sample size increases. B) The margin
More informationChapter 9: Hypothesis Testing Sections
Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 TwoSided Alternatives 9.6 Comparing the
More informationAn Introduction to Statistics Course (ECOE 1302) Spring Semester 2011 Chapter 10 TWOSAMPLE TESTS
The Islamic University of Gaza Faculty of Commerce Department of Economics and Political Sciences An Introduction to Statistics Course (ECOE 130) Spring Semester 011 Chapter 10 TWOSAMPLE TESTS Practice
More informationAnalysis of numerical data S4
Basic medical statistics for clinical and experimental research Analysis of numerical data S4 Katarzyna Jóźwiak k.jozwiak@nki.nl 3rd November 2015 1/42 Hypothesis tests: numerical and ordinal data 1 group:
More informationSimple Linear Regression Inference
Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation
More informationTesting a claim about a population mean
Introductory Statistics Lectures Testing a claim about a population mean One sample hypothesis test of the mean Department of Mathematics Pima Community College Redistribution of this material is prohibited
More informationChapter 5: Basic Statistics and Hypothesis Testing
Chapter 5: Basic Statistics and Hypothesis Testing In this chapter: 1. Viewing the tvalue from an OLS regression (UE 5.2.1) 2. Calculating critical tvalues and applying the decision rule (UE 5.2.2) 3.
More informationNull Hypothesis H 0. The null hypothesis (denoted by H 0
Hypothesis test In statistics, a hypothesis is a claim or statement about a property of a population. A hypothesis test (or test of significance) is a standard procedure for testing a claim about a property
More informationHypoTesting. Name: Class: Date: Multiple Choice Identify the choice that best completes the statement or answers the question.
Name: Class: Date: HypoTesting Multiple Choice Identify the choice that best completes the statement or answers the question. 1. A Type II error is committed if we make: a. a correct decision when the
More informationSample Size and Power in Clinical Trials
Sample Size and Power in Clinical Trials Version 1.0 May 011 1. Power of a Test. Factors affecting Power 3. Required Sample Size RELATED ISSUES 1. Effect Size. Test Statistics 3. Variation 4. Significance
More informationClass 19: Two Way Tables, Conditional Distributions, ChiSquare (Text: Sections 2.5; 9.1)
Spring 204 Class 9: Two Way Tables, Conditional Distributions, ChiSquare (Text: Sections 2.5; 9.) Big Picture: More than Two Samples In Chapter 7: We looked at quantitative variables and compared the
More informationNonparametric Methods for Two Samples. Nonparametric Methods for Two Samples
Nonparametric Methods for Two Samples An overview In the independent twosample ttest, we assume normality, independence, and equal variances. This ttest is robust against nonnormality, but is sensitive
More informationChapter 7 Part 2. Hypothesis testing Power
Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship
More informationChapter 9, Part A Hypothesis Tests. Learning objectives
Chapter 9, Part A Hypothesis Tests Slide 1 Learning objectives 1. Understand how to develop Null and Alternative Hypotheses 2. Understand Type I and Type II Errors 3. Able to do hypothesis test about population
More informationChapter 1112 1 Review
Chapter 1112 Review Name 1. In formulating hypotheses for a statistical test of significance, the null hypothesis is often a statement of no effect or no difference. the probability of observing the data
More informationDevelop hypothesis and then research to find out if it is true. Derived from theory or primary question/research questions
Chapter 12 Hypothesis Testing Learning Objectives Examine the process of hypothesis testing Evaluate research and null hypothesis Determine one or twotailed tests Understand obtained values, significance,
More information7 Hypothesis testing  one sample tests
7 Hypothesis testing  one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X
More informationMind on Statistics. Chapter 12
Mind on Statistics Chapter 12 Sections 12.1 Questions 1 to 6: For each statement, determine if the statement is a typical null hypothesis (H 0 ) or alternative hypothesis (H a ). 1. There is no difference
More informationProbability of rejecting the null hypothesis when
Sample Size The first question faced by a statistical consultant, and frequently the last, is, How many subjects (animals, units) do I need? This usually results in exploring the size of the treatment
More informationStudy Guide for the Final Exam
Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make
More informationNull Hypothesis Significance Testing Signifcance Level, Power, ttests. 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom
Null Hypothesis Significance Testing Signifcance Level, Power, ttests 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom Simple and composite hypotheses Simple hypothesis: the sampling distribution is
More informationRecall this chart that showed how most of our course would be organized:
Chapter 4 OneWay ANOVA Recall this chart that showed how most of our course would be organized: Explanatory Variable(s) Response Variable Methods Categorical Categorical Contingency Tables Categorical
More informationHYPOTHESIS TESTING WITH SPSS:
HYPOTHESIS TESTING WITH SPSS: A NONSTATISTICIAN S GUIDE & TUTORIAL by Dr. Jim Mirabella SPSS 14.0 screenshots reprinted with permission from SPSS Inc. Published June 2006 Copyright Dr. Jim Mirabella CHAPTER
More informationOutline of Topics. Statistical Methods I. Types of Data. Descriptive Statistics
Statistical Methods I Tamekia L. Jones, Ph.D. (tjones@cog.ufl.edu) Research Assistant Professor Children s Oncology Group Statistics & Data Center Department of Biostatistics Colleges of Medicine and Public
More informationAP Statistics 2002 Scoring Guidelines
AP Statistics 2002 Scoring Guidelines The materials included in these files are intended for use by AP teachers for course and exam preparation in the classroom; permission for any other use must be sought
More informationDesign and Analysis of Equivalence Clinical Trials Via the SAS System
Design and Analysis of Equivalence Clinical Trials Via the SAS System Pamela J. Atherton Skaff, Jeff A. Sloan, Mayo Clinic, Rochester, MN 55905 ABSTRACT An equivalence clinical trial typically is conducted
More informationModule 5 Hypotheses Tests: Comparing Two Groups
Module 5 Hypotheses Tests: Comparing Two Groups Objective: In medical research, we often compare the outcomes between two groups of patients, namely exposed and unexposed groups. At the completion of this
More information6. Statistical Inference: Significance Tests
6. Statistical Inference: Significance Tests Goal: Use statistical methods to check hypotheses such as Women's participation rates in elections in France is higher than in Germany. (an effect) Ethnic divisions
More informationUCLA STAT 13 Statistical Methods  Final Exam Review Solutions Chapter 7 Sampling Distributions of Estimates
UCLA STAT 13 Statistical Methods  Final Exam Review Solutions Chapter 7 Sampling Distributions of Estimates 1. (a) (i) µ µ (ii) σ σ n is exactly Normally distributed. (c) (i) is approximately Normally
More informationMultiple Hypothesis Testing: The Ftest
Multiple Hypothesis Testing: The Ftest Matt Blackwell December 3, 2008 1 A bit of review When moving into the matrix version of linear regression, it is easy to lose sight of the big picture and get lost
More informationChapter 7 Notes  Inference for Single Samples. You know already for a large sample, you can invoke the CLT so:
Chapter 7 Notes  Inference for Single Samples You know already for a large sample, you can invoke the CLT so: X N(µ, ). Also for a large sample, you can replace an unknown σ by s. You know how to do a
More informationHYPOTHESIS TESTING (ONE SAMPLE)  CHAPTER 7 1. used confidence intervals to answer questions such as...
HYPOTHESIS TESTING (ONE SAMPLE)  CHAPTER 7 1 PREVIOUSLY used confidence intervals to answer questions such as... You know that 0.25% of women have red/green color blindness. You conduct a study of men
More informationHYPOTHESIS TESTING (ONE SAMPLE)  CHAPTER 7 1. used confidence intervals to answer questions such as...
HYPOTHESIS TESTING (ONE SAMPLE)  CHAPTER 7 1 PREVIOUSLY used confidence intervals to answer questions such as... You know that 0.25% of women have red/green color blindness. You conduct a study of men
More informationAP STATISTICS 2009 SCORING GUIDELINES (Form B)
AP STATISTICS 2009 SCORING GUIDELINES (Form B) Question 5 Intent of Question The primary goals of this question were to assess students ability to (1) state the appropriate hypotheses, (2) identify and
More informationChapter III. Testing Hypotheses
Chapter III Testing Hypotheses R (Introduction) A statistical hypothesis is an assumption about a population parameter This assumption may or may not be true The best way to determine whether a statistical
More informationHYPOTHESIS TESTING AND TYPE I AND TYPE II ERROR
HYPOTHESIS TESTING AND TYPE I AND TYPE II ERROR Hypothesis is a conjecture (an inferring) about one or more population parameters. Null Hypothesis (H 0 ) is a statement of no difference or no relationship
More informationIntroduction to Hypothesis Testing OPRE 6301
Introduction to Hypothesis Testing OPRE 6301 Motivation... The purpose of hypothesis testing is to determine whether there is enough statistical evidence in favor of a certain belief, or hypothesis, about
More informationIntroduction. Hypothesis Testing. Hypothesis Testing. Significance Testing
Introduction Hypothesis Testing Mark Lunt Arthritis Research UK Centre for Ecellence in Epidemiology University of Manchester 13/10/2015 We saw last week that we can never know the population parameters
More informationCHAPTER 11 SECTION 2: INTRODUCTION TO HYPOTHESIS TESTING
CHAPTER 11 SECTION 2: INTRODUCTION TO HYPOTHESIS TESTING MULTIPLE CHOICE 56. In testing the hypotheses H 0 : µ = 50 vs. H 1 : µ 50, the following information is known: n = 64, = 53.5, and σ = 10. The standardized
More informationHomework 6 Solutions
Math 17, Section 2 Spring 2011 Assignment Chapter 20: 12, 14, 20, 24, 34 Chapter 21: 2, 8, 14, 16, 18 Chapter 20 20.12] Got Milk? The student made a number of mistakes here: Homework 6 Solutions 1. Null
More informationChapter 4: Statistical Hypothesis Testing
Chapter 4: Statistical Hypothesis Testing Christophe Hurlin November 20, 2015 Christophe Hurlin () Advanced Econometrics  Master ESA November 20, 2015 1 / 225 Section 1 Introduction Christophe Hurlin
More informationIntroduction to. Hypothesis Testing CHAPTER LEARNING OBJECTIVES. 1 Identify the four steps of hypothesis testing.
Introduction to Hypothesis Testing CHAPTER 8 LEARNING OBJECTIVES After reading this chapter, you should be able to: 1 Identify the four steps of hypothesis testing. 2 Define null hypothesis, alternative
More informationSelected Nonparametric and Parametric Statistical Tests for TwoSample Cases 1
Selected Nonparametric and Parametric Statistical Tests for TwoSample Cases The Tstatistic is used to test differences in the means of two groups. The grouping variable is categorical and data for the
More informationHypothesis Testing COMP 245 STATISTICS. Dr N A Heard. 1 Hypothesis Testing 2 1.1 Introduction... 2 1.2 Error Rates and Power of a Test...
Hypothesis Testing COMP 45 STATISTICS Dr N A Heard Contents 1 Hypothesis Testing 1.1 Introduction........................................ 1. Error Rates and Power of a Test.............................
More informationNonparametric Test Procedures
Nonparametric Test Procedures 1 Introduction to Nonparametrics Nonparametric tests do not require that samples come from populations with normal distributions or any other specific distribution. Hence
More informationSAMPLING & INFERENTIAL STATISTICS. Sampling is necessary to make inferences about a population.
SAMPLING & INFERENTIAL STATISTICS Sampling is necessary to make inferences about a population. SAMPLING The group that you observe or collect data from is the sample. The group that you make generalizations
More informationStatistics  Written Examination MEC Students  BOVISA
Statistics  Written Examination MEC Students  BOVISA Prof.ssa A. Guglielmi 26.0.2 All rights reserved. Legal action will be taken against infringement. Reproduction is prohibited without prior consent.
More informationPower and Sample Size Determination
Power and Sample Size Determination Bret Hanlon and Bret Larget Department of Statistics University of Wisconsin Madison November 3 8, 2011 Power 1 / 31 Experimental Design To this point in the semester,
More informationStatistical Significance and Bivariate Tests
Statistical Significance and Bivariate Tests BUS 735: Business Decision Making and Research 1 1.1 Goals Goals Specific goals: Refamiliarize ourselves with basic statistics ideas: sampling distributions,
More informationStatistical Inference
Statistical Inference Idea: Estimate parameters of the population distribution using data. How: Use the sampling distribution of sample statistics and methods based on what would happen if we used this
More informationHypothesis Testing Summary
Hypothesis Testing Summary Hypothesis testing begins with the drawing of a sample and calculating its characteristics (aka, statistics ). A statistical test (a specific form of a hypothesis test) is an
More informationHypothesis Testing and Confidence Interval Estimation
Biostatistics for Health Care Researchers: A Short Course Hypothesis Testing and Confidence Interval Estimation Presented ed by: Susan M. Perkins, Ph.D. Division of Biostatistics Indiana University School
More information1 SAMPLE SIGN TEST. NonParametric Univariate Tests: 1 Sample Sign Test 1. A nonparametric equivalent of the 1 SAMPLE TTEST.
NonParametric Univariate Tests: 1 Sample Sign Test 1 1 SAMPLE SIGN TEST A nonparametric equivalent of the 1 SAMPLE TTEST. ASSUMPTIONS: Data is nonnormally distributed, even after log transforming.
More informationLecture 13 More on hypothesis testing
Lecture 13 More on hypothesis testing Thais Paiva STA 111  Summer 2013 Term II July 22, 2013 1 / 27 Thais Paiva STA 111  Summer 2013 Term II Lecture 13, 07/22/2013 Lecture Plan 1 Type I and type II error
More informationChapter 9: Hypothesis Testing Sections
Chapter 9: Hypothesis Testing Sections  we are still here Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 TwoSided Alternatives 9.5 The t Test 9.6 Comparing the
More informationMONT 107N Understanding Randomness Solutions For Final Examination May 11, 2010
MONT 07N Understanding Randomness Solutions For Final Examination May, 00 Short Answer (a) (0) How are the EV and SE for the sum of n draws with replacement from a box computed? Solution: The EV is n times
More informationChapter 8 Hypothesis Testing Chapter 8 Hypothesis Testing 81 Overview 82 Basics of Hypothesis Testing
Chapter 8 Hypothesis Testing 1 Chapter 8 Hypothesis Testing 81 Overview 82 Basics of Hypothesis Testing 83 Testing a Claim About a Proportion 85 Testing a Claim About a Mean: s Not Known 86 Testing
More information