Chapter 4: Statistical Hypothesis Testing

Chapter 4: Statistical Hypothesis Testing Christophe Hurlin November 20, 2015 Christophe Hurlin () Advanced Econometrics - Master ESA November 20, 2015 1 / 225

Section 1 Introduction Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 2 / 225

1. Introduction The outline of this chapter is the following: Section 2. Statistical hypothesis testing Section 3. Tests in the multiple linear regression model Subsection 3.1. The Student test Subsection 3.2. The Fisher test Section 4. MLE and Inference Subsection 4.1. The Likelihood Ratio (LR) test Subsection 4.2.The Wald test Subsection 4.3. The Lagrange Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 3 / 225

1. Introduction References Amemiya T. (1985), Advanced Econometrics. Harvard University Press. Greene W. (2007), Econometric Analysis, sixth edition, Pearson - Prentice Hil (recommended) Ruud P., (2000) An introduction to Classical Econometric Theory, Oxford University Press. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 4 / 225

1. Introduction Notations: In this chapter, I will (try to...) follow some conventions of notation. f Y (y) F Y (y) Pr () y Y probability density or mass function cumulative distribution function probability vector matrix Be careful: in this chapter, I don t distinguish between a random vector (matrix) and a vector (matrix) of deterministic elements (except in section 2). For more appropriate notations, see: Abadir and Magnus (2002), Notation in econometrics: a proposal for a standard, Econometrics Journal. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 5 / 225

Section 2 Statistical hypothesis testing Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 6 / 225

2. Statistical hypothesis testing Objectives The objective of this section is to de ne the following concepts: 1 Null and alternative hypotheses 2 One-sided and two-sided tests 3 Rejection region, test statistic and critical value 4 Size, power and power function 5 Uniformly most powerful (UMP) test 6 Neyman Pearson lemma 7 Consistent test and unbiased test 8 p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 7 / 225

2. Statistical hypothesis testing Introduction 1 A statistical hypothesis test is a method of making decisions or a rule of decision (as concerned a statement about a population parameter) using the data of sample. 2 Statistical hypothesis tests de ne a procedure that controls ( xes) the probability of incorrectly deciding that a default position (null hypothesis) is incorrect based on how likely it would be for a set of observations to occur if the null hypothesis were true. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 8 / 225

2. Statistical hypothesis testing Introduction (cont d) In general we distinguish two types of tests: 1 The parametric tests assume that the data have come from a type of probability distribution and makes inferences about the parameters of the distribution 2 The non-parametric tests refer to tests that do not assume the data or population have any characteristic structure or parameters. In this course, we only consider the parametric tests. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 9 / 225

2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 10 / 225

2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 11 / 225

2. Statistical hypothesis testing De nition (Hypothesis) A hypothesis is a statement about a population parameter. The formal testing procedure involves a statement of the hypothesis, usually in terms of a null or maintained hypothesis and an alternative, conventionally denoted H 0 and H 1, respectively. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 12 / 225

2. Statistical hypothesis testing Introduction 1 The null hypothesis refers to a general or default position: that there is no relationship between two measured phenomena or that a potential medical treatment has no e ect. 2 The costs associated to the violation of the null must be higher than the cost of a violation of the alternative. Example (Choice of the null hypothesis) In a credit scoring problem, in general we have: H 0 : the client is not risky(acceptance of the loan) versus H 1 : the client is risky (refusal of the loan). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 13 / 225

2. Statistical hypothesis testing De nition (Simple and composite hypotheses) A simple hypothesis speci es the population distribution completely. A composite hypothesis does not specify the population distribution completely. Example (Simple and composite hypotheses) If X t (θ), H 0 : θ = θ 0 is a simple hypothesis. H 1 : θ > θ 0, H 1 : θ < θ 0, and H 1 : θ 6= θ 0 are composite hypotheses. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 14 / 225

2. Statistical hypothesis testing De nition (One-sided test) A one-sided test has the general form: H 0 : θ = θ 0 or H 0 : θ = θ 0 H 1 : θ < θ 0 H 1 : θ > θ 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 15 / 225

2. Statistical hypothesis testing De nition (Two-sided test) A two-sided test has the general form: H 0 : θ = θ 0 H 1 : θ 6= θ 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 16 / 225

2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 17 / 225

2. Statistical hypothesis testing De nition (Rejection region) The rejection region is the set of values of the test statistic (or equivalently the set of samples) for which the null hypothesis is rejected. The rejection region is denoted W. For example, a standard rejection region W is of the form: or equivalently W = fx : T (x) > cg W = fx 1,.., x N : T (x 1,.., x N ) > cg where x denotes a sample fx 1,.., x N g, T (x) the realisation of a test statistic and c the critical value. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 18 / 225

2. Statistical hypothesis testing Remarks 1 A (hypothesis) test is thus a rule that speci es: 1 For which sample values the decision is made to fail to reject H0 as true; 2 For which sample values the decision is made to reject H0. 3 Never say "Accept H1", "fail to reject H1" etc.. 2 The complement of the rejection region is the non-rejection region. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 19 / 225

2. Statistical hypothesis testing Remark The rejection region is de ned as to be: W = fx : T (x) {z } test statistic 7 c {z} g critical value T (x) is the realisation of the statistic (random variable): T (X ) = T (X 1,.., X N ) The test statistic T (X ) has an exact or an asymptotic distribution D under the null H 0. T (X ) H0 D or T (X ) d! H0 D Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 20 / 225

2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 21 / 225

2. Statistical hypothesis testing Decision Fail to reject H 0 Reject H 0 Truth H 0 Correct decision Type I error H 1 Type II error Correct decision Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 22 / 225

2. Statistical hypothesis testing De nition (Size) The probability of a type I error is the (nominal) size of the test. This is conventionally denoted α and is also called the signi cance level. α = Pr (Wj H 0 ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 23 / 225

2. Statistical hypothesis testing Remark For a simple null hypothesis: α = Pr (Wj H 0 ) For a composite null hypothesis: α = sup Pr (Wj H 0 ) θ 0 2H 0 A test is said to have level if its size is less than or equal to α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 24 / 225

2. Statistical hypothesis testing De nition (Power) The power of a test is the probability that it will correctly lead to rejection of a false null hypothesis: power = Pr (Wj H 1 ) = 1 β where β denotes the probability of type II error, i.e. β = Pr W denotes the non-rejection region. W H 1 and Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 25 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m = m 1 with m 1 < m 0. An econometrician propose the following rule of decision: W = fx : x N < cg where X N = N 1 N i=1 X i denotes the sample mean and c is a constant (critical value). Question: calculate the size and the power of this test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 26 / 225

2. Statistical hypothesis testing Solution The rejection region is W= fx : x N < cg. Under the null H 0 : m = m 0 : X N H0 N So, the size of the test is equal to: m 0, σ2 N α = Pr (Wj H 0 ) = Pr X N < c H 0 X = N m 0 Pr σ/ p N < c m 0 σ/ p N H 0 c m0 = Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 27 / 225

2. Statistical hypothesis testing Solution (cont d) The rejection region is W= fx : x N < cg. Under the alternative H 1 : m = m 1 : X N N m 1, σ2 H1 N So, the power of the test is equal to: power = Pr (Wj H 1 ) X = N m 1 Pr σ/ p N < c m 1 σ/ p N H 1 c m1 = Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 28 / 225

2. Statistical hypothesis testing Solution (cont d) In conclusion: c m0 α = Φ σ/ p N c m1 β = 1 power = 1 Φ σ/ p N We have a system of two equations with three parameters: α, β (or power) and the critical value c. 1 There is a trade-o between the probabilities of the errors of type I and II, i.e. α and β : if c decreases, α decreases but β increases. 2 A solution is to impose a size α and determine the critical value and the power. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 29 / 225

2. Statistical hypothesis testing Solution (cont d) In order to illustrate the tradeo between α and β given the critical value c, take an example with σ 2 = 1 and N = 100: H 0 : m = m 0 = 1.2 H 1 : m = m 1 = 1 X N N m 0, σ2 X N N m 1, σ2 H0 N H1 N We have W = fx : x N < cg c m0 α = Pr (Wj H 0 ) = Φ σ/ p = Φ (10 (c 1.2)) N β = Pr W c m1 H 1 = 1 Φ σ/ p = 1 Φ (10 (c 1)) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 30 / 225

2. Statistical hypothesis testing 4.5 4 Density under H0 Density under H1 3.5 3 2.5 α=pr(w[h0)=5% β=1 Pr(W H1)=36.12% 2 1.5 1 0.5 0 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 31 / 225

2. Statistical hypothesis testing Click me! Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 32 / 225

2. Statistical hypothesis testing Fact (Critical value) The (nominal) size α is xed by the analyst and the critical value is deduced from α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 33 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2, N = 100 and σ 2 = 1. We want to test H 0 : m = 1.2 H 1 : m = 1 An econometrician propose the following rule of decision: W = fx : x N < cg where X N = N 1 N i=1 X i denotes the sample mean and c is a constant (critical value). Questions: (1) what is the critical value of the test of size α = 5%? (2) what is the power of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 34 / 225

2. Statistical hypothesis testing Solution We know that: c m0 α = Pr (Wj H 0 ) = Φ σ/ p N So, the critical value that corresponds to a signi cance level of α is: c = m 0 + σ p N Φ 1 (α) NA: if m 0 = 1.2, m 1 = 1, N = 100, σ 2 = 1 and α = 5%, then the rejection region is W = fx : x N < 1.0355g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 35 / 225

2. Statistical hypothesis testing Solution (cont d) W = x : x N < m 0 + p σ Φ 1 (α) N The power of the test is: c m1 power = Pr (Wj H 1 ) = Φ σ/ p N Given the critical value, we have: m0 m 1 power = Φ σ/ p N + Φ 1 (α) NA: if m 0 = 1.2, m 1 = 1, N = 100, σ 2 = 1 and α = 5%: 1.2 1 power = Φ 1/ p 100 + Φ 1 (0.05) = 0.6388 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 36 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 with σ 2 = 1 and N = 100. We want to test H 0 : m = 1.2 H 1 : m = 1 The rejection region for a signi cance level α = 5% is: W = fx : x N < 1.0355g where X N = N 1 N i=1 X i denotes the sample mean. Question: if the realisation of the sample mean is equal to 1.13, what is the conclusion of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 37 / 225

2. Statistical hypothesis testing Solution (cont d) For a nominal size α = 5%, the rejection region is: W = fx : x N < 1.0355g If we observe x N = 1.13 This realisation does not belong to the rejection region: x N /2 W For a level of 5%, we do not reject the null hypothesis H 0 : m = 1.2. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 38 / 225

2. Statistical hypothesis testing De nition (Power function) In general, the alternative hypothesis is composite. In this case, the power is a function of the value of the parameter under the alternative. power = P (θ) 8θ 2 H 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 39 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m < m 0 Consider the following rule of decision: W = x : x N < m 0 + p σ Φ 1 (α) N Questions: What is the power function of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 40 / 225

2. Statistical hypothesis testing Solution As in the previous case, we have: m0 m power = P (m) = Φ σ/ p N + Φ 1 (α) with m < m 0 NA: if m 0 = 1.2, N = 100, σ 2 = 1 and α = 5%. 1.2 m P (m) = Φ 1.6449 with m < m 0 1/10 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 41 / 225

2. Statistical hypothesis testing Power function P (m) 1.2 1 0.8 0.6 0.4 0.2 0 0.7 0.8 0.9 1 1.1 1.2 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 42 / 225

2. Statistical hypothesis testing Example (Power function) Consider a test H 0 : θ = θ 0 versus H 1 : θ 6= θ 0, the power function has this general form: Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 43 / 225

2. Statistical hypothesis testing De nition (Most powerful test) A test (denoted A) is uniformly most powerful (UMP) if it has greater power than any other test of the same size for all admissible values of the parameter. α A = α B = α for any test B of size α. β A β B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 44 / 225

2. Statistical hypothesis testing UMP tests How to derive the rejection region of the UMP test of size α? => the Neyman Pearson lemma Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 45 / 225

2. Statistical hypothesis testing Lemma (Neyman Pearson) Consider a hypothesis test between two point hypotheses H 0 : θ = θ 0 and H 1 : θ = θ 1. The uniformly most powerful (UMP) test has a rejection region de ned by: W = xj L N (θ 0 ; x) L N (θ 1 ; x) < K where L N (θ 0 ; x) denotes the likelihood of the sample x and K is a constant determined by the size α such that: LN (θ 0 ; X ) Pr L N (θ 1 ; X ) < K H 0 = α Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 46 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m = m 1 with m 1 > m 0. Question: What is the rejection region of the UMP test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 47 / 225

2. Statistical hypothesis testing Solution Since X 1,.., X N are N.i.d. m, σ 2, the likelihood of the sample fx 1,.., x N g is de ned as to be (cf. chapter 2):! N 1 L N (θ; x) = σ N (2π) N /2 exp 1 2σ 2 (x i m) 2 i=1 Given the Neyman Pearson lemma the rejection region of the UMP test of size α is given by: L N (θ 0 ; x) L N (θ 1 ; x) < K where K is a constant determined by the size α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 48 / 225

2. Statistical hypothesis testing Solution (cont d) 1 exp σ N (2π) N /2 1 σ N (2π) N /2 exp 1 2σ 2 N i=1 (x i 1 2σ 2 N i=1 (x i m 0 ) 2 m 1 ) 2 < K This expression can rewritten as: 1 exp 2σ 2 N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K () N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K 1 where K 1 = 2σ 2 ln (K ) is a constant. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 49 / 225

2. Statistical hypothesis testing Solution (cont d) N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K 1 () 2 (m 0 m 1 ) N i=1 x i + N m 2 1 m 2 0 < K1 () (m 0 m 1 ) N i=1 x i < K 2 where K 2 = K 1 N m 2 1 m 2 0 /2 is a constant. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 50 / 225

2. Statistical hypothesis testing Solution (cont d) Since m 1 > m 0, we have where K 3 = K 2 / (N (m 0 (m 0 m 1 ) N i=1 x i < K 2 1 N N i=1 x i > K 3 m 1 )) is a constant. The rejection region of the UMP test for H 0 : m = m 0 against H 0 : m = m 1 with m 1 > m 0 has the general form: where A is a constant. W = fx : x N > Ag Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 51 / 225

2. Statistical hypothesis testing Solution (cont d) W = fx : x N > Ag Determine the critical value A from the nominal size: α = Pr (Wj H 0 ) = Pr (x N > Aj H 0 ) X = N m 0 1 Pr σ/ p N < A m 0 σ/ p N H 0 A m0 = 1 Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 52 / 225

2. Statistical hypothesis testing Solution (cont d) So, we have α = 1 A = m 0 + A m0 Φ σ/ p N σ p N Φ 1 (1 α) The rejection region of the UMP test of size α for H 0 : m = m 0 against H 0 : m = m 1 with m 1 > m 0 is: W = x : x N > m 0 + p σ Φ 1 (1 α) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 53 / 225

2. Statistical hypothesis testing Fact (UMP one-sided test) For a one-sided test H 0 : θ = θ 0 against H 1 : θ > θ 0 (or H 1 : θ < θ 1 ) the rejection region W of the UMP test is equivalent to the rejection region obtained for the test H 0 : θ = θ 0 against H 1 : θ = θ 1 with for θ 1 > θ 0 (or θ 1 < θ 0 ) if this region does not depend on the value of θ 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 54 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m > m 0 Question: What is the rejection region of the UMP test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 55 / 225

2. Statistical hypothesis testing Solution Consider the test: H 0 : m = m 0 H 1 : m = m 1 with m 1 > m 0. The rejection region of the UMP test of size α is: W = x : x N > m 0 + p σ Φ 1 (1 α) N W does not depend on m 1. It is also the rejection region of the UMP one-sided test for H 0 : m = m 0 H 1 : m > m 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 56 / 225

2. Statistical hypothesis testing Fact (Two-sided test) For a two-sided test H 0 : θ = θ 0 against H 1 : θ 6= θ 0 the non rejection region W of the test of size α is the intersection of the non rejection regions of the corresponding one-sided UMP tests of size α/2 Test A: H 0 : θ = θ 0 against H 1 : θ > θ 0 Test B: H 0 : θ = θ 0 against H 1 : θ < θ 0 So, we have: W = W A \ W B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 57 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m 6= m 0 Question: What is the rejection region of the test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 58 / 225

2. Statistical hypothesis testing Solution Consider the one-sided tests: Test A: H 0 : m = m 0 against H 1 : m < m 0 Test B: H 0 : m = m 0 against H 1 : m > m 0 The rejection regions of the UMP test of size α/2 are: W A = x : x N < m 0 + p σ Φ 1 (α/2) N W B = x : x N > m 0 + p σ Φ 1 (1 N α/2) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 59 / 225

2. Statistical hypothesis testing Solution (cont d) The non-rejection regions of the UMP test of size α/2 are: W A = x : x N m 0 + p σ Φ 1 (α/2) N W B = x : x N m 0 + p σ Φ 1 (1 N α/2) The non rejection region of the two-sided test corresponds to the intersection of these two regions: W = W A \ W B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 60 / 225

2. Statistical hypothesis testing Solution (cont d) So, non rejection region of the two-sided test of size α is: W = x : m 0 + p σ Φ 1 (α/2) x N m 0 + p σ Φ 1 (1 N N α/2) Since, Φ 1 (α/2) = Φ 1 (1 α/2), this region can be rewritten as: W = x : jx N m 0 j p σ Φ 1 (1 α/2) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 61 / 225

2. Statistical hypothesis testing Solution (cont d) W = x : jx N m 0 j σ p N Φ 1 (1 α/2) Finally, the rejection region of the two-sided test of size α is: W = x : jx N m 0 j > p σ Φ 1 (1 α/2) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 62 / 225

2. Statistical hypothesis testing Solution (cont d) W = x : jx N m 0 j > σ p N Φ 1 (1 α/2) NA: if m 0 = 1.2, N = 100, σ 2 = 1 and α = 5%: W = x : jx N 1.2j > 1 10 Φ 1 (0.975) W = fx : jx N 1.2j > 0.1960g If the realisation of jx N 1.2j is larger than 0.1960, we reject the null H 0 : m = 1.2 for a signi cance level of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 63 / 225

2. Statistical hypothesis testing De nition (Unbiased Test) A test is unbiased if its power P (θ) is greater than or equal to its size α for all values of the parameter θ. P (θ) α 8θ 2 H 1 By construction, we have P (θ 0 ) = Pr (Wj Hj 0 ) = α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 64 / 225

2. Statistical hypothesis testing De nition (Consistent Test) A test is consistent if its power goes to one as the sample size grows to in nity. lim N! P (θ) = 1 8θ 2 H 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 65 / 225

2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m < m 0 The rejection region of the UMP test of size α is W = x : x N < m 0 + p σ Φ 1 (α) N Question: show that this test is (1) unbiased and (2) consistent. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 66 / 225

2. Statistical hypothesis testing Solution W = x : x N < m 0 + p σ Φ 1 (α) N The power function of the test is de ned as to be: P (m) = Pr (Wj H 1 ) = Pr X N < m 0 + p σ Φ 1 (α) N m < m 0 m0 m = Φ σ/ p N + Φ 1 (α) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 67 / 225

2. Statistical hypothesis testing Solution m0 m P (m) = Φ σ/ p N + Φ 1 (α) The test is consistent since: 8m < m 0 lim P (m) = 1 N! The test is unbiased since P (m) α 8m < m 0 lim m!m 0 P (m) = Φ Φ 1 (α) = α Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 68 / 225

2. Statistical hypothesis testing Solution The decision Reject H0 or fail to reject H0 is not so informative! Indeed, there is some arbitrariness to the choice of α (level). Another strategy is to ask, for every α, whether the test rejects at that level. Another alternative is to use the so-called p-value the smallest level of signi cance at which H 0 would be rejected given the value of the test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 69 / 225

2. Statistical hypothesis testing De nition (p-value) Suppose that for every α 2 [0, 1], one has a size α test with rejection region W α. Then, the p-value is de ned to be: p-value = inf fα : T (y) 2 W α g The p-value is the smallest level at which one can reject H 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 70 / 225

2. Statistical hypothesis testing The p-value is a measure of evidence against H 0 : p-value evidence < 0.01 Very strong evidence against H 0 0.01 0.05 Strong evidence against H 0 0.05 0.10 Weak evidence against H 0 > 0.10 Little or no evidence against H 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 71 / 225

2. Statistical hypothesis testing Remarks 1 A large p-value does not mean strong evidence in favor of H0. 2 A large p-value can occur for two reasons: 1 H0 is true; 2 H0 is false but the test has low power. 3 The p-value is not the probability that the null hypothesis is true! Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 72 / 225

2. Statistical hypothesis testing For a nominal size of 5%, we reject the null H 0 : β SP500 = 0. For a nominal size of 5%, we fail to reject the null H 0 : β C = 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 73 / 225

2. Statistical hypothesis testing Summary Hypothesis testing is de ned by the following general procedure Step 1: State the relevant null and alternative hypotheses (misstating the hypotheses muddies the rest of the procedure!); Step 2: Consider the statistical assumptions being made about the sample in doing the test (independence, distributions, etc.) incorrect assumptions mean that the test is invalid! Step 3: Choose the appropriate test (exact or asymptotic tests) and thus state the relevant test statistic (say, T). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 74 / 225

2. Statistical hypothesis testing Summary (cont d) Step 4: Derive the distribution of the test statistic under the null hypothesis (sometimes it is well-known, sometimes it is more tedious!) for example, the Student t-distribution or the Fisher distribution. Step 5: Determine the critical value (and thus the critical region). Step 6: Compute (using the observations!) the observed value of the test statistic T, say t obs. Step 7: Decide to either fail to reject the null hypothesis or reject in favor of the alternative assumption the decision rule is to reject the null hypothesis H 0 if the observed value of the test statistic, t obs is in the critical region, and to fail to reject the null hypothesis otherwise Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 75 / 225

2. Statistical hypothesis testing Key concepts 1 Null and alternative hypotheses 2 Simple and composite hypotheses 3 One-sided and two-sided tests 4 Rejection region, test statistic and critical value 5 Type I and type II errors 6 Size, power and power function 7 Uniformly most powerful (UMP) test 8 Neyman Pearson lemma 9 Consistent test and unbiased test 10 p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 76 / 225

Section 3 Tests in the multiple linear regression model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 77 / 225

3. Tests in the multiple linear regression model Objectives In the context of the multiple linear regression model (cf. chapter 3), the objective of this section is to present : 1 the Student test 2 the t-statistic and the z-statistic 3 the Fisher test 4 the global F-test 5 To distinguish the case with normality assumption and the case without any assumption on the distribution of the error term (semi-parametric speci cation) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 78 / 225

3. Tests in the multiple linear regression model Be careful: in this section, I don t distinguish between a random vector (matrix) and a vector (matrix) of deterministic elements. For more appropriate notations, see: Abadir and Magnus (2002), Notation in econometrics: a proposal for a standard, Econometrics Journal. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 79 / 225

3. Tests in the multiple linear regression model Model Consider the (population) multiple linear regression model: where (cf. chapter 3): y = Xβ + ε y is a N 1 vector of observations y i for i = 1,.., N X is a N K matrix of K explicative variables x ik for k = 1,., K and i = 1,.., N ε is a N 1 vector of error terms ε i. β = (β 1..β K ) > is a K 1 vector of parameters Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 80 / 225

3. Tests in the multiple linear regression model Assumptions Fact (Assumptions) We assume that the multiple linear regression model satisfy the assumptions A1-A5 (cf. chapter 3) We distinguish two cases: 1 Case 1: assumption A6 (Normality) holds and ε N 0, σ 2 I N 2 Case 2: the distribution of ε is unknown (semi-parametric speci cation) and ε?? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 81 / 225

3. Tests in the multiple linear regression model Parametric tests The β k are unknown features of the population, but: 1 One can formulate a hypothesis about their value; 2 One can construct a test statistic with a known nite sample distribution (case 1) or an asymptotic distribution (case 2); 3 One can take a decision meaning reject H0 if the value of the test statistic is too unlikely. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 82 / 225

3. Tests in the multiple linear regression model Three tests of interest: H 0 : β k = a k or H 0 : β k = a k H 1 : β k < a k H 1 : β k > a k H 0 : β k = a k H 1 : β k 6= a k where a k = 0 or a k 6= 0. H 0 : Rβ = q H 1 : Rβ 6= q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 83 / 225

3. Tests in the multiple linear regression model For that, we introduce two types of test 1 The Student test or t-test 2 The Fisher test of F-test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 84 / 225

Subsection 3.1 The Student test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 85 / 225

3.1. The Student test Case 1: Normality assumption A6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 86 / 225

3.1. The Student test Assumption 6 (normality): the disturbances are normally distributed. εj X N 0 N 1, σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 87 / 225

3.1. The Student test Reminder (cf. chapter 3) Fact (Linear regression model) Under the assumption A6 (normality), the estimators β b and bσ 2 have a nite sample distribution given by: bβ 1 N β,σ 2 X > X bσ 2 σ 2 (N K ) χ2 (N K ) Moreover, b β and bσ 2 are independent. This result holds whether or not the matrix X is considered as random. In this last case, the distribution of b β is conditional to X. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 88 / 225

3.1. The Student test Remarks 1 Any linear combination of β b is also normally distributed: Aβ b 1 N Aβ,σ 2 A X > X A > 2 Any subset of b β has a joint normal distribution. bβ k N β k, σ 2 m kk where m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 89 / 225

3.1. The Student test Reminder If X and Y are two independent random variables such that then the variable Z de ned as to be X N (0, 1) Y χ 2 (θ) Z = X p Y /θ has a Student s t-distribution with θ degrees of freedom Z t (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 90 / 225

3.1. The Student test Student test statistic Consider a test with the null: Under the null H 0 : H 0 : β k = a k bβ k a k σ p N (0, 1) m kk H0 bσ 2 σ 2 (N K ) H 0 χ 2 (N K ) and these two variables are independent... Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 91 / 225

3.1. The Student test Student test statistic (cont d) So, under the null H 0 we have: bβ k a k σ p N (0, 1) m kk H0 bσ 2 σ 2 (N K ) H 0 χ 2 (N K ) bβ k a k σ p m q kk bσ 2 (N K ) σ 2 (N K ) = b β k a k bσ p m kk H0 t (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 92 / 225

3.1. The Student test De nition (Student t-statistic) Under the null H 0 : β k = a k, the Student test-statistic or t-statistic is de ned to be: a k T k = b β k t (N K ) bse b β H0 k where N is the number of observations, K is the number of explanatory variables (including the constant term), t (N K ) is the Student t-distribution with N K degrees of freedom and bse b β k = bσ p m kk with m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 93 / 225

3.1. The Student test Remarks 1 Under the assumption A6 (normality) and under the null H 0 : β k = a k, the Student test-statistic has an exact ( nite sample) distribution. T k t (N K ) H0 2 The term bse b β k denotes the estimator of the standard error of the OLS estimator bβ k and it corresponds to the square root of the k th diagonal element of bv bβ (cf. chapter 3): bv 1 bβ = bσ 2 X > X Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 94 / 225

3.1. The Student test Consider the one-sided test: k (y) < Aj T k t (N H0 H 0 : β k = a k H 1 : β k < a k The rejection region is de ned as to be: W = fy : T k (y) < Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr T K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 95 / 225

3.1. The Student test α = Pr T k (y) < Aj T k t (N K ) = F N K (A) H0 where F N K (.) denotes the cdf of the Student s t-distribution with N K degrees of freedom. Denote c α the α-quantile of this distribution:. A = FN 1 K (α) = c α The rejection region of the test of size α is de ned as to be: W = fy : T k (y) < c α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 96 / 225

3.1. The Student test De nition (One-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k < a k at the α (say, 5%) signi cance level if: W = fy : T k (y) < c α g where c α is the α (say, 5%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 97 / 225

3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT < 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 98 / 225

3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = 1.9898 1 = 3.1501 bse b β 0.3142 MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : T k (y) < 1.7341g Conclusion: for a signi cance level of 5%, we fail to reject the null H 0 : β MSFT = 1 against H 1 : β MSFT < 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 99 / 225

3.1. The Student test Solution (cont d) 0.5 0.45 Density of Ts under H0 0.4 0.35 0.3 0.25 0.2 α=5% 0.15 0.1 0.05 DF = 18 Critical value = 1.7341 0 4 3 2 1 0 1 2 3 4 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 100 / 225

3.1. The Student test Consider the one-sided test k (y) > Aj T k t (N H0 H 0 : β k = a k H 1 : β k > a k The rejection region is de ned as to be: W = fy : T k (y) > Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr T K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 101 / 225

3.1. The Student test α = 1 Pr T k (y) < Aj T k t (N K ) H0 or equivalently 1 α = F N K (A) where F N K (.) denotes the cdf of the Student s t-distribution with N K degrees of freedom. Denote c 1 α the 1 α quantile of this distribution: A = F 1 N K (1 α) = c 1 α The rejection region of the test of size α is de ned as to be: W = fy : T k (y) > c 1 α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 102 / 225

3.1. The Student test De nition (One-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k > a k at the α (say, 5%) signi cance level if: W = fy : T k (y) > c 1 α g where c 1 α is the 1 α (say, 95%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 103 / 225

3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT > 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 104 / 225

3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = 1.9898 1 = 3.1501 bse b β 0.3142 MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : T k (y) > 1.7341g Conclusion: for a signi cance level of 5%, we reject the null H 0 : β MSFT = 1 against H 1 : β MSFT > 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 105 / 225

3.1. The Student test Solution (cont d) 0.5 0.45 0.4 Density of Ts under H0 0.35 0.3 0.25 0.2 0.15 0.1 0.05 DF = 18 Critical value = 1.7341 α=5% 0 4 3 2 1 0 1 2 3 4 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 106 / 225

3.1. The Student test Consider the two-sided test H 0 : β k = a k H 1 : β k 6= a k The non-rejection region is de ned as the intersection of the two non-rejection regions of the one-sided test of level α/2: W = W A \ W B H 0 : β k = a k against H 1 : β k < a k W A = fy : T k (y) > c α/2 g H 0 : β k = a k against H 1 : β k > a k W B = fy : T k (y) < c 1 α/2 g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 107 / 225

3.1. The Student test W = fy : c α/2 < T k (y) < c 1 α/2 g Since the Student s t-distribution is symmetric, c α/2 = c 1 α/2 W = fy : c 1 α/2 < T k (y) < c 1 α/2 g The rejection region is then de ned as to be: W = fy : jt k (y)j > c 1 α/2 g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 108 / 225

3.1. The Student test De nition (Two-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k 6= a k at the α (say, 5%) signi cance level if: W = fy : jt k (y)j > c 1 α/2 g where c 1 α/2 is the 1 α/2 (say, 97.5%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 109 / 225

3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT 6= 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 110 / 225

3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = 1.9898 1 = 3.1501 bse b β 0.3142 MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : jt k (y)j > 2.1009g Conclusion: for a signi cance level of 5%, we reject the null H 0 : β MSFT = 1 against H 1 : β MSFT 6= 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 111 / 225

3.1. The Student test 0.5 0.45 Density of Ts under H0 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 α=2.5% DF = 18 Critical value = 2.1009 α=2.5% 0 4 3 2 1 0 1 2 3 4 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 112 / 225

3.1. The Student test Rejection regions H 0 H 1 Rejection region β k = a k β k > a k W = fy : T k (y) > c 1 α g β k = a k β k < a k W = fy : T k (y) < c α g β k = a k β k 6= a k W = fy : jt k (y)j > c 1 α/2 g where c β denotes the β-quantile (critical value) of the Student t-distribution with N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 113 / 225

3.1. The Student test De nition (P-values) The p-values of Student tests are equal to: Two-sided test: p-value = 2 F N K ( jt k (y)j) Right tailed test: p-value = 1 F N K (T k (y)) Left tailed test: p-value = F N K ( T k (y)) where T k (y) is the realisation of the Student test-statistic and F N K (.) the cdf of the Student s t-distribution with N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 114 / 225

3.1. The Student test Example (One-sided test) Consider the previous CAPM model. We want to test: H 0 : c = 0 against H 1 : c 6= 0 H 0 : β MSFT = 0 against H 1 : β MSFT 6= 0 Question: nd the corresponding p-values. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 115 / 225

3.1. The Student test Solution Since we consider two-sided tests with N = 20 and K = 2: p-value c = 2 F 18 ( jt c (y)j) = 2 F 18 ( 0.9868) = 0.3368 p-value c = 2 F 18 ( jt MSFT (y)j) = 2 F 18 ( 6.3326) = 5.7e 006 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 116 / 225

3.1. The Student test Fact (Student test with large sample) For a large sample size N T k t (N K ) N (0, 1) H0 Then, the rejection region for a Student two-sided test becomes W = y : jt k (y)j > Φ 1 (1 α/2) where Φ (.) denotes the cdf of the standard normal distribution. For α = 5%, Φ 1 (0.975) = 1.96, so we have: W = fy : jt k (y)j > 1.96g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 117 / 225

3.1. The Student test Case 2: Semi-parametric model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 118 / 225

3.1. The Student test Assumption 6 (normality): the distribution of the disturbances is unknown, but satisfy (assumptions A1-A5): E ( εj X) = 0 N 1 V ( εj X) = σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 119 / 225

3.1. The Student test Problem 1 The exact ( nite sample) distribution of bβ k and bσ 2 are unknown. 2 As a consequence the nite sample distribution of T k (y) is also unknown. 3 But, we can use the asymptotic properties of the OLS estimators (cf. chapter 3). In particular, we have: p N bβ β d! N 0, σ 2 Q 1 where Q = p lim 1 N X> X = E X x i x > i Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 120 / 225

3.1. The Student test De nition (Z-statistic) Under the null H 0 : β k = a k, if the assumptions A1-A5 hold (cf. chapter 3), the z-statistic de ned by Z k = b β k a k bse asy b β k d! H0 N (0, 1) where bse asy b β k = bσ p m kk denotes the estimator of the asymptotic standard error of the estimator bβ k and m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 121 / 225

3.1. The Student test Rejection regions The rejection regions have the same form as for the t-test (except for the distribution) H 0 H 1 Rejection region β k = a k β k > a k W = y : Z k (y) > Φ 1 (1 α) β k = a k β k < a k W = y : Z k (y) < Φ 1 (α) β k = a k β k 6= a k W = y : jz k (y)j > Φ 1 (1 α/2) where Φ (.) denotes the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 122 / 225

3.1. The Student test De nition (P-values) The p-values of the Z-tests are equal to: Two-sided test: p-value = 2 Φ ( jz k (y)j) right tailed test: p-value = 1 left tailed test: p-value = Φ ( Φ (Z k (y)) Z k (y)) where Z k (y) is the realisation of the Z-statistic and Φ (.) the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 123 / 225

3.1. The Student test Summary Normality Assumption Non Assumption Test-statistic t-statistic z-statistic De nition T k = b β k a k bσ p m kk Z k = b β k a k bσ p m kk Exact distribution T k H0 t (N K ) Asymptotic distribution Z K d!h0 N (0, 1) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 124 / 225

3.1. The Student test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 125 / 225

Subsection 3.2 The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 126 / 225

3.2. The Fisher test Consider the two-sided test associated to p linear constraints on the parameters β k : H 0 : Rβ = q H 1 : Rβ 6= q where R is a p K matrix and q is a p 1 vector. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 127 / 225

3.2. The Fisher test Example (Linear constraints) If K = 4 and if we want to test H 0 : β 1 + β 2 = 0 and β 2 we have p = 2 linear constraints with: 3β 3 = 4, then R β (24) (4,1) 1 1 0 0 0 1 3 0 0 B @ = q (21) β 1 β 2 β 3 β 4 1 C A = 0 4 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 128 / 225

3.2. The Fisher test Example (Linear constraints) If K = 4 and if we want to test H 0 : β 2 = β 3 = β 4 = 0, then we have p = 3 linear constraints with: 0 @ R β (34) (4,1) 0 1 0 0 0 0 1 0 0 0 0 1 0 1 A B @ = q (31) β 1 β 2 β 3 β 4 1 C A = 0 @ 0 0 0 1 A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 129 / 225

3.1. The Student test Case 1: Normality assumption A6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 130 / 225

3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic is de ned as to be: F = 1 Rβ p b > 1 1 q bσ 2 R X > X R > Rβ b q where b β denotes the OLS estimator. Under the null H 0 : Rβ = q, the F -statistic has a Fisher exact ( nite sample) distribution F H0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 131 / 225

3.2. The Fisher test Reminder If X and Y are two independent random variables such that then the variable Z de ned by X χ 2 (θ 1 ) Y χ 2 (θ 2 ) Z = X /θ 1 Y /θ 2 has a Fisher distribution with θ 1 and θ 2 degrees of freedom Z F (θ1,θ 2 ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 132 / 225

3.2. The Fisher test Proof Under assumption A6, we have the following (conditional to X) distribution bβ 1 N β,σ 2 X > X bσ 2 σ 2 (N K ) χ2 (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 133 / 225

3.2. The Fisher test Proof (cont d) Consider the vector m = R b β q. Under the null H 0 : Rβ = q We have E (m) = RE bβ q = Rβ q = 0 V (m) = E Rβ b q Rβ b bβ = RV = σ 2 R R > 1 X > X R > > q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 134 / 225

3.2. The Fisher test Proof (cont d) We can base the test of H 0 on the Wald criterion: W (11) = m > 1 (V (m)) (1p) pp > = R b β q σ 2 R m p1 1 X > X R > 1 R b β q Under assumption A6 (normality) W H0 χ 2 (p) These two variables are independent. bσ 2 σ 2 (N K ) χ2 (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 135 / 225

3.2. The Fisher test Proof (cont d) W H0 χ 2 (p) bσ 2 σ 2 (N K ) χ2 (N K ) So, the ratio of these two variables has a Fisher distribution F = W p bσ 2 (N K ) σ 2 (N K ) H 0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 136 / 225

3.2. The Fisher test Proof (cont d) R b > 1 1 β q σ 2 R X > X R > Rβ b q /p F = bσ 2 (N K ) / (N K ) σ 2 After simpli cation, the F-statistic is de ned by: F = 1 p R b β Under the null H 0 : Rβ =q : > 1 1 q bσ 2 R X > X R > Rβ b q F H0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 137 / 225

3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can be de ned as a function of the SSR of the constrained (H 0 ) and unconstrained model (H 1 ): SSR0 SSR 1 N K F = SSR 1 p where SSR 0 denotes the sum of squared residuals of the constrained model estimated under H 0 and SSR 1 denotes the sum of squared residuals of the unconstrained model estimated under H 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 138 / 225

3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can be de ned as to be: F = 1 > bβh1 bβ bσ 2 H0 X > X bβh1 bβ H0 p where β b H0 denotes the OLS estimator obtained in the constrained model (under H 0 ) and β b H1 denotes the OLS estimator obtained in the unconstrained model (under H 1 ). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 139 / 225

3.2. The Fisher test De nition (Constrained OLS estimator) Under suitable regularity conditions, the constrained OLS estimator b β C of β, obtained under the constraint Rβ = q, is given by: bβ C = b β UC X > X 1 R > 1 1 R X > X R > Rβ b UC q where b β UC is the unconstrained OLS estimator. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 140 / 225

3.2. The Fisher test Example (Fisher test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t where r MSFT,t is the excess return for Microsoft, r SP500,t for the SP500, r Ford,t for Ford and r GE,t for general electric. We want to test the following linear constraints: H 0 : β 2 = 1 and β 3 = β 4 Question: write a Matlab code to compute the F-statistic according to the three alternative de nitions. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 141 / 225

3.2. The Fisher test Solution In this problem, the null H 0 : β 2 = 1 and β 3 = β 4 can be written as: R β (24) (4,1) 0 1 0 0 0 0 1 1 0 B @ = q (21) β 1 β 2 β 3 β 4 1 C A = 1 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 142 / 225

3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 143 / 225

3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 144 / 225

3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 145 / 225

3.2. The Fisher test Consider the Fisher test H 0 : Rβ = q H 1 : Rβ 6= q Since the Fisher test-statistic is always positive, the rejection region is de ned as to be: W = fy : F (y) > Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr F (y) > Aj F F (p,n K ) H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 146 / 225

3.2. The Fisher test α = Pr (Wj H 0 ) = Pr F (y) > Aj F F (p,n K ) H0 or equivalently α = 1 Pr F (y) < Aj F F (p,n K ) H0 Denote d 1 α the 1 α quantile of the Fisher distribution with p and N K degrees of freedom. A = d 1 α The rejection region of the test of size α is de ned as to be: W = fy : F (y) > d 1 α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 147 / 225

3.2. The Fisher test De nition (Rejection region of a Fisher test) The critical region of the Fisher test is that H 0 : Rβ = q is rejected in favor of H 1 : Rβ 6= q at the α (say, 5%) signi cance level if: W = fy : F (y) > d 1 α g where d 1 α is the 1 α critical value (say 95%) of the Fisher distribution with p and N K degrees of freedom and F k (y) is the realisation of the Fisher test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 148 / 225

3.2. The Fisher test Example (Fisher test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t where r MSFT,t is the excess return for Microsoft, r SP500,t for the SP500, r Ford,t for Ford and r GE,t for general electric. We want to test the following linear constraints: H 0 : β 2 = 1 and β 3 = β 4 Question: given the realisation of the Fisher test-statistic (cf. previous example), conclude for a signi cance level α = 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 149 / 225

3.2. The Fisher test Solution Step 1: compute the F-statistic (cf. Matlab code) F (y) = 4.3406 Step 2: Determine the rejection region for a nominal size α = 5% for N = 24, K = 4 and p = 2 F H0 F (2,20) W = fy : F (y) > 3.4928g Conclusion: for a signi cance level of 5%, we reject the null H 0 : Rβ = q against H 1 : Rβ 6= q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 150 / 225

3.2. The Fisher test 1.2 1 Density of F under H0 0.8 DF1 = 2 DF2 = 20 Critical value = 3.4928 0.6 0.4 α=5% 0.2 0 0 1 2 3 4 5 6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 151 / 225

3.2. The Fisher test De nition (Student test-statistic and Fisher test-statistic ) Consider the test H 0 : β k = a k versus H 1 : β k 6= a k the Fisher test-statistic corresponds to the squared of the corresponding Student s test-statistic F = T 2 k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 152 / 225

3.2. The Fisher test Proof Consider the test H 0 : β k = a k against H 1 : β k 6= a k, then we have: R = 0 0.. 1 k th position q = a k 0 0 As a consequence : Rβ b q = bβ k a k 1 bσ 2 R X X > R > = bv b β k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 153 / 225

3.2. The Fisher test Proof (cont d) So, for a test H 0 : β k = a k against H 1 : β k 6= a k, the Fisher test-statistic becomes F = Rβ b > 1 1 q bσ 2 R X > X R > Rβ b q So, we have: b β k a k 2 F = bv b β k and the F test-statistic is equal to the squared t-statistic: F = T 2 k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 154 / 225

3.2. The Fisher test De nition (P-values) The p-value of the F-test is equal to: p-value = 1 F p,n K (F (y)) where F(y) is the realisation of the F-statistic and F p,n K (.) the cdf of the Fisher distribution with p and N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 155 / 225

3.2. The Fisher test De nition (Global F-test) In a multiple linear regression model with a constant term y i = β 1 + K k=2 β k x ik + ε i the global F-test corresponds to the test of signi cance of all the explicative variables: H 0 : β 2 =.. = β K = 0 Under the assumption A6 (normality), the global F-test-statistic satis es: F H0 F (K 1,N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 156 / 225

3.2. The Fisher test Remarks 1 The global F-test is a test designed to see if the model is useful overall. 2 The null H 0 : β 2 =.. = β K = 0 can be written as: 0 B @ R β (K 1K ) (K,1) 0 1 0 0.. 0 0 0 1 0.. 0.... 0 1................ 0.. 0 0.. 1 = q (K 11) 1 0 B C @ A β 1.... β K 1 C A = 0 B @ 0.... 0 1 C A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 157 / 225

3.2. The Fisher test Corollary (Global F-test) In a multiple linear regression model with a constant term y i = β 1 + K k=2 β k x ik + ε i the global F-test-statistic can also be de ned as: R 2 N K F = 1 R 2 K 1 where R 2 denotes the (unadjusted) coe cient of determination. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 158 / 225

3.2. The Fisher test Example (Global F-test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t Question: write a Matlab code to compute the global F-test, the critical value for α = 5% and the p-value. Compare your results with Eviews. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 159 / 225

3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 160 / 225

3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 161 / 225

3.2. The Fisher test Case 2: Semi-parametric model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 162 / 225

3.2. The Fisher test Assumption 6 (normality): the distribution of the disturbances is unknown, but satisfy (assumptions A1-A5): E ( εj X) = 0 N 1 V ( εj X) = σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 163 / 225

3.2. The Fisher test Problem 1 The exact ( nite sample) distribution of bβ k and bσ 2 are unknown. As a consequence the nite sample distribution of F(y) is also unknown. 2 But, we can express the F-statistic as a linear function of the Wald statistic. 3 The Wald statistic has a chi-squared asymptotic distribution (cf. next section) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 164 / 225

3.2. The Fisher test De nition (F-test-statistic and Wald statistic) The Fisher test-statistic can expressed as a linear function of the Wald test-statistic as F = 1 p Wald Wald = 1 p R b β > 1 1 q R V asy bβ R > Rβ b q Under assumptions A1-A5, the Wald test-statistic converges to a chi-squared distribution Wald! d χ 2 (p) H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 165 / 225

3. Tests in the multiple linear regression model Key concepts of Section 3 1 Student test 2 Fisher test 3 t-statistic and z-statistic 4 Global F-test 5 Exact ( nite sample) distribution under the normality assumption 6 Asymptotic distribution Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 166 / 225

Section 4 MLE and Inference Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 167 / 225

4. MLE and inference Introduction Consider a parametric model, linear or nonlinear (GARCH, probit, logit, etc.), with a vector of parameters θ = (θ 1 :.. : θ K ) > We assume that the problem is regular (cf. chapter 2) and we consider a ML estimator bθ The nite sample distribution of bθ is unknown, but bθ is asymptotically normally distributed (cf. chapter 2). We want to test a set of linear or nonlinear constraints on the true parameters (population) θ 1,.., θ K. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 168 / 225

4. MLE and inference De nition (Null hypothesis) Consider a null hypothesis of p linear and/or nonlinear constraints H 0 : c (θ) = 0 p1 {z} p1 where c (θ) is a vectorial function de ned as: c : R K! R p θ 7! c (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 169 / 225

4. MLE and inference Notations 1 c (θ) is a p 1 vector of functions c 1 (θ),.., c p (θ): c (θ) = 0 B @ c 1 (θ) c 2 (θ).. c p (θ) 2 In the case of p linear constraints, we have: 1 C A H 0 : c (θ) = Rθ q = 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 170 / 225

4. MLE and inference Example (Linear constraints) Consider the two linear constraints θ 1 = θ 2 + θ 3 and θ 2 + θ 4 = 1. We have p = 2 constraints such that: θ1 θ H 0 : c (θ) = 2 θ 3 0 = θ (2,1) 2 + θ 4 1 0 The function c (θ) can be written as Rθ = (θ 1 θ 2 θ 3 θ 4 ) >, we have c (θ) = Rθ q = 1 1 1 0 0 1 0 1 q. For instance if K = 4 and θ 0 B @ θ 1 θ 2 θ 3 θ 4 1 C A 0 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 171 / 225

4. MLE and inference Example (Nonlinear constraints) Consider the linear and nonlinear constraints θ 1 θ 2 = 0 θ 2 1 θ 3 = 0 We have p = 2 constraints such that: θ1 θ H 0 : c (θ) = 2 (2,1) θ 2 1 θ 3 = 0 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 172 / 225

4. MLE and inference Assumptions 1 The functions c 1 (θ),.., c p (θ) are di erentiable. 2 There is no redundant constraint (identi cation assumption). Formally, we have c (θ) (row) rank θ > = p 8θ 2 Θ with c (θ) θ > (p,k ) 0 = B @ c 1 (θ) θ 1 c 2 (θ) θ 1 c 1 (θ) θ 2.. c 2 (θ) θ 2.. c 1 (θ) θ K c 2 (θ) θ K........ c p (θ) θ 1 c p (θ) θ 2.. c p (θ) θ K 1 C A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 173 / 225

4. MLE and inference Consider the two-sided test H 0 : c (θ) = 0 versus H 1 : c (θ) 6= 0 We introduce three di erent asymptotic tests (the trilogy..) 1 The Likelihood Ratio (LR) test 2 The Wald test 3 The Lagrance Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 174 / 225

4. MLE and inference For each of the three tests, we will present: 1 the test-statistic 2 its asymptotic distribution under the null 3 the (asymptotic) rejection region 4 the (asymptotic) p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 175 / 225

Subsection 4.1 The Likelihood Ratio (LR) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 176 / 225

4.1. The Likelihood Ratio (LR) test De nition (Likelihood Ratio (LR) test statistic) The likelihood ratio (LR) test-statistic is de ned by as to be: LR = 2 b θ H0 ; yj x b θ H1 ; yj x `N where `N (θ; yj x) denotes the (conditional) log-likelihood of the sample y, bθ H0 and bθ H1 are respectively the maximum likelihood estimator of θ under the alternative and the null hypothesis. `N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 177 / 225

4.1. The Likelihood Ratio (LR) test Comments Consider the ratio of likelihoods under H 1 (no constraint) and under H 0 (with c (θ) = 0). b θ H0 ; yj x λ = L N L N b θ H1 ; yj x 1 λ > 0 since both likelihood are positive. 2 λ < 1 since L N (H 0 ) cannot be larger than L N (H 1 ). A restricted optimum is never superior to an unrestricted one. 3 If λ is too small, then doubt is cast on the restrictions c (θ) = 0. 4 Consider the statistic LR= 2 ln (λ): if λ is "too small", then LR is large (rejection of the null)... Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 178 / 225

4.1. The Likelihood Ratio (LR) test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the LR test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): LR d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : LR (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and LR(y) is the realisation of the LR test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 179 / 225

4.1. The Likelihood Ratio (LR) test De nition (p-value of the LRT test) The p-value of the LR test is equal to: p-value = 1 G p (LR (y)) where LR(y) is the realisation of the LR test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 180 / 225

4.1. The Likelihood Ratio (LR) test Example (LRT and Poisson distribution) Suppose that X 1,X 2,,X N are i.i.d. discrete random variables, such that X i Pois (θ) with a pmf (probability mass function) de ned as: Pr (X i = x i ) = exp ( x i! θ) θx i where θ is an unknown parameter to estimate. We have a sample (realisation) of size N = 10 given by f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g. Question: use a LR test to test the null H 0 : θ = 1.8 against H 1 : θ 6= 1.8 and give a conclusion for signi cance level of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 181 / 225

4.1. The Likelihood Ratio (LR) test Solution The log-likelihood function is de ned as to be: `N (θ; x) = θn + ln (θ) N i=1 x i ln N i=1 x i! In the chapter 2, we found that the ML estimator of θ is the sample mean: bθ = 1 N N X i i=1 Given the sample f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g, the estimate of θ (under H 1, with non constraint) is bθ H1 = 2, and the corresponding log-likelihood is equal to: b θ H1 ; x = ln (0.104) `N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 182 / 225

4.1. The Likelihood Ratio (LR) test Solution (cont d) Under the null H 0 : θ = 1.8, we don t need to estimate θ and the log-likelihood is equal to: `N (θ H0 ; x) = 1.8N + ln (1.8) The LR test-statistic is equal to: LR (y) = 2 ln N i=1 x i ln N i=1 0.0936 = 0.21072 0.104 x i! = ln (0.0936) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 183 / 225

4.1. The Likelihood Ratio (LR) test Solution (cont d) LR (y) = 0.21072 For N = 10, p = 1 (one restriction) and α = 0.05, the critical region is: and the p-value is W = y : LR (y) > χ 2 0.95 (1) = 3.8415 pvalue = 1 G 1 (0.21072) = 0.6462 where G 1 (.) is the cdf of the χ 2 (1) distribution. Conclusion: for a signi cance level of 5%, we fail to reject the null H 0 : θ = 1.8. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 184 / 225

Subsection 4.2 The Wald test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 185 / 225

4.2. The Wald test De nition (Wald test-statistic) The Wald test-statistic associated to the test of H 0 : c (θ) = 0 is de ned as to be: > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 where bθ H1 is the maximum likelihood estimator of θ under the alternative hypothesis (unconstrained model) and bv asy b θ H1 is an estimator of its asymptotic variance covariance matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 186 / 225

4.2. The Wald test Remark 0 > Wald = c b c θ H1 B b {z } @ θ > θ H1 bv asy b c > θ H1 b {z } {z } θ > θ H1 C A {z } 1p pk K K K p 1 1 c b θ H1 {z } p1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 187 / 225

4.2. The Wald test Example (Wald test-statistic) Consider a model with K = 3 parameters θ = (θ 1 : θ 2 : θ 3 ) > with θ 1 θ 2 = 0 θ 2 1 θ 3 = 0 We have two constraints (p = 2) and: H 0 : c (θ) (2,1) θ1 θ = 2 θ 2 1 θ 3 = 0 0 Denote bθ H1 = (θ 1 : θ 2 : θ 3 ) > the ML estimator of θ under the alternative hypothesis and bv asy b θ H1 the estimator of its asymptotic variance covariance matrix. Question: write the Wald test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 188 / 225

4.2. The Wald test Solution Here we have K = 3 and p = 2 c b θ H1 = b θ 1 bθ 2 bθ 2 1 bθ 3! c b 1 1 0 θ > θ H1 = 2bθ 1 0 1 > Wald = c b c θ H1 b θ > θ H1 V b asy b c > θ H1 b θ > θ H1 1 c b θ H1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 189 / 225

4.2. The Wald test Remark In the case of linear constraints H 0 : Rθ q = 0 we have with H 0 : c (θ) = 0 c (θ) = Rθ q c θ > (θ) = R Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 190 / 225

4.2. The Wald test De nition (Wald test-statistic and linear constraints) Consider the test of linear constraints H 0 : c (θ) = Rθ q = 0. The Wald test-statistic is de ned as to be: > Wald = Rbθ H1 q R bv asy b θ H1 R > 1 Rbθ H1 q where bθ H1 is the maximum likelihood estimator of θ under the alternative hypothesis (unconstrained model) and bv asy b θ H1 is an estimator of its asymptotic variance covariance matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 191 / 225

4.2. The Wald test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the Wald test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): Wald d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : Wald (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and Wald(y) is the realisation of the Wald test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 192 / 225

4.2. The Wald test Proof Under some regularity conditions, we have p N b d! θ H1 θ 0 N 0, I 1 (θ 0 ) We use the delta method for the function c (.). The function c (.) is a continuous and continuously di erentiable function not involving N, then p N c b d! c θ H1 c (θ 0 ) N 0, θ > (θ 0) I 1 (θ 0 ) c θ > (θ 0) > Under the null H 0 : c (θ 0 ) = 0, we have c θ > (θ 0) I 1 (θ 0 ) c 1/2 pnc θ > (θ 0) > b d! θ H1 N (0, Ip ) where I p is the identity matrix of size p. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 193 / 225

4.2. The Wald test Proof (cont d) The Wald criteria is de ned as to be: Wald criteria > = N c b θ H1 c θ > (θ 0) I = N c c θ > (θ 0) I 1 (θ 0 ) c! 1/2 > θ > (θ 0) > 1 (θ 0 ) c 1/2 θ > (θ 0) > c b θ H1 1 (θ 0 ) c 1 θ > (θ 0) > c b θ H1 b θ H1 > c θ > (θ 0) I So, under the null H 0 : c (θ 0 ) = 0,, we have Wald criteria d! H0 χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 194 / 225

4.2. The Wald test Proof (cont d) Wald Criteria = > N c b θ H1 c θ > (θ 0) I A feasible Wald test-statistic is given by 1 (θ 0 ) c 1 θ > (θ 0) > c b θ H1 Wald = > N c b θ H1 c b θ > θ H1 b I 1 c > 1 bθ H1 b θ > θ H1 c b θ H1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 195 / 225

4.2. The Wald test Proof (cont d) Since We have nally bv asy b θ H1 = N 1 bi 1 bθ H1 > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 and Wald d! H0 χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 196 / 225

4.2. The Wald test De nition (p-value of the Wald test) The p-value of the Wald test is equal to: p-value = 1 G p (Wald (y)) where Wald(y) is the realisation of the Wald test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 197 / 225

4.2. The Wald test De nition (z-statistic) Consider the test H 0 : θ k = a k versus H 1 : θ k 6= a k. The z-statistic corresponds to the square root of the Wald test-statistic and satis es b θ k a k Z k = r! d N (0, 1) H0 bv asy b θ k where bθ k is the ML estimator of θ k obtained under H 1 (unconstrained model). The critical region for a signi cance level of α is: n W = y : jz k (y)j > Φ 1 α o 1 2 where Φ (.) denotes the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 198 / 225

4.2. The Wald test Computational issues The Wald test-statistic depends on the estimator of the asymptotic variance covariance matrix: > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 bv asy b θ H1 = N 1 1 bi bθ H1 where I b θ H1 denotes the average Fisher information matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 199 / 225

4.2. The Wald test Computational issues (cont d) Three estimators are available for the average Fisher information matrix: Actual Average Fisher Matrix: bi A b θ = 1 N BHHH estimator: bi B b θ = 1 N N i=1 Hessian based estimator: bi c b θ = 1 N `i (θ; y i j x i ) θ N i=1 b θ N i=1 bi i b θ >! `i (θ; y i j x i ) θ 2`i (θ; y i j x i ) θ θ > b θ bθ Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 200 / 225

4.2. The Wald test Computational issues (cont d) 1 These estimators are asymptotically equivalent, but the corresponding estimates may be very di erent in small samples. 2 Thus, we can obtain three di erent values for the Wald statistic given the choice of the estimator for V asy b θ H1 (cf. exercises). 3 In general, the estimator A is rarely available and the estimator B (BHHH) gives erratic results. 4 Most of the software use the estimator C (Hessian based estimator). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 201 / 225

4.2. The Wald test Computational issues (cont d) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 202 / 225

Subsection 4.3 The Lagrange Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 203 / 225

4.3. The Lagrange Multiplier (LM) test Introduction Consider the set of constraints c (θ) = 0. Let λ be a vector of Lagrange multipliers and de ne the Lagrangian function `N (θ ; yj x) = `N (θ; yj x) + λc (θ) The solution to the constrained maximization problem is the root of `N (θ ; yj x) θ = `N (θ; yj x) θ + c (θ) > θ > λ `N (θ ; yj x) λ = c (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 204 / 225

4.3. The Lagrange Multiplier (LM) test Introduction (cont d) `N (θ ; yj x) θ = `N (θ; yj x) θ + c (θ) > θ > λ 1 If the restrictions are valid, then imposing them will not lead to a signi cant di erence in the maximized value of the likelihood function. In the rst-order conditions, the meaning is that the second term in the derivative vector will be small. In particular, λ will be small. 2 We could test this directly, that is, test H 0 : λ = 0 which leads to the Lagrange multiplier test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 205 / 225

4.3. The Lagrange Multiplier (LM) test Introduction (cont d) There is an equivalent simpler formulation, however. If the restrictions c (θ) = 0 are valid, the derivatives of the log-likelihood of the unconstrained model evaluated at the restricted parameter vector will be approximately zero. `N (θ; yj x) θ = 0 b θ H0 The vector of rst derivatives of the log-likelihood is the vector of (e cient) scores. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 206 / 225

4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) For these reasons, this test is called the score test as well as the Lagrange multiplier test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 207 / 225

4.3. The Lagrange Multiplier (LM) test Guess Let us assume that θ is scalar, i.e. K = 1, then the LM statistic is simply de ned as: 2 b θ H0 ; Y j x LM = V s N b θ H0 ; Y j x Since bi N b θ H0 = V s N b θ H0 ; Y j x, we have: s N LM = s N b θ H0 ; Y j x 2 bi N b θ H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 208 / 225

4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) The LM test-statistic or score test associated to the test of H 0 : c (θ) = 0 is de ned as to be: LM = s N b θ H0 ; Y j x > b I 1 N b θ H0 s N b θ H0 ; Y j x where bθ H0 is the maximum likelihood estimator of θ under the null hypothesis (constrained model), s N (θ; Y j x) is the score vector of the unconstrained model and bi N b θ H0 is an estimator of the Fisher information matrix of the sample evaluated at bθ H0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 209 / 225

4.3. The Lagrange Multiplier (LM) test Remark Since: bv asy b θ H0 = bi 1 N b θ H0 there is another expression for the LM statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 210 / 225

4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) The LM test-statistic or score test associated to the test of H 0 : c (θ) = 0 is de ned as to be: > LM = s N b θ H0 ; Y j x V b asy b θ H0 s N b θ H0 ; Y j x where bθ H0 is the maximum likelihood estimator of θ under the null hypothesis (constrained model), s N (θ; Y j x) is the score vector of the unconstrained model and bv asy b θ H0 is an estimator of the asymptotic variance covariance matrix of bθ H0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 211 / 225

4.3. The Lagrange Multiplier (LM) test Remark The LM test-statistic can also be de ned by: LM = λ > c θ > b θ H0 b V asy b θ H0 c θ > b θ H0 > λ where λ denotes the Lagrange Multiplier associated to the constraints c (θ) = 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 212 / 225

4.3. The Lagrange Multiplier (LM) test The LM test-statistic can be obtained from the following auxiliary procedure: Step 1: Estimate the constrained model and obtain bθ H0. Step 2: Form the gradients for each observation of the unrestricted model evaluated at bθ H0 g i b θ H0 ; y i j x i 8i = 1,..N Step 3: Run the regression of a vector of 1 on the variables g i b θ H0 ; y i j x i 8i = 1,..N, then LM = N R 2 where R 2 denotes the (unadjusted) coe cient of determination of this auxiliary regression. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 213 / 225

4.3. The Lagrange Multiplier (LM) test Computational issues 1 The LM test-statistic depends on the estimator of the asymptotic variance covariance matrix: > LM = s N b θ H0 ; Y j x V b asy b θ H0 s N b θ H0 ; Y j x bv asy b θ H0 = N 1 1 bi bθ H0 where I b θ H0 denotes the average Fisher information matrix. 2 Thus, we can obtain three di erent values for the LM statistic given the choice of the estimator for V asy b θ H0 (cf. exercises). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 214 / 225

4.3. The Lagrange Multiplier (LM) test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the LM test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): LM d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : LM (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and LM(y) is the realisation of the LM test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 215 / 225

4.3. The Lagrange Multiplier (LM) test De nition (p-value of the LM test) The p-value of the LM test is equal to: p-value = 1 G p (LM (y)) where LM(y) is the realisation of the LM test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 216 / 225

Subsection 4.4 A comparison of the three tests Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 217 / 225

4.4. A comparison of the three tests Source: Pelgrin (2010), Lecture notes, Advanced Econometrics Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 218 / 225

4.4. A comparison of the three tests Summary Test Requires estimation under LRT H 0 and H 1 Wald H 1 LM H 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 219 / 225

4.4. A comparison of the three tests Computational problems If the ML maximisation problem is complex (with local extrema) and if it has no closed form solution (nonlinear models: GARCH, Markov Switching models etc.), it may be particularly di cult to get a ML estimates bθ through a numerical optimisation of the log-likelihood. If the constraints c (θ) = 0 are not valid in the data, the (numerical) convergence of the optimisation algorithm may be very problematic under the null H 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 220 / 225

4.4. A comparison of the three tests Asymptotic comparison The three tests have the same asymptotic distribution under the null H 0 : c (θ) = 0: LRT d! H0 χ 2 (p) Wald d! H0 LM d! H0 χ 2 (p) χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 221 / 225

4.4. A comparison of the three tests Theorem (Asymptotic comparison) The three tests are asymptotically equivalent. Under some regularity conditions and under the null H 0 : c (θ) = 0, the di erences between the three test statistics converge to 0 as N tends to in nity: LRT LM p! H0 0 LRT Wald p! H0 0 LM Wald p! H0 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 222 / 225

4.4. A comparison of the three tests Fact (Finite sample properties) The nite sample properties of the three tests may be di erent, especially in small samples. For small sample size, they can lead to opposite conclusion about the rejection of the null hypothesis. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 223 / 225

4. MLE and inference Key concepts of Section 4 1 Likelihood Ratio (LR) test 2 Wald test 3 Lagrange Multiplier (LM) test 4 Computational issues 5 Comparison of the three tests (the trilogy) in nite samples Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 224 / 225

End of Chapter 4 Christophe Hurlin (University of Orléans) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, 2015 225 / 225