Chapter 4: Statistical Hypothesis Testing

Transcription

1 Chapter 4: Statistical Hypothesis Testing Christophe Hurlin November 20, 2015 Christophe Hurlin () Advanced Econometrics - Master ESA November 20, / 225

2 Section 1 Introduction Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

3 1. Introduction The outline of this chapter is the following: Section 2. Statistical hypothesis testing Section 3. Tests in the multiple linear regression model Subsection 3.1. The Student test Subsection 3.2. The Fisher test Section 4. MLE and Inference Subsection 4.1. The Likelihood Ratio (LR) test Subsection 4.2.The Wald test Subsection 4.3. The Lagrange Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

4 1. Introduction References Amemiya T. (1985), Advanced Econometrics. Harvard University Press. Greene W. (2007), Econometric Analysis, sixth edition, Pearson - Prentice Hil (recommended) Ruud P., (2000) An introduction to Classical Econometric Theory, Oxford University Press. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

5 1. Introduction Notations: In this chapter, I will (try to...) follow some conventions of notation. f Y (y) F Y (y) Pr () y Y probability density or mass function cumulative distribution function probability vector matrix Be careful: in this chapter, I don t distinguish between a random vector (matrix) and a vector (matrix) of deterministic elements (except in section 2). For more appropriate notations, see: Abadir and Magnus (2002), Notation in econometrics: a proposal for a standard, Econometrics Journal. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

6 Section 2 Statistical hypothesis testing Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

7 2. Statistical hypothesis testing Objectives The objective of this section is to de ne the following concepts: 1 Null and alternative hypotheses 2 One-sided and two-sided tests 3 Rejection region, test statistic and critical value 4 Size, power and power function 5 Uniformly most powerful (UMP) test 6 Neyman Pearson lemma 7 Consistent test and unbiased test 8 p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

8 2. Statistical hypothesis testing Introduction 1 A statistical hypothesis test is a method of making decisions or a rule of decision (as concerned a statement about a population parameter) using the data of sample. 2 Statistical hypothesis tests de ne a procedure that controls ( xes) the probability of incorrectly deciding that a default position (null hypothesis) is incorrect based on how likely it would be for a set of observations to occur if the null hypothesis were true. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

9 2. Statistical hypothesis testing Introduction (cont d) In general we distinguish two types of tests: 1 The parametric tests assume that the data have come from a type of probability distribution and makes inferences about the parameters of the distribution 2 The non-parametric tests refer to tests that do not assume the data or population have any characteristic structure or parameters. In this course, we only consider the parametric tests. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

10 2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

11 2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

12 2. Statistical hypothesis testing De nition (Hypothesis) A hypothesis is a statement about a population parameter. The formal testing procedure involves a statement of the hypothesis, usually in terms of a null or maintained hypothesis and an alternative, conventionally denoted H 0 and H 1, respectively. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

13 2. Statistical hypothesis testing Introduction 1 The null hypothesis refers to a general or default position: that there is no relationship between two measured phenomena or that a potential medical treatment has no e ect. 2 The costs associated to the violation of the null must be higher than the cost of a violation of the alternative. Example (Choice of the null hypothesis) In a credit scoring problem, in general we have: H 0 : the client is not risky(acceptance of the loan) versus H 1 : the client is risky (refusal of the loan). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

14 2. Statistical hypothesis testing De nition (Simple and composite hypotheses) A simple hypothesis speci es the population distribution completely. A composite hypothesis does not specify the population distribution completely. Example (Simple and composite hypotheses) If X t (θ), H 0 : θ = θ 0 is a simple hypothesis. H 1 : θ > θ 0, H 1 : θ < θ 0, and H 1 : θ 6= θ 0 are composite hypotheses. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

15 2. Statistical hypothesis testing De nition (One-sided test) A one-sided test has the general form: H 0 : θ = θ 0 or H 0 : θ = θ 0 H 1 : θ < θ 0 H 1 : θ > θ 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

16 2. Statistical hypothesis testing De nition (Two-sided test) A two-sided test has the general form: H 0 : θ = θ 0 H 1 : θ 6= θ 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

17 2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

18 2. Statistical hypothesis testing De nition (Rejection region) The rejection region is the set of values of the test statistic (or equivalently the set of samples) for which the null hypothesis is rejected. The rejection region is denoted W. For example, a standard rejection region W is of the form: or equivalently W = fx : T (x) > cg W = fx 1,.., x N : T (x 1,.., x N ) > cg where x denotes a sample fx 1,.., x N g, T (x) the realisation of a test statistic and c the critical value. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

19 2. Statistical hypothesis testing Remarks 1 A (hypothesis) test is thus a rule that speci es: 1 For which sample values the decision is made to fail to reject H0 as true; 2 For which sample values the decision is made to reject H0. 3 Never say "Accept H1", "fail to reject H1" etc.. 2 The complement of the rejection region is the non-rejection region. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

20 2. Statistical hypothesis testing Remark The rejection region is de ned as to be: W = fx : T (x) {z } test statistic 7 c {z} g critical value T (x) is the realisation of the statistic (random variable): T (X ) = T (X 1,.., X N ) The test statistic T (X ) has an exact or an asymptotic distribution D under the null H 0. T (X ) H0 D or T (X ) d! H0 D Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

21 2. Statistical hypothesis testing Introduction (cont d) A statistical test is based on three elements: 1 A null hypothesis and an alternative hypothesis 2 A rejection region based on a test statistic and a critical value 3 A type I error and a type II error Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

22 2. Statistical hypothesis testing Decision Fail to reject H 0 Reject H 0 Truth H 0 Correct decision Type I error H 1 Type II error Correct decision Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

23 2. Statistical hypothesis testing De nition (Size) The probability of a type I error is the (nominal) size of the test. This is conventionally denoted α and is also called the signi cance level. α = Pr (Wj H 0 ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

24 2. Statistical hypothesis testing Remark For a simple null hypothesis: α = Pr (Wj H 0 ) For a composite null hypothesis: α = sup Pr (Wj H 0 ) θ 0 2H 0 A test is said to have level if its size is less than or equal to α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

25 2. Statistical hypothesis testing De nition (Power) The power of a test is the probability that it will correctly lead to rejection of a false null hypothesis: power = Pr (Wj H 1 ) = 1 β where β denotes the probability of type II error, i.e. β = Pr W denotes the non-rejection region. W H 1 and Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

26 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m = m 1 with m 1 < m 0. An econometrician propose the following rule of decision: W = fx : x N < cg where X N = N 1 N i=1 X i denotes the sample mean and c is a constant (critical value). Question: calculate the size and the power of this test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

27 2. Statistical hypothesis testing Solution The rejection region is W= fx : x N < cg. Under the null H 0 : m = m 0 : X N H0 N So, the size of the test is equal to: m 0, σ2 N α = Pr (Wj H 0 ) = Pr X N < c H 0 X = N m 0 Pr σ/ p N < c m 0 σ/ p N H 0 c m0 = Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

28 2. Statistical hypothesis testing Solution (cont d) The rejection region is W= fx : x N < cg. Under the alternative H 1 : m = m 1 : X N N m 1, σ2 H1 N So, the power of the test is equal to: power = Pr (Wj H 1 ) X = N m 1 Pr σ/ p N < c m 1 σ/ p N H 1 c m1 = Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

29 2. Statistical hypothesis testing Solution (cont d) In conclusion: c m0 α = Φ σ/ p N c m1 β = 1 power = 1 Φ σ/ p N We have a system of two equations with three parameters: α, β (or power) and the critical value c. 1 There is a trade-o between the probabilities of the errors of type I and II, i.e. α and β : if c decreases, α decreases but β increases. 2 A solution is to impose a size α and determine the critical value and the power. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

30 2. Statistical hypothesis testing Solution (cont d) In order to illustrate the tradeo between α and β given the critical value c, take an example with σ 2 = 1 and N = 100: H 0 : m = m 0 = 1.2 H 1 : m = m 1 = 1 X N N m 0, σ2 X N N m 1, σ2 H0 N H1 N We have W = fx : x N < cg c m0 α = Pr (Wj H 0 ) = Φ σ/ p = Φ (10 (c 1.2)) N β = Pr W c m1 H 1 = 1 Φ σ/ p = 1 Φ (10 (c 1)) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

31 2. Statistical hypothesis testing Density under H0 Density under H α=pr(w[h0)=5% β=1 Pr(W H1)=36.12% Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

32 2. Statistical hypothesis testing Click me! Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

33 2. Statistical hypothesis testing Fact (Critical value) The (nominal) size α is xed by the analyst and the critical value is deduced from α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

34 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2, N = 100 and σ 2 = 1. We want to test H 0 : m = 1.2 H 1 : m = 1 An econometrician propose the following rule of decision: W = fx : x N < cg where X N = N 1 N i=1 X i denotes the sample mean and c is a constant (critical value). Questions: (1) what is the critical value of the test of size α = 5%? (2) what is the power of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

35 2. Statistical hypothesis testing Solution We know that: c m0 α = Pr (Wj H 0 ) = Φ σ/ p N So, the critical value that corresponds to a signi cance level of α is: c = m 0 + σ p N Φ 1 (α) NA: if m 0 = 1.2, m 1 = 1, N = 100, σ 2 = 1 and α = 5%, then the rejection region is W = fx : x N < g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

36 2. Statistical hypothesis testing Solution (cont d) W = x : x N < m 0 + p σ Φ 1 (α) N The power of the test is: c m1 power = Pr (Wj H 1 ) = Φ σ/ p N Given the critical value, we have: m0 m 1 power = Φ σ/ p N + Φ 1 (α) NA: if m 0 = 1.2, m 1 = 1, N = 100, σ 2 = 1 and α = 5%: power = Φ 1/ p Φ 1 (0.05) = Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

37 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 with σ 2 = 1 and N = 100. We want to test H 0 : m = 1.2 H 1 : m = 1 The rejection region for a signi cance level α = 5% is: W = fx : x N < g where X N = N 1 N i=1 X i denotes the sample mean. Question: if the realisation of the sample mean is equal to 1.13, what is the conclusion of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

38 2. Statistical hypothesis testing Solution (cont d) For a nominal size α = 5%, the rejection region is: W = fx : x N < g If we observe x N = 1.13 This realisation does not belong to the rejection region: x N /2 W For a level of 5%, we do not reject the null hypothesis H 0 : m = 1.2. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

39 2. Statistical hypothesis testing De nition (Power function) In general, the alternative hypothesis is composite. In this case, the power is a function of the value of the parameter under the alternative. power = P (θ) 8θ 2 H 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

40 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m < m 0 Consider the following rule of decision: W = x : x N < m 0 + p σ Φ 1 (α) N Questions: What is the power function of the test? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

41 2. Statistical hypothesis testing Solution As in the previous case, we have: m0 m power = P (m) = Φ σ/ p N + Φ 1 (α) with m < m 0 NA: if m 0 = 1.2, N = 100, σ 2 = 1 and α = 5%. 1.2 m P (m) = Φ with m < m 0 1/10 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

42 2. Statistical hypothesis testing Power function P (m) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

43 2. Statistical hypothesis testing Example (Power function) Consider a test H 0 : θ = θ 0 versus H 1 : θ 6= θ 0, the power function has this general form: Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

44 2. Statistical hypothesis testing De nition (Most powerful test) A test (denoted A) is uniformly most powerful (UMP) if it has greater power than any other test of the same size for all admissible values of the parameter. α A = α B = α for any test B of size α. β A β B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

45 2. Statistical hypothesis testing UMP tests How to derive the rejection region of the UMP test of size α? => the Neyman Pearson lemma Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

46 2. Statistical hypothesis testing Lemma (Neyman Pearson) Consider a hypothesis test between two point hypotheses H 0 : θ = θ 0 and H 1 : θ = θ 1. The uniformly most powerful (UMP) test has a rejection region de ned by: W = xj L N (θ 0 ; x) L N (θ 1 ; x) < K where L N (θ 0 ; x) denotes the likelihood of the sample x and K is a constant determined by the size α such that: LN (θ 0 ; X ) Pr L N (θ 1 ; X ) < K H 0 = α Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

47 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m = m 1 with m 1 > m 0. Question: What is the rejection region of the UMP test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

48 2. Statistical hypothesis testing Solution Since X 1,.., X N are N.i.d. m, σ 2, the likelihood of the sample fx 1,.., x N g is de ned as to be (cf. chapter 2):! N 1 L N (θ; x) = σ N (2π) N /2 exp 1 2σ 2 (x i m) 2 i=1 Given the Neyman Pearson lemma the rejection region of the UMP test of size α is given by: L N (θ 0 ; x) L N (θ 1 ; x) < K where K is a constant determined by the size α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

49 2. Statistical hypothesis testing Solution (cont d) 1 exp σ N (2π) N /2 1 σ N (2π) N /2 exp 1 2σ 2 N i=1 (x i 1 2σ 2 N i=1 (x i m 0 ) 2 m 1 ) 2 < K This expression can rewritten as: 1 exp 2σ 2 N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K () N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K 1 where K 1 = 2σ 2 ln (K ) is a constant. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

50 2. Statistical hypothesis testing Solution (cont d) N i=1 (x i m 1 ) 2 N i=1 (x i m 0 ) 2 < K 1 () 2 (m 0 m 1 ) N i=1 x i + N m 2 1 m 2 0 < K1 () (m 0 m 1 ) N i=1 x i < K 2 where K 2 = K 1 N m 2 1 m 2 0 /2 is a constant. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

51 2. Statistical hypothesis testing Solution (cont d) Since m 1 > m 0, we have where K 3 = K 2 / (N (m 0 (m 0 m 1 ) N i=1 x i < K 2 1 N N i=1 x i > K 3 m 1 )) is a constant. The rejection region of the UMP test for H 0 : m = m 0 against H 0 : m = m 1 with m 1 > m 0 has the general form: where A is a constant. W = fx : x N > Ag Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

52 2. Statistical hypothesis testing Solution (cont d) W = fx : x N > Ag Determine the critical value A from the nominal size: α = Pr (Wj H 0 ) = Pr (x N > Aj H 0 ) X = N m 0 1 Pr σ/ p N < A m 0 σ/ p N H 0 A m0 = 1 Φ σ/ p N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

53 2. Statistical hypothesis testing Solution (cont d) So, we have α = 1 A = m 0 + A m0 Φ σ/ p N σ p N Φ 1 (1 α) The rejection region of the UMP test of size α for H 0 : m = m 0 against H 0 : m = m 1 with m 1 > m 0 is: W = x : x N > m 0 + p σ Φ 1 (1 α) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

54 2. Statistical hypothesis testing Fact (UMP one-sided test) For a one-sided test H 0 : θ = θ 0 against H 1 : θ > θ 0 (or H 1 : θ < θ 1 ) the rejection region W of the UMP test is equivalent to the rejection region obtained for the test H 0 : θ = θ 0 against H 1 : θ = θ 1 with for θ 1 > θ 0 (or θ 1 < θ 0 ) if this region does not depend on the value of θ 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

55 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m > m 0 Question: What is the rejection region of the UMP test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

56 2. Statistical hypothesis testing Solution Consider the test: H 0 : m = m 0 H 1 : m = m 1 with m 1 > m 0. The rejection region of the UMP test of size α is: W = x : x N > m 0 + p σ Φ 1 (1 α) N W does not depend on m 1. It is also the rejection region of the UMP one-sided test for H 0 : m = m 0 H 1 : m > m 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

57 2. Statistical hypothesis testing Fact (Two-sided test) For a two-sided test H 0 : θ = θ 0 against H 1 : θ 6= θ 0 the non rejection region W of the test of size α is the intersection of the non rejection regions of the corresponding one-sided UMP tests of size α/2 Test A: H 0 : θ = θ 0 against H 1 : θ > θ 0 Test B: H 0 : θ = θ 0 against H 1 : θ < θ 0 So, we have: W = W A \ W B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

58 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m 6= m 0 Question: What is the rejection region of the test of size α? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

59 2. Statistical hypothesis testing Solution Consider the one-sided tests: Test A: H 0 : m = m 0 against H 1 : m < m 0 Test B: H 0 : m = m 0 against H 1 : m > m 0 The rejection regions of the UMP test of size α/2 are: W A = x : x N < m 0 + p σ Φ 1 (α/2) N W B = x : x N > m 0 + p σ Φ 1 (1 N α/2) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

60 2. Statistical hypothesis testing Solution (cont d) The non-rejection regions of the UMP test of size α/2 are: W A = x : x N m 0 + p σ Φ 1 (α/2) N W B = x : x N m 0 + p σ Φ 1 (1 N α/2) The non rejection region of the two-sided test corresponds to the intersection of these two regions: W = W A \ W B Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

61 2. Statistical hypothesis testing Solution (cont d) So, non rejection region of the two-sided test of size α is: W = x : m 0 + p σ Φ 1 (α/2) x N m 0 + p σ Φ 1 (1 N N α/2) Since, Φ 1 (α/2) = Φ 1 (1 α/2), this region can be rewritten as: W = x : jx N m 0 j p σ Φ 1 (1 α/2) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

62 2. Statistical hypothesis testing Solution (cont d) W = x : jx N m 0 j σ p N Φ 1 (1 α/2) Finally, the rejection region of the two-sided test of size α is: W = x : jx N m 0 j > p σ Φ 1 (1 α/2) N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

63 2. Statistical hypothesis testing Solution (cont d) W = x : jx N m 0 j > σ p N Φ 1 (1 α/2) NA: if m 0 = 1.2, N = 100, σ 2 = 1 and α = 5%: W = x : jx N 1.2j > 1 10 Φ 1 (0.975) W = fx : jx N 1.2j > g If the realisation of jx N 1.2j is larger than , we reject the null H 0 : m = 1.2 for a signi cance level of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

64 2. Statistical hypothesis testing De nition (Unbiased Test) A test is unbiased if its power P (θ) is greater than or equal to its size α for all values of the parameter θ. P (θ) α 8θ 2 H 1 By construction, we have P (θ 0 ) = Pr (Wj Hj 0 ) = α. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

65 2. Statistical hypothesis testing De nition (Consistent Test) A test is consistent if its power goes to one as the sample size grows to in nity. lim N! P (θ) = 1 8θ 2 H 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

66 2. Statistical hypothesis testing Example (Test on the mean) Consider a sequence X 1,.., X N of i.i.d. continuous random variables with X i N m, σ 2 where σ 2 is known. We want to test H 0 : m = m 0 H 1 : m < m 0 The rejection region of the UMP test of size α is W = x : x N < m 0 + p σ Φ 1 (α) N Question: show that this test is (1) unbiased and (2) consistent. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

67 2. Statistical hypothesis testing Solution W = x : x N < m 0 + p σ Φ 1 (α) N The power function of the test is de ned as to be: P (m) = Pr (Wj H 1 ) = Pr X N < m 0 + p σ Φ 1 (α) N m < m 0 m0 m = Φ σ/ p N + Φ 1 (α) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

68 2. Statistical hypothesis testing Solution m0 m P (m) = Φ σ/ p N + Φ 1 (α) The test is consistent since: 8m < m 0 lim P (m) = 1 N! The test is unbiased since P (m) α 8m < m 0 lim m!m 0 P (m) = Φ Φ 1 (α) = α Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

69 2. Statistical hypothesis testing Solution The decision Reject H0 or fail to reject H0 is not so informative! Indeed, there is some arbitrariness to the choice of α (level). Another strategy is to ask, for every α, whether the test rejects at that level. Another alternative is to use the so-called p-value the smallest level of signi cance at which H 0 would be rejected given the value of the test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

70 2. Statistical hypothesis testing De nition (p-value) Suppose that for every α 2 [0, 1], one has a size α test with rejection region W α. Then, the p-value is de ned to be: p-value = inf fα : T (y) 2 W α g The p-value is the smallest level at which one can reject H 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

71 2. Statistical hypothesis testing The p-value is a measure of evidence against H 0 : p-value evidence < 0.01 Very strong evidence against H Strong evidence against H Weak evidence against H 0 > 0.10 Little or no evidence against H 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

72 2. Statistical hypothesis testing Remarks 1 A large p-value does not mean strong evidence in favor of H0. 2 A large p-value can occur for two reasons: 1 H0 is true; 2 H0 is false but the test has low power. 3 The p-value is not the probability that the null hypothesis is true! Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

73 2. Statistical hypothesis testing For a nominal size of 5%, we reject the null H 0 : β SP500 = 0. For a nominal size of 5%, we fail to reject the null H 0 : β C = 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

74 2. Statistical hypothesis testing Summary Hypothesis testing is de ned by the following general procedure Step 1: State the relevant null and alternative hypotheses (misstating the hypotheses muddies the rest of the procedure!); Step 2: Consider the statistical assumptions being made about the sample in doing the test (independence, distributions, etc.) incorrect assumptions mean that the test is invalid! Step 3: Choose the appropriate test (exact or asymptotic tests) and thus state the relevant test statistic (say, T). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

75 2. Statistical hypothesis testing Summary (cont d) Step 4: Derive the distribution of the test statistic under the null hypothesis (sometimes it is well-known, sometimes it is more tedious!) for example, the Student t-distribution or the Fisher distribution. Step 5: Determine the critical value (and thus the critical region). Step 6: Compute (using the observations!) the observed value of the test statistic T, say t obs. Step 7: Decide to either fail to reject the null hypothesis or reject in favor of the alternative assumption the decision rule is to reject the null hypothesis H 0 if the observed value of the test statistic, t obs is in the critical region, and to fail to reject the null hypothesis otherwise Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

76 2. Statistical hypothesis testing Key concepts 1 Null and alternative hypotheses 2 Simple and composite hypotheses 3 One-sided and two-sided tests 4 Rejection region, test statistic and critical value 5 Type I and type II errors 6 Size, power and power function 7 Uniformly most powerful (UMP) test 8 Neyman Pearson lemma 9 Consistent test and unbiased test 10 p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

77 Section 3 Tests in the multiple linear regression model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

78 3. Tests in the multiple linear regression model Objectives In the context of the multiple linear regression model (cf. chapter 3), the objective of this section is to present : 1 the Student test 2 the t-statistic and the z-statistic 3 the Fisher test 4 the global F-test 5 To distinguish the case with normality assumption and the case without any assumption on the distribution of the error term (semi-parametric speci cation) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

79 3. Tests in the multiple linear regression model Be careful: in this section, I don t distinguish between a random vector (matrix) and a vector (matrix) of deterministic elements. For more appropriate notations, see: Abadir and Magnus (2002), Notation in econometrics: a proposal for a standard, Econometrics Journal. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

80 3. Tests in the multiple linear regression model Model Consider the (population) multiple linear regression model: where (cf. chapter 3): y = Xβ + ε y is a N 1 vector of observations y i for i = 1,.., N X is a N K matrix of K explicative variables x ik for k = 1,., K and i = 1,.., N ε is a N 1 vector of error terms ε i. β = (β 1..β K ) > is a K 1 vector of parameters Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

81 3. Tests in the multiple linear regression model Assumptions Fact (Assumptions) We assume that the multiple linear regression model satisfy the assumptions A1-A5 (cf. chapter 3) We distinguish two cases: 1 Case 1: assumption A6 (Normality) holds and ε N 0, σ 2 I N 2 Case 2: the distribution of ε is unknown (semi-parametric speci cation) and ε?? Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

82 3. Tests in the multiple linear regression model Parametric tests The β k are unknown features of the population, but: 1 One can formulate a hypothesis about their value; 2 One can construct a test statistic with a known nite sample distribution (case 1) or an asymptotic distribution (case 2); 3 One can take a decision meaning reject H0 if the value of the test statistic is too unlikely. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

83 3. Tests in the multiple linear regression model Three tests of interest: H 0 : β k = a k or H 0 : β k = a k H 1 : β k < a k H 1 : β k > a k H 0 : β k = a k H 1 : β k 6= a k where a k = 0 or a k 6= 0. H 0 : Rβ = q H 1 : Rβ 6= q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

84 3. Tests in the multiple linear regression model For that, we introduce two types of test 1 The Student test or t-test 2 The Fisher test of F-test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

85 Subsection 3.1 The Student test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

86 3.1. The Student test Case 1: Normality assumption A6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

87 3.1. The Student test Assumption 6 (normality): the disturbances are normally distributed. εj X N 0 N 1, σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

88 3.1. The Student test Reminder (cf. chapter 3) Fact (Linear regression model) Under the assumption A6 (normality), the estimators β b and bσ 2 have a nite sample distribution given by: bβ 1 N β,σ 2 X > X bσ 2 σ 2 (N K ) χ2 (N K ) Moreover, b β and bσ 2 are independent. This result holds whether or not the matrix X is considered as random. In this last case, the distribution of b β is conditional to X. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

89 3.1. The Student test Remarks 1 Any linear combination of β b is also normally distributed: Aβ b 1 N Aβ,σ 2 A X > X A > 2 Any subset of b β has a joint normal distribution. bβ k N β k, σ 2 m kk where m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

90 3.1. The Student test Reminder If X and Y are two independent random variables such that then the variable Z de ned as to be X N (0, 1) Y χ 2 (θ) Z = X p Y /θ has a Student s t-distribution with θ degrees of freedom Z t (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

91 3.1. The Student test Student test statistic Consider a test with the null: Under the null H 0 : H 0 : β k = a k bβ k a k σ p N (0, 1) m kk H0 bσ 2 σ 2 (N K ) H 0 χ 2 (N K ) and these two variables are independent... Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

92 3.1. The Student test Student test statistic (cont d) So, under the null H 0 we have: bβ k a k σ p N (0, 1) m kk H0 bσ 2 σ 2 (N K ) H 0 χ 2 (N K ) bβ k a k σ p m q kk bσ 2 (N K ) σ 2 (N K ) = b β k a k bσ p m kk H0 t (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

93 3.1. The Student test De nition (Student t-statistic) Under the null H 0 : β k = a k, the Student test-statistic or t-statistic is de ned to be: a k T k = b β k t (N K ) bse b β H0 k where N is the number of observations, K is the number of explanatory variables (including the constant term), t (N K ) is the Student t-distribution with N K degrees of freedom and bse b β k = bσ p m kk with m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

94 3.1. The Student test Remarks 1 Under the assumption A6 (normality) and under the null H 0 : β k = a k, the Student test-statistic has an exact ( nite sample) distribution. T k t (N K ) H0 2 The term bse b β k denotes the estimator of the standard error of the OLS estimator bβ k and it corresponds to the square root of the k th diagonal element of bv bβ (cf. chapter 3): bv 1 bβ = bσ 2 X > X Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

95 3.1. The Student test Consider the one-sided test: k (y) < Aj T k t (N H0 H 0 : β k = a k H 1 : β k < a k The rejection region is de ned as to be: W = fy : T k (y) < Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr T K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

96 3.1. The Student test α = Pr T k (y) < Aj T k t (N K ) = F N K (A) H0 where F N K (.) denotes the cdf of the Student s t-distribution with N K degrees of freedom. Denote c α the α-quantile of this distribution:. A = FN 1 K (α) = c α The rejection region of the test of size α is de ned as to be: W = fy : T k (y) < c α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

97 3.1. The Student test De nition (One-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k < a k at the α (say, 5%) signi cance level if: W = fy : T k (y) < c α g where c α is the α (say, 5%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

98 3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT < 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

99 3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = = bse b β MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : T k (y) < g Conclusion: for a signi cance level of 5%, we fail to reject the null H 0 : β MSFT = 1 against H 1 : β MSFT < 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

100 3.1. The Student test Solution (cont d) Density of Ts under H α=5% DF = 18 Critical value = Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

101 3.1. The Student test Consider the one-sided test k (y) > Aj T k t (N H0 H 0 : β k = a k H 1 : β k > a k The rejection region is de ned as to be: W = fy : T k (y) > Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr T K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

102 3.1. The Student test α = 1 Pr T k (y) < Aj T k t (N K ) H0 or equivalently 1 α = F N K (A) where F N K (.) denotes the cdf of the Student s t-distribution with N K degrees of freedom. Denote c 1 α the 1 α quantile of this distribution: A = F 1 N K (1 α) = c 1 α The rejection region of the test of size α is de ned as to be: W = fy : T k (y) > c 1 α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

103 3.1. The Student test De nition (One-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k > a k at the α (say, 5%) signi cance level if: W = fy : T k (y) > c 1 α g where c 1 α is the 1 α (say, 95%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

104 3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT > 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

105 3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = = bse b β MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : T k (y) > g Conclusion: for a signi cance level of 5%, we reject the null H 0 : β MSFT = 1 against H 1 : β MSFT > 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

106 3.1. The Student test Solution (cont d) Density of Ts under H DF = 18 Critical value = α=5% Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

107 3.1. The Student test Consider the two-sided test H 0 : β k = a k H 1 : β k 6= a k The non-rejection region is de ned as the intersection of the two non-rejection regions of the one-sided test of level α/2: W = W A \ W B H 0 : β k = a k against H 1 : β k < a k W A = fy : T k (y) > c α/2 g H 0 : β k = a k against H 1 : β k > a k W B = fy : T k (y) < c 1 α/2 g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

108 3.1. The Student test W = fy : c α/2 < T k (y) < c 1 α/2 g Since the Student s t-distribution is symmetric, c α/2 = c 1 α/2 W = fy : c 1 α/2 < T k (y) < c 1 α/2 g The rejection region is then de ned as to be: W = fy : jt k (y)j > c 1 α/2 g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

109 3.1. The Student test De nition (Two-sided Student test) The critical region of the Student test is that H 0 : β k = a k is rejected in favor of H 1 : β k 6= a k at the α (say, 5%) signi cance level if: W = fy : jt k (y)j > c 1 α/2 g where c 1 α/2 is the 1 α/2 (say, 97.5%) critical value of a Student t-distribution with N K degrees of freedom and T k (y) is the realisation of the Student test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

110 3.1. The Student test Example (One-sided test) Consider the CAPM model (cf. chapter 1) and the following results (Eviews). We want to test the beta of MSFT as H 0 : β MSFT = 1 against H 1 : β MSFT 6= 1 Question: give a conclusion for a nominal size of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

111 3.1. The Student test Solution Step 1: compute the t-statistic T MSFT (y) = b β MSFT 1 = = bse b β MSFT Step 2: Determine the rejection region for a nominal size α = 5%. T MSFT H0 t (20 2) W = fy : jt k (y)j > g Conclusion: for a signi cance level of 5%, we reject the null H 0 : β MSFT = 1 against H 1 : β MSFT 6= 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

112 3.1. The Student test Density of Ts under H α=2.5% DF = 18 Critical value = α=2.5% Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

113 3.1. The Student test Rejection regions H 0 H 1 Rejection region β k = a k β k > a k W = fy : T k (y) > c 1 α g β k = a k β k < a k W = fy : T k (y) < c α g β k = a k β k 6= a k W = fy : jt k (y)j > c 1 α/2 g where c β denotes the β-quantile (critical value) of the Student t-distribution with N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

114 3.1. The Student test De nition (P-values) The p-values of Student tests are equal to: Two-sided test: p-value = 2 F N K ( jt k (y)j) Right tailed test: p-value = 1 F N K (T k (y)) Left tailed test: p-value = F N K ( T k (y)) where T k (y) is the realisation of the Student test-statistic and F N K (.) the cdf of the Student s t-distribution with N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

115 3.1. The Student test Example (One-sided test) Consider the previous CAPM model. We want to test: H 0 : c = 0 against H 1 : c 6= 0 H 0 : β MSFT = 0 against H 1 : β MSFT 6= 0 Question: nd the corresponding p-values. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

116 3.1. The Student test Solution Since we consider two-sided tests with N = 20 and K = 2: p-value c = 2 F 18 ( jt c (y)j) = 2 F 18 ( ) = p-value c = 2 F 18 ( jt MSFT (y)j) = 2 F 18 ( ) = 5.7e 006 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

117 3.1. The Student test Fact (Student test with large sample) For a large sample size N T k t (N K ) N (0, 1) H0 Then, the rejection region for a Student two-sided test becomes W = y : jt k (y)j > Φ 1 (1 α/2) where Φ (.) denotes the cdf of the standard normal distribution. For α = 5%, Φ 1 (0.975) = 1.96, so we have: W = fy : jt k (y)j > 1.96g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

118 3.1. The Student test Case 2: Semi-parametric model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

119 3.1. The Student test Assumption 6 (normality): the distribution of the disturbances is unknown, but satisfy (assumptions A1-A5): E ( εj X) = 0 N 1 V ( εj X) = σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

120 3.1. The Student test Problem 1 The exact ( nite sample) distribution of bβ k and bσ 2 are unknown. 2 As a consequence the nite sample distribution of T k (y) is also unknown. 3 But, we can use the asymptotic properties of the OLS estimators (cf. chapter 3). In particular, we have: p N bβ β d! N 0, σ 2 Q 1 where Q = p lim 1 N X> X = E X x i x > i Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

121 3.1. The Student test De nition (Z-statistic) Under the null H 0 : β k = a k, if the assumptions A1-A5 hold (cf. chapter 3), the z-statistic de ned by Z k = b β k a k bse asy b β k d! H0 N (0, 1) where bse asy b β k = bσ p m kk denotes the estimator of the asymptotic standard error of the estimator bβ k and m kk is k th diagonal element of X > X 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

122 3.1. The Student test Rejection regions The rejection regions have the same form as for the t-test (except for the distribution) H 0 H 1 Rejection region β k = a k β k > a k W = y : Z k (y) > Φ 1 (1 α) β k = a k β k < a k W = y : Z k (y) < Φ 1 (α) β k = a k β k 6= a k W = y : jz k (y)j > Φ 1 (1 α/2) where Φ (.) denotes the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

123 3.1. The Student test De nition (P-values) The p-values of the Z-tests are equal to: Two-sided test: p-value = 2 Φ ( jz k (y)j) right tailed test: p-value = 1 left tailed test: p-value = Φ ( Φ (Z k (y)) Z k (y)) where Z k (y) is the realisation of the Z-statistic and Φ (.) the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

124 3.1. The Student test Summary Normality Assumption Non Assumption Test-statistic t-statistic z-statistic De nition T k = b β k a k bσ p m kk Z k = b β k a k bσ p m kk Exact distribution T k H0 t (N K ) Asymptotic distribution Z K d!h0 N (0, 1) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

125 3.1. The Student test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

126 Subsection 3.2 The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

127 3.2. The Fisher test Consider the two-sided test associated to p linear constraints on the parameters β k : H 0 : Rβ = q H 1 : Rβ 6= q where R is a p K matrix and q is a p 1 vector. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

128 3.2. The Fisher test Example (Linear constraints) If K = 4 and if we want to test H 0 : β 1 + β 2 = 0 and β 2 we have p = 2 linear constraints with: 3β 3 = 4, then R β (24) (4,1) = q (21) β 1 β 2 β 3 β 4 1 C A = 0 4 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

129 3.2. The Fisher test Example (Linear constraints) If K = 4 and if we want to test H 0 : β 2 = β 3 = β 4 = 0, then we have p = 3 linear constraints with: R β (34) (4,1) A = q (31) β 1 β 2 β 3 β 4 1 C A = A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

130 3.1. The Student test Case 1: Normality assumption A6 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

131 3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic is de ned as to be: F = 1 Rβ p b > 1 1 q bσ 2 R X > X R > Rβ b q where b β denotes the OLS estimator. Under the null H 0 : Rβ = q, the F -statistic has a Fisher exact ( nite sample) distribution F H0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

132 3.2. The Fisher test Reminder If X and Y are two independent random variables such that then the variable Z de ned by X χ 2 (θ 1 ) Y χ 2 (θ 2 ) Z = X /θ 1 Y /θ 2 has a Fisher distribution with θ 1 and θ 2 degrees of freedom Z F (θ1,θ 2 ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

133 3.2. The Fisher test Proof Under assumption A6, we have the following (conditional to X) distribution bβ 1 N β,σ 2 X > X bσ 2 σ 2 (N K ) χ2 (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

134 3.2. The Fisher test Proof (cont d) Consider the vector m = R b β q. Under the null H 0 : Rβ = q We have E (m) = RE bβ q = Rβ q = 0 V (m) = E Rβ b q Rβ b bβ = RV = σ 2 R R > 1 X > X R > > q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

135 3.2. The Fisher test Proof (cont d) We can base the test of H 0 on the Wald criterion: W (11) = m > 1 (V (m)) (1p) pp > = R b β q σ 2 R m p1 1 X > X R > 1 R b β q Under assumption A6 (normality) W H0 χ 2 (p) These two variables are independent. bσ 2 σ 2 (N K ) χ2 (N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

136 3.2. The Fisher test Proof (cont d) W H0 χ 2 (p) bσ 2 σ 2 (N K ) χ2 (N K ) So, the ratio of these two variables has a Fisher distribution F = W p bσ 2 (N K ) σ 2 (N K ) H 0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

137 3.2. The Fisher test Proof (cont d) R b > 1 1 β q σ 2 R X > X R > Rβ b q /p F = bσ 2 (N K ) / (N K ) σ 2 After simpli cation, the F-statistic is de ned by: F = 1 p R b β Under the null H 0 : Rβ =q : > 1 1 q bσ 2 R X > X R > Rβ b q F H0 F (p,n K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

138 3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can be de ned as a function of the SSR of the constrained (H 0 ) and unconstrained model (H 1 ): SSR0 SSR 1 N K F = SSR 1 p where SSR 0 denotes the sum of squared residuals of the constrained model estimated under H 0 and SSR 1 denotes the sum of squared residuals of the unconstrained model estimated under H 1. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

139 3.2. The Fisher test De nition (Fisher test-statistic) Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can be de ned as to be: F = 1 > bβh1 bβ bσ 2 H0 X > X bβh1 bβ H0 p where β b H0 denotes the OLS estimator obtained in the constrained model (under H 0 ) and β b H1 denotes the OLS estimator obtained in the unconstrained model (under H 1 ). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

140 3.2. The Fisher test De nition (Constrained OLS estimator) Under suitable regularity conditions, the constrained OLS estimator b β C of β, obtained under the constraint Rβ = q, is given by: bβ C = b β UC X > X 1 R > 1 1 R X > X R > Rβ b UC q where b β UC is the unconstrained OLS estimator. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

141 3.2. The Fisher test Example (Fisher test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t where r MSFT,t is the excess return for Microsoft, r SP500,t for the SP500, r Ford,t for Ford and r GE,t for general electric. We want to test the following linear constraints: H 0 : β 2 = 1 and β 3 = β 4 Question: write a Matlab code to compute the F-statistic according to the three alternative de nitions. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

142 3.2. The Fisher test Solution In this problem, the null H 0 : β 2 = 1 and β 3 = β 4 can be written as: R β (24) (4,1) = q (21) β 1 β 2 β 3 β 4 1 C A = 1 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

143 3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

144 3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

145 3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

146 3.2. The Fisher test Consider the Fisher test H 0 : Rβ = q H 1 : Rβ 6= q Since the Fisher test-statistic is always positive, the rejection region is de ned as to be: W = fy : F (y) > Ag where A is a constant determined by the nominal size α. α = Pr (Wj H 0 ) = Pr F (y) > Aj F F (p,n K ) H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

147 3.2. The Fisher test α = Pr (Wj H 0 ) = Pr F (y) > Aj F F (p,n K ) H0 or equivalently α = 1 Pr F (y) < Aj F F (p,n K ) H0 Denote d 1 α the 1 α quantile of the Fisher distribution with p and N K degrees of freedom. A = d 1 α The rejection region of the test of size α is de ned as to be: W = fy : F (y) > d 1 α g Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

148 3.2. The Fisher test De nition (Rejection region of a Fisher test) The critical region of the Fisher test is that H 0 : Rβ = q is rejected in favor of H 1 : Rβ 6= q at the α (say, 5%) signi cance level if: W = fy : F (y) > d 1 α g where d 1 α is the 1 α critical value (say 95%) of the Fisher distribution with p and N K degrees of freedom and F k (y) is the realisation of the Fisher test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

149 3.2. The Fisher test Example (Fisher test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t where r MSFT,t is the excess return for Microsoft, r SP500,t for the SP500, r Ford,t for Ford and r GE,t for general electric. We want to test the following linear constraints: H 0 : β 2 = 1 and β 3 = β 4 Question: given the realisation of the Fisher test-statistic (cf. previous example), conclude for a signi cance level α = 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

150 3.2. The Fisher test Solution Step 1: compute the F-statistic (cf. Matlab code) F (y) = Step 2: Determine the rejection region for a nominal size α = 5% for N = 24, K = 4 and p = 2 F H0 F (2,20) W = fy : F (y) > g Conclusion: for a signi cance level of 5%, we reject the null H 0 : Rβ = q against H 1 : Rβ 6= q Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

151 3.2. The Fisher test Density of F under H0 0.8 DF1 = 2 DF2 = 20 Critical value = α=5% Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

152 3.2. The Fisher test De nition (Student test-statistic and Fisher test-statistic ) Consider the test H 0 : β k = a k versus H 1 : β k 6= a k the Fisher test-statistic corresponds to the squared of the corresponding Student s test-statistic F = T 2 k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

153 3.2. The Fisher test Proof Consider the test H 0 : β k = a k against H 1 : β k 6= a k, then we have: R = k th position q = a k 0 0 As a consequence : Rβ b q = bβ k a k 1 bσ 2 R X X > R > = bv b β k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

154 3.2. The Fisher test Proof (cont d) So, for a test H 0 : β k = a k against H 1 : β k 6= a k, the Fisher test-statistic becomes F = Rβ b > 1 1 q bσ 2 R X > X R > Rβ b q So, we have: b β k a k 2 F = bv b β k and the F test-statistic is equal to the squared t-statistic: F = T 2 k Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

155 3.2. The Fisher test De nition (P-values) The p-value of the F-test is equal to: p-value = 1 F p,n K (F (y)) where F(y) is the realisation of the F-statistic and F p,n K (.) the cdf of the Fisher distribution with p and N K degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

156 3.2. The Fisher test De nition (Global F-test) In a multiple linear regression model with a constant term y i = β 1 + K k=2 β k x ik + ε i the global F-test corresponds to the test of signi cance of all the explicative variables: H 0 : β 2 =.. = β K = 0 Under the assumption A6 (normality), the global F-test-statistic satis es: F H0 F (K 1,N K ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

157 3.2. The Fisher test Remarks 1 The global F-test is a test designed to see if the model is useful overall. 2 The null H 0 : β 2 =.. = β K = 0 can be written as: 0 R β (K 1K ) (K,1) = q (K 11) 1 0 B A β β K 1 C A = C A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

158 3.2. The Fisher test Corollary (Global F-test) In a multiple linear regression model with a constant term y i = β 1 + K k=2 β k x ik + ε i the global F-test-statistic can also be de ned as: R 2 N K F = 1 R 2 K 1 where R 2 denotes the (unadjusted) coe cient of determination. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

159 3.2. The Fisher test Example (Global F-test and CAPM model) Consider the extended CAPM model ( le: Chapter4_data.xls): r MSFT,t = β 1 + β 2 r SP500,t + β 3 r Ford,t + β 4 r GE,t + ε t Question: write a Matlab code to compute the global F-test, the critical value for α = 5% and the p-value. Compare your results with Eviews. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

160 3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

161 3.2. The Fisher test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

162 3.2. The Fisher test Case 2: Semi-parametric model Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

163 3.2. The Fisher test Assumption 6 (normality): the distribution of the disturbances is unknown, but satisfy (assumptions A1-A5): E ( εj X) = 0 N 1 V ( εj X) = σ 2 I N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

164 3.2. The Fisher test Problem 1 The exact ( nite sample) distribution of bβ k and bσ 2 are unknown. As a consequence the nite sample distribution of F(y) is also unknown. 2 But, we can express the F-statistic as a linear function of the Wald statistic. 3 The Wald statistic has a chi-squared asymptotic distribution (cf. next section) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

165 3.2. The Fisher test De nition (F-test-statistic and Wald statistic) The Fisher test-statistic can expressed as a linear function of the Wald test-statistic as F = 1 p Wald Wald = 1 p R b β > 1 1 q R V asy bβ R > Rβ b q Under assumptions A1-A5, the Wald test-statistic converges to a chi-squared distribution Wald! d χ 2 (p) H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

166 3. Tests in the multiple linear regression model Key concepts of Section 3 1 Student test 2 Fisher test 3 t-statistic and z-statistic 4 Global F-test 5 Exact ( nite sample) distribution under the normality assumption 6 Asymptotic distribution Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

167 Section 4 MLE and Inference Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

168 4. MLE and inference Introduction Consider a parametric model, linear or nonlinear (GARCH, probit, logit, etc.), with a vector of parameters θ = (θ 1 :.. : θ K ) > We assume that the problem is regular (cf. chapter 2) and we consider a ML estimator bθ The nite sample distribution of bθ is unknown, but bθ is asymptotically normally distributed (cf. chapter 2). We want to test a set of linear or nonlinear constraints on the true parameters (population) θ 1,.., θ K. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

169 4. MLE and inference De nition (Null hypothesis) Consider a null hypothesis of p linear and/or nonlinear constraints H 0 : c (θ) = 0 p1 {z} p1 where c (θ) is a vectorial function de ned as: c : R K! R p θ 7! c (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

170 4. MLE and inference Notations 1 c (θ) is a p 1 vector of functions c 1 (θ),.., c p (θ): c (θ) = 0 c 1 (θ) c 2 (θ).. c p (θ) 2 In the case of p linear constraints, we have: 1 C A H 0 : c (θ) = Rθ q = 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

171 4. MLE and inference Example (Linear constraints) Consider the two linear constraints θ 1 = θ 2 + θ 3 and θ 2 + θ 4 = 1. We have p = 2 constraints such that: θ1 θ H 0 : c (θ) = 2 θ 3 0 = θ (2,1) 2 + θ The function c (θ) can be written as Rθ = (θ 1 θ 2 θ 3 θ 4 ) >, we have c (θ) = Rθ q = q. For instance if K = 4 and θ 0 θ 1 θ 2 θ 3 θ 4 1 C A 0 1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

172 4. MLE and inference Example (Nonlinear constraints) Consider the linear and nonlinear constraints θ 1 θ 2 = 0 θ 2 1 θ 3 = 0 We have p = 2 constraints such that: θ1 θ H 0 : c (θ) = 2 (2,1) θ 2 1 θ 3 = 0 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

173 4. MLE and inference Assumptions 1 The functions c 1 (θ),.., c p (θ) are di erentiable. 2 There is no redundant constraint (identi cation assumption). Formally, we have c (θ) (row) rank θ > = p 8θ 2 Θ with c (θ) θ > (p,k ) 0 = c 1 (θ) θ 1 c 2 (θ) θ 1 c 1 (θ) θ 2.. c 2 (θ) θ 2.. c 1 (θ) θ K c 2 (θ) θ K c p (θ) θ 1 c p (θ) θ 2.. c p (θ) θ K 1 C A Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

174 4. MLE and inference Consider the two-sided test H 0 : c (θ) = 0 versus H 1 : c (θ) 6= 0 We introduce three di erent asymptotic tests (the trilogy..) 1 The Likelihood Ratio (LR) test 2 The Wald test 3 The Lagrance Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

175 4. MLE and inference For each of the three tests, we will present: 1 the test-statistic 2 its asymptotic distribution under the null 3 the (asymptotic) rejection region 4 the (asymptotic) p-value Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

176 Subsection 4.1 The Likelihood Ratio (LR) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

177 4.1. The Likelihood Ratio (LR) test De nition (Likelihood Ratio (LR) test statistic) The likelihood ratio (LR) test-statistic is de ned by as to be: LR = 2 b θ H0 ; yj x b θ H1 ; yj x `N where `N (θ; yj x) denotes the (conditional) log-likelihood of the sample y, bθ H0 and bθ H1 are respectively the maximum likelihood estimator of θ under the alternative and the null hypothesis. `N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

178 4.1. The Likelihood Ratio (LR) test Comments Consider the ratio of likelihoods under H 1 (no constraint) and under H 0 (with c (θ) = 0). b θ H0 ; yj x λ = L N L N b θ H1 ; yj x 1 λ > 0 since both likelihood are positive. 2 λ < 1 since L N (H 0 ) cannot be larger than L N (H 1 ). A restricted optimum is never superior to an unrestricted one. 3 If λ is too small, then doubt is cast on the restrictions c (θ) = 0. 4 Consider the statistic LR= 2 ln (λ): if λ is "too small", then LR is large (rejection of the null)... Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

179 4.1. The Likelihood Ratio (LR) test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the LR test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): LR d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : LR (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and LR(y) is the realisation of the LR test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

180 4.1. The Likelihood Ratio (LR) test De nition (p-value of the LRT test) The p-value of the LR test is equal to: p-value = 1 G p (LR (y)) where LR(y) is the realisation of the LR test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

181 4.1. The Likelihood Ratio (LR) test Example (LRT and Poisson distribution) Suppose that X 1,X 2,,X N are i.i.d. discrete random variables, such that X i Pois (θ) with a pmf (probability mass function) de ned as: Pr (X i = x i ) = exp ( x i! θ) θx i where θ is an unknown parameter to estimate. We have a sample (realisation) of size N = 10 given by f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g. Question: use a LR test to test the null H 0 : θ = 1.8 against H 1 : θ 6= 1.8 and give a conclusion for signi cance level of 5%. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

182 4.1. The Likelihood Ratio (LR) test Solution The log-likelihood function is de ned as to be: `N (θ; x) = θn + ln (θ) N i=1 x i ln N i=1 x i! In the chapter 2, we found that the ML estimator of θ is the sample mean: bθ = 1 N N X i i=1 Given the sample f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g, the estimate of θ (under H 1, with non constraint) is bθ H1 = 2, and the corresponding log-likelihood is equal to: b θ H1 ; x = ln (0.104) `N Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

183 4.1. The Likelihood Ratio (LR) test Solution (cont d) Under the null H 0 : θ = 1.8, we don t need to estimate θ and the log-likelihood is equal to: `N (θ H0 ; x) = 1.8N + ln (1.8) The LR test-statistic is equal to: LR (y) = 2 ln N i=1 x i ln N i= = x i! = ln (0.0936) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

184 4.1. The Likelihood Ratio (LR) test Solution (cont d) LR (y) = For N = 10, p = 1 (one restriction) and α = 0.05, the critical region is: and the p-value is W = y : LR (y) > χ (1) = pvalue = 1 G 1 ( ) = where G 1 (.) is the cdf of the χ 2 (1) distribution. Conclusion: for a signi cance level of 5%, we fail to reject the null H 0 : θ = 1.8. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

185 Subsection 4.2 The Wald test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

186 4.2. The Wald test De nition (Wald test-statistic) The Wald test-statistic associated to the test of H 0 : c (θ) = 0 is de ned as to be: > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 where bθ H1 is the maximum likelihood estimator of θ under the alternative hypothesis (unconstrained model) and bv asy b θ H1 is an estimator of its asymptotic variance covariance matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

187 4.2. The Wald test Remark 0 > Wald = c b c θ H1 B b {z θ > θ H1 bv asy b c > θ H1 b {z } {z } θ > θ H1 C A {z } 1p pk K K K p 1 1 c b θ H1 {z } p1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

188 4.2. The Wald test Example (Wald test-statistic) Consider a model with K = 3 parameters θ = (θ 1 : θ 2 : θ 3 ) > with θ 1 θ 2 = 0 θ 2 1 θ 3 = 0 We have two constraints (p = 2) and: H 0 : c (θ) (2,1) θ1 θ = 2 θ 2 1 θ 3 = 0 0 Denote bθ H1 = (θ 1 : θ 2 : θ 3 ) > the ML estimator of θ under the alternative hypothesis and bv asy b θ H1 the estimator of its asymptotic variance covariance matrix. Question: write the Wald test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

189 4.2. The Wald test Solution Here we have K = 3 and p = 2 c b θ H1 = b θ 1 bθ 2 bθ 2 1 bθ 3! c b θ > θ H1 = 2bθ > Wald = c b c θ H1 b θ > θ H1 V b asy b c > θ H1 b θ > θ H1 1 c b θ H1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

190 4.2. The Wald test Remark In the case of linear constraints H 0 : Rθ q = 0 we have with H 0 : c (θ) = 0 c (θ) = Rθ q c θ > (θ) = R Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

191 4.2. The Wald test De nition (Wald test-statistic and linear constraints) Consider the test of linear constraints H 0 : c (θ) = Rθ q = 0. The Wald test-statistic is de ned as to be: > Wald = Rbθ H1 q R bv asy b θ H1 R > 1 Rbθ H1 q where bθ H1 is the maximum likelihood estimator of θ under the alternative hypothesis (unconstrained model) and bv asy b θ H1 is an estimator of its asymptotic variance covariance matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

192 4.2. The Wald test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the Wald test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): Wald d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : Wald (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and Wald(y) is the realisation of the Wald test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

193 4.2. The Wald test Proof Under some regularity conditions, we have p N b d! θ H1 θ 0 N 0, I 1 (θ 0 ) We use the delta method for the function c (.). The function c (.) is a continuous and continuously di erentiable function not involving N, then p N c b d! c θ H1 c (θ 0 ) N 0, θ > (θ 0) I 1 (θ 0 ) c θ > (θ 0) > Under the null H 0 : c (θ 0 ) = 0, we have c θ > (θ 0) I 1 (θ 0 ) c 1/2 pnc θ > (θ 0) > b d! θ H1 N (0, Ip ) where I p is the identity matrix of size p. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

194 4.2. The Wald test Proof (cont d) The Wald criteria is de ned as to be: Wald criteria > = N c b θ H1 c θ > (θ 0) I = N c c θ > (θ 0) I 1 (θ 0 ) c! 1/2 > θ > (θ 0) > 1 (θ 0 ) c 1/2 θ > (θ 0) > c b θ H1 1 (θ 0 ) c 1 θ > (θ 0) > c b θ H1 b θ H1 > c θ > (θ 0) I So, under the null H 0 : c (θ 0 ) = 0,, we have Wald criteria d! H0 χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

195 4.2. The Wald test Proof (cont d) Wald Criteria = > N c b θ H1 c θ > (θ 0) I A feasible Wald test-statistic is given by 1 (θ 0 ) c 1 θ > (θ 0) > c b θ H1 Wald = > N c b θ H1 c b θ > θ H1 b I 1 c > 1 bθ H1 b θ > θ H1 c b θ H1 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

196 4.2. The Wald test Proof (cont d) Since We have nally bv asy b θ H1 = N 1 bi 1 bθ H1 > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 and Wald d! H0 χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

197 4.2. The Wald test De nition (p-value of the Wald test) The p-value of the Wald test is equal to: p-value = 1 G p (Wald (y)) where Wald(y) is the realisation of the Wald test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

198 4.2. The Wald test De nition (z-statistic) Consider the test H 0 : θ k = a k versus H 1 : θ k 6= a k. The z-statistic corresponds to the square root of the Wald test-statistic and satis es b θ k a k Z k = r! d N (0, 1) H0 bv asy b θ k where bθ k is the ML estimator of θ k obtained under H 1 (unconstrained model). The critical region for a signi cance level of α is: n W = y : jz k (y)j > Φ 1 α o 1 2 where Φ (.) denotes the cdf of the standard normal distribution. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

199 4.2. The Wald test Computational issues The Wald test-statistic depends on the estimator of the asymptotic variance covariance matrix: > Wald = c b c θ H1 b θ > θ H1 V b asy b c > 1 θ H1 b θ > θ H1 c b θ H1 bv asy b θ H1 = N 1 1 bi bθ H1 where I b θ H1 denotes the average Fisher information matrix. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

200 4.2. The Wald test Computational issues (cont d) Three estimators are available for the average Fisher information matrix: Actual Average Fisher Matrix: bi A b θ = 1 N BHHH estimator: bi B b θ = 1 N N i=1 Hessian based estimator: bi c b θ = 1 N `i (θ; y i j x i ) θ N i=1 b θ N i=1 bi i b θ >! `i (θ; y i j x i ) θ 2`i (θ; y i j x i ) θ θ > b θ bθ Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

201 4.2. The Wald test Computational issues (cont d) 1 These estimators are asymptotically equivalent, but the corresponding estimates may be very di erent in small samples. 2 Thus, we can obtain three di erent values for the Wald statistic given the choice of the estimator for V asy b θ H1 (cf. exercises). 3 In general, the estimator A is rarely available and the estimator B (BHHH) gives erratic results. 4 Most of the software use the estimator C (Hessian based estimator). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

202 4.2. The Wald test Computational issues (cont d) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

203 Subsection 4.3 The Lagrange Multiplier (LM) test Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

204 4.3. The Lagrange Multiplier (LM) test Introduction Consider the set of constraints c (θ) = 0. Let λ be a vector of Lagrange multipliers and de ne the Lagrangian function `N (θ ; yj x) = `N (θ; yj x) + λc (θ) The solution to the constrained maximization problem is the root of `N (θ ; yj x) θ = `N (θ; yj x) θ + c (θ) > θ > λ `N (θ ; yj x) λ = c (θ) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

205 4.3. The Lagrange Multiplier (LM) test Introduction (cont d) `N (θ ; yj x) θ = `N (θ; yj x) θ + c (θ) > θ > λ 1 If the restrictions are valid, then imposing them will not lead to a signi cant di erence in the maximized value of the likelihood function. In the rst-order conditions, the meaning is that the second term in the derivative vector will be small. In particular, λ will be small. 2 We could test this directly, that is, test H 0 : λ = 0 which leads to the Lagrange multiplier test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

206 4.3. The Lagrange Multiplier (LM) test Introduction (cont d) There is an equivalent simpler formulation, however. If the restrictions c (θ) = 0 are valid, the derivatives of the log-likelihood of the unconstrained model evaluated at the restricted parameter vector will be approximately zero. `N (θ; yj x) θ = 0 b θ H0 The vector of rst derivatives of the log-likelihood is the vector of (e cient) scores. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

207 4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) For these reasons, this test is called the score test as well as the Lagrange multiplier test. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

208 4.3. The Lagrange Multiplier (LM) test Guess Let us assume that θ is scalar, i.e. K = 1, then the LM statistic is simply de ned as: 2 b θ H0 ; Y j x LM = V s N b θ H0 ; Y j x Since bi N b θ H0 = V s N b θ H0 ; Y j x, we have: s N LM = s N b θ H0 ; Y j x 2 bi N b θ H0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

209 4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) The LM test-statistic or score test associated to the test of H 0 : c (θ) = 0 is de ned as to be: LM = s N b θ H0 ; Y j x > b I 1 N b θ H0 s N b θ H0 ; Y j x where bθ H0 is the maximum likelihood estimator of θ under the null hypothesis (constrained model), s N (θ; Y j x) is the score vector of the unconstrained model and bi N b θ H0 is an estimator of the Fisher information matrix of the sample evaluated at bθ H0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

210 4.3. The Lagrange Multiplier (LM) test Remark Since: bv asy b θ H0 = bi 1 N b θ H0 there is another expression for the LM statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

211 4.3. The Lagrange Multiplier (LM) test De nition (LM or score test) The LM test-statistic or score test associated to the test of H 0 : c (θ) = 0 is de ned as to be: > LM = s N b θ H0 ; Y j x V b asy b θ H0 s N b θ H0 ; Y j x where bθ H0 is the maximum likelihood estimator of θ under the null hypothesis (constrained model), s N (θ; Y j x) is the score vector of the unconstrained model and bv asy b θ H0 is an estimator of the asymptotic variance covariance matrix of bθ H0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

212 4.3. The Lagrange Multiplier (LM) test Remark The LM test-statistic can also be de ned by: LM = λ > c θ > b θ H0 b V asy b θ H0 c θ > b θ H0 > λ where λ denotes the Lagrange Multiplier associated to the constraints c (θ) = 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

213 4.3. The Lagrange Multiplier (LM) test The LM test-statistic can be obtained from the following auxiliary procedure: Step 1: Estimate the constrained model and obtain bθ H0. Step 2: Form the gradients for each observation of the unrestricted model evaluated at bθ H0 g i b θ H0 ; y i j x i 8i = 1,..N Step 3: Run the regression of a vector of 1 on the variables g i b θ H0 ; y i j x i 8i = 1,..N, then LM = N R 2 where R 2 denotes the (unadjusted) coe cient of determination of this auxiliary regression. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

214 4.3. The Lagrange Multiplier (LM) test Computational issues 1 The LM test-statistic depends on the estimator of the asymptotic variance covariance matrix: > LM = s N b θ H0 ; Y j x V b asy b θ H0 s N b θ H0 ; Y j x bv asy b θ H0 = N 1 1 bi bθ H0 where I b θ H0 denotes the average Fisher information matrix. 2 Thus, we can obtain three di erent values for the LM statistic given the choice of the estimator for V asy b θ H0 (cf. exercises). Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

215 4.3. The Lagrange Multiplier (LM) test De nition (Asymptotic distribution and critical region) Under some regularity conditions (cf. chapter 2) and under the null H 0 : c (θ) = 0, the LM test-statistic converges to a chi-squared distribution with p degrees of freedom (the number of restrictions imposed): LM d! H0 χ 2 (p) The (asymptotic) critical region for a signi cance level of α is: W = y : LM (y) > χ 2 1 α (p) where χ 2 1 α (p) is the 1 α critical value of the chi-squared distribution with p degrees of freedom and LM(y) is the realisation of the LM test-statistic. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

216 4.3. The Lagrange Multiplier (LM) test De nition (p-value of the LM test) The p-value of the LM test is equal to: p-value = 1 G p (LM (y)) where LM(y) is the realisation of the LM test-statistic and G p (.) is the cdf of the chi-squared distribution with p degrees of freedom. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

217 Subsection 4.4 A comparison of the three tests Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

218 4.4. A comparison of the three tests Source: Pelgrin (2010), Lecture notes, Advanced Econometrics Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

219 4.4. A comparison of the three tests Summary Test Requires estimation under LRT H 0 and H 1 Wald H 1 LM H 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

220 4.4. A comparison of the three tests Computational problems If the ML maximisation problem is complex (with local extrema) and if it has no closed form solution (nonlinear models: GARCH, Markov Switching models etc.), it may be particularly di cult to get a ML estimates bθ through a numerical optimisation of the log-likelihood. If the constraints c (θ) = 0 are not valid in the data, the (numerical) convergence of the optimisation algorithm may be very problematic under the null H 0. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

221 4.4. A comparison of the three tests Asymptotic comparison The three tests have the same asymptotic distribution under the null H 0 : c (θ) = 0: LRT d! H0 χ 2 (p) Wald d! H0 LM d! H0 χ 2 (p) χ 2 (p) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

222 4.4. A comparison of the three tests Theorem (Asymptotic comparison) The three tests are asymptotically equivalent. Under some regularity conditions and under the null H 0 : c (θ) = 0, the di erences between the three test statistics converge to 0 as N tends to in nity: LRT LM p! H0 0 LRT Wald p! H0 0 LM Wald p! H0 0 Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

223 4.4. A comparison of the three tests Fact (Finite sample properties) The nite sample properties of the three tests may be di erent, especially in small samples. For small sample size, they can lead to opposite conclusion about the rejection of the null hypothesis. Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

224 4. MLE and inference Key concepts of Section 4 1 Likelihood Ratio (LR) test 2 Wald test 3 Lagrange Multiplier (LM) test 4 Computational issues 5 Comparison of the three tests (the trilogy) in nite samples Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225

225 End of Chapter 4 Christophe Hurlin (University of Orléans) Christophe Hurlin (University of Orléans) Advanced Econometrics - Master ESA November 20, / 225