Test of Hypotheses. Since the Neyman-Pearson approach involves two statistical hypotheses, one has to decide which one

Transcription

1 Test of Hypotheses Hypothesis, Test Statistic, and Rejection Region Imagine that you play a repeated Bernoulli game: you win $1 if head and lose $1 if tail. After 10 plays, you lost $2 in net (4 heads and 6 tails). Would you complain that the coin is not a fair coin? What if you lost $3? $4? $10? If you think the coin is not fair, how would you present your case? A statistical test uses a random sample from an unknown distribution to determine the validity of an assertion (or a theory) about the distribution. In parametric tests, it is known that the sample is from a known family of distribution function f(x;2). That is, the distribution function is known (e.g., Bernoulli), but its parameter(s) 2 is unknown 1. And we wish test an assertion such as 2=1/2. The assertion about the distribution function we wish to test is called a null hypothesis or a maintained hypothesis, and it is typically denoted by H 0. The distribution specified by the null hypothesis is called the null distribution. A null hypothesis may completely specify the distribution as in H 0 : 2=1/2 for a Bernoulli distribution, or may specify a set of distributions as in H 0 : 2<1/2. The former is called a simple null hypothesis, and the latter a composite null hypothesis. When a distribution involves multiple parameters, a null hypothesis may involve multiple statements such as H 0 : :=1, F 2 =2 for a normal distribution. This is called the joint null hypotheses. There are two approaches to the test of a null hypothesis. The test of significance advocated by R. A. Fisher will reject a hypothesis, but never accept one. A significance tester s objective is to disprove a null hypothesis on the basis of data. The hypothesis may be that a rise in the minimum wage rate has no effect on the entry level employment; the welfare reform of 1996 has no effect on the number of welfare recipients; the US aggregate production function has a constant returns to scale; there is no change in the monetary policy rule between before and after 1987, etc. When the test rejects the null hypothesis we say that the test is significant, or that the effect of the minimum wage is significantly different from zero for the null hypothesis described above. When the test does not reject the null hypothesis, no conclusion would be drawn, or we say that there is not enough sample information to reject the null hypothesis. The approach developed by J. Neyman and E. S. Pearson tests a null hypothesis against another statistical hypothesis, which is called an alternative hypothesis. An alternative hypothesis can be either simple or composite, and it is typically denoted by either H 1 or H a. Examples of the null and alternative hypotheses are... a simple null against a simple alternative... a simple null against a two-sided composite alternative... a one-sided composite null against a one-sided composite alternative More generally, let 1 be the parameter space, and 1 0 and 1 1 be mutually exclusive subsets of 1. Then, we can express the null and alternative hypotheses in a general form as Since the Neyman-Pearson approach involves two statistical hypotheses, one has to decide which one 1 In many application problems, the distribution function of the population is unknown. In such cases, we either assume a certain family of distribution and conduct parametric tests, or apply nonparametric tests. We will not discuss the latter in this class.

2 should be called the null hypothesis. One may call 2>2 0 the null hypothesis, and 2#2 0 the alternative hypothesis in the last example. If both statements are treated symmetrically in the construction of the test, it should not matter which one is selected as the null hypothesis. In the Neyman-Pearson approach, the two hypotheses are treated asymmetrically in their importance, and the choice of the null hypothesis makes a difference in the final conclusion of a test. Once the null and alternative hypotheses are specified in the Neyman-Pearson test, we need to choose 2 (i) a test statistic (ii) a decision rule that tells us to either reject or accept H 0. The choice of the test statistic determines how we use the sample information for the test. This choice can be arbitrary, but a function of the estimator of parameter under consideration is commonly used as a test statistic. For example, we may use the sample mean as a test statistic in testing the Bernoulli parameter 2, since is an unbiased estimator of 2. 1 Given the test statistic, the decision rule specifies when to reject and when to accept H 0. For example, we may decide to reject H 0 if the test statistic for some constant c, and accept H 0 if. These decision rules partition the potential sample observations {x 1,þ,x n } into two subsets. The set of sample observations for which H 0 will be rejected is called a rejection region, or a critical region, of the test. The rejection region usually takes the form of a one-sided or a two-sided rejection region: One-sided rejection region: Two-sided rejection region:, or, where the boundary values c, c R and c u are called the critical values. Tests based on these types of critical regions are called the one-sided test and the two-sided test, respectively 2. It should be emphasized that an acceptance of a hypothesis does not mean that the hypothesis is proved in any rigorous sense. It simply means that the data is compatible with the hypothesis, and one is willing to act as if the hypothesis were true. A rejection of the null hypothesis implies that the data and the hypothesis are not compatible. Example. Let X 1,...,X n be a random sample from a Bernoulli distribution wish to test, x=0 or 1. We (A) H 0 : 2=1/2, H 1 : 2=1/4 1 Alternatively, we may derive a test statistic from one of the three classical methods: the Lkielihhod Ratio Test, the Lagrange Multiplier Test, and the Wald (type) Test. These tests will be discussed below in detail. Properties of a test may vary with the choice of the test statistic, rendering some tests better than other tests. 2 Some authors use the term one-tailed and two-tailed test. We will use these terms interchangeably.

3 3 (B) H 0 : 2=1/2, H 1 : 2 1/2 Test (A) is a test of a simple null against a simple alternative hypothesis. Since the sample mean is an unbiased estimator of 2, it seems reasonable to reject H 0 if is closer to the alternative value 1/4 than to the null value 1/2. That is, we may choose as a test statistic, and the one-sided rejection region } where the midpoint between the two hypothesized values are chosen as the critical value. We reject H 0 if is less than or equal to 3/8, and accept H 0 if is greater than 3/8. As noted earlier, acceptance of the null hypothesis does not mean that we have proved 2=1/2. Consider a sample of size 2 (i.e., toss a coin twice). The sample mean is greater than 3/8 when the outcomes of the two tosses are either {HT}, or {TH}, or {HH}. Realization of one of these outcomes clearly does not prove 2=1/2, because all three outcomes are possible outcomes for all 2,(0,1). Test (B) is a test of a simple null against a composite alternative hypothesis. It is intuitively reasonable to reject H 0 if the sample mean is sufficiently far from the null value 1/2 in either direction. This suggests a two-sided rejection region of the form and we reject H 0 if either #1/8 or $7/8. Though our choices of the test statistic and the rejection region in these examples seem to be reasonable, there is no a priori reason why they are better than some other choices. For example, is the critical value 3/8 in Test (A) better than 5/16 or 7/16? What if we use median as the test statistic? To address these issues, we need to consider the consequences of alternative choices. Properties of Test Correct decisions in a test of hypothesis is to reject the null hypothesis when it is wrong and accept it when it is correct. However, one bounds to make one of the two types of errors in a typical test: Type I error - reject a true null hypothesis Type II error - not reject (or accept) a false null hypothesis The probability of rejecting a true null hypothesis (Type I error) is called the size of the test, or the level of significance (significance level), and typically denoted by ". The probability of rejecting a false null hypothesis (correct decision) is called the power of the test, and it is usually denoted by B. The probability of accepting a false null hypothesis (Type II error) is usually denoted by $=1-B. Size of test (significance level): " = P(reject a true H 0 ) = P(type I error)

4 4 Power of test: B = P(reject a false H 0 ) = 1 - P(accept a false H 0 ) = 1 - P(type II error) = 1 - $ When the null hypothesis is composite, the probability of rejecting a true H 0 varies in general with the value of 2 in the null parameter set 1 0. The significance level of a test is then defined as the maximum (or supremum) of the probability of rejecting a true H 0. Similarly, when the alternative hypothesis is composite, the probability of type II error and the power of the test also vary with the value of 2 in the alternative parameter set 1 1. To make this functional relationship explicit, the probability of Type II error is denoted by $(2), and the power of the test by B(2), for The function B(2), 201 1, is called the power function of the test 1. Significance Level : Power Function: When we construct a test, it is desirable to minimize the probabilities of the two types of errors. An ideal test 2 is the one that has "=0 and $(2)=0. An ideal test seldom exists except in trivial cases. In general, there is a trade-off between these two probabilities when the sample size is fixed. If we try to make the significance level small, the probability of type II error increases, and vice versa. The Neyman -Pearson approach resolves this conflict by treating the null and alternative hypotheses asymmetrically. The significance level " is first selected, and then a rejection region is chosen that has the highest power among all rejection regions of same level of significance. A test of a simple null against a simple alternative hypothesis is called the most powerful test of significance level " if it has more power than any other test of same significance level ". When the alternative hypothesis is composite, a test of significance level " is called the uniformly most powerful test of significance level " if it has more power than any other test of same significance level " for all The term uniformly refers to all alternative parameter values. A test is called an unbiased test if the probability of rejecting wrong H 0 is always higher than the probability of rejecting correct H 0, that is, the power of the test is always higher than its significance level. These concepts are illustrated in the following examples. Example. Consider the previous example of testing the parameter 2. Suppose we have a sample of size 2. The rejection regions are for test A and for Test B. Following table summarizes the sample points and their probabilities. 1 Some authors define the power function as B(2)=P(reject H 0 ) for all 201. A graph of the probability of accepting H 0, 1-B(2), is called the OC-curve (Operating Characteristic curve). 2 When the power function is defined as B(2) for all 201, an ideal power function is defined as a function B(2)=0 for all and B(2)=1 for all

5 5 Sample Point P(T i ) P(T i H 0 ) P(T i H 1A ) T 1 ={0,0} 0 (1-2) 2 1/4 9/16 T 2 ={0,1} 1/2 2(1-2) 1/4 3/16 T 3 ={1,0} 1/2 2(1-2) 1/4 3/16 T 4 ={1,1} /4 1/16 For Test A, the rejection region is. Hence, For Test B, the rejection region is. The significance level of this test is The probability of Type II error (accepting a false H 0 ) is, and the power function of this test is, where 2 is any value admissible under H 1 : 2 ½. This is illustrated below. Now consider a composite hypothesis Test C: and suppose we use the rejection region The probability of type I error then becomes The supremum of this probability is 1/2, and hence the significance level of this test is "=1/2. The power function is In these examples we determined the rejection region and then calculated the significance level of the test. The Neyman -Pearson approach selects the significance level of the test first, and then determines the

6 rejection region. When the distribution is discrete, we may not be able to achieve the given significance level. Therefore, we will consider a continuous distribution to determine the rejection region. 6 Example. One-sided Test. Let variance is known. We wish to test be a random sample from a normal distribution N(:, F 2 ), where the where : 0 and : 1 are known constants and. We will use the unbiased estimator of : as a test statistic, and choose a one-sided rejection region. That is, we reject H 0 if the sample mean is far away from the null value : 0 toward the alternative value : 1. The critical value c will be chosen such that where " is the significance level. The conventional choice of " is either 0.01, or 0.05, or 0.1. We have shown earlier that. Hence, is distributed as under the null hypothesis, and where M is the cdf of a standard normal random variable, and is distributed as a standard normal under H 0. The standard normal table gives that c for "=0.05, and c for "=0.1, and the critical value is computed by. We reject H 0 if, and accept H 0 otherwise. Note that the critical value c is a decreasing function of the significance level " and sample size n, and an increasing function of variance F 2. For a numerical example, consider : 0 =0, F 2 =4 and n=25. The null hypothesis is rejected at the 5% significance level if the observed sample mean is greater than or equal to 0.656, and rejected at the 10% significance level if it is greater than or equal to If the observed sample mean were 0.59, the null hypothesis is rejected at 10% significance level, but accepted at 5% significance level. At what level of significance would the null hypothesis have been rejected if the observed sample mean were 0.59? The minimum significance level at which H 0 would have been rejected, given the observed value of the test statistic, is called the p-value of the test. The p-value of the test under consideration can be found by computing the probability

7 7 where is the observed sample mean. For : 0 =0, F 2 =4 and n=25, the p-value of =0.59 is equal to Thus, the null hypothesis will be rejected at 7% significance level if the observed sample mean were When the test result is reported as rejection of H 0 at a given significance level, it does not indicate how strongly H 0 is rejected, because H 0 is rejected regardless of whether the sample mean is slightly or substantially greater than : 0 +c. The p-value captures the variations in the strength of the test result, and hence provide more information about the test result than just saying that H 0 is rejected or not. Now we find the probability of type II error and the power of the test. The probability of type II error (accepting a false H 0 ) is given by Since is distributed as N(: 1, F 2 /n) under the alternative hypothesis, we can rewrite $ as where Since Z 1 is a standard normal under H 1, one can easily compute the probability of type II error, the cumulative cdf M(c 1 ), by using a standard computer software like GAUSS. This probability clearly depends on the alternative value : 1. The power of the test is then computed by 1-$. For example, let : 0 =0, : 1 =1, F 2 =4 and n=25. For the 5% significance level, c 0 =1.64 and c 1 = Therefore,, The pdf s of under the null and alternative hypotheses are depicted in the figure below. The size of the test " is the area under the pdf for, the probability $ of type II error is the area under the pdf for, and the power B of the test is the area under the pdf for.

8 8 It is obvious from the figure that, if we reduce the significance level ", then the critical value c increases and consequently, the probability of type II error $ increases and the power of the test decreases. This illustrates the trade-off between the two types of errors. It is also clear that the power of this test is always greater than the significance level for any choice of ". Therefore, this test is an unbiased test. This test is also the most powerful test, i.e., any other rejection region of significance level " will have a lower power than the current test. This result is due to the Neyman-Pearson Lemma, which is presented below. Intuitively, one can convince oneself about this result by trying to move some of the probability mass " to somewhere else, and then checking the power of the new test. Example. Two-sided test. Let X 1,þ,X n be a random sample from a normal distribution N(:, F 2 ), where the variance is known. We wish to test This is a test of a simple null against a composite alternative hypothesis. We will use the unbiased estimator of : as a test statistic, and choose a two-sided rejection region. That is, we reject H 0 if the sample mean is far away from the null value : 0 in either direction. The critical value c will be chosen such that for a given size of the test ". From the discussion in the previous example, we can write Since Z 0 is distributed as a standard normal under H 0, we can find the value of c 0 from the standard normal table: c for "=0.05, and c for "=0.1. The critical value of the rejection region is then computed from c=fc 0 /. The rejection region is illustrated in the figure below as the areas under the pdf for the range of and, where each tail has probability "/2. The probability of type II error is the area under the pdf for the interval, and the power B of the test is the area under the pdf

9 9 for the range of and. Since the alternative hypothesis is composite in this case, the probability of type II error and the power of the test vary with the alternative value : 1. For any given : 1, the probability of type II error is found from is a standard normal under H 1 and hence one can easily compute $(: 1 ) for each : 1.The power of the test is then computed by 1-$(: 1 ) for each : 1. Though we will not prove it here, this test is the UMPU (uniformly most powerful unbiased) test 1. Three examples of the power function are illustrated below where : 0 =0 and F 2 =4. These examples show that the power of the test decreases uniformly as " becomes smaller and it increases uniformly with the size of the sample. 1 See chapter 13 in Roussas s A First Course in Mathematical Statistics.

10 10 Example. Two-sided test with unknown F. The critical values in the previous examples involve the standard deviation F. When F is unknown, we can not calculate the critical values. As in the construction of the confidence interval, we need to replace F or F 2 with an estimate. where S 2 is the sample variance. Since Z 0 is distributed as a central t(n-1) under H 0, we can find the value of c 0 from the table of central t-distribution. For example, "=5% "=10% n= n= The critical value of the rejection region is then computed from c=sc 0 /. Therefore, we reject H 0 if the sample mean deviates in either direction from the null value : 0 by more than 2.262S/ for n=10 at the 5% significance level. This is a UMPU test. Example. Test of Difference in Means with Known Variances. Consider two independent normal populations and, where variances are known. Let where a and b are known constants. We wish to test where * 0 is a known constant. If a=1, b=-1 and * 0 =0, we are testing the equality of the two means. Let and be the sample means of sample size n and m, respectively, from the two populations. An unbiased estimator of * is. We will use this estimator as a test statistic, and choose a two-sided rejection region. The critical value c will be chosen such that. Since and are independent, is distributed as a normal with mean * and variance. From the discussion in the previous example, we can write Since Z 0 is distributed as a standard normal under H 0, we can find the value of c 0 from the standard normal table as in the previous example. This is a UMPU test Example. Difference in Means with Unknown Common Variance. Suppose in the previous example that the two populations have the same variance,, but F 2 is unknown. We have shown earlier in the construction of the confidence interval that is the pooled estimator of

11 11 F 2, and has a central P 2 (n+m-2) distribution which is independent of. Therefore, The critical value of the rejection region is then found from where T 0 has a t(n+m-2) distribution under H 0. This is a UMPU test Methods of Construction of Test Tests in these examples are constructed by selecting rejection regions that are intuitively reasonable, and they turn out to be uniformly most powerful unbiased tests. We now consider a systematic method of finding a most powerful test of a simple null against a simple alternative hypothesis. A test chooses a subset of the sample space as a rejection region. Suppose that each potential observation of the sample has a positive probability under the null as well as under the alternative hypothesis. Every time a potential observation is put into the rejection region, it increases the probability of type I error as well as the power. We may interpret the probability of type I error as a cost and the power as a benefit of selecting a potential observation as an element of the rejection region. Since we wish to make the probability of type I error small and power of the test large, we need to select the sets of potential observations which have the lowest cost/benefit ratio. This idea of constructing a most powerful rejection region by selecting observations of low cost/benefit ratio is due to Neyman and Pearson as stated in the following lemma. Neyman-Pearson Lemma. Let X 1,þ,X n be a random sample from a population f(x;2). The most powerful test of size " of a simple null hypothesis H 0 :2=2 0 versus a simple alternative hypothesis H 1 :2=2 1 is given by the rejection region R where is the likelihood function under H i, and k is a positive constant that satisfies the condition for a given significance level ". The Neyman-Pearson lemma measures the cost/benefit ratio by the likelihood ratio of the null to the alternative hypothesis. Recall that the likelihood function is the joint pdf of random sample. Thus, we are comparing the probability of an observation (x 1, x 2, þ, x n ) under the null to that under the alternative hypothesis. How many potential sample points should be selected? That is, what value of k should be chosen? We need to pick just enough number of them to meet the requirement that the joint probability of selected

12 observations under the null hypothesis is equal to the given significance level. This will determine the value of k. There may not exist a constant k that gives the size of test exactly equal to ", particularly in the case of discrete distribution. We consider an example of a continuous distribution below. 12 Example. Let X 1,þ,X n be a random sample from a normal distribution N(:, F 2 ), where the variance is known. We wish to test H 0 ::=: 0 versus H 1 ::=: 1 at the level of significance ". The ratio of the normal likelihood functions is The rejection region is the subset of the sample space that satisfies 8#k, which can be rewritten as where Thus, when : 0 <: 1, we only need find a value of k * such that, which can also be expressed as, where c=k * -: 0. This example shows that the test we used in an earlier example is the most powerful test of H 0 against H 1. The rejection region in this example does not depend on the parameter value under H 1, and thus provides for the most powerful test of against any alternative value : 1 as long as : 1 >: 0. In other words, the rejection region gives the most powerful test of a simple null hypothesis against all values of : in a composite alternative hypothesis. Such a test is called a uniformly most powerful (UMP) test, where the term uniformly refers to all alternative parameter values. Therefore, the one-sided UMP test of against is the rejection region and the one-sided UMP test of the same null hypothesis against is the rejection region. This implies that no single rejection region for the two-sided test of the same simple null hypothesis against can dominate other rejection regions in power for all parameter values under the alternative hypothesis. That is, a UMP test does not exist for such a two-sided test. When the null hypothesis is a composite hypothesis,, the significance level is defined by the supremum of the probability of Type I error over the parameter set under H 0. For a given rejection region, it is easy to see that the probability of Type I error increases as : increases, because increases

13 for any given as the mean value under the null increases 1. Hence, the significance level of this test is determined at. Given this significance level, the UMP test shown above for testing against alternative hypothesis is also the UMP test of against the alternative hypothesis. 13 Construction of Classical Tests of Nested Hypotheses One of the major drawback of the Neyman-Pearson Lemma is that it cannot be applied if there are other parameters in the likelihood function whose values are not specified by the hypotheses. When the likelihood function involves some unknown parameters, the ratio of likelihood cannot be computed and the Lemma becomes useless. Furthermore, the test we encounter most often in practice is a two-sided test for which the UMP test does not exist in general, though we can find the UMP test among all unbiased test (UMPU). There are three classical methods of constructing a test: Likelihood Ratio (LR) test, Lagrange Multiplier (LM) test, and Wald test. All three methods are widely used in econometrics, and they produce identical test in many cases. Let be the likelihood function where 2 is a vector of parameters in the unrestricted parameter space. The unknown parameters can be estimated by using the MLE method. The null hypothesis imposes some restrictions on the parameters so that the parameter space under the null hypothesis is a nested subset 1 0 of 1. When the null hypothesis does not specify specific values for all parameters, we can estimate the remaining parameters by the MLE method. The test statistics and rejection regions of the three methods are based on the restricted and unrestricted MLE. The LR test requires both restricted and unrestricted estimators. The LM test requires only the restricted estimators while the Wald test requires only the unrestricted estimators. In general, the finite sample distribution of these test statistics are either unknown or difficult to derive. Hence, we use their asymptotic distributions, and all three test statistics have the same asymptotic null distributions. The unrestricted estimators of the parameters are the usual MLE that are the solutions to the first order conditions where denotes the unrestricted estimator. Substituting into the likelihood function, we derive the value of likelihood function or. The restricted estimators subject a set of restrictions, where r is a known constants, are derived by maximizing the Lagrange equation where 8 is a vector of Lagrange multipliers. Taking the derivatives with respect to 2 and 8, we have the first order conditions 1 Move the pdf curve to the right as : 0 increases. For any given critical value, it is easy to see that the area of right tail of the null pdf increases.

14 14 where and denote the restricted parameter estimator and the estimator of the Lagrange multipliers. The value of the likelihood function at the restricted estimator is denoted by or. Likelihood Ratio Test The idea of the likelihood ratio test is that, if the restrictions under the null are true, then and must be close each other. In fact, in a certainty case, they must be identical. In the case of random events, however, the restricted and unrestricted estimators will be different with probability one even if the restriction is correct. For example, in the case of tossing a truly fair coin, the outcome may be {H,H,H,T}, which will yield the unrestricted MLE of 2 higher than 0.5. On the average, however, the two estimators should yield the likelihood values quite close each other if the restrictions are correct. The likelihood ratio test is based on the comparison of the two likelihood values: reject the null hypothesis if the difference between them is sufficiently large. The rejection region is thus constructed by or where k is a constant. Note that the unrestricted likelihood will always greater than the restricted likelihood. Therefore, the ratio is greater than one, and the log difference will be positive. The choice of k depends on the level of significance and the distribution function of the ratio of the likelihood values or the distribution function of the difference in log-likelihood. The finite sample distribution function is in general either unknown or difficult to derive. In such a case, we use the asymptotic distribution function where J is the number of restrictions under the null hypothesis. In many cases, however, we can find a finite sample distribution of a transformation of the likelihood ratio and can construct a rejection region that is equivalent to a likelihood ratio test. The following example illustrates such a case. Example. Let X 1,þ,X n be a random sample from a normal distribution N(:, F 2 ), where both parameters are unknown. We wish to test

15 15 Th likelihood and the log-likelihood functions are The unconstrained MLE are derived from the gradients The constrained MLE subject to :=: 0 are Therefore, the likelihood values are The rejection region for the likelihood ratio test is thus The rejection region can be rewritten as which leads to Note that the statistic has a central t distribution with (n-1) degrees of freedom under the null hypothesis. Therefore, we can find the critical value for a given level of significance from

16 16 by using the distribution. This is the test we examined before. Exercise: Find the LR test for against. Lagrange Multiplier Test The Lagrange multiplier in the constrained optimization represents the shadow price of the constraint. In the case of certainty, the value of Lagrange multiplier will be zero if the constraints are not binding. In the context of testing hypothesis, non-binding constraints imply that the parameter space specified by the null hypothesis is acceptable. Based on this, the LM test reject the null hypothesis if the estimator is sufficiently different from zero. The finite sample distribution of is in general unknown or difficult to derive. Therefore, we often use its asymptotic distribution to construct the rejection region. Here, we will consider only the previous example. The Lagrange equation for the example is where 8 is the Lagrange multiplier for the restriction :=: 0. The derivative of with respect to parameters are Solutions of these first order conditions for :, F 2 and 8 are the constrained estimators:,, Note that the right hand side of is, the first derivative of the likelihood function evaluated at the constrained estimators of : and F 2. Thus, testing on is equivalent to testing on the first derivative of the log-likelihood function, which is called the efficient scores. When the LM test is formed based on the efficient scores, the test is called the score test. Since we are testing whether is close to zero or not, we may construct a rejection region on the basis of some constant times. That is, is equivalent to for some constant value a. Thus, consider Testing a sufficient deviation of from zero is equivalent to testing a sufficient deviation of the right hand

17 side standard normal statistic. If F is known, we may use a two sided normal test, rejecting the null hypothesis if the right hand side statistic is at the tail outside of the critical values. Since F is unknown, we need to replace it with its estimate. Consider then a statistic 17 where is the sample variance, which is based on the unconstrained estimator. This gives a two-sided t-test. For the LM test, we usually use the constrained estimator. However, the distribution of is complicated. Hence, the LM test is usually based on the asymptotic variable Wald Test This test uses only the unrestricted estimators. If the restrictions imposed by the null hypothesis are correct, the unrestricted (consistent) estimators must satisfy the restrictions very closely. If the unrestricted estimators are sufficiently different from the restrictions, then we reject the null hypothesis. Since the unrestricted MLE of : is the sample mean, the Wald test asks whether is close to zero or not. Thus, the rejection region of the Wald test takes a form of The choice of k depends on the distribution of, which is Since this distribution involves the unknown parameter F 2, we need to replace it with its unrestricted estimator. We have seen before that this procedure leads to where is the sample variance. Hence, we can use the two-sided t-test.