1 Another method of estimation: least squares

Size: px
Start display at page:

Download "1 Another method of estimation: least squares"

Transcription

1 1 Another method of estimation: least squares erm: -estim.tex, Dec8, 009: 6 p.m. (draft - typos/writos likely exist) Corrections, comments, suggestions welcome. 1.1 Least squares in general Assume Y i is some rv with nite mean yi might not, know the form of f Yi (y : ). and variance y, where we might, or If one has a sample of n observations from this population, the least-squares estimator(s) of are those,, that minimize 1 (yi E[y i : x i ; ]) where the x i is a vector of observed explanatory variables, not random variables ( xed in repeated samples). Finding the least-squares estimate of requires that we specify the form of E[y i : x i ; ] but does not require that we specify f Yi (y i ; x i ; ). Note that maximum likelihood estimation typically requires that we specify f Yi (y i ; x i ; ), which implies E[y i : x i ; ]. For example, consider the following common additive speci cation for y i y i g(x i : ) + i i 1; ; :::; n and is a rv with zero mean (E[] 0) and nite variance, y. 3 We have a data set that consists of n fy i ; x i g pairs. Since E[y i : x i ] g(x i : ) the least-squares estimator(s) of are those that minimize (Yi g(x i : )) SSR where SSR denotes the sum of squared residua. Some books call it RSS (for example, Gujarati, page 171) 1 Note that I did not say random sample. While a random sample would be nice, leastsquares estimation is well de ned even if the sample is not random. That said, the least-squares estimators might lack desirable properties if the sample is not random. For a given x i, all the randomness in y i is invoked by the randomness in 3 Note that here I am not being completely general. I am assuming the random component is additive, which is not required for l.s. estimation. 1

2 Things to note about least-squares estimators if one is willing to assume y i g(x i : ) + i i 1; ; :::; n where i is a rv with zero mean (E[] 0) and nite variance, y: One does not need to assume a speci c distribution for (normal or otherwise), but one needs to put the above few restrictions on. g(x i : ) does not have to be linear in the, but that is the speci cation that you are most accustomed to. Some of the properties of the estimators of the will depend on what one assumes about the disribution of (normal or otherwise) and/or whether one assumes the Y i in the sample are independent of one another An aside: Note that while we are not accustomed to thinking this way, all that one needs to do least squares is to assume the rv of interest Y, has a density function such that E[Y ] exists. For example, one could assume Y has the Poisson distribution, f Y (y) e y y! for y 0; 1; ; 3; ::::, and use least squares to estimate, the expected value of Y (and ao its variance). 4 The least-squares estimator of,, is that that minimizes (Yi ). 5 A fun, and instructive exercise would be to nd the least squares estimator(s) for a rv Y assuming a few di erent forms for f Y (y : ). For example, could one proceed with least-square assuming Y has a Bernoulli distribution? Try it and see what happens. 1. Revert to the standard assumption that y i g(x i : ) + i, but now be more restrictive: assume linearity and x i a scalar g(x i : ) + x i In which case y i + x i + i i 1; ; :::; n 4 Note that here the random term is not additive. While assuming an additive term (y i g(x i : ) + i i 1; ; :::; n) is typical in least-squares, it, as I noted above, it is not necessary. 5 We know from earlier, that the maximium likelihood estimator of, ml, is the sample average. Is the least-squares estimator of ao the sample average?

3 where E[] 0 and has nite variance. This model is called the linear regression model (MGB 485, 486). It has three parameters:, and Y. Contrast the linear regression model with the classical linear regression model, which adds the assumption N(0; ). The least-squares estimates of and are those estimates, and, that minimize (yi E(y : x i )) (yi ( + x i )) 1..1 Let s nd these estimates. Minimize (let me know if you nd an typos in the following derivations) SSR (yi ( + x i )) wrt and. Since, we have put no restrictions on the ranges of and, we are looking for an interior solution in terms of these (y i x i )( 1) (y i x i ) X n y i n [ny n nx] # x i Set this equal to zero, and solve for to obtain y x (y i x i )( x i ) n # X y i x i x i x i 3

4 Set this equal to zero and solve for 6 which Implies 0 y i x i x i x i y i x i x i y i x i nx y ix i nx x i x i x i Substitute in y x for to obtain which implies y ix i nx(y x) x i y ix i nxy + nx P n x i y ix i nx x i x i nxy y ix i + x n x i x i nxy Note the following rearrangement of the lhs, nx x i nx x i x i x i nx x i x i nx x i 6 ) ) ) ) (y i x i )( x i ) 0 (y i x i )(x i ) 0 (y i x i )(x i ) 0 (y i y i )(x i ) 0 (^ i )(x i ) 0 One could check that one s least squares estimates imply this. It is a good check on your math. 4

5 so, replacing nx x i with [ x i nx ] x i, one obtains. x i nx x i y ix i x i nxy multiplying through one gets n # X x i nx y i x i nxy which implies that y ix i nxy P n y ix i nxy P x i nx n (x i x) This is the least-squares estimate for assuming g(x i : ) + x i. By substitution, the least-squares estimate for,, is y x Note that, in this case, and ml ml where ml and ml are the maximum likelihood estimates assuming the classical linear regression model. That is, if one assumes a classic linear-regression model, the ml estimators exist and are equal to the estimators, but if one assumes only the linear regression model (don t add the assumption that N(0; )), the estimators exist, but not the ml estimators. There are a number of di erent ways to write, they are all equal. 5

6 y ix i nxy x i nx y ix i nxy (x i x) ~x iy i ~x i ~x i~y i ~x i where ~x i x i x and ~y i y i y. One uses di erent characterizations in di erent situations - depending on what one wants to demonstrate. 1.. There is no least-squares estimate of y Note that since (y i x i )) is not a function of y, there is not a least-squares estimator for y. That is, what one minimizes to obtain the estimates is not a function of y. However, given and, one can estimate y with ^ y ^ (y i x i )) n The intuition for dividing by n the calculation of and. is that one loses two degrees of freedom in ^ is not a least-squares estimator, but is based on the least-squares estimators of and. It is possible to show that E[^ ]. See, for example, Gujarati Basic Econometrics, the appendix to chapter Remember Least-squares estimators exist even if g(x i : ) 6 + x i That is, g(x i : ) can be nonlinear in. For example, one could assume g(x i : ) e xi 6

7 so which is highly nonlinear x ie xi In which case, y i e xi + i i 1; ; :::; n where E[] 0 and has nite variance. and the least-squares estimates of is that estimate,, that minimizes SSR (yi e xi ) This is an example of nonlinear least squares Some properties of least-squares estimators of the form y i + x i + i where E[] 0 and has nite variance. i 1; ; :::; n Assume that the the Y i in the sample are independent of one another (we have a random sample) From above, and assuming the above linear form ~x iy i ~x i ~x i k y i where k ~x i w i y i where w i ~xi k. In words, is a linear combination (weighted sum) of the n random variables, y 1 ; y ; :::; y n, where the weights are a function of the x 0 s. We call estimators with this property linear estimators. 8. Note that determining it was a linear estimator did not require that f() have a particular form. 7 In contrast, note that if one assumed y i x i + i it would still be linear least-squares because the function is linear in. 8 Looking ahead, this is part of the famous, Gauss-Markov theorem. 7

8 Given that w iy i, and given the x i the y i are independent E[ ] E[ w i y i ] w i E[y i ] since the w i are constants: they vary with x but the x are assumed xed in repeated samples. Since E[y i ] + x i Since w i ~xi k k since ~x i (x i x) 0 Because ~x i x i Because ~x i 0 And because k ~x i w i ( + x i ) w i + ~x i + k k x ) x i ~x i + x w i x i ~x i x i ~x i x i ~x i (~x i + x) k n # X ~x i + x ~x i k k ~x i ~x i ~x i That is E[ ] 8

9 In words, is an unbiased estimator of. Note that this proof did not require that we assume a speci c distribution for. We need only the assumptions of the linear regression model, and a random sample (independent y i ). Note that at this point we have demonstrated that is a linear unbiased estimator, and this result does not depend on a normality assumption. It is ao possible to show that E[ ] I leave that as an exercise for you. In summary, the least-squares estimators of the parameters are linear and unbiased estimators. It is possible to show that E[^ ], but remember that ^ not a least-squares estimate. (yi x i)) n is 9

10 So, assuming the linear regression model and a random sample, and are linear estimators and unbiased estimators. This is good. It is possible to further show that in the class of linear unbiased estimators, the least-squares estimators have minimum variance. This earns them the adjective best. So, assuming the linear regression model and a random sample, and are BLUE (best linear unbiased estimators). This is the Gauss-Markov theorem. If one assumes E[] N(0; ) the estimators gain more desirable properties because they are ao the ml estimators. 10

11 1.4.1 One can use and to predict values of y j conditional on x j y j + x j y j is a random variable that, for xed x s, will vary from sample to sample. Since and are both unbiased estimates, y j is an unbiased estimate of y j ; that is E[y j ] y j. Think about the sampling distribution of y j, which is conditioned on x j 11

12 1.5 The variances of the least-squares estimators The variance of The least-squares estimate of is a statistic and will vary from sample to sample, so has a sampling distribution, f (v). An issue at hand is determining the variance of this sampling distribution. 9 An important issue is whether we proceed assuming a knowledge of, or only knowledge of its estimate, b. We will start assuming knowledge of, and afterwards discuss how the variance of di ers when it is expressed as a function of b rather than. Knowing is atypical, but easier, so we start there. To emphasize that we are conditioning on, in the shortrun, denote f (v) more speci cally as f (v ) and write var( ) ( ). 10 Above we showed that is a linear estimator. That is, it can be written w i y i where the w i can be treated as constants. We ao know that var(ax) a var(x) if a is a constant. Combining these two pieces of information, along with knowledge of : var( ) wi y wi Recollect that y i + x i + i where E[] 0 and has a nite variance so y. 9 More generally, we would like to know the form of f (v). 10 In contrast to f (v b ) and var( b ) (b ) 1

13 Proceeding, var( ) wi ~x i k ~x i k ~x i ( ~x i ) ~x i (x i x) since k ~x i, and the standard error of is se of ( ) [var( )] :5 Notice that var( ) decreases as (x i x) increases What did we assume to derive var( ) ( ) P n? We ~x i assumed that y i + x i + i where E[] 0 and has a nite variance, and the Y i are independent. We did not need to assume that has a speci c distribution, such as the normal. It is ao possible to derive the var( ) as a function of P n var( ) ( ) ( n )( x i ) ~x i Note again that we cannot calculate var( ) or var( ) unless we assume a speci c value for Note that one can ao calculate cov ;. It is not 0 because both and are a function of. cov ; E [( E[ ]) ( E[ ])] E [( ) ( )] 13

14 Note that var( ) ( )) ~x i is a function of the x s in the sample, but not the y s. Therefore, if one makes the typical assumption that the x s are xed in repeated samples, ( ) is not a random variable. By the same argument, neither is ( ). This is because we are assuming knowledge of. This is an important point. ( ) and ( ) are not statistics and not things that are estimated; they are calculated given knowledge of and knowledge of the x leve in the data. Said another way, while the least-squares estimates of and will vary from sample to sample, ( ) and ( ) do not vary from sample to sample (assuming the x s are xed in repeated samples). Soon, we we will consider the problem of estimating var( b ) and var( b ). But rst, 1.5. The variance of y j as a function of One can ao show that (for example, Gujarati, Essentia page 185) # var y j yj ( ) 1 n + (x j x) ~x j Note that this is a function of and the x s but not the y s, so not something that is estimated. It is not a random variable. But, y x and y x, so cov ; E [y x (y x)) ( )] E [( x + x) ( )] xe h( ) i x (x i x) x ~x i The covariance decreases as the variation in the x s increases. 14

15 1.6 What is implied if one adds to the above the assumption that ~N(o; ). If y i + x i + i where ~N(o; ), then y i ~N( + x i ; ). From earlier we know that w iy i, so is a linear combination of normally distributed random variables, so is normally distributed. Speci cally ~N(; ) ~x i By the same logic ~N(; ( n )( x i )) ~x i When is known, neither or has a t distribution, both are normally distributed. I say this here because some people incorrectly believe that and always have a t distribution If ~N(; ) then ( ~x i ):5 ~N(0; 1) 15

16 So, if we assumed a value for, we could calculate (not estimate it) and then calculate a con dence interval for and test the null hypothesis that takes some speci c value such as zero. For example, since ~N (0; 1) Pr 1:96 1:96 :95 ) Pr 1:96 1:96 :95 ) Pr 1:96 1:96 :95 ) Pr 1:96 + 1:96 :95 1:96 + 1:96 is the 95% con dence interval for based on, and the assumption that is normally distributed. How do we interpret this interval? Note that this interval depends on which is a random variable, so the con dence interval is a random variable. 95% 1 13 of these interva will contain. Assuming that ~N(o; ), it follows that y j is ao normally distributed (it is a linear function of two normally distributed random variables ( and ). Speci cally, #! y j ~N + x j ; 1 n + (x j x) ~x j So one can ao get a con dence interval for y i 1 Note that one cannot say that there is a 95% chance that the true is between 1:96 ) and ( + 1:96. Further note that since is not a random variable if the x s are xed in repeated sample, the position of this con dence interval is a random variable, but not its length. 13 Note that none of the above has anything to do with the t distribution. 16

17 1.7 However, we don t typically assume a value for but estimate it with ^ Continue, for now, to assume that ~N(o; ), so assume the CLR model, but that we do not know, so have to estimate it ^ (y i x i ) (n ) and note the important distinction between ^ and, the rst is a rv, the second is a constant. The rst thing to note, as we demonstrate below, is even though ~N(o; ) where N (0; 1) ^ ^ ^ ~x i Toooooo bad Note that ~N(; ) because we are assuming ~N(o; ) 17

18 Since it is not normal, what distribution does ^ have? Let s try and demonstrate that it has a t distribution. The following is a bit di cult - think of it as walking backwards from the end of the trail back to your car, forgetting where you started. What I am doing is deriving the distribution of ^. Remember that that ~N (0; 1). 18

19 Now de ne another random variable, G, remember we are going backwards, such that G (n )^ note that I have de ned a function that is a linear function of the ratio ^ P (n ) (Y x i) (n ) the reason for (n ) above was so it would cancel here P (Yi x i ) P (Yi E[y j jx i ]) P (Yi E[Y i jx i ]) P (Yi E[Y i jx i ]) y (Yi E[Y i jx i ]) y because y j jx i is an unbiased estimate of E[Y i jx i ] Note that ^ does not explicitly appear in this last term - we started with it, but it disappeared. Further note that (yi E[y i jx i ]) ~N(0; 1) because y i ~N(E[y i jx i ]; y). y 19

20 Note, this is critical, our created random variable, G P n (Yi E[Y ijx i ]), y is the sum of the squares of a bunch of standard normal random variables. That means it has a distribution. 15 The important thing to remember at this point is that we have created a random variable G that is a linear function of the ratio ^ and we know its density function. You want to learn what you can about the Chi-squared distribution (keep in mind, saying k is the degrees of freedom of the distribution is just another way of saying the density function has one parameter, k). Speci cally, G (Yi E[Y i jx i ]) y (n )^ ~ n It is (n ) because of the parameter (number of degrees of freedom) is not the number of terms in the sum, but the number of independent terms in the sum, which is (n ) because we lose two degrees of freedom to get E[Y i jx i ] + x i. That is, G (n )^ parameter (n ). is a rv with a Chi-squared distribution with (The bottom line is someone worked backward and gured out a rv that was a function of ^ and, and that had a Chi-square distribution.) Note that neither ^ nor is a parameter in the Chi-square, which is important. 15 See Gujarati page 114 and MGB pages 41-43). Theorem 7 (MGB page 4) states that If random variables X i, i 1; ; ::; k, are normally distributed with means i and variances i, then U P n Xi i has a chi-square distribution with parameter k (k degress of i freedom). A collarary is that if X 1 ; X ; ::; X n is a random sample from a normal distribution with mean mean and variance then P n Xi has a chi-square distribution with n degrees of freedom. A special case is that Xi has a chi-square distribution with 1 degree of freedom. 0

21 So what do we know at this point? and ( ~x i (n )^ ~ n ):5 ~N(0; 1) So, now let s mention the t distribution. MGB tell us n N(0; 1) :5 ~t n (n ) That is, a standard normal rv, e.g., divided by the square root of a rv (divided by its parameter) has a t distribution with that parameter Theorem 10 (MGB page 50) states that If the rv Z has a standard normal distribution, if the rv U has a chi-squared distribution with (degrees of freedom k), and if Z and U are independent, Z (Uk) :5 has a Student t distribution with parameter k (degrees of freedom). A relevant Corollary is on page 50. 1

22 So, let s divide and see what simpli es. De ne the rv W W N(0; 1) :5 (n ) n (n ) :5 n ( ~x i ):5 n ( ~x i ):5 (n )^ (n ) :5 ( ~x i ):5 (^ :5 (n ) :5 ( ~x i ):5 ^ : ^ Note that cance out; ( P n ~x i ):5 this is critical since we don t know it. ^ ( ~x i ):5 ~t n ^ if y i + x i + i where ~N(0; ).

23 So, to say it explicitly, we have determined that ^ has a t distribution with parameter (n ) 17 It took a lot of what we have learned to derive this. Consider an example. If 18 n 3 ~t 30 ^ In which case Pr(t 30 > :04) :05 and Pr(t 30 < :04) :05 from the t table. So, Pr :04 < < :04 :95 ^ () Pr :04^ < < + :04^ :95 The interval :04^ to + :04^ is the 95% con dence interval for based on ^ rather than. This interval is a random variable; 95% of these interva will include. Contrast this con dence interval with which we derived earlier. Pr 1:96 < < + 1:96 :95 A hypothesis test How would one determine whether they can reject the null hypothesis that 4? One can derive the con dence interval for and see if its includes 4. Alternatively, one can directly use ~t n ^ If 4, the null is correct 4 ^ ~t n 17 This is close but di erent from saying that has a t distribution. 18 If ~t ^ n, E[ n ] 0 and it variance is ^ (n ) n. In explanation, all t (n 4) distributions have a mean of zero, and n is the variance of all t distributions. (n 4) 3

24 Note that since a value of is assumed, this is a calculable number. For example, if n 3, 8, and ^ then 4 ^ 8 4 If one chooses a two-tailed test (:05 in each tail), the critical value of t is :04. In this case, :0 < :045 and one fai to reject the null hypothesis that f() (B B)/sigmahatB ( )b has a t distribution Most basic OLS regression packages print out the t values corresponding to the null hypothesis 0. Be aware that these t statistics don t mean much unless you are willing to assume that is normally distributed. 19 Note that these t values make no sense if one does not assume ~N 0;. That is, if one does not adopt this assumption, the random variable ^ does not have a t distribution. Said a di erent way, if you are unwilling to assume ~N 0; you better not be paying any attention to the t values your OLS package printed out. 4

25 Now derive the 95% con dence interval for assuming n 3 We are still assuming the CLR model and no knowledge of. Earlier we showed that G (n )^ ~ n Using the table one can determine that Pr 30 > 46:98 :05 and Pr 30 < 16:79 :05 Below is the density function for 30; :5% of the area is to the left of 16:79 and :5% is to the right of 46:98. f(g) G has a ChiSquared distribution g We are still assuming the CLR model and no knowledge of. Earlier we showed that G (n )^ ~ n ) ) So Pr 16:79 30^ 46:98 :95 16:79 Pr 30^ 1 46:98 30^ :95 30^ Pr 16:79 30^ :95 46:98 5

26 ) ) 30^ Pr 46:98 30^ :95 16:79 Pr :638^ 1:786^ :95 So, we have derived a con dence interval on the population parameter as a function of ^. Note that the con dence interval, :638^ 1:786^, is a random variable; 95% of these interva will include. If one wanted to test the null hypothesis that takes some speci c value, e.g. 4, one can either see if 4 is in the interval :638^ 1:786^. Or one can directly use the fact that Plugging in the 4 and n 3 (n )^ ~ n (30)^ 4 7:5^ ~ n From above, for a two-tailed test at the :05 signi cance level, the critical values of 30 are 16:79 and 46:98. So if ^ 46:98 7:5 6:6, one would reject the null hypothesis that 4. One would ao reject this null hypothesis if ^ 16:798 7:5 : 6

27 How about a con dence interval for y i, conditional on x i, assuming ~N(0; ) and no knowldege of? From above we know that #! y j ~N + x j ; 1 n + (x j x) ~x j If we replace with ^ it no longer has a normal distribution. But, by the same argument as above y j E[y j jx j ] ^ yj ~t n This implies, still assuming n 3, Pr :04 < y! j E[y j jx j ] < :04 ^ yj :95 ) Pr y j :04^ yj < y j jx j < y j + :04^ yj :95 So, 95% of the interva, y j :04^ yj < y j jx j < y j + :04^ yj, will contain the true y j conditional on x j. Note that this interval takes it minimum value when x j x, decreases as x j! x. How do I know this? 7

28 1.7.1 What if I don t know the distribution of but am willing to assume E[] 0 and that has a nite but unknown variance? We are now assuming a LRM, but not a CLRM. y j In this case we still can do OLS estimation, and, as we saw,,, and are BLUE estimators. We can ao calculate ^ (y i x i ) (n ) and ^ ^ ~x i To do hypothesis tests or interval estimation on, we need to determine the distribution of Note that we cannot assume that ^ ~N (0; 1). If it were normal, one can determine (above) that ^ ~t n, but now we can t determine the distribution. To do so we need to know the distribution of, which we do not. ^ 8

29 1.7. What if we know the distribution of and it is not normal? Now we are assuming a LRM and knowledge of f (), which is not normal. So we are not assuming the CLR model. Assume for example, ~S 0; where S denotes the Snerd distribution, where the Snerd is not the normal distribution - to start, assume you know. In this case, is ~S (0; 1)? That is, does it have a standardized Snerd distribution? The answer is sometimes but not always. 0 If you could show that ~S (0; 1) one could do con dence interva and hypothesis tests for assumed values of. If one replaces with ^, ^ will de nitely not have a Snerd distribution or a student t distribution. In theory, one could gure out the distribution of this rv (along the lines we did it assuming normality) and then do hypothesis tests and con dence interva. This could be tough. Now again assume you know, continuing to assume has a Snerd distribution To simulate estimated con dence interva for and, one might proceed as follows: Assume the data-generating process for your realworld population of interest is the LRM with ~S 0;, where the value of is known - S 0; is completely speci ed. Estimate and for this sample. Then assume, and are the population parameters; that is, your suedo data-generating process is y i + x i + i where ~S 0;. For the vector of x, x 1 ; x ; :::x i ; :::; x n generate S di erent random samples of size n based on the suedo data-generating process; make S a large number. For each sample s, estimate s and s. Plot the distribution of the S s and the distribution of the S s. The former is an estimated sampling distribution for, centered on, the latter an estimated sampling distribution for, centered on. A 95% con dence for each can be estimated by lopping o the top and bottom :5% of each of these estimated distributions. 0 For example if had a t distribution, would not have a t distribution. But we know that if is normal then is normal. 9

30 Note, these estimated con dence interva are a function of the initial random sample from your population, the assumption that one has a LRM, the assumption that ~S 0;, that is known, and n: it is de nitely a function of the Snerd assumption and. The larger n the shorter the con dence interval. Note, one does not need to theoretically derive either f( ) or f( ): the latter was derived by simulation. Now continue to assume has a Snerd distribution but now assum the value of is unknown. To simulate estimated con dences interva for,, b one might proceed as follows: Assume the data-generating process for your real-world population of interest is the LRM with ~S 0;, where the value of is unknown. Estimate and for this sample, and use these to estimate, b. Then assume, and b are the population parameters; that is, your suedo data-generating process is y i + x i + i where ~S 0; b. For the vector of x, x 1 ; x ; :::x i ; :::; x n generate S di erent random samples of size n based on the suedo data-generating process; make S a large number. For each sample s, estimate s and s, and then use them to estimate s b Plot the distribution of the S s, the distribution of the S s, and the distribution of the S s b. The rst is an estimated sampling distribution for, centered on, the second is an estimated sampling distribution for, centered on, and the third is the sampling distribution of b, centered on b. A 95% con dence for each can be estimated by lopping o the top and bottom :5% of each of these estimated distributions. Note, these estimated con dence interva are a function of the initial random sample from your population, the assumption that one has a LRM, the assumption that ~S 0; and n: it is de nitely a function of the Snerd assumption. It is not a function of the value of. The larger n the shorter these con dence interva. Note, one does not need to theoretically derive either f( b ) or f( ): the latter was derived by simulation. 30

1. The Classical Linear Regression Model: The Bivariate Case

1. The Classical Linear Regression Model: The Bivariate Case Business School, Brunel University MSc. EC5501/5509 Modelling Financial Decisions and Markets/Introduction to Quantitative Methods Prof. Menelaos Karanasos (Room SS69, Tel. 018956584) Lecture Notes 3 1.

More information

OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance

OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance Lecture 5: Hypothesis Testing What we know now: OLS is not only unbiased it is also the most precise (efficient) unbiased estimation technique - ie the estimator has the smallest variance (if the Gauss-Markov

More information

Exact Nonparametric Tests for Comparing Means - A Personal Summary

Exact Nonparametric Tests for Comparing Means - A Personal Summary Exact Nonparametric Tests for Comparing Means - A Personal Summary Karl H. Schlag European University Institute 1 December 14, 2006 1 Economics Department, European University Institute. Via della Piazzuola

More information

Representation of functions as power series

Representation of functions as power series Representation of functions as power series Dr. Philippe B. Laval Kennesaw State University November 9, 008 Abstract This document is a summary of the theory and techniques used to represent functions

More information

EC 6310: Advanced Econometric Theory

EC 6310: Advanced Econometric Theory EC 6310: Advanced Econometric Theory July 2008 Slides for Lecture on Bayesian Computation in the Nonlinear Regression Model Gary Koop, University of Strathclyde 1 Summary Readings: Chapter 5 of textbook.

More information

Chapter 3: The Multiple Linear Regression Model

Chapter 3: The Multiple Linear Regression Model Chapter 3: The Multiple Linear Regression Model Advanced Econometrics - HEC Lausanne Christophe Hurlin University of Orléans November 23, 2013 Christophe Hurlin (University of Orléans) Advanced Econometrics

More information

CAPM, Arbitrage, and Linear Factor Models

CAPM, Arbitrage, and Linear Factor Models CAPM, Arbitrage, and Linear Factor Models CAPM, Arbitrage, Linear Factor Models 1/ 41 Introduction We now assume all investors actually choose mean-variance e cient portfolios. By equating these investors

More information

Linear and Piecewise Linear Regressions

Linear and Piecewise Linear Regressions Tarigan Statistical Consulting & Coaching statistical-coaching.ch Doctoral Program in Computer Science of the Universities of Fribourg, Geneva, Lausanne, Neuchâtel, Bern and the EPFL Hands-on Data Analysis

More information

Limitations of regression analysis

Limitations of regression analysis Limitations of regression analysis Ragnar Nymoen Department of Economics, UiO 8 February 2009 Overview What are the limitations to regression? Simultaneous equations bias Measurement errors in explanatory

More information

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

More information

Two-Variable Regression: Interval Estimation and Hypothesis Testing

Two-Variable Regression: Interval Estimation and Hypothesis Testing Two-Variable Regression: Interval Estimation and Hypothesis Testing Jamie Monogan University of Georgia Intermediate Political Methodology Jamie Monogan (UGA) Confidence Intervals & Hypothesis Testing

More information

Review of Bivariate Regression

Review of Bivariate Regression Review of Bivariate Regression A.Colin Cameron Department of Economics University of California - Davis accameron@ucdavis.edu October 27, 2006 Abstract This provides a review of material covered in an

More information

Chapter 4: Statistical Hypothesis Testing

Chapter 4: Statistical Hypothesis Testing Chapter 4: Statistical Hypothesis Testing Christophe Hurlin November 20, 2015 Christophe Hurlin () Advanced Econometrics - Master ESA November 20, 2015 1 / 225 Section 1 Introduction Christophe Hurlin

More information

Introduction. Agents have preferences over the two goods which are determined by a utility function. Speci cally, type 1 agents utility is given by

Introduction. Agents have preferences over the two goods which are determined by a utility function. Speci cally, type 1 agents utility is given by Introduction General equilibrium analysis looks at how multiple markets come into equilibrium simultaneously. With many markets, equilibrium analysis must take explicit account of the fact that changes

More information

The Delta Method and Applications

The Delta Method and Applications Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and

More information

Simple Linear Regression Inference

Simple Linear Regression Inference Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

More information

Econometrics The Multiple Regression Model: Inference

Econometrics The Multiple Regression Model: Inference Econometrics The Multiple Regression Model: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, March 2011 1 / 24 in

More information

Hypothesis Testing for Beginners

Hypothesis Testing for Beginners Hypothesis Testing for Beginners Michele Piffer LSE August, 2011 Michele Piffer (LSE) Hypothesis Testing for Beginners August, 2011 1 / 53 One year ago a friend asked me to put down some easy-to-read notes

More information

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson Contents What Are These Demos About? How to Use These Demos If This Is Your First Time Using Fathom Tutorial: An Extended Example

More information

Inference in Regression Analysis. Dr. Frank Wood

Inference in Regression Analysis. Dr. Frank Wood Inference in Regression Analysis Dr. Frank Wood Inference in the Normal Error Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters

More information

Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood

Regression Estimation - Least Squares and Maximum Likelihood. Dr. Frank Wood Regression Estimation - Least Squares and Maximum Likelihood Dr. Frank Wood Least Squares Max(min)imization Function to minimize w.r.t. b 0, b 1 Q = n (Y i (b 0 + b 1 X i )) 2 i=1 Minimize this by maximizing

More information

OLS in Matrix Form. Let y be an n 1 vector of observations on the dependent variable.

OLS in Matrix Form. Let y be an n 1 vector of observations on the dependent variable. OLS in Matrix Form 1 The True Model Let X be an n k matrix where we have observations on k independent variables for n observations Since our model will usually contain a constant term, one of the columns

More information

Chapter 2. Dynamic panel data models

Chapter 2. Dynamic panel data models Chapter 2. Dynamic panel data models Master of Science in Economics - University of Geneva Christophe Hurlin, Université d Orléans Université d Orléans April 2010 Introduction De nition We now consider

More information

1 Hypothesis Testing. H 0 : population parameter = hypothesized value:

1 Hypothesis Testing. H 0 : population parameter = hypothesized value: 1 Hypothesis Testing In Statistics, a hypothesis proposes a model for the world. Then we look at the data. If the data are consistent with that model, we have no reason to disbelieve the hypothesis. Data

More information

1 Density functions, cummulative density functions, measures of central tendency, and measures of dispersion

1 Density functions, cummulative density functions, measures of central tendency, and measures of dispersion Density functions, cummulative density functions, measures of central tendency, and measures of dispersion densityfunctions-intro.tex October, 9 Note tat tis section of notes is limitied to te consideration

More information

DEPARTMENT OF ECONOMICS. Unit ECON 12122 Introduction to Econometrics. Notes 4 2. R and F tests

DEPARTMENT OF ECONOMICS. Unit ECON 12122 Introduction to Econometrics. Notes 4 2. R and F tests DEPARTMENT OF ECONOMICS Unit ECON 11 Introduction to Econometrics Notes 4 R and F tests These notes provide a summary of the lectures. They are not a complete account of the unit material. You should also

More information

How to Conduct a Hypothesis Test

How to Conduct a Hypothesis Test How to Conduct a Hypothesis Test The idea of hypothesis testing is relatively straightforward. In various studies we observe certain events. We must ask, is the event due to chance alone, or is there some

More information

Quantitative Methods for Economics Tutorial 9. Katherine Eyal

Quantitative Methods for Economics Tutorial 9. Katherine Eyal Quantitative Methods for Economics Tutorial 9 Katherine Eyal TUTORIAL 9 4 October 2010 ECO3021S Part A: Problems 1. In Problem 2 of Tutorial 7, we estimated the equation ŝleep = 3, 638.25 0.148 totwrk

More information

Master s Theory Exam Spring 2006

Master s Theory Exam Spring 2006 Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

More information

Heteroskedasticity and Weighted Least Squares

Heteroskedasticity and Weighted Least Squares Econ 507. Econometric Analysis. Spring 2009 April 14, 2009 The Classical Linear Model: 1 Linearity: Y = Xβ + u. 2 Strict exogeneity: E(u) = 0 3 No Multicollinearity: ρ(x) = K. 4 No heteroskedasticity/

More information

A Logic of Prediction and Evaluation

A Logic of Prediction and Evaluation 5 - Hypothesis Testing in the Linear Model Page 1 A Logic of Prediction and Evaluation 5:12 PM One goal of science: determine whether current ways of thinking about the world are adequate for predicting

More information

Wooldridge, Introductory Econometrics, 4th ed. Multiple regression analysis:

Wooldridge, Introductory Econometrics, 4th ed. Multiple regression analysis: Wooldridge, Introductory Econometrics, 4th ed. Chapter 4: Inference Multiple regression analysis: We have discussed the conditions under which OLS estimators are unbiased, and derived the variances of

More information

Data Mining and Data Warehousing. Henryk Maciejewski. Data Mining Predictive modelling: regression

Data Mining and Data Warehousing. Henryk Maciejewski. Data Mining Predictive modelling: regression Data Mining and Data Warehousing Henryk Maciejewski Data Mining Predictive modelling: regression Algorithms for Predictive Modelling Contents Regression Classification Auxiliary topics: Estimation of prediction

More information

7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

More information

ECON 142 SKETCH OF SOLUTIONS FOR APPLIED EXERCISE #2

ECON 142 SKETCH OF SOLUTIONS FOR APPLIED EXERCISE #2 University of California, Berkeley Prof. Ken Chay Department of Economics Fall Semester, 005 ECON 14 SKETCH OF SOLUTIONS FOR APPLIED EXERCISE # Question 1: a. Below are the scatter plots of hourly wages

More information

Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

More information

The Simple Linear Regression Model: Specification and Estimation

The Simple Linear Regression Model: Specification and Estimation Chapter 3 The Simple Linear Regression Model: Specification and Estimation 3.1 An Economic Model Suppose that we are interested in studying the relationship between household income and expenditure on

More information

Simple Regression Theory II 2010 Samuel L. Baker

Simple Regression Theory II 2010 Samuel L. Baker SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

More information

Summary of Probability

Summary of Probability Summary of Probability Mathematical Physics I Rules of Probability The probability of an event is called P(A), which is a positive number less than or equal to 1. The total probability for all possible

More information

The Method of Lagrange Multipliers

The Method of Lagrange Multipliers The Method of Lagrange Multipliers S. Sawyer October 25, 2002 1. Lagrange s Theorem. Suppose that we want to maximize (or imize a function of n variables f(x = f(x 1, x 2,..., x n for x = (x 1, x 2,...,

More information

2. What are the theoretical and practical consequences of autocorrelation?

2. What are the theoretical and practical consequences of autocorrelation? Lecture 10 Serial Correlation In this lecture, you will learn the following: 1. What is the nature of autocorrelation? 2. What are the theoretical and practical consequences of autocorrelation? 3. Since

More information

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1. MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

More information

Chapter Additional: Standard Deviation and Chi- Square

Chapter Additional: Standard Deviation and Chi- Square Chapter Additional: Standard Deviation and Chi- Square Chapter Outline: 6.4 Confidence Intervals for the Standard Deviation 7.5 Hypothesis testing for Standard Deviation Section 6.4 Objectives Interpret

More information

496 STATISTICAL ANALYSIS OF CAUSE AND EFFECT

496 STATISTICAL ANALYSIS OF CAUSE AND EFFECT 496 STATISTICAL ANALYSIS OF CAUSE AND EFFECT * Use a non-parametric technique. There are statistical methods, called non-parametric methods, that don t make any assumptions about the underlying distribution

More information

4.6 Null Space, Column Space, Row Space

4.6 Null Space, Column Space, Row Space NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

More information

160 CHAPTER 4. VECTOR SPACES

160 CHAPTER 4. VECTOR SPACES 160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

More information

Variance of OLS Estimators and Hypothesis Testing. Randomness in the model. GM assumptions. Notes. Notes. Notes. Charlie Gibbons ARE 212.

Variance of OLS Estimators and Hypothesis Testing. Randomness in the model. GM assumptions. Notes. Notes. Notes. Charlie Gibbons ARE 212. Variance of OLS Estimators and Hypothesis Testing Charlie Gibbons ARE 212 Spring 2011 Randomness in the model Considering the model what is random? Y = X β + ɛ, β is a parameter and not random, X may be

More information

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4) Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

More information

Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ).

Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ). Correlation Regression Bivariate Normal Suppose that X and Y are r.v. s with joint density f(x y) and suppose that the means of X and Y are respectively µ 1 µ 2 and the variances are 1 2. Definition The

More information

Ordinary Least Squares and Poisson Regression Models by Luc Anselin University of Illinois Champaign-Urbana, IL

Ordinary Least Squares and Poisson Regression Models by Luc Anselin University of Illinois Champaign-Urbana, IL Appendix C Ordinary Least Squares and Poisson Regression Models by Luc Anselin University of Illinois Champaign-Urbana, IL This note provides a brief description of the statistical background, estimators

More information

Chapter 11: Two Variable Regression Analysis

Chapter 11: Two Variable Regression Analysis Department of Mathematics Izmir University of Economics Week 14-15 2014-2015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions

More information

" Y. Notation and Equations for Regression Lecture 11/4. Notation:

 Y. Notation and Equations for Regression Lecture 11/4. Notation: Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

More information

Statistics in Geophysics: Linear Regression II

Statistics in Geophysics: Linear Regression II Statistics in Geophysics: Linear Regression II Steffen Unkel Department of Statistics Ludwig-Maximilians-University Munich, Germany Winter Term 2013/14 1/28 Model definition Suppose we have the following

More information

3.6: General Hypothesis Tests

3.6: General Hypothesis Tests 3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

More information

Chapter 1. Vector autoregressions. 1.1 VARs and the identi cation problem

Chapter 1. Vector autoregressions. 1.1 VARs and the identi cation problem Chapter Vector autoregressions We begin by taking a look at the data of macroeconomics. A way to summarize the dynamics of macroeconomic data is to make use of vector autoregressions. VAR models have become

More information

Statistiek (WISB361)

Statistiek (WISB361) Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17

More information

Class 19: Two Way Tables, Conditional Distributions, Chi-Square (Text: Sections 2.5; 9.1)

Class 19: Two Way Tables, Conditional Distributions, Chi-Square (Text: Sections 2.5; 9.1) Spring 204 Class 9: Two Way Tables, Conditional Distributions, Chi-Square (Text: Sections 2.5; 9.) Big Picture: More than Two Samples In Chapter 7: We looked at quantitative variables and compared the

More information

Chapter 8: Interval Estimates and Hypothesis Testing

Chapter 8: Interval Estimates and Hypothesis Testing Chapter 8: Interval Estimates and Hypothesis Testing Chapter 8 Outline Clint s Assignment: Taking Stock Estimate Reliability: Interval Estimate Question o Normal Distribution versus the Student t-distribution:

More information

Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

More information

Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares

Outline. Topic 4 - Analysis of Variance Approach to Regression. Partitioning Sums of Squares. Total Sum of Squares. Partitioning sums of squares Topic 4 - Analysis of Variance Approach to Regression Outline Partitioning sums of squares Degrees of freedom Expected mean squares General linear test - Fall 2013 R 2 and the coefficient of correlation

More information

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part II)

Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part II) Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part II) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December

More information

IDENTIFICATION IN A CLASS OF NONPARAMETRIC SIMULTANEOUS EQUATIONS MODELS. Steven T. Berry and Philip A. Haile. March 2011 Revised April 2011

IDENTIFICATION IN A CLASS OF NONPARAMETRIC SIMULTANEOUS EQUATIONS MODELS. Steven T. Berry and Philip A. Haile. March 2011 Revised April 2011 IDENTIFICATION IN A CLASS OF NONPARAMETRIC SIMULTANEOUS EQUATIONS MODELS By Steven T. Berry and Philip A. Haile March 2011 Revised April 2011 COWLES FOUNDATION DISCUSSION PAPER NO. 1787R COWLES FOUNDATION

More information

Lesson 1: Comparison of Population Means Part c: Comparison of Two- Means

Lesson 1: Comparison of Population Means Part c: Comparison of Two- Means Lesson : Comparison of Population Means Part c: Comparison of Two- Means Welcome to lesson c. This third lesson of lesson will discuss hypothesis testing for two independent means. Steps in Hypothesis

More information

Advanced Microeconomics

Advanced Microeconomics Advanced Microeconomics Ordinal preference theory Harald Wiese University of Leipzig Harald Wiese (University of Leipzig) Advanced Microeconomics 1 / 68 Part A. Basic decision and preference theory 1 Decisions

More information

Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

More information

HYPOTHESIS TESTING: POWER OF THE TEST

HYPOTHESIS TESTING: POWER OF THE TEST HYPOTHESIS TESTING: POWER OF THE TEST The first 6 steps of the 9-step test of hypothesis are called "the test". These steps are not dependent on the observed data values. When planning a research project,

More information

Quadratic forms Cochran s theorem, degrees of freedom, and all that

Quadratic forms Cochran s theorem, degrees of freedom, and all that Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us

More information

Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

More information

c 2008 Je rey A. Miron We have described the constraints that a consumer faces, i.e., discussed the budget constraint.

c 2008 Je rey A. Miron We have described the constraints that a consumer faces, i.e., discussed the budget constraint. Lecture 2b: Utility c 2008 Je rey A. Miron Outline: 1. Introduction 2. Utility: A De nition 3. Monotonic Transformations 4. Cardinal Utility 5. Constructing a Utility Function 6. Examples of Utility Functions

More information

Mathematics for Economics (Part I) Note 5: Convex Sets and Concave Functions

Mathematics for Economics (Part I) Note 5: Convex Sets and Concave Functions Natalia Lazzati Mathematics for Economics (Part I) Note 5: Convex Sets and Concave Functions Note 5 is based on Madden (1986, Ch. 1, 2, 4 and 7) and Simon and Blume (1994, Ch. 13 and 21). Concave functions

More information

1. Introduction to multivariate data

1. Introduction to multivariate data . Introduction to multivariate data. Books Chat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall Krzanowski, W.J. Principles of multivariate analysis. Oxford.000 Johnson,

More information

Hypothesis Testing in the Linear Regression Model An Overview of t tests, D Prescott

Hypothesis Testing in the Linear Regression Model An Overview of t tests, D Prescott Hypothesis Testing in the Linear Regression Model An Overview of t tests, D Prescott 1. Hypotheses as restrictions An hypothesis typically places restrictions on population regression coefficients. Consider

More information

The Multiple Regression Model: Hypothesis Tests and the Use of Nonsample Information

The Multiple Regression Model: Hypothesis Tests and the Use of Nonsample Information Chapter 8 The Multiple Regression Model: Hypothesis Tests and the Use of Nonsample Information An important new development that we encounter in this chapter is using the F- distribution to simultaneously

More information

Multivariate Normal Distribution

Multivariate Normal Distribution Multivariate Normal Distribution Lecture 4 July 21, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #4-7/21/2011 Slide 1 of 41 Last Time Matrices and vectors Eigenvalues

More information

Regression Analysis: A Complete Example

Regression Analysis: A Complete Example Regression Analysis: A Complete Example This section works out an example that includes all the topics we have discussed so far in this chapter. A complete example of regression analysis. PhotoDisc, Inc./Getty

More information

14.773 Problem Set 2 Due Date: March 7, 2013

14.773 Problem Set 2 Due Date: March 7, 2013 14.773 Problem Set 2 Due Date: March 7, 2013 Question 1 Consider a group of countries that di er in their preferences for public goods. The utility function for the representative country i is! NX U i

More information

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics

Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics Institute of Actuaries of India Subject CT3 Probability and Mathematical Statistics For 2015 Examinations Aim The aim of the Probability and Mathematical Statistics subject is to provide a grounding in

More information

Sampling and Hypothesis Testing

Sampling and Hypothesis Testing Population and sample Sampling and Hypothesis Testing Allin Cottrell Population : an entire set of objects or units of observation of one sort or another. Sample : subset of a population. Parameter versus

More information

Panel Data Econometrics

Panel Data Econometrics Panel Data Econometrics Master of Science in Economics - University of Geneva Christophe Hurlin, Université d Orléans University of Orléans January 2010 De nition A longitudinal, or panel, data set is

More information

Section 2: Estimation, Confidence Intervals and Testing Hypothesis

Section 2: Estimation, Confidence Intervals and Testing Hypothesis Section 2: Estimation, Confidence Intervals and Testing Hypothesis Carlos M. Carvalho The University of Texas at Austin McCombs School of Business http://faculty.mccombs.utexas.edu/carlos.carvalho/teaching/

More information

Bias in the Estimation of Mean Reversion in Continuous-Time Lévy Processes

Bias in the Estimation of Mean Reversion in Continuous-Time Lévy Processes Bias in the Estimation of Mean Reversion in Continuous-Time Lévy Processes Yong Bao a, Aman Ullah b, Yun Wang c, and Jun Yu d a Purdue University, IN, USA b University of California, Riverside, CA, USA

More information

Economics 241B Hypothesis Testing: Large Sample Inference

Economics 241B Hypothesis Testing: Large Sample Inference Economics 241B Hyothesis Testing: Large Samle Inference Statistical inference in large-samle theory is base on test statistics whose istributions are nown uner the truth of the null hyothesis. Derivation

More information

Inference for Regression

Inference for Regression Simple Linear Regression Inference for Regression The simple linear regression model Estimating regression parameters; Confidence intervals and significance tests for regression parameters Inference about

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

Domain of a Composition

Domain of a Composition Domain of a Composition Definition Given the function f and g, the composition of f with g is a function defined as (f g)() f(g()). The domain of f g is the set of all real numbers in the domain of g such

More information

Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

More information

Linear combinations of parameters

Linear combinations of parameters Linear combinations of parameters Suppose we want to test the hypothesis that two regression coefficients are equal, e.g. β 1 = β 2. This is equivalent to testing the following linear constraint (null

More information

Empirical Methods in Applied Economics

Empirical Methods in Applied Economics Empirical Methods in Applied Economics Jörn-Ste en Pischke LSE October 2005 1 Observational Studies and Regression 1.1 Conditional Randomization Again When we discussed experiments, we discussed already

More information

Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

More information

Regression Analysis: Basic Concepts

Regression Analysis: Basic Concepts The simple linear model Regression Analysis: Basic Concepts Allin Cottrell Represents the dependent variable, y i, as a linear function of one independent variable, x i, subject to a random disturbance

More information

2 Testing under Normality Assumption

2 Testing under Normality Assumption 1 Hypothesis testing A statistical test is a method of making a decision about one hypothesis (the null hypothesis in comparison with another one (the alternative using a sample of observations of known

More information

Veri cation and Validation of Simulation Models

Veri cation and Validation of Simulation Models of of Simulation Models mpressive slide presentations Faculty of Math and CS - UBB 1st Semester 2010-2011 Other mportant Validate nput- Hypothesis Type Error Con dence nterval Using Historical nput of

More information

Joint Exam 1/P Sample Exam 1

Joint Exam 1/P Sample Exam 1 Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question

More information

Math 181 Spring 2007 HW 1 Corrected

Math 181 Spring 2007 HW 1 Corrected Math 181 Spring 2007 HW 1 Corrected February 1, 2007 Sec. 1.1 # 2 The graphs of f and g are given (see the graph in the book). (a) State the values of f( 4) and g(3). Find 4 on the x-axis (horizontal axis)

More information

3 Multiple Discrete Random Variables

3 Multiple Discrete Random Variables 3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f

More information

LOGNORMAL MODEL FOR STOCK PRICES

LOGNORMAL MODEL FOR STOCK PRICES LOGNORMAL MODEL FOR STOCK PRICES MICHAEL J. SHARPE MATHEMATICS DEPARTMENT, UCSD 1. INTRODUCTION What follows is a simple but important model that will be the basis for a later study of stock prices as

More information

The Exponential Family

The Exponential Family The Exponential Family David M. Blei Columbia University November 3, 2015 Definition A probability density in the exponential family has this form where p.x j / D h.x/ expf > t.x/ a./g; (1) is the natural

More information

Business Statistics. Lecture 8: More Hypothesis Testing

Business Statistics. Lecture 8: More Hypothesis Testing Business Statistics Lecture 8: More Hypothesis Testing 1 Goals for this Lecture Review of t-tests Additional hypothesis tests Two-sample tests Paired tests 2 The Basic Idea of Hypothesis Testing Start

More information

6.042/18.062J Mathematics for Computer Science December 12, 2006 Tom Leighton and Ronitt Rubinfeld. Random Walks

6.042/18.062J Mathematics for Computer Science December 12, 2006 Tom Leighton and Ronitt Rubinfeld. Random Walks 6.042/8.062J Mathematics for Comuter Science December 2, 2006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Random Walks Gambler s Ruin Today we re going to talk about one-dimensional random walks. In

More information