Logit, Probit and Tobit: Models for Categorical and Limited

Transcription

1 Logit, Probit and Tobit: Models for Categorical and Limited Dependent Variables By Rajulton Fernando Presented at PLCS/RDC Statistics and Data Series at Western March 23, 2011

2 Introduction In social science research, categorical data are often collected through surveys. Categorical Nominal and Ordinal variables They take only a few values that do NOT have a metric. A) Binary Case Many dependent variables of interest take only two values (a dichotomous variable), denoting an event or non-event and coded as 1 and 0 respectively. Some examples: The labor force status of a person. Voting behavior of a person (in favor of a new policy). Whether a person got married or divorced. Whether a person involved in criminal behaviour, etc.

3 Introduction With such variables, we can build models that describe the response probabilities, say P(y i = 1), of the dependent variable y i. y i For a sample of N independently and identically distributed observations i = 1,...,N and a (K+1)-dimensional vector x i of explanatory variables, the probability bilit that t y takes value 1 is modeled as P ( yi = 1 xi ) = F ( xi β ) = F ( zi where β is a (K + 1)-dimensional column vector of parameters. The transformation function F is crucial. It maps the linear combination into [0,1] and satisfies in general F( ) =0 0, F(+ ) =1, andδf(z)/δz > 0 [that is, it is a cumulative distribution function]. )

4 The Logit and Probit Models When the transformation function F is the logistic function, the response probabilities are given by P( y i = 1 x i ) = x β i e x β And, when the transformation function F is the cumulative density function (cdf) of the standard normal distribution, the response probabilities are x β x β i given by i s P ( yi = 1 xi ) = Φ ( xi β ) = Φ ( s ) ds = e 2 The Logit and Probit models are almost identical (see the Figure next slide) and the choice of the model is arbitrary, although logit model has certain advantages (simplicity and ease of interpretation) 1+ e i 2π ds

5 Source: J.S. Long, 1997

6 The Logit and Probit Models However, the parameters of the two models are scaled differently. The parameter estimates in a logistic regression tend to be 1.6 to 1.8 times higher than they are in a corresponding probit model. The probit and logit models are estimated by maximum likelihood (ML), assuming independence across observations. The ML estimator of β is consistent and asymptotically normally distributed. However, the estimation rests on the strong assumption that the latent error term is normally distributed and homoscedastic. If homoscedasticity is violated, no easy solution.

7 The Logit and Probit Models Note: The response function (logistic or probit) is an S-shaped function, which implies a fixed change in X has a smaller impact on the probability when it is near zero than when it is near the middle. Thus, it is a non-linear response function. How to interpret the coefficients : In both models, If b > 0 p increases as X increases If b <0 p decreases as X increases As mentioned above, b cannot be interpreted as a simple slope as in ordinary regression. Because the rate at which the curve ascends or descends changes according to the value of X. In other words, it is not a constant change as in ordinary regression. The greatest rate of change is at p = 0.5

8 The Logit and Probit Models In the logit model, we can interpret b as an effect on the odds. That is, every unit increase in X results in a multiplicative effect of e b on the odds. Example: If b = 0.25, then e.25 = Thus, when X changes by one unit, p increases by a factor of 1.28, or changes by 28%. - In the probit model, use the Z-score terminology. For every unit increase in X, the Z-score (or the Probit of success ) increases by b units. [Or, we can also say that an increase in X changes Z by b standard deviation units.] - If you like, you can convert the z-score to probabilities y,y p using the normal table.

9 Models for Polytomous Data B) Polytomous Case Here we need to distinguish between purely nominal variables and really ordinal variables. When the variable is purely nominal, we can extend the dichotomous logit model, using one of the categories as reference and modeling the other responses j=1,2,..m-1 compared to the reference. Example: In the case of 3 categories, using the 3 rd category as the reference, logit p 1 = ln(p 1 /p 3 ) and logit p 2 = ln(p 2 /p 3 ), which will give two sets of parameter estimates. P( y P ( y P( y = exp( β 1x) = 1) = 1 + exp( β 1x) + exp( β 2 x) exp( β 2 x) = 2) = 1 + exp( β x) + exp( β x) 3) = exp( β x) exp( β x) 2

10 Polytomous Case When the variable is really ordinal, we use cumulative logits (or probits). The logits in this model are for cumulative categories at each point, contrasting categories above with categories below. Example: Suppose Y has 4 categories; then, logit (p 1 ) = ln{p 1 /(1p (1-p 1 )} = a 1 + bx logit (p 1 + p 2 ) = ln{(p 1 + p 2 )/(1-p 1 p 2 )} = a 2 + bx logit (p 1 +p 2 +p 3 ) = ln{(p 1 + p 2 + p 3 )/(1-p 1 p 2 p 3 )} = a 3 + bx Since these are cumulative logits, the probabilities are attached to being in category j and lower. Since the right side changes only in the intercepts, and not in the slope coefficient, this model is known as Proportional odds model. Thus, in ordered logistic, we need to test the assumption of proportionality as well.

11 Ordinal Logistic a 1, a 2, a 3 are the intercepts that satisfy the property a 1 < a 2 < a 3 interpreted as thresholds of the latent variable. Interpretation of parameter estimates depends on the software used! Check the software manual. If the RHS = a + bx, a positive coefficient is associated more with lower order categories and a negative coefficient is associated more with higher order categories. If the RHS = a bx, a negative coefficient is more associated with lower ordered categories, and a positive coefficient is more associated with higher ordered categories.

12 Model for Limited Dependent Variable C) Tobit Model This model is for metric dependent variable and when it is limited in the sense we observe it only if it is above or below some cut off level. For example, the wages may be limited from below by the minimum wage The donation amount give to charity Top coding income at, say, at $300,000 Time use and leisure activity of individuals Extramarital affairs It is also called censored regression model. Censoring can be from below or from above, also called left and right censoring. [Do not confuse the term censoring with the one used in dynamic modeling.]

13 The Tobit Model The model is called Tobit because it was first proposed by Tobin (1958), and involves aspects of Probit analysis a term coined by Goldberger for Tobin s Probit. Reasoning behind: If we include the censored observations as y = 0, the censored observations on the left will pull down the end of the line, resulting in underestimates of the intercept and overestimates of the slope. If we exclude the censored observations and just use the observations for which y>0 (that is, truncating the sample), it will overestimate the intercept and underestimate the slope. The degree of bias in both will increase as the number of observations that take on the value of zero increases. (see Figure next slide)

14 Source: J.S. Long

15 The Tobit Model The Tobit model uses all of the information, including info on censoring and provides consistent estimates. It is also a nonlinear model and similar to the probit model. It is estimated using maximum likelihood estimation techniques. The likelihood function for the tobit model takes the form: This is an unusual function, it consists of two terms, the first for non-censored observations (it is the pdf), and dthe second dfor censored observations (iti is the cdf).

16 The Tobit Model The estimated tobit coefficients are the marginal effects of a change in x j on y*, the unobservable latent variable and can be interpreted in the same way as in a linear regression model. But such an interpretation may not be useful since we are interested in the effect of X on the observable y (or change in the censored outcome). It can be shown that t change in y is found by multiplying l i the coefficient with Pr(a<y*<b), that is, the probability of being uncensored. Since this probability is a fraction, the marginal effect is actually attenuated. In the above, a and b denote lower and upper censoring points. For example, in left censoring, the limits will be: a =0, b=+.

17 Illustrations for logit, probit and tobit models, using womenwk.dta from Baum available at Descriptive Statistics N Minimum Maximum Mean Std. Deviation age education married children wagefull wage lw work lwf Valid N (listwise) 1343 Binary Logistic Regression Model Summary Step -2 Log likelihood Cox & Snell R Square Nagelkerke R Square a a. Estimation terminated at iteration number 5 because parameter estimates changed by less than.001. Hosmer and Lemeshow Test Step Chi-square df Sig Variables in the Equation B S.E. Wald df Sig. Exp(B) Step 1 a age education married children Constant a. Variable(s) entered on step 1: age, education, married, children.

18 Binary Probit Regression (in SPSS, use the ordinal regression menu and select probit link function. Ignore the test of parallel lines, etc.) Model Fitting Information Model -2 Log Likelihood Chi-Square df Sig. Intercept Only Final Link function: Probit. Parameter Estimates 95% Confidence Interval Estimate Std. Error Wald df Sig. Lower Bound Upper Bound Threshold [work = 0] Location age education children [married=0] [married=1] 0 a Link function: Probit. a. This parameter is set to zero because it is redundant. Tobit regression cannot be done in SPSS. Use Stata. Here are the Stata commands. First, fit simple OLS Regression of the variable lwf (just to check). regress lwf age married children education Source SS df MS Number of obs = F( 4, 1995) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = lwf Coef. Std. Err. t P> t [95% Conf. Interval] age married children education _cons tobit lwf age married children education, ll(0)

19 Tobit regression Number of obs = 2000 LR chi2(4) = Prob > chi2 = Log likelihood = Pseudo R2 = lwf Coef. Std. Err. t P> t [95% Conf. Interval] age married children education _cons /sigma Obs. summary: 657 left-censored observations at lwf<= uncensored observations 0 right-censored observations. mfx compute, predict(pr(0,.)) Marginal effects after tobit y = Pr(lwf>0) (predict, pr(0,.)) = variable dy/dx Std. Err. z P> z [ 95% C.I. ] X age married* children educat~n (*) dy/dx is for discrete change of dummy variable from 0 to 1. mfx compute, predict(e(0,.)) Marginal effects after tobit y = E(lwf lwf>0) (predict, e(0,.)) = variable dy/dx Std. Err. z P> z [ 95% C.I. ] X age married* children educat~n (*) dy/dx is for discrete change of dummy variable from 0 to 1