# Ordinal Regression. Chapter

Size: px
Start display at page:

Transcription

1 Ordinal Regression Chapter 4 Many variables of interest are ordinal. That is, you can rank the values, but the real distance between categories is unknown. Diseases are graded on scales from least severe to most severe. Survey respondents choose answers on scales from strongly agree to strongly disagree. Students are graded on scales from A to F. You can use ordinal categorical variables as predictors, or factors, in many statistical procedures, such as linear regression. However, you have to make difficult decisions. Should you forget the ordering of the values and treat your categorical variables as if they are nominal? Should you substitute some sort of scale (for example, numbers 1 to 5) and pretend the variables are interval? Should you use some other transformation of the values hoping to capture some of that extra information in the ordinal scale? When your dependent variable is ordinal you also face a quandary. You can forget about the ordering and fit a multinomial logit model that ignores any ordering of the values of the dependent variable. You fit the same model if your groups are defined by color of car driven or severity of a disease. You estimate coefficients that capture differences between all possible pairs of groups. Or you can apply a model that incorporates the ordinal nature of the dependent variable. The SPSS Ordinal Regression procedure, or PLUM (Polytomous Universal Model), is an extension of the general linear model to ordinal categorical data. You can specify five link functions as well as scaling parameters. The procedure can be used to fit heteroscedastic probit and logit models. 69

2 70 Chapter 4 Fitting an Ordinal Logit Model Before delving into the formulation of ordinal regression models as specialized cases of the general linear model, let s consider a simple example. To fit a binary logistic regression model, you estimate a set of regression coefficients that predict the probability of the outcome of interest. The same logistic model can be written in different ways. The version that shows what function of the probabilities results in a linear combination of parameters is prob(event) ln ( 1 prob(event) ) = β + β X + β X + + β X k k The quantity to the left of the equal sign is called a logit. It s the log of the odds that an event occurs. (The odds that an event occurs is the ratio of the number of people who experience the event to the number of people who do not. This is what you get when you divide the probability that the event occurs by the probability that the event does not occur, since both probabilities have the same denominator and it cancels, leaving the number of events divided by the number of non-events.) The coefficients in the logistic regression model tell you how much the logit changes based on the values of the predictor variables. When you have more than two events, you can extend the binary logistic regression model, as described in Chapter 3. For ordinal categorical variables, the drawback of the multinomial regression model is that the ordering of the categories is ignored. Modeling Cumulative Counts You can modify the binary logistic regression model to incorporate the ordinal nature of a dependent variable by defining the probabilities differently. Instead of considering the probability of an individual event, you consider the probability of that event and all events that are ordered before it. Consider the following example. A random sample of Vermont voters was asked to rate their satisfaction with the criminal justice system in the state (Doble, 1999). They rated judges on the scale: Poor (1), Only fair (2), Good (3), and Excellent (4). They also indicated whether they or anyone in their family was a crime victim in the last three years. You want to model the relationship between their rating and having a crime victim in the household.

3 71 Ordinal Regression Defining the Event In ordinal logistic regression, the event of interest is observing a particular score or less. For the rating of judges, you model the following odds: θ 1 = prob(score of 1) / prob(score greater than 1) θ 2 = prob(score of 1 or 2) / prob(score greater than 2) θ 3 = prob(score of 1, 2, or 3) / prob(score greater than 3) The last category doesn t have an odds associated with it since the probability of scoring up to and including the last score is 1. All of the odds are of the form: θ j = prob( score j ) / prob(score > j) You can also write the equation as θ j = prob( score j ) / (1 prob( score j )), since the probability of a score greater than j is 1 probability of a score less than or equal to j. Ordinal Model The ordinal logistic model for a single independent variable is then ln( θ j ) = α j β X where j goes from 1 to the number of categories minus 1. It is not a typo that there is a minus sign before the coefficients for the predictor variables, instead of the customary plus sign. That is done so that larger coefficients indicate an association with larger scores. When you see a positive coefficient for a dichotomous factor, you know that higher scores are more likely for the first category. A negative coefficient tells you that lower scores are more likely. For a continuous variable, a positive coefficient tells you that as the values of the variable increase, the likelihood of larger scores increases. An association with higher scores means smaller cumulative probabilities for lower scores, since they are less likely to occur. Each logit has its own α j term but the same coefficient β. That means that the effect of the independent variable is the same for different logit functions. That s an assumption you have to check. That s also the reason the model is also called the proportional odds model. The terms, called the threshold values, often aren t of much interest. Their α j

4 72 Chapter 4 values do not depend on the values of the independent variable for a particular case. They are like the intercept in a linear regression, except that each logit has its own. They re used in the calculations of predicted values. From the previous equations, you also see that combining adjacent scores into a single category won t change the results for the groups that aren t involved in the merge. That s a desirable feature. Examining Observed Cumulative Counts Before you start building any model, you should examine the data. Figure 4-1 is a cumulative percentage plot of the ratings, with separate curves for those whose households experienced crime and those who didn t. The lines for those who experienced crime are above the lines for those who didn t. Figure 4-1 also helps you visualize the ordinal regression model. It models a function of those two curves. Consider the rating Poor. A larger percentage of crime victims than non-victims chose this response. (Because it is the first response, the cumulative percentage is just the observed percentage for the response.) As additional percentages are added (the cumulative percentage for Only fair is the sum of Poor and Only fair), the cumulative percentages for the crime victim households remain larger than for those without crime. It s only at the end, when both groups must reach 100%, that they must join. Because the victims assign lower scores, you expect to see a negative coefficient for the predictor variable, hhcrime (household crime experience). Figure 4-1 Plot of observed cumulative percentages Cumulative percent Poor Only fair Good Rating of judges Excellent Anyone in HH crime victim within past 3 years? Yes No

5 73 Ordinal Regression Specifying the Analysis To fit the cumulative logit model, open the file vermontcrime.sav and from the menus choose: Analyze Regression Ordinal... A Dependent: rating A Factors: hhcrime Options... Link: Logit Output... Display Goodness of fit statistics Summary statistics Parameter estimates Cell information Test of Parallel Lines Saved Variables Estimated response probabilities Parameter Estimates Figure 4-2 contains the estimated coefficients for the model. The estimates labeled Threshold are the α j s, the intercept equivalent terms. The estimates labeled Location are the ones you re interested in. They are the coefficients for the predictor variables. The coefficient for hhcrime (coded 1 = yes, 2 = no), the independent variable in the model, is As is always the case with categorical predictors in models with intercepts, the number of coefficients displayed is one less than the number of categories of the variable. In this case, the coefficient is for the value of 1. Category 2 is the reference category and has a coefficient of 0. The coefficient for those whose household experienced crime in the past three years is negative, as you expected from Figure 4-1. That means it s associated with poorer scores on the rankings of judges. If you calculate e β, that s the ratio of the odds for lower to higher scores for those experiencing crime and those not experiencing crime. In this example, exp(0.633) = This ratio stays the same over all of the ratings. The Wald statistic is the square of the ratio of the coefficient to its standard error. Based on the small observed significance level, you can reject the null hypothesis that

6 74 Chapter 4 it is zero. There appears to be a relationship between household crime and ratings of judges. For any rating level, people who experience crime score judges lower than those who don t experience crime. Figure 4-2 Parameter estimates 95% Confidence Interval Threshold Location [rating = 1] [rating = 2] [rating = 3] [hhcrime=1] [hhcrime=2] Estimate Std. Error Wald df Sig. Lower Bound Upper Bound Link function: Logit. 1. This parameter is set to zero because it is redundant. Testing Parallel Lines When you fit an ordinal regression you assume that the relationships between the independent variables and the logits are the same for all the logits. That means that the results are a set of parallel lines or planes one for each category of the outcome variable. You can check this assumption by allowing the coefficients to vary, estimating them, and then testing whether they are all equal. The result of the test of parallelism is in Figure 4-3. The row labeled Null Hypothesis contains 2 log-likelihood for the constrained model, the model that assumes the lines are parallel. The row labeled General is for the model with separate lines or planes. You want to know whether the general model results in a sizeable improvement in fit from the null hypothesis model. The entry labeled Chi-Square is the difference between the two 2 log-likelihood values. If the lines or planes are parallel, the observed significance level for the change should be large, since the general model doesn t improve the fit very much. The parallel model is adequate. You don t want to reject the null hypothesis that the lines are parallel. From Figure 4-3, you see that the assumption is plausible for this problem. If you do reject the null hypothesis, it is possible that the link function selected is incorrect for the data or that the relationships between the independent variables and logits are not the same for all logits.

7 75 Ordinal Regression Figure 4-3 Test of parallel lines Test of Parallel Lines 1 Model Null Hypothesis General -2 Log Likelihood Chi-Square df Sig The null hypothesis states that the location parameters (slope coefficients) are the same across response categories. 1. Link function: Logit. Does the Model Fit? A standard statistical maneuver for testing whether a model fits is to compare observed and expected values. That is what s done here as well. Calculating Expected Values You can use the coefficients in Figure 4-2 to calculate cumulative predicted probabilities from the logistic model for each case: ( e α j βx) prob(event j) = 1 / (1 + ) Remember that the events in an ordinal logistic model are not individual scores but cumulative scores. First, calculate the predicted probabilities for those who didn t experience household crime. That means that β is 0, and all you have to worry about are the intercept terms. prob(score 1) = 1 / (1 + e ) = prob(score 1 or 2) = 1 / (1 + e ) = prob(score 1 or 2 or 3) = 1 / (1 + ) = prob(score 1 or 2 or 3 or 4) = 1 e 2.59 From the estimated cumulative probabilities, you can easily calculate the estimated probabilities of the individual scores for those whose households did not experience crime. You calculate the probabilities for the individual scores by subtraction, using the formula: prob(score = j) = prob(score less than or equal to j) prob(score less than j).

8 76 Chapter 4 The probability for score 1 doesn t require any modifications. For the remaining scores, you calculate the differences between cumulative probabilities: prob(score = 2) = prob(score = 1 or 2) prob(score = 1) = prob(score = 3) = prob(score 1, 2, 3) prob(score 1, 2) = prob(score = 4) = 1 prob(score 1, 2, 3) = You calculate the probabilities for those whose households experienced crime in the same way. The only difference is that you have to include the value of β in the equation. That is, prob(score = 1) = 1 / (1 + e ( ) ) = prob(score = 1 or 2 ) = 1 / (1 + e ( ) ) = prob(score = 1, 2, or 3) = 1 / (1 + e ( ) ) = prob(score = 1, 2, 3, or 4) = 1 Of course, you don t have to do any of the actual calculations, since SPSS will do them for you. In the Options dialog box, you can ask that the predicted probabilities for each score be saved. Figure 4-4 gives the predicted probabilities for each cell. The output is from the Means procedure with the saved predicted probabilities (EST1_1, EST2_1, EST3_1, and EST4_1) as the dependent variables and hhcrime as the factor variable. All cases with the same value of hhcrime have the same predicted probabilities for all of the response categories. That s why the standard deviation in each cell is 0. For each rating, the estimated probabilities for everybody combined are the same as the observed marginals for the rating variable. Figure 4-4 Estimated response probabilities Anyone in HH crime victim within past 3 years? Yes No Total Mean N Std. Deviation Mean N Std. Deviation Mean N Std. Deviation Estimated Cell Probability for Response Category: 1 Estimated Cell Probability for Response Category: 2 Estimated Cell Probability for Response Category: 3 Estimated Cell Probability for Response Category:

9 77 Ordinal Regression For each rating, the estimated odds of the cumulative ratings for those who experience crime divided by the estimated odds of the cumulative ratings for those who didn t experience crime is e β = For the first response, the odds ratio is ( ) = ( ) 1.88 For the cumulative probability of the second response, the odds ratio is ( ) ( ) = ( ) ( ) 1.88 Comparing Observed and Expected Counts You can use the previously estimated probabilities to calculate the number of cases you expect in each of the cells of a two-way crosstabulation of rating and crime in the household. You multiply the expected probabilities for those without a history by 490, the number of respondents who didn t report a history. The expected probabilities for those with a history are multiplied by 76, the number of people reporting a household history of crime. These are the numbers you see in Figure 4-5 in the row labeled Expected. The row labeled Observed is the actual count. The Pearson residual is a standardized difference between the observed and predicted values: O Pearson residual = ij E ij n i pˆ ij ( 1 pˆ ij ) Figure 4-5 Cell information Frequency Anyone in HH crime victim within past 3 years? Yes No Link function: Logit. Observed Expected Pearson Residual Observed Expected Pearson Residual Rating of judges Poor Only fair Good Excellent

10 78 Chapter 4 Goodness-of-Fit Measures From the observed and expected frequencies, you can compute the usual Pearson and Deviance goodness-of-fit measures. The Pearson goodness-of-fit statistic is χ 2 ( ΣΣ O ij E ij ) 2 = E ij The deviance measure is O D = 2ΣΣO ln ij ij E ij Both of the goodness-of-fit statistics should be used only for models that have reasonably large expected values in each cell. If you have a continuous independent variable or many categorical predictors or some predictors with many values, you may have many cells with small expected values. SPSS warns you about the number of empty cells in your design. In this situation, neither statistic provides a dependable goodness-of-fit test. If your model fits well, the observed and expected cell counts are similar, the value of each statistic is small, and the observed significance level is large. You reject the null hypothesis that the model fits if the observed significance level for the goodnessof-fit statistics is small. Good models have large observed significance levels. In Figure 4-6, you see that the goodness-of-fit measures have large observed significance levels, so it appears that the model fits. Figure 4-6 Goodness-of-fit statistics Pearson Deviance Chi-Square df Sig Link function: Logit. Including Additional Predictor Variables A single predictor variable example makes explaining the basics easier, but real problems almost always involve more than one predictor. Consider what happens when

11 79 Ordinal Regression additional factor variables such as sex, age2 (two categories), and educ5 (five categories) are included as well. Recall the Ordinal Regression dialog box and select: A Dependent: rating A Factors: hhcrime, sex, age2, educ5 Options... Link: Logit Output... Display Goodness of fit statistics Summary statistics Parameter estimates Test of Parallel Lines Saved Variables Predicted category The dimensions of the problem have quickly escalated. You ve gone from eight cells, defined by the four ranks and two crime categories, to 160 cells. The number of cases with valid values for all of the variables is 536, so cells with small observed and predicted frequencies will be a problem for the tests that evaluate the goodness of fit of the model. That s why the warning in Figure 4-7 appears. Figure 4-7 Warning for empty cells There are 44 (29.7%) cells (i.e., dependent variable levels by combinations of predictor variable values) with zero frequencies. Overall Model Test Before proceeding to examine the individual coefficients, you want to look at an overall test of the null hypothesis that the location coefficients for all of the variables in the model are 0. You can base this on the change in 2 log-likelihood when the variables are added to a model that contains only the intercept. The change in likelihood function has a chi-square distribution even when there are cells with small observed and predicted counts. From Figure 4-8, you see that the difference between the two log-likelihoods the chi square has an observed significance level of less than This means that

12 80 Chapter 4 you can reject the null hypothesis that the model without predictors is as good as the model with the predictors. Figure 4-8 Model-fitting information Model Intercept Only Final -2 Log Likelihood Chi-Square df Sig Link function: Logit. You also want to test the assumption that the regression coefficients are the same for all four categories. If you reject the assumption of parallelism, you should consider using multinomial regression, which estimates separate coefficients for each category. Since the observed significance level in Figure 4-9 is large, you don t have sufficient evidence to reject the parallelism hypothesis. Figure 4-9 Test of parallelism Model Null Hypothesis General -2 Log Likelihood Chi-Square df Sig The null hypothesis states that the location parameters (slope coefficients) are the same across response categories. Examining the Coefficients From the observed significance levels in Figure 4-10, you see that sex, education, and household history of crime are all related to the ratings. They all have negative coefficients. Men (code 1) are less likely to assign higher ratings than women, people with less education are less likely to assign higher ratings than people with graduate education (code 5), and persons whose households have been victims of crime are less likely to assign higher ratings than those in crime-free households. Age doesn t appear to be related to the rating.

13 81 Ordinal Regression Figure 4-10 Parameter estimates for the model Threshold Location [rating = 1] [rating = 2] [rating = 3] [hhcrime=1] [hhcrime=2] [sex=1] [sex=2] [age2=0] [age2=1] [educ5=1] [educ5=2] [educ5=3] [educ5=4] [educ5=5] Estimate Std. Error Wald df Sig Link function: Logit. 1. This parameter is set to zero because it is redundant. Measuring Strength of Association There are several -like statistics that can be used to measure the strength of the association between the dependent variable and the predictor variables. They are not as useful as the R 2 statistic in regression, since their interpretation is not straightforward. Three commonly used statistics are: Cox and Snell Nagelkerke s R 2 R 2 R 2 L( B ( 0) ) CS = L( Bˆ ) R 2 N 2 R CS R 2 = L( B ( 0) ) 2 n 2 -- n

14 82 Chapter 4 McFadden s where L( Bˆ ) is the log-likelihood function for the model with the estimated parameters and L( B ( 0) ) is the log-likelihood with just the thresholds, and n is the number of cases (sum of all weights). For this example, the values of all of the pseudo R-square statistics are small. Figure 4-11 Pseudo R-square R 2 R 2 L( Bˆ ) M = L( B ( 0) ) Cox and Snell Nagelkerke McFadden Link function: Logit. Classifying Cases You can use the predicted probability of each response category to assign cases to categories. A case is assigned to the response category for which it has the largest predicted probability. Figure 4-12 is the classification table, which is obtained by crosstabulating rating by pre_1. (This is sometimes called the confusion matrix.) Figure 4-12 Classification table Count Rating of judges Poor Only fair Good Excellent Total Predicted Response Category Only fair Good Total

15 83 Ordinal Regression Of the 198 people who selected the response Only fair, only 42 are correctly assigned to the category using the predicted probability. Of the 279 who selected Good, 246 are correctly assigned. None of the respondents who selected Poor or Excellent are correctly assigned. If the goal of your analysis is to study the association between the grouping variable and the predictor variables, the poor classification should not concern you. If your goal is to target marketing or collections efforts, the correct classification rate may be more important. Generalized Linear Models The ordinal logistic model is one of many models subsumed under the rubric of generalized linear models for ordinal data. The model is based on the assumption that there is a latent continuous outcome variable and that the observed ordinal outcome arises from discretizing the underlying continuum into j-ordered groups. The thresholds estimate these cutoff values. The basic form of the generalized linear model is link( γ j ) γ j θ j [ β 1 x 1 + β 2 x β k x k ] = exp( τ 1 z 1 + τ 2 z τ m z m ) where is the cumulative probability for the jth category, is the threshold for the jth category, β 1 β k are the regression coefficients, x 1 x k are the predictor variables, and k is the number of predictors. The numerator on the right side determines the location of the model. The denominator of the equation specifies the scale. The τ 1 τ m are coefficients for the scale component and z 1 z m are m predictor variables for the scale component (chosen from the same set of variables as the x s). The scale component accounts for differences in variability for different values of the predictor variables. For example, if certain groups have more variability than others in their ratings, using a scale component to account for this may improve your model. θ j Link Function The link function is the function of the probabilities that results in a linear model in the parameters. It defines what goes on the left side of the equation. It s the link between the random component on the left side of the equation and the systematic component

16 84 Chapter 4 on the right. In the criminal rating example, the link function is the logit function, since the log of the odds results is equal to the linear combination of the parameters. That is, prob( event) ln ( 1 prob( event) ) = β + β x + β x + + β x k k Five different link functions are available in the Ordinal Regression procedure in SPSS. They are summarized in the following table. The symbol γ represents the probability that the event occurs. Remember that in ordinal regression, the probability of an event is redefined in terms of cumulative probabilities. Function Form Typical application Logit ln γ γ Evenly distributed categories Complementary log-log ln( ln( 1 γ) ) Higher categories more probable Negative log-log ln( ln( γ) ) Lower categories more probable Probit ( ) Φ 1 γ Analyses with explicit normally distributed latent variable Cauchit (inverse Cauchy) tan( πγ ( 0.5) ) Outcome with many extreme values If you select the probit link function, you fit the model described in Chapter 5. The observed probabilities are replaced with the value of the standard normal curve below which the observed proportion of the area is found. Probit and logit models are reasonable choices when the changes in the cumulative probabilities are gradual. If there are abrupt changes, other link functions should be used. The complementary log-log link may be a good model when the cumulative probabilities increase from 0 fairly slowly and then rapidly approach 1. If the opposite is true, namely that the cumulative probability for lower scores is high and the approach to 1 is slow, the negative log-log link may describe the data. If the complementary log-log model describes the probability of an event occurring, the loglog model describes the probability of the event not occurring.

17 85 Ordinal Regression Fitting a Heteroscedastic Probit Model Probit models are useful for analyzing signal detection data. Signal detection describes the process of detecting an event in the face of uncertainty or noise. You must decide whether a signal is present or absent. For example, a radiologist has to decide whether a tumor is present or not based on inspecting images. You can model the uncertainty in the decision-making process by asking subjects to report how confident they are in their decision. You postulate the existence of two normal distributions: one for the probability of detecting a signal when only noise is present and one for detecting the signal when both the signal and the noise are present. The difference between the means of the two distributions is called d, a measure of the sensitivity of the person to the signal. The general probit model is py ( kx) Φ c k d n X = σ s x where Y is the dependent variable, such as a confidence rating, with values from 1 to K, X is a 0 1 variable that indicates whether the signal was present or absent, c k are ordered distances from the noise distribution, d n is the scaled distance parameter, and is the standard deviation of the signal distribution. The model can be rewritten as σ s [ py ( kx) ] = Φ 1 Φ 1 c k e ax d n X where is the inverse of the cumulative normal distribution and a is the natural log of σ s. The numerator models the location; the denominator, the scale. If the noise and signal distributions have different variances, you must include this information in the model. Otherwise, the parameter estimates are biased and inconsistent. Even large sample sizes won t set things right. Modeling Signal Detection Consider data reported from a light detection study by Swets, et al. (1961) and discussed by DeCarlo (2003). Data are for a single individual who rated his confidence that a signal was present in 591 trials when the signal was absent and 597 trials when the signal was present.

18 86 Chapter 4 In Figure 4-13, you see the cumulative distribution of the ratings under the two conditions (signal absent and signal present). The noise curve is above the signal curve, indicating that the low confidence ratings were more frequent when a signal was not present. Figure 4-13 Plot of cumulative confidence ratings 100% 80% Cumulative percent 60% 40% 20% 0% Signal or noise Noise Signal Not very confident Very confident Rating Cases weighted by count Fitting a Location-Only Model If you assume that the variance of the noise and signal distributions are equal, you can fit the usual probit model. Open the file swets.sav. The data are aggregated. For each possible combination of signal and response, there is a count of the number of times that each response was chosen. You must weight the data file before proceeding. From the menus choose: Data Weight Cases... Weight cases by A count

19 87 Ordinal Regression Analyze Regression Ordinal... A Dependent: response A Covariate(s): signal Options... Link: Probit Output... Display Parameter Estimates Goodness of fit statistics Examining the Goodness of Fit Since the model has only 12 cells and none of them have zero frequencies, you can examine the goodness-of-fit statistics without concern that the expected counts are too small for the chi-square approximation to be valid. From Figure 4-14, you see that the model does not fit well. The observed significance level is less than Figure 4-14 Goodness-of-fit statistics Pearson Deviance Chi-Square df Sig Link function: Probit. One of the reasons the model may fit poorly is because the variance of the two distributions of responses may be different. You need a model that allows the variance of the underlying variable to vary as a function of one or more of the independent variables. You can select a model for the standard deviation such that σ i = e Z i γ where Z i is a vector of covariates selected from the predictor variables.

20 88 Chapter 4 Fitting a Scale Parameter To fit a model that allows for different variances in the two groups, you must specify a model for the scale parameters, the denominator in the previous equation. To fit a model with different variances in the two groups, recall the dialog box and select: Scale... Scale model: signal Because you have only one independent variable, signal, separate variances are estimated for each of the two categories of signal. If you have several predictor variables, you can specify a separate model for the scale component. The goodness-of-fit statistics in Figure 4-15 indicate that the model fits much better than the location-only model. The variability of the distributions are an important consideration in this problem. Figure 4-15 Goodness of fit with scale model Pearson Deviance Chi-Square df Sig Link function: Probit. Parameter Estimates When you fit a model with scale parameters as well as location parameters, parameter estimates for both are displayed. Figure 4-16 Parameter estimates for model with location and scale parameters Parameter Estimates 95% Confidence Interval Threshold Location Scale [response = 1] [response = 2] [response = 3] [response = 4] [response = 5] signal signal Estimate Std. Error Wald df Sig. Lower Bound Upper Bound Link function: Probit.

21 89 Ordinal Regression The threshold values are distances of the response criteria from the mean of the noise distribution. The location parameter estimate is the estimate of the detection parameter, d n. To convert the scale parameter to an estimate of the ratio of the noise to signal standard deviations, you must compute e 0.348, which is Model-Fitting Information The overall test of the model is shown in Figure When there is a scale parameter, the null hypothesis is that both the location parameters and the scale parameters are 0. A scale parameter of 0 means that the variances are equal. Based on the small observed significance level, you can reject this composite null hypothesis. Consult DeCarlo (2003) for further discussion of this example and for other examples of using the Ordinal Regression procedure in signal detection. Figure 4-17 Model-fitting information Model Intercept Only Final -2 Log Likelihood Chi-Square df Sig Link function: Probit.

22

### 11. Analysis of Case-control Studies Logistic Regression

Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

### SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

### Multinomial and Ordinal Logistic Regression

Multinomial and Ordinal Logistic Regression ME104: Linear Regression Analysis Kenneth Benoit August 22, 2012 Regression with categorical dependent variables When the dependent variable is categorical,

### Overview Classes. 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7)

Overview Classes 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7) 2-4 Loglinear models (8) 5-4 15-17 hrs; 5B02 Building and

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### Chapter 5 Analysis of variance SPSS Analysis of variance

Chapter 5 Analysis of variance SPSS Analysis of variance Data file used: gss.sav How to get there: Analyze Compare Means One-way ANOVA To test the null hypothesis that several population means are equal,

### Logit Models for Binary Data

Chapter 3 Logit Models for Binary Data We now turn our attention to regression models for dichotomous data, including logistic regression and probit analysis. These models are appropriate when the response

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

### Two Correlated Proportions (McNemar Test)

Chapter 50 Two Correlated Proportions (Mcemar Test) Introduction This procedure computes confidence intervals and hypothesis tests for the comparison of the marginal frequencies of two factors (each with

### VI. Introduction to Logistic Regression

VI. Introduction to Logistic Regression We turn our attention now to the topic of modeling a categorical outcome as a function of (possibly) several factors. The framework of generalized linear models

### Simple Regression Theory II 2010 Samuel L. Baker

SIMPLE REGRESSION THEORY II 1 Simple Regression Theory II 2010 Samuel L. Baker Assessing how good the regression equation is likely to be Assignment 1A gets into drawing inferences about how close the

### CHAPTER 14 ORDINAL MEASURES OF CORRELATION: SPEARMAN'S RHO AND GAMMA

CHAPTER 14 ORDINAL MEASURES OF CORRELATION: SPEARMAN'S RHO AND GAMMA Chapter 13 introduced the concept of correlation statistics and explained the use of Pearson's Correlation Coefficient when working

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### LOGISTIC REGRESSION ANALYSIS

LOGISTIC REGRESSION ANALYSIS C. Mitchell Dayton Department of Measurement, Statistics & Evaluation Room 1230D Benjamin Building University of Maryland September 1992 1. Introduction and Model Logistic

### The Dummy s Guide to Data Analysis Using SPSS

The Dummy s Guide to Data Analysis Using SPSS Mathematics 57 Scripps College Amy Gamble April, 2001 Amy Gamble 4/30/01 All Rights Rerserved TABLE OF CONTENTS PAGE Helpful Hints for All Tests...1 Tests

### Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus

Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives

### Binary Logistic Regression

Binary Logistic Regression Main Effects Model Logistic regression will accept quantitative, binary or categorical predictors and will code the latter two in various ways. Here s a simple model including

### Poisson Models for Count Data

Chapter 4 Poisson Models for Count Data In this chapter we study log-linear models for count data under the assumption of a Poisson error structure. These models have many applications, not only to the

### SPSS Guide: Regression Analysis

SPSS Guide: Regression Analysis I put this together to give you a step-by-step guide for replicating what we did in the computer lab. It should help you run the tests we covered. The best way to get familiar

### SPSS Explore procedure

SPSS Explore procedure One useful function in SPSS is the Explore procedure, which will produce histograms, boxplots, stem-and-leaf plots and extensive descriptive statistics. To run the Explore procedure,

### Crosstabulation & Chi Square

Crosstabulation & Chi Square Robert S Michael Chi-square as an Index of Association After examining the distribution of each of the variables, the researcher s next task is to look for relationships among

### LOGIT AND PROBIT ANALYSIS

LOGIT AND PROBIT ANALYSIS A.K. Vasisht I.A.S.R.I., Library Avenue, New Delhi 110 012 amitvasisht@iasri.res.in In dummy regression variable models, it is assumed implicitly that the dependent variable Y

### CONTINGENCY TABLES ARE NOT ALL THE SAME David C. Howell University of Vermont

CONTINGENCY TABLES ARE NOT ALL THE SAME David C. Howell University of Vermont To most people studying statistics a contingency table is a contingency table. We tend to forget, if we ever knew, that contingency

### Bivariate Statistics Session 2: Measuring Associations Chi-Square Test

Bivariate Statistics Session 2: Measuring Associations Chi-Square Test Features Of The Chi-Square Statistic The chi-square test is non-parametric. That is, it makes no assumptions about the distribution

### Lesson 1: Comparison of Population Means Part c: Comparison of Two- Means

Lesson : Comparison of Population Means Part c: Comparison of Two- Means Welcome to lesson c. This third lesson of lesson will discuss hypothesis testing for two independent means. Steps in Hypothesis

### MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS MSR = Mean Regression Sum of Squares MSE = Mean Squared Error RSS = Regression Sum of Squares SSE = Sum of Squared Errors/Residuals α = Level of Significance

### INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA)

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) As with other parametric statistics, we begin the one-way ANOVA with a test of the underlying assumptions. Our first assumption is the assumption of

### Chapter 15. Mixed Models. 15.1 Overview. A flexible approach to correlated data.

Chapter 15 Mixed Models A flexible approach to correlated data. 15.1 Overview Correlated data arise frequently in statistical analyses. This may be due to grouping of subjects, e.g., students within classrooms,

### Session 7 Bivariate Data and Analysis

Session 7 Bivariate Data and Analysis Key Terms for This Session Previously Introduced mean standard deviation New in This Session association bivariate analysis contingency table co-variation least squares

### Introduction to Quantitative Methods

Introduction to Quantitative Methods October 15, 2009 Contents 1 Definition of Key Terms 2 2 Descriptive Statistics 3 2.1 Frequency Tables......................... 4 2.2 Measures of Central Tendencies.................

### This chapter will demonstrate how to perform multiple linear regression with IBM SPSS

CHAPTER 7B Multiple Regression: Statistical Methods Using IBM SPSS This chapter will demonstrate how to perform multiple linear regression with IBM SPSS first using the standard method and then using the

### Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study)

Cairo University Faculty of Economics and Political Science Statistics Department English Section Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study) Prepared

### Study Guide for the Final Exam

Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make

### Tests for Two Survival Curves Using Cox s Proportional Hazards Model

Chapter 730 Tests for Two Survival Curves Using Cox s Proportional Hazards Model Introduction A clinical trial is often employed to test the equality of survival distributions of two treatment groups.

### Gamma Distribution Fitting

Chapter 552 Gamma Distribution Fitting Introduction This module fits the gamma probability distributions to a complete or censored set of individual or grouped data values. It outputs various statistics

### HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

### II. DISTRIBUTIONS distribution normal distribution. standard scores

Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,

### Simple linear regression

Simple linear regression Introduction Simple linear regression is a statistical method for obtaining a formula to predict values of one variable from another where there is a causal relationship between

### 1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number

1) Write the following as an algebraic expression using x as the variable: Triple a number subtracted from the number A. 3(x - x) B. x 3 x C. 3x - x D. x - 3x 2) Write the following as an algebraic expression

### To give it a definition, an implicit function of x and y is simply any relationship that takes the form:

2 Implicit function theorems and applications 21 Implicit functions The implicit function theorem is one of the most useful single tools you ll meet this year After a while, it will be second nature to

### Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Introduction to Analysis of Variance (ANOVA) The Structural Model, The Summary Table, and the One- Way ANOVA Limitations of the t-test Although the t-test is commonly used, it has limitations Can only

### Logistic regression modeling the probability of success

Logistic regression modeling the probability of success Regression models are usually thought of as only being appropriate for target variables that are continuous Is there any situation where we might

### Basic Statistics and Data Analysis for Health Researchers from Foreign Countries

Basic Statistics and Data Analysis for Health Researchers from Foreign Countries Volkert Siersma siersma@sund.ku.dk The Research Unit for General Practice in Copenhagen Dias 1 Content Quantifying association

### Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY

Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY 1. Introduction Besides arriving at an appropriate expression of an average or consensus value for observations of a population, it is important to

### Introduction to Regression and Data Analysis

Statlab Workshop Introduction to Regression and Data Analysis with Dan Campbell and Sherlock Campbell October 28, 2008 I. The basics A. Types of variables Your variables may take several forms, and it

### Chapter Seven. Multiple regression An introduction to multiple regression Performing a multiple regression on SPSS

Chapter Seven Multiple regression An introduction to multiple regression Performing a multiple regression on SPSS Section : An introduction to multiple regression WHAT IS MULTIPLE REGRESSION? Multiple

### Part 2: Analysis of Relationship Between Two Variables

Part 2: Analysis of Relationship Between Two Variables Linear Regression Linear correlation Significance Tests Multiple regression Linear Regression Y = a X + b Dependent Variable Independent Variable

### Logistic Regression. http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests

Logistic Regression http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests Overview Binary (or binomial) logistic regression is a form of regression which is used when the dependent is a dichotomy

### I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

Beckman HLM Reading Group: Questions, Answers and Examples Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Linear Algebra Slide 1 of

### NCSS Statistical Software Principal Components Regression. In ordinary least squares, the regression coefficients are estimated using the formula ( )

Chapter 340 Principal Components Regression Introduction is a technique for analyzing multiple regression data that suffer from multicollinearity. When multicollinearity occurs, least squares estimates

Mplus Short Courses Topic 2 Regression Analysis, Eploratory Factor Analysis, Confirmatory Factor Analysis, And Structural Equation Modeling For Categorical, Censored, And Count Outcomes Linda K. Muthén

### Using An Ordered Logistic Regression Model with SAS Vartanian: SW 541

Using An Ordered Logistic Regression Model with SAS Vartanian: SW 541 libname in1 >c:\=; Data first; Set in1.extract; A=1; PROC LOGIST OUTEST=DD MAXITER=100 ORDER=DATA; OUTPUT OUT=CC XBETA=XB P=PROB; MODEL

### Factor Analysis. Principal components factor analysis. Use of extracted factors in multivariate dependency models

Factor Analysis Principal components factor analysis Use of extracted factors in multivariate dependency models 2 KEY CONCEPTS ***** Factor Analysis Interdependency technique Assumptions of factor analysis

### The Probit Link Function in Generalized Linear Models for Data Mining Applications

Journal of Modern Applied Statistical Methods Copyright 2013 JMASM, Inc. May 2013, Vol. 12, No. 1, 164-169 1538 9472/13/\$95.00 The Probit Link Function in Generalized Linear Models for Data Mining Applications

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

### Odds ratio, Odds ratio test for independence, chi-squared statistic.

Odds ratio, Odds ratio test for independence, chi-squared statistic. Announcements: Assignment 5 is live on webpage. Due Wed Aug 1 at 4:30pm. (9 days, 1 hour, 58.5 minutes ) Final exam is Aug 9. Review

### Research Methods & Experimental Design

Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and

### Generalized Linear Models

Generalized Linear Models We have previously worked with regression models where the response variable is quantitative and normally distributed. Now we turn our attention to two types of models where the

### SUGI 29 Statistics and Data Analysis

Paper 194-29 Head of the CLASS: Impress your colleagues with a superior understanding of the CLASS statement in PROC LOGISTIC Michelle L. Pritchard and David J. Pasta Ovation Research Group, San Francisco,

### Simple Predictive Analytics Curtis Seare

Using Excel to Solve Business Problems: Simple Predictive Analytics Curtis Seare Copyright: Vault Analytics July 2010 Contents Section I: Background Information Why use Predictive Analytics? How to use

### ANALYSING LIKERT SCALE/TYPE DATA, ORDINAL LOGISTIC REGRESSION EXAMPLE IN R.

ANALYSING LIKERT SCALE/TYPE DATA, ORDINAL LOGISTIC REGRESSION EXAMPLE IN R. 1. Motivation. Likert items are used to measure respondents attitudes to a particular question or statement. One must recall

### CALCULATIONS & STATISTICS

CALCULATIONS & STATISTICS CALCULATION OF SCORES Conversion of 1-5 scale to 0-100 scores When you look at your report, you will notice that the scores are reported on a 0-100 scale, even though respondents

### Chapter 13. Chi-Square. Crosstabs and Nonparametric Tests. Specifically, we demonstrate procedures for running two separate

1 Chapter 13 Chi-Square This section covers the steps for running and interpreting chi-square analyses using the SPSS Crosstabs and Nonparametric Tests. Specifically, we demonstrate procedures for running

### Chapter 13 Introduction to Linear Regression and Correlation Analysis

Chapter 3 Student Lecture Notes 3- Chapter 3 Introduction to Linear Regression and Correlation Analsis Fall 2006 Fundamentals of Business Statistics Chapter Goals To understand the methods for displaing

### CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS

Examples: Regression And Path Analysis CHAPTER 3 EXAMPLES: REGRESSION AND PATH ANALYSIS Regression analysis with univariate or multivariate dependent variables is a standard procedure for modeling relationships

### Weight of Evidence Module

Formula Guide The purpose of the Weight of Evidence (WoE) module is to provide flexible tools to recode the values in continuous and categorical predictor variables into discrete categories automatically,

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### 5. Linear Regression

5. Linear Regression Outline.................................................................... 2 Simple linear regression 3 Linear model............................................................. 4

### Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables

### X X X a) perfect linear correlation b) no correlation c) positive correlation (r = 1) (r = 0) (0 < r < 1)

CORRELATION AND REGRESSION / 47 CHAPTER EIGHT CORRELATION AND REGRESSION Correlation and regression are statistical methods that are commonly used in the medical literature to compare two or more variables.

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### HURDLE AND SELECTION MODELS Jeff Wooldridge Michigan State University BGSE/IZA Course in Microeconometrics July 2009

HURDLE AND SELECTION MODELS Jeff Wooldridge Michigan State University BGSE/IZA Course in Microeconometrics July 2009 1. Introduction 2. A General Formulation 3. Truncated Normal Hurdle Model 4. Lognormal

### Errata and updates for ASM Exam C/Exam 4 Manual (Sixteenth Edition) sorted by page

Errata for ASM Exam C/4 Study Manual (Sixteenth Edition) Sorted by Page 1 Errata and updates for ASM Exam C/Exam 4 Manual (Sixteenth Edition) sorted by page Practice exam 1:9, 1:22, 1:29, 9:5, and 10:8

### CHAPTER 12 EXAMPLES: MONTE CARLO SIMULATION STUDIES

Examples: Monte Carlo Simulation Studies CHAPTER 12 EXAMPLES: MONTE CARLO SIMULATION STUDIES Monte Carlo simulation studies are often used for methodological investigations of the performance of statistical

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Doing Multiple Regression with SPSS. In this case, we are interested in the Analyze options so we choose that menu. If gives us a number of choices:

Doing Multiple Regression with SPSS Multiple Regression for Data Already in Data Editor Next we want to specify a multiple regression analysis for these data. The menu bar for SPSS offers several options:

### 1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96

1 Final Review 2 Review 2.1 CI 1-propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years

### Statistics in Retail Finance. Chapter 2: Statistical models of default

Statistics in Retail Finance 1 Overview > We consider how to build statistical models of default, or delinquency, and how such models are traditionally used for credit application scoring and decision

### Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@

Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@ Yanchun Xu, Andrius Kubilius Joint Commission on Accreditation of Healthcare Organizations,

### Logit and Probit. Brad Jones 1. April 21, 2009. University of California, Davis. Bradford S. Jones, UC-Davis, Dept. of Political Science

Logit and Probit Brad 1 1 Department of Political Science University of California, Davis April 21, 2009 Logit, redux Logit resolves the functional form problem (in terms of the response function in the

### Factor Analysis. Chapter 420. Introduction

Chapter 420 Introduction (FA) is an exploratory technique applied to a set of observed variables that seeks to find underlying factors (subsets of variables) from which the observed variables were generated.

### Standard errors of marginal effects in the heteroskedastic probit model

Standard errors of marginal effects in the heteroskedastic probit model Thomas Cornelißen Discussion Paper No. 320 August 2005 ISSN: 0949 9962 Abstract In non-linear regression models, such as the heteroskedastic

### SCHOOL OF HEALTH AND HUMAN SCIENCES DON T FORGET TO RECODE YOUR MISSING VALUES

SCHOOL OF HEALTH AND HUMAN SCIENCES Using SPSS Topics addressed today: 1. Differences between groups 2. Graphing Use the s4data.sav file for the first part of this session. DON T FORGET TO RECODE YOUR

### Classification Problems

Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems

### 6 3 The Standard Normal Distribution

290 Chapter 6 The Normal Distribution Figure 6 5 Areas Under a Normal Distribution Curve 34.13% 34.13% 2.28% 13.59% 13.59% 2.28% 3 2 1 + 1 + 2 + 3 About 68% About 95% About 99.7% 6 3 The Distribution Since

### International Statistical Institute, 56th Session, 2007: Phil Everson

Teaching Regression using American Football Scores Everson, Phil Swarthmore College Department of Mathematics and Statistics 5 College Avenue Swarthmore, PA198, USA E-mail: peverso1@swarthmore.edu 1. Introduction

### HLM software has been one of the leading statistical packages for hierarchical

Introductory Guide to HLM With HLM 7 Software 3 G. David Garson HLM software has been one of the leading statistical packages for hierarchical linear modeling due to the pioneering work of Stephen Raudenbush

### Logistic Regression (1/24/13)

STA63/CBB540: Statistical methods in computational biology Logistic Regression (/24/3) Lecturer: Barbara Engelhardt Scribe: Dinesh Manandhar Introduction Logistic regression is model for regression used

### Point Biserial Correlation Tests

Chapter 807 Point Biserial Correlation Tests Introduction The point biserial correlation coefficient (ρ in this chapter) is the product-moment correlation calculated between a continuous random variable

### It is important to bear in mind that one of the first three subscripts is redundant since k = i -j +3.

IDENTIFICATION AND ESTIMATION OF AGE, PERIOD AND COHORT EFFECTS IN THE ANALYSIS OF DISCRETE ARCHIVAL DATA Stephen E. Fienberg, University of Minnesota William M. Mason, University of Michigan 1. INTRODUCTION

### 7 Generalized Estimating Equations

Chapter 7 The procedure extends the generalized linear model to allow for analysis of repeated measurements or other correlated observations, such as clustered data. Example. Public health of cials can

### CREDIT SCORING MODEL APPLICATIONS:

Örebro University Örebro University School of Business Master in Applied Statistics Thomas Laitila Sune Karlsson May, 2014 CREDIT SCORING MODEL APPLICATIONS: TESTING MULTINOMIAL TARGETS Gabriela De Rossi

### Section 14 Simple Linear Regression: Introduction to Least Squares Regression

Slide 1 Section 14 Simple Linear Regression: Introduction to Least Squares Regression There are several different measures of statistical association used for understanding the quantitative relationship

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max