# LOGISTIC REGRESSION ANALYSIS

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 LOGISTIC REGRESSION ANALYSIS C. Mitchell Dayton Department of Measurement, Statistics & Evaluation Room 1230D Benjamin Building University of Maryland September Introduction and Model Logistic regression analysis (LRA) extends the techniques of multiple regression analysis to research situations in which the outcome variable is categorical. In practice, situations involving categorical outcomes are quite common. In the setting of evaluating an educational program, for example, predictions may be made for the dichotomous outcome of success/failure or improved/not-improved. Similarly, in a medical setting, an outcome might be presence/absence of disease. The focus of this document is on situations in which the outcome variable is dichotomous, although extension of the techniques of LRA to outcomes with three or more categories (e.g., improved, same, or worse) is possible (see, for example, Hosmer & Lemeshow, 1989, Chapter 8). In this section, we review the multiple regression model and, then, present the model for LRA. The fundamental model underlying multiple regression analysis (MRA) posits that a continuous outcome variable is, in theory, a linear combination of a set of predictors and error. Thus, for an outcome variable, Y, and a set of p predictor variables, X 1,...,X p, the MRA model is of the form: where is the Y-intercept (i.e., the expected value of Y when all X's are set to 0), β j is a multiple (partial) regression coefficient (i.e., the expected change in Y per unit change in X j assuming all other X's are held constant) and ε is the error of prediction. If error is omitted, the resulting model represents the expected, or predicted, value of Y: Note that Y=Y +ε. Thus, we can interpret the MRA model as follows: each observed score, Y, is made up of an expected, or predictable, component, Y, that is a function of the predictor variables X 1,...,X p, and an error, or unpredictable component, ε, that represents error of measurement (i.e., unreliability) and/or error in the selection of the model (i.e., mis-specification). The MRA model summarized above is applicable when the outcome variable, Y, is continuous, but is not appropriate for situations in which Y is categorical. For example, if Y takes on the value 1 for "success" and 0 for "failure," the multiple regression equation would not result in predicted values restricted to exactly 1 or 0. In fact, these predicted values would be spread out over an interval that contains uninterpretable values such as.5 or.33 and could even include negative values and/or values greater than 1. The model for logistic regression analysis, described below, is a more realistic representation of the situation when an outcome variable is categorical. The model for logistic regression analysis assumes that the outcome variable, Y, is categorical (e.g., dichotomous), but LRA does not model this outcome variable directly. Rather, LRA is based on probabilities associated with the values of Y. For simplicity, and because it is the case most commonly encountered in practice, we assume that Y is dichotomous, taking on values of 1 (i.e., the positive outcome, or success) and 0 (i.e., the negative outcome, or failure). In theory, the hypothetical, population proportion of cases for which Y = 1 is defined as π = P(Y =1). Then, the theoretical proportion of cases for which Y = 0 is 1 - π = P(Y = 0). In the absence of other information, we would estimate π by the sample proportion of cases for which Y = 1. However, in the regression context, it is assumed that there is a set of predictor variables, X 1,...,X p, that are related to Y and, therefore, provide additional information for predicting Y. For theoretical, mathematical reasons, LRA is based on a linear model for the natural logarithm of the odds (i.e., the log-odds) in favor of Y = 1 (see Appendix A for a further explanation of odds): file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (1 of 9) [4/23/ :31:29 AM]

2 Note that in the LRA model, π is a conditional probability of the form P(Y=1 X 1,...,X p ). That is, it is assumed that "success" is more or less likely depending on combinations of values of the predictor variables. The log-odds, as defined above is also known as the logit transformation of π and the analytical approach described here is sometimes known as logit analysis. The LRA model above is identical to the MRA model except that the log-odds in favor of Y = 1 replaces the expected value of Y. There are two basic reasons underlying the development of the model above. First, probabilities and odds obey multiplicative, rather than additive, rules. However, taking the logarithm of the odds allows for the simpler, additive model since logarithms convert multiplication into addition. And, second, there is a (relatively) simple exponential transformation for converting log-odds back to probability. In particular, the inverse transformation is the logistic function of the form: Due to the mathematical relationship, e a /(1+e a ) = 1/(1+e -a ), the logistic function for LRA is sometimes presented in the form: Due to the mathematical relation, 1- e a /(1+e a ) = 1/(1+e a ), the probability for a 0 response is: 2. Fitting the LRA Model to Data As in MRA, there are two important stages in the analysis of data. First, estimates for the parameters in the model must be obtained and, second, some determination must be made of how well the model actually fits the observed data. In MRA, the parameter estimates are obtained using the least-squares principle and assessment of fit is based on significance tests for the regression coefficients as well as on interpreting the multiple correlation coefficient. The process is analogous for LRA, although specific details of these stages of analysis are somewhat different as outlined in this section. The parameters that must be estimated from the available data are the constant, α, and the logistic regression coefficients, β j. Because of the nature of the model, estimation is based on the maximum likelihood principle rather than on the least-squares principle. In the context of LRA, maximum likelihood estimation (MLE) involves the following. First, we define the likelihood, L, of the sample data as the product, across all sampled cases, of the probabilities for success or for failure: Note that Y is the 0/1 outcome for the i th case and, X i1,...,x ip are the values of the predictor variables for the i th case based on a sample of n cases. The use of Y i and 1-Y i as exponents in the equation above includes in the likelihood the appropriate probability term dependent upon whether Y i =1 or Y i =0 (note that F 0 =1 for any expression, F). Using the methods of calculus, a set of values for α and the β j can be calculated that maximize L and these resulting values are known as maximum likelihood estimates (MLE s). This maximization process is somewhat more complicated than the corresponding minimization procedure in MRA for finding least-square estimates. However, the general approach involves establishing initial guesses for the unknown parameters and then continuously adjusting these estimates until the maximum value of L is found. This iterative solution procedure is available in popular statistical procedures such as those found in SPSS and SAS. file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (2 of 9) [4/23/ :31:29 AM]

3 To distinguish them from parameters, we denote the MLE s as a and b j. Given that these estimates have been calculated for a real data set, tests of significance for individual logistic regression coefficients can be set up as in MRA. That is, for the hypothesis, H: β j = 0, a statistic of the form z = b j /S j can be calculated based on the estimated standard error, S j, for b j (SPSS and SAS report χ 1 2 = z 2 and label these values as Wald statistics). Similarly, the usefulness of the model as a whole can be assessed by testing the hypothesis that, simultaneously, all of the partial logistic regression coefficients are 0; i.e., H: β j = 0 for all j. In effect, we can compare the general model given above with the restricted model Log e (π /(1-π )) = α. This test, that is equivalent to testing the significance of the multiple R in MRA, is based on a chi-squared statistic (SPSS labels this value as "Model Chi-Square"). Finally, different LRA models fitted to the same set of data can be compared statistically in a simple manner if the models are hierarchical. The hierarchy principle requires that the model with the larger number of predictors include among its predictors all of the predictors from the simpler model (e.g., predictors X 1 and X 2 in the simpler model and predictors X 1, X 2, X 3 and X 4 in the more complex model). Given this condition, the difference in model chi-squared values is (approximately) distributed as chi-squared with degrees of freedom equal to the difference in degrees of freedom for the two models (e.g., for the above example, the degrees of freedom would be 2). In effect, this procedure tests a conditional null hypothesis that, for the example, would be: H: β 3 = β 4 = 0 β 1, β 2. That is, the values of the logistic regression coefficients associated with X 1 and X 2 are unrestricted, but the logistic regression coefficients associated with X 3 and X 4 are assumed by hypothesis to be 0. If the models are specified in a series of "blocks" in SPSS, an "Improvement" chi-square value is computed for each successive model and this can be used to test whether or not the additional predictors result in significantly better fit of the model to the data. 3. Interpreting the Fitted LRA Model We may interpret the results from a LRA at three different levels. First, each term in the equation represents contributions to estimated log-odds. Thus, for each one unit increase (decrease) in X j, there is predicted to be an increase (decrease) of b j units in the log-odds in favor of Y = 1. Also, if all predictors are set equal to 0, the predicted log-odds in favor of Y = 1 would be the constant term, a. Second, since most people do not find it natural to think in terms of log-odds, the LRA equation can be transformed to odds by exponentiation: With respect to odds, the influence of each predictor is multiplicative. Thus, for each one unit increase in X j, the predicted odds is increased by a factor of exp(b j ) If X is decreased by one unit, the multiplicative factor is exp(-b j ). Note that exp(c) = e c. Similarly, if all predictors are set equal to 0, the predicted odds are exp(a). Finally, the results can be expressed in terms of probabilities by use of the logistic function. With estimates substituted the equation above becomes: Although this form of the model may seem simplest because it results in predicted probabilities for Y = 1, there is no simple interpretation for the logistic regression coefficients in this form. However, for the case of a single predictor, X, a plot can be generated with X represented by the abscissa (i.e., horizontal axis) and P(Y X) represented by the ordinate (i.e., vertical axis). The resulting curve is S-shaped and is illustrated in one of the exemplary analyses presented below. 4. Illustrative Examples The database used for purposes of illustration is provided with SPSS and is based on 50 premature infants, some of whom did not survive infancy. The SPSS setup for this database, including the raw data, is contained in Appendix B. The outcome variable is dichotomous with "1" representing survival of the infant and "0" representing death of the infant. For the first exemplary analysis, a single independent variable, weight at birth in kilograms, was utilized. For the second analysis, three additional predictors were entered as described below. (1) Single Predictor Example Using birth weight as the predictor, the logistic regression equation for log-odds in favor of survival is estimated to be (the SPSS LOGISTIC REGRESSION printout for this block is provided in Appendix C): To estimate odds per se, the equation is exponentiated: file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (3 of 9) [4/23/ :31:29 AM]

4 Finally, the probability of survival is obtained by applying the logistic transformation: In the database, birth weight, in kilograms, ranged from 1.03 to 3.64 (i.e., from 2.27 to 8.02 pounds). Thus, for the lowest weight recorded, 1.03, the log-odds in favor of survival are , the odds in favor of survival are.1640 and probability of survival is.14. At the other extreme, for birth weight of 3.64 kilograms, the log-odds in favor of survival are , the odds in favor of survival are and the probability of survival is.94. The relation between birth weight and survival of infants is displayed in Figure 1. This graph was constructed by systematically varying birth weight from 1 to 4 kilograms (shown on the abscissa) and calculating the estimated probability of survival (shown on the ordinate). Note from reading the graph that infants with a birth weight of slightly over 2 kilograms (actually, 2.07) have an estimated odds of 1:1 in favor of survival (i.e., for π is estimated to equal.5 for a birth weight of 2.07 kilograms). The estimated logistic regression coefficient for birth weight is and the exponential of this value is e = This indicates that for a one kilogram increase in birth weight, the odds in favor of survival are estimated to be increased by a multiplicative factor of 5.7. The reported standard error for b is.5786 and statistical significance can be assessed by the Wald chi-squared statistic, (1.7406/.5786) 2 = 9.05, that, with 1 degree of freedom, is significant at conventional levels (the empirical two-tailed p-value is reported to be.0026 in Appendix C). Thus, this study supports the conclusion that birth weight is a useful predictor of infant survival. To aid in interpretation, a classification table can be constructed by predicting the survival or death of each infant based on whether or not the odds for survival are greater or less than 1.0, and comparing these predictions to the actual outcome for each infant. As shown in Appendix C, the percents of correct decisions are 56.5 for infants who survived, 74.1 for infants who died, and 66.0 overall. This overall result can be compared with a rate of 54.0% that would be obtained by simply predicting death as the outcome for every infant (i.e., since 27 of the 50 infants died, this is the prediction with the greater likelihood of being correct). (2) Multiple Predictor Example Building on the previous example, the following three predictors, in addition to birth weight, were included in the model: gestation age, sex of infant, and the APGAR score (a measure of infant well being). Using this set of four predictors, the logistic regression equation for the log-odds in favor of survival is estimated to be (note: the SPSS LOGISTIC REGRESSION printout for this block is provided in Appendix D): To estimate odds, the equation is exponentiated: Finally, the probability of survival is obtained by applying the logistic transformation: Unlike the case of a single predictor, there is no simple graphical display that can be used. However, log-odds, odds and probabilities can be calculated for various combinations of the predictors. In the present case, there is more interest in deciding whether or not the three predictors, in addition to birth weight, have improved the predictive efficiency of the model. The Wald chi-squared statistics, as taken from Appendix D, are non-significant for age, sex and APGAR (i.e., p-values of.23,.60 and.58, respectively), whereas the chi-squared value for weight is significant at the.05 level (i.e., p-value of.02). Thus, given that the other predictors remain in the model, removing weight as a predictor would result in significantly poorer predictive efficiency, although removing any of the other predictors does not have a significant impact. A second, global way of comparing the single and multiple predictor models is based on the improvement chi-squared value that is reported as based on 3 degrees of freedom This value is non-significant at conventional levels and, thus, we conclude that the predictive efficiency of the 4-predictor model is no greater than that of the 1-predictor model. It should be noted that the classification table results are exactly the same using four predictors as for using only a single predictor (e.g., 66.0% correct classifications overall). file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (4 of 9) [4/23/ :31:29 AM]

5 Suggested References Fienberg, S. E. (1980). The Analysis of Cross-Classified Categorical Data (Second Edition). Cambridge, MA: The MIT Press Fleiss, J. (1981). Statistical Methods for Rates and Proportions (Second Edition). New York: Wiley Hosmer, D. W. & Lemeshow, S. (1989). Applied Logistic Regression, New York: Wiley. Appendix A: The Meaning of Odds Odds represent the relative frequency with which different outcomes occur. Odds are sometimes expressed as a ratio of the form a:b. For example, odds of 3:1 in favor of the first outcome means that the first outcome occurs 3 times for each single occurrence of the second outcome. Similarly, odds of 5:2 means that the first outcome occurs 5 times for each 2 occurrences of the second outcome. Odds are directly related to probabilities and can be translated back and forth using these relations: Probability = a/(a+b) when odds are expressed as a:b; or Probability = Odds/(1 + Odds) when odds are expressed in decimal form (e.g., 3:1 becomes 3.0 and 5:2 becomes 2.5) Odds = Probability/(1 - Probability ) Some examples: The probability of rolling a 1 with a true die is 1/6 = The odds in favor of rolling a one are 1/6 5/6 =.20, or 1:5. Viewed the other way, the odds against rolling a 1 with a true die are 5:1, or 5.0. The odds in favor of not drawing a face card from a standard deck of playing cards (i.e., drawing a card other than a Jack, Queen, King or Ace) are 36:16 or 9:4. The corresponding probability is 9/(9+4) = 9/13 = Appendix B: Exemplary Database (SPSS Syntaz) TITLE 'RDS: Respiratory Distress Syndrome'. DATA LIST /CASEID 1-2 SURVIVAL 4 TREATMNT 6 TIME 8-10(1) WEIGHT 12-15(3) APGAR SEX 20 AGE PH 33-35(2) RESP 37. RECODE SURVIVAL (1=2) (0=1). VARIABLE LABELS SURVIVAL 'Infant Survival' TREATMNT 'Treatment Administered' TIME 'Time to Spontaneous Respiration' WEIGHT 'Birthweight in Kilograms' APGAR 'APGAR Score' SEX 'Sex of Infant' PH 'PH Level' RESP 'Respiratory Therapy'. MISSING VALUE RESP(9). VALUE LABELS TREATMNT 1 'Tham' 0 'Sodium Bicarbonate' /SEX 1 'Male' 0 'Female' /RESP 1 'Yes' 0 'No' 9 'No Answer' /SURVIVAL 2 'Survive' 1 'Die'. BEGIN DATA file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (5 of 9) [4/23/ :31:29 AM]

6 END DATA. Appendix C: Exemplary SPSS Analysis for a Single Predictor Total number of cases: 50 (Unweighted) Number of selected cases: 50 Number of unselected cases: 0 Number of selected cases: 50 Number rejected because of missing data: 0 Number of cases included in the analysis: 50 Dependent Variable Encoding: Original Internal Value Value Dependent Variable.. SURVIVE Beginning Block Number 0. Initial Log Likelihood Function -2 Log Likelihood * Constant is included in the model. Beginning Block Number 1. Method: Enter Variable(s) Entered on Step Number 1.. WEIGHT Estimation terminated at iteration number 3 because file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (6 of 9) [4/23/ :31:29 AM]

7 Log Likelihood decreased by less than.01 percent. -2 Log Likelihood Goodness of Fit Cox & Snell - R^2.214 Nagelkerke - R^2.286 Chi-Square df Significance Chi-Square df Significance Model Block Step Variables in the Equation Variable B S.E. Wald df Sig R Exp(B) WEIGHT Constant Beginning Block Number 2. Method: Enter Variable(s) Entered on Step Number 1.. GESTAGE APGAR SEX Estimation terminated at iteration number 4 because Log Likelihood decreased by less than.01 percent. -2 Log Likelihood Goodness of Fit Cox & Snell - R^2.239 Nagelkerke - R^2.319 Chi-Square df Significance Chi-Square df Significance Model Block Step Appendix D: Exemplary SPSS Analysis for Multiple Predictors file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (7 of 9) [4/23/ :31:29 AM]

8 Variables in the Equation Variable B S.E. Wald df Sig R Exp(B) WEIGHT GESTAGE APGAR SEX Constant file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (8 of 9) [4/23/ :31:29 AM]

9 Figure 1 file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (9 of 9) [4/23/ :31:29 AM]

### ANNOTATED OUTPUT--SPSS Logistic Regression

Logistic Regression Logistic regression is a variation of the regression model. It is used when the dependent response variable is binary in nature. Logistic regression predicts the probability of the

### Overview Classes. 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7)

Overview Classes 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7) 2-4 Loglinear models (8) 5-4 15-17 hrs; 5B02 Building and

### Wednesday PM. Multiple regression. Multiple regression in SPSS. Presentation of AM results Multiple linear regression. Logistic regression

Wednesday PM Presentation of AM results Multiple linear regression Simultaneous Stepwise Hierarchical Logistic regression Multiple regression Multiple regression extends simple linear regression to consider

### Logistic Regression. Introduction. The Purpose Of Logistic Regression

Logistic Regression...1 Introduction...1 The Purpose Of Logistic Regression...1 Assumptions Of Logistic Regression...2 The Logistic Regression Equation...3 Interpreting Log Odds And The Odds Ratio...4

### 1/2/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 When and why do we use logistic regression? Binary Multinomial Theory behind logistic regression Assessing the model Assessing predictors

### (Logistic Regression) 2 *! %\$%!&2.!0(%%##%#\$" 2 \$ (dichotomy) %%!0(%-"! #\$""\$1 \$%,#*"./'(0(6 *-16*7" 22% "1 / 4"2'(0(-17")* "(4(*:#5* \$" \$2

(Logistic Regression)!"#\$ %!&'()* *%!+,-*("./.\$ Multiple Regression +%!& %%!*"0(%-"! %\$..(/" + 2 *! %\$%!&2.!0(%%##%#\$" 2 \$ (dichotomy) %%!0(%-"! #\$""\$ '()*\$%#\$"#0( 2 \$(4( 54( \$#%)#*" \$"-\$(*( \$%,#*"./'(0(6

### Chapter 6: Answers. Omnibus Tests of Model Coefficients. Chi-square df Sig Step Block Model.

Task Chapter 6: Answers Recent research has shown that lecturers are among the most stressed workers. A researcher wanted to know exactly what it was about being a lecturer that created this stress and

### 11. Analysis of Case-control Studies Logistic Regression

Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

### CRJ Doctoral Comprehensive Exam Statistics Friday August 23, :00pm 5:30pm

CRJ Doctoral Comprehensive Exam Statistics Friday August 23, 23 2:pm 5:3pm Instructions: (Answer all questions below) Question I: Data Collection and Bivariate Hypothesis Testing. Answer the following

### Multinomial and Ordinal Logistic Regression

Multinomial and Ordinal Logistic Regression ME104: Linear Regression Analysis Kenneth Benoit August 22, 2012 Regression with categorical dependent variables When the dependent variable is categorical,

### Logistic Regression. Introduction CHAPTER The Logistic Regression Model 14.2 Inference for Logistic Regression

Logistic Regression Introduction The simple and multiple linear regression methods we studied in Chapters 10 and 11 are used to model the relationship between a quantitative response variable and one or

### Survival Analysis Using SPSS. By Hui Bian Office for Faculty Excellence

Survival Analysis Using SPSS By Hui Bian Office for Faculty Excellence Survival analysis What is survival analysis Event history analysis Time series analysis When use survival analysis Research interest

### The Probit Link Function in Generalized Linear Models for Data Mining Applications

Journal of Modern Applied Statistical Methods Copyright 2013 JMASM, Inc. May 2013, Vol. 12, No. 1, 164-169 1538 9472/13/\$95.00 The Probit Link Function in Generalized Linear Models for Data Mining Applications

### Multivariate Logistic Regression

1 Multivariate Logistic Regression As in univariate logistic regression, let π(x) represent the probability of an event that depends on p covariates or independent variables. Then, using an inv.logit formulation

### Binary Logistic Regression

Binary Logistic Regression Main Effects Model Logistic regression will accept quantitative, binary or categorical predictors and will code the latter two in various ways. Here s a simple model including

### Ordinal Regression. Chapter

Ordinal Regression Chapter 4 Many variables of interest are ordinal. That is, you can rank the values, but the real distance between categories is unknown. Diseases are graded on scales from least severe

### A correlation exists between two variables when one of them is related to the other in some way.

Lecture #10 Chapter 10 Correlation and Regression The main focus of this chapter is to form inferences based on sample data that come in pairs. Given such paired sample data, we want to determine whether

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### STAT 5817: Logistic Regression and Odds Ratio

A cohort of people is a group of people whose membership is clearly defined. A prospective study is one in which a cohort of people is followed for the occurrence or nonoccurrence of specified endpoints

### Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.)

Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.) Logistic regression generalizes methods for 2-way tables Adds capability studying several predictors, but Limited to

### 11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables.

Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection

### Statistics in Retail Finance. Chapter 2: Statistical models of default

Statistics in Retail Finance 1 Overview > We consider how to build statistical models of default, or delinquency, and how such models are traditionally used for credit application scoring and decision

### Generalized Linear Models

Generalized Linear Models We have previously worked with regression models where the response variable is quantitative and normally distributed. Now we turn our attention to two types of models where the

### Multiple logistic regression analysis of cigarette use among high school students

Multiple logistic regression analysis of cigarette use among high school students ABSTRACT Joseph Adwere-Boamah Alliant International University A binary logistic regression analysis was performed to predict

### Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study)

Cairo University Faculty of Economics and Political Science Statistics Department English Section Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study) Prepared

### Statistics 305: Introduction to Biostatistical Methods for Health Sciences

Statistics 305: Introduction to Biostatistical Methods for Health Sciences Modelling the Log Odds Logistic Regression (Chap 20) Instructor: Liangliang Wang Statistics and Actuarial Science, Simon Fraser

### Application of Odds Ratio and Logistic Models in Epidemiology and Health Research

Application of Odds Ratio and Logistic Models in Epidemiology and Health Research By: Min Qi Wang, PhD; James M. Eddy, DEd; Eugene C. Fitzhugh, PhD Wang, M.Q., Eddy, J.M., & Fitzhugh, E.C.* (1995). Application

### Logistic Regression: Basics

Evidence is no evidence if based solely on p value Logistic Regression: Basics Prediction Model: Binary Outcomes Nemours Stats 101 Laurens Holmes, Jr. General Linear Model OUTCOME Continuous Counts Data

### Paper Let the Data Speak: New Regression Diagnostics Based on Cumulative Residuals

Paper 255-28 Let the Data Speak: New Regression Diagnostics Based on Cumulative Residuals Gordon Johnston and Ying So SAS Institute Inc. Cary, North Carolina, USA Abstract Residuals have long been used

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### %DISCIT Macro: Pre-screening continuous variables for subsequent binary logistic regression analysis through visualization

ABSTRACT Paper 92 %DISCIT Macro: Pre-screening continuous variables for subsequent binary logistic regression analysis through visualization Mohamed S. Anany, Videa (Cox Media Group subsidiary) In binary

### DEPARTMENT OF HEALTH AND HUMAN SCIENCES HS900 RESEARCH METHODS

DEPARTMENT OF HEALTH AND HUMAN SCIENCES HS900 RESEARCH METHODS Using SPSS Session 2 Topics addressed today: 1. Recoding data missing values, collapsing categories 2. Making a simple scale 3. Standardisation

### Two Correlated Proportions (McNemar Test)

Chapter 50 Two Correlated Proportions (Mcemar Test) Introduction This procedure computes confidence intervals and hypothesis tests for the comparison of the marginal frequencies of two factors (each with

### Module 4 - Multiple Logistic Regression

Module 4 - Multiple Logistic Regression Objectives Understand the principles and theory underlying logistic regression Understand proportions, probabilities, odds, odds ratios, logits and exponents Be

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### JOB SATISFACTION AMONG FEMALE TEACHERS IN PUBLIC AND PRIVATE SECTORS (A CROSS-SECTIONAL STUDY)

Volume 2, 2013, Page 88-97 JOB SATISFACTION AMONG FEMALE TEACHERS IN PUBLIC AND PRIVATE SECTORS (A CROSS-SECTIONAL STUDY) Naila Amjad, Shazia Qasim Department of Statistics, Lahore College for Women University,

### SPSS Multivariable Linear Models and Logistic Regression

1 SPSS Multivariable Linear Models and Logistic Regression Multivariable Models Single continuous outcome (dependent variable), one main exposure (independent) variable, and one or more potential confounders

### Yiming Peng, Department of Statistics. February 12, 2013

Regression Analysis Using JMP Yiming Peng, Department of Statistics February 12, 2013 2 Presentation and Data http://www.lisa.stat.vt.edu Short Courses Regression Analysis Using JMP Download Data to Desktop

### The first three steps in a logistic regression analysis with examples in IBM SPSS. Steve Simon P.Mean Consulting www.pmean.com

The first three steps in a logistic regression analysis with examples in IBM SPSS. Steve Simon P.Mean Consulting www.pmean.com 2. Why do I offer this webinar for free? I offer free statistics webinars

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Loglinear Models. Angela Jeansonne. Brief History. Overview. This page last updated

Loglinear Models Angela Jeansonne This page last updated Brief History Until the late 1960 s, contingency tables - two-way tables formed by cross classifying categorical variables - were typically analyzed

### 5. Ordinal regression: cumulative categories proportional odds. 6. Ordinal regression: comparison to single reference generalized logits

Lecture 23 1. Logistic regression with binary response 2. Proc Logistic and its surprises 3. quadratic model 4. Hosmer-Lemeshow test for lack of fit 5. Ordinal regression: cumulative categories proportional

### Research Variables. Measurement. Scales of Measurement. Chapter 4: Data & the Nature of Measurement

Chapter 4: Data & the Nature of Graziano, Raulin. Research Methods, a Process of Inquiry Presented by Dustin Adams Research Variables Variable Any characteristic that can take more than one form or value.

### Logit Models for Binary Data

Chapter 3 Logit Models for Binary Data We now turn our attention to regression models for dichotomous data, including logistic regression and probit analysis. These models are appropriate when the response

### SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

### 7. Tests of association and Linear Regression

7. Tests of association and Linear Regression In this chapter we consider 1. Tests of Association for 2 qualitative variables. 2. Measures of the strength of linear association between 2 quantitative variables.

### Review of Fundamental Mathematics

Review of Fundamental Mathematics As explained in the Preface and in Chapter 1 of your textbook, managerial economics applies microeconomic theory to business decision making. The decision-making tools

### Biostatistics 305. Multinomial logistic regression

B a s i c S t a t i s t i c s F o r D o c t o r s Singapore Med J 2005; 46(6) : 259 CME Article Biostatistics 305. Multinomial logistic regression Y H Chan Multinomial logistic regression is the extension

### Data Analysis for categorical variables and its application to happiness studies

Data Analysis for categorical variables and its application to happiness studies Thanawit Bunsit Department of Economics, University of Bath The economics of happiness and wellbeing workshop: building

### Logistic Regression. http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests

Logistic Regression http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests Overview Binary (or binomial) logistic regression is a form of regression which is used when the dependent is a dichotomy

### Linear Regression in SPSS

Linear Regression in SPSS Data: mangunkill.sav Goals: Examine relation between number of handguns registered (nhandgun) and number of man killed (mankill) checking Predict number of man killed using number

### Lecture 12: Generalized Linear Models for Binary Data

Lecture 12: Generalized Linear Models for Binary Data Dipankar Bandyopadhyay, Ph.D. BMTRY 711: Analysis of Categorical Data Spring 2011 Division of Biostatistics and Epidemiology Medical University of

### Simple Linear Regression One Binary Categorical Independent Variable

Simple Linear Regression Does sex influence mean GCSE score? In order to answer the question posed above, we want to run a linear regression of sgcseptsnew against sgender, which is a binary categorical

### Algebra I: Lesson 5-4 (5074) SAS Curriculum Pathways

Two-Variable Quantitative Data: Lesson Summary with Examples Bivariate data involves two quantitative variables and deals with relationships between those variables. By plotting bivariate data as ordered

### LEARNING OBJECTIVES SCALES OF MEASUREMENT: A REVIEW SCALES OF MEASUREMENT: A REVIEW DESCRIBING RESULTS DESCRIBING RESULTS 8/14/2016

UNDERSTANDING RESEARCH RESULTS: DESCRIPTION AND CORRELATION LEARNING OBJECTIVES Contrast three ways of describing results: Comparing group percentages Correlating scores Comparing group means Describe

### II. DISTRIBUTIONS distribution normal distribution. standard scores

Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,

### CHAPTER 11 CHI-SQUARE: NON-PARAMETRIC COMPARISONS OF FREQUENCY

CHAPTER 11 CHI-SQUARE: NON-PARAMETRIC COMPARISONS OF FREQUENCY The hypothesis testing statistics detailed thus far in this text have all been designed to allow comparison of the means of two or more samples

### INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

### CHAPTER 2 AND 10: Least Squares Regression

CHAPTER 2 AND 0: Least Squares Regression In chapter 2 and 0 we will be looking at the relationship between two quantitative variables measured on the same individual. General Procedure:. Make a scatterplot

### It is important to bear in mind that one of the first three subscripts is redundant since k = i -j +3.

IDENTIFICATION AND ESTIMATION OF AGE, PERIOD AND COHORT EFFECTS IN THE ANALYSIS OF DISCRETE ARCHIVAL DATA Stephen E. Fienberg, University of Minnesota William M. Mason, University of Michigan 1. INTRODUCTION

### where b is the slope of the line and a is the intercept i.e. where the line cuts the y axis.

Least Squares Introduction We have mentioned that one should not always conclude that because two variables are correlated that one variable is causing the other to behave a certain way. However, sometimes

### Chapter 8 Hypothesis Testing

Chapter 8 Hypothesis Testing Chapter problem: Does the MicroSort method of gender selection increase the likelihood that a baby will be girl? MicroSort: a gender-selection method developed by Genetics

3.2 LOGARITHMIC FUNCTIONS AND THEIR GRAPHS Copyright Cengage Learning. All rights reserved. What You Should Learn Recognize and evaluate logarithmic functions with base a. Graph logarithmic functions.

### Lecture - 32 Regression Modelling Using SPSS

Applied Multivariate Statistical Modelling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 32 Regression Modelling Using SPSS (Refer

### High School Algebra 1 Common Core Standards & Learning Targets

High School Algebra 1 Common Core Standards & Learning Targets Unit 1: Relationships between Quantities and Reasoning with Equations CCS Standards: Quantities N-Q.1. Use units as a way to understand problems

### SUMAN DUVVURU STAT 567 PROJECT REPORT

SUMAN DUVVURU STAT 567 PROJECT REPORT SURVIVAL ANALYSIS OF HEROIN ADDICTS Background and introduction: Current illicit drug use among teens is continuing to increase in many countries around the world.

### III. INTRODUCTION TO LOGISTIC REGRESSION. a) Example: APACHE II Score and Mortality in Sepsis

III. INTRODUCTION TO LOGISTIC REGRESSION 1. Simple Logistic Regression a) Example: APACHE II Score and Mortality in Sepsis The following figure shows 30 day mortality in a sample of septic patients as

### Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

### VI. Introduction to Logistic Regression

VI. Introduction to Logistic Regression We turn our attention now to the topic of modeling a categorical outcome as a function of (possibly) several factors. The framework of generalized linear models

### Lecture 16: Logistic regression diagnostics, splines and interactions. Sandy Eckel 19 May 2007

Lecture 16: Logistic regression diagnostics, splines and interactions Sandy Eckel seckel@jhsph.edu 19 May 2007 1 Logistic Regression Diagnostics Graphs to check assumptions Recall: Graphing was used to

### SPSS: Descriptive and Inferential Statistics. For Windows

For Windows August 2012 Table of Contents Section 1: Summarizing Data...3 1.1 Descriptive Statistics...3 Section 2: Inferential Statistics... 10 2.1 Chi-Square Test... 10 2.2 T tests... 11 2.3 Correlation...

### Semester 1 Statistics Short courses

Semester 1 Statistics Short courses Course: STAA0001 Basic Statistics Blackboard Site: STAA0001 Dates: Sat. March 12 th and Sat. April 30 th (9 am 5 pm) Assumed Knowledge: None Course Description Statistical

### Tips for surviving the analysis of survival data. Philip Twumasi-Ankrah, PhD

Tips for surviving the analysis of survival data Philip Twumasi-Ankrah, PhD Big picture In medical research and many other areas of research, we often confront continuous, ordinal or dichotomous outcomes

### Bivariate Statistics Session 2: Measuring Associations Chi-Square Test

Bivariate Statistics Session 2: Measuring Associations Chi-Square Test Features Of The Chi-Square Statistic The chi-square test is non-parametric. That is, it makes no assumptions about the distribution

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Hypothesis Testing - Relationships

- Relationships Session 3 AHX43 (28) 1 Lecture Outline Correlational Research. The Correlation Coefficient. An example. Considerations. One and Two-tailed Tests. Errors. Power. for Relationships AHX43

### 13. Poisson Regression Analysis

136 Poisson Regression Analysis 13. Poisson Regression Analysis We have so far considered situations where the outcome variable is numeric and Normally distributed, or binary. In clinical work one often

### Minitab Guide. This packet contains: A Friendly Guide to Minitab. Minitab Step-By-Step

Minitab Guide This packet contains: A Friendly Guide to Minitab An introduction to Minitab; including basic Minitab functions, how to create sets of data, and how to create and edit graphs of different

### Equations. #1-10 Solve for the variable. Inequalities. 1. Solve the inequality: 2 5 7. 2. Solve the inequality: 4 0

College Algebra Review Problems for Final Exam Equations #1-10 Solve for the variable 1. 2 1 4 = 0 6. 2 8 7 2. 2 5 3 7. = 3. 3 9 4 21 8. 3 6 9 18 4. 6 27 0 9. 1 + log 3 4 5. 10. 19 0 Inequalities 1. Solve

### Statistics in Retail Finance. Chapter 6: Behavioural models

Statistics in Retail Finance 1 Overview > So far we have focussed mainly on application scorecards. In this chapter we shall look at behavioural models. We shall cover the following topics:- Behavioural

### There are some general common sense recommendations to follow when presenting

Presentation of Data The presentation of data in the form of tables, graphs and charts is an important part of the process of data analysis and report writing. Although results can be expressed within

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

### Categorical Data Analysis: Logistic Regression

Categorical Data Analysis: Logistic Regression Haitao Chu, M.D., Ph.D. University of Minnesota H. Chu (UM) PubH 7406: Advanced Regression 1 / 30 5.1 Model Interpretation 5.1.1 Model Interpretation The

### How Do We Test Multiple Regression Coefficients?

How Do We Test Multiple Regression Coefficients? Suppose you have constructed a multiple linear regression model and you have a specific hypothesis to test which involves more than one regression coefficient.

### Simple Linear Regression

Inference for Regression Simple Linear Regression IPS Chapter 10.1 2009 W.H. Freeman and Company Objectives (IPS Chapter 10.1) Simple linear regression Statistical model for linear regression Estimating

### Calculating the Probability of Returning a Loan with Binary Probability Models

Calculating the Probability of Returning a Loan with Binary Probability Models Associate Professor PhD Julian VASILEV (e-mail: vasilev@ue-varna.bg) Varna University of Economics, Bulgaria ABSTRACT The

### The three statistics of interest comparing the exposed to the not exposed are:

Review: Common notation for a x table: Not Exposed Exposed Disease a b a+b No Disease c d c+d a+c b+d N Warning: Rosner presents the transposed table so b and c have their meanings reversed in the text.

### Florida Department of Education/Office of Assessment January 2012. Grade 7 FCAT 2.0 Mathematics Achievement Level Descriptions

Florida Department of Education/Office of Assessment January 2012 Grade 7 FCAT 2.0 Mathematics Grade 7 FCAT 2.0 Mathematics Reporting Category Geometry and Measurement Students performing at the mastery

### Regression Analysis: A Complete Example

Regression Analysis: A Complete Example This section works out an example that includes all the topics we have discussed so far in this chapter. A complete example of regression analysis. PhotoDisc, Inc./Getty

### Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@

Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@ Yanchun Xu, Andrius Kubilius Joint Commission on Accreditation of Healthcare Organizations,

### Statistics revision. Dr. Inna Namestnikova. Statistics revision p. 1/8

Statistics revision Dr. Inna Namestnikova inna.namestnikova@brunel.ac.uk Statistics revision p. 1/8 Introduction Statistics is the science of collecting, analyzing and drawing conclusions from data. Statistics

### Elementary Statistics. Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination

Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination What is a Scatter Plot? A Scatter Plot is a plot of ordered pairs (x, y) where the horizontal axis is used

### Statistics and research

Statistics and research Usaneya Perngparn Chitlada Areesantichai Drug Dependence Research Center (WHOCC for Research and Training in Drug Dependence) College of Public Health Sciences Chulolongkorn University,

### Logistic Regression 1. y log( ) logit( y) 1 y = = +

Written by: Robin Beaumont e-mail: robin@organplayers.co.uk Date Thursday, 05 July 2012 Version: 3 Logistic Regression model Can be converted to a probability dependent variable = outcome/event log odds

### UNDERSTANDING MULTIPLE REGRESSION

UNDERSTANDING Multiple regression analysis (MRA) is any of several related statistical methods for evaluating the effects of more than one independent (or predictor) variable on a dependent (or outcome)

### The scatterplot indicates a positive linear relationship between waist size and body fat percentage:

STAT E-150 Statistical Methods Multiple Regression Three percent of a man's body is essential fat, which is necessary for a healthy body. However, too much body fat can be dangerous. For men between the

### A Tutorial on Logistic Regression

A Tutorial on Logistic Regression Ying So, SAS Institute Inc., Cary, NC ABSTRACT Many procedures in SAS/STAT can be used to perform logistic regression analysis: CATMOD, GENMOD,LOGISTIC, and PROBIT. Each