# LOGISTIC REGRESSION ANALYSIS

Size: px
Start display at page:

## Transcription

1 LOGISTIC REGRESSION ANALYSIS C. Mitchell Dayton Department of Measurement, Statistics & Evaluation Room 1230D Benjamin Building University of Maryland September Introduction and Model Logistic regression analysis (LRA) extends the techniques of multiple regression analysis to research situations in which the outcome variable is categorical. In practice, situations involving categorical outcomes are quite common. In the setting of evaluating an educational program, for example, predictions may be made for the dichotomous outcome of success/failure or improved/not-improved. Similarly, in a medical setting, an outcome might be presence/absence of disease. The focus of this document is on situations in which the outcome variable is dichotomous, although extension of the techniques of LRA to outcomes with three or more categories (e.g., improved, same, or worse) is possible (see, for example, Hosmer & Lemeshow, 1989, Chapter 8). In this section, we review the multiple regression model and, then, present the model for LRA. The fundamental model underlying multiple regression analysis (MRA) posits that a continuous outcome variable is, in theory, a linear combination of a set of predictors and error. Thus, for an outcome variable, Y, and a set of p predictor variables, X 1,...,X p, the MRA model is of the form: where is the Y-intercept (i.e., the expected value of Y when all X's are set to 0), β j is a multiple (partial) regression coefficient (i.e., the expected change in Y per unit change in X j assuming all other X's are held constant) and ε is the error of prediction. If error is omitted, the resulting model represents the expected, or predicted, value of Y: Note that Y=Y +ε. Thus, we can interpret the MRA model as follows: each observed score, Y, is made up of an expected, or predictable, component, Y, that is a function of the predictor variables X 1,...,X p, and an error, or unpredictable component, ε, that represents error of measurement (i.e., unreliability) and/or error in the selection of the model (i.e., mis-specification). The MRA model summarized above is applicable when the outcome variable, Y, is continuous, but is not appropriate for situations in which Y is categorical. For example, if Y takes on the value 1 for "success" and 0 for "failure," the multiple regression equation would not result in predicted values restricted to exactly 1 or 0. In fact, these predicted values would be spread out over an interval that contains uninterpretable values such as.5 or.33 and could even include negative values and/or values greater than 1. The model for logistic regression analysis, described below, is a more realistic representation of the situation when an outcome variable is categorical. The model for logistic regression analysis assumes that the outcome variable, Y, is categorical (e.g., dichotomous), but LRA does not model this outcome variable directly. Rather, LRA is based on probabilities associated with the values of Y. For simplicity, and because it is the case most commonly encountered in practice, we assume that Y is dichotomous, taking on values of 1 (i.e., the positive outcome, or success) and 0 (i.e., the negative outcome, or failure). In theory, the hypothetical, population proportion of cases for which Y = 1 is defined as π = P(Y =1). Then, the theoretical proportion of cases for which Y = 0 is 1 - π = P(Y = 0). In the absence of other information, we would estimate π by the sample proportion of cases for which Y = 1. However, in the regression context, it is assumed that there is a set of predictor variables, X 1,...,X p, that are related to Y and, therefore, provide additional information for predicting Y. For theoretical, mathematical reasons, LRA is based on a linear model for the natural logarithm of the odds (i.e., the log-odds) in favor of Y = 1 (see Appendix A for a further explanation of odds): file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (1 of 9) [4/23/ :31:29 AM]

2 Note that in the LRA model, π is a conditional probability of the form P(Y=1 X 1,...,X p ). That is, it is assumed that "success" is more or less likely depending on combinations of values of the predictor variables. The log-odds, as defined above is also known as the logit transformation of π and the analytical approach described here is sometimes known as logit analysis. The LRA model above is identical to the MRA model except that the log-odds in favor of Y = 1 replaces the expected value of Y. There are two basic reasons underlying the development of the model above. First, probabilities and odds obey multiplicative, rather than additive, rules. However, taking the logarithm of the odds allows for the simpler, additive model since logarithms convert multiplication into addition. And, second, there is a (relatively) simple exponential transformation for converting log-odds back to probability. In particular, the inverse transformation is the logistic function of the form: Due to the mathematical relationship, e a /(1+e a ) = 1/(1+e -a ), the logistic function for LRA is sometimes presented in the form: Due to the mathematical relation, 1- e a /(1+e a ) = 1/(1+e a ), the probability for a 0 response is: 2. Fitting the LRA Model to Data As in MRA, there are two important stages in the analysis of data. First, estimates for the parameters in the model must be obtained and, second, some determination must be made of how well the model actually fits the observed data. In MRA, the parameter estimates are obtained using the least-squares principle and assessment of fit is based on significance tests for the regression coefficients as well as on interpreting the multiple correlation coefficient. The process is analogous for LRA, although specific details of these stages of analysis are somewhat different as outlined in this section. The parameters that must be estimated from the available data are the constant, α, and the logistic regression coefficients, β j. Because of the nature of the model, estimation is based on the maximum likelihood principle rather than on the least-squares principle. In the context of LRA, maximum likelihood estimation (MLE) involves the following. First, we define the likelihood, L, of the sample data as the product, across all sampled cases, of the probabilities for success or for failure: Note that Y is the 0/1 outcome for the i th case and, X i1,...,x ip are the values of the predictor variables for the i th case based on a sample of n cases. The use of Y i and 1-Y i as exponents in the equation above includes in the likelihood the appropriate probability term dependent upon whether Y i =1 or Y i =0 (note that F 0 =1 for any expression, F). Using the methods of calculus, a set of values for α and the β j can be calculated that maximize L and these resulting values are known as maximum likelihood estimates (MLE s). This maximization process is somewhat more complicated than the corresponding minimization procedure in MRA for finding least-square estimates. However, the general approach involves establishing initial guesses for the unknown parameters and then continuously adjusting these estimates until the maximum value of L is found. This iterative solution procedure is available in popular statistical procedures such as those found in SPSS and SAS. file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (2 of 9) [4/23/ :31:29 AM]

3 To distinguish them from parameters, we denote the MLE s as a and b j. Given that these estimates have been calculated for a real data set, tests of significance for individual logistic regression coefficients can be set up as in MRA. That is, for the hypothesis, H: β j = 0, a statistic of the form z = b j /S j can be calculated based on the estimated standard error, S j, for b j (SPSS and SAS report χ 1 2 = z 2 and label these values as Wald statistics). Similarly, the usefulness of the model as a whole can be assessed by testing the hypothesis that, simultaneously, all of the partial logistic regression coefficients are 0; i.e., H: β j = 0 for all j. In effect, we can compare the general model given above with the restricted model Log e (π /(1-π )) = α. This test, that is equivalent to testing the significance of the multiple R in MRA, is based on a chi-squared statistic (SPSS labels this value as "Model Chi-Square"). Finally, different LRA models fitted to the same set of data can be compared statistically in a simple manner if the models are hierarchical. The hierarchy principle requires that the model with the larger number of predictors include among its predictors all of the predictors from the simpler model (e.g., predictors X 1 and X 2 in the simpler model and predictors X 1, X 2, X 3 and X 4 in the more complex model). Given this condition, the difference in model chi-squared values is (approximately) distributed as chi-squared with degrees of freedom equal to the difference in degrees of freedom for the two models (e.g., for the above example, the degrees of freedom would be 2). In effect, this procedure tests a conditional null hypothesis that, for the example, would be: H: β 3 = β 4 = 0 β 1, β 2. That is, the values of the logistic regression coefficients associated with X 1 and X 2 are unrestricted, but the logistic regression coefficients associated with X 3 and X 4 are assumed by hypothesis to be 0. If the models are specified in a series of "blocks" in SPSS, an "Improvement" chi-square value is computed for each successive model and this can be used to test whether or not the additional predictors result in significantly better fit of the model to the data. 3. Interpreting the Fitted LRA Model We may interpret the results from a LRA at three different levels. First, each term in the equation represents contributions to estimated log-odds. Thus, for each one unit increase (decrease) in X j, there is predicted to be an increase (decrease) of b j units in the log-odds in favor of Y = 1. Also, if all predictors are set equal to 0, the predicted log-odds in favor of Y = 1 would be the constant term, a. Second, since most people do not find it natural to think in terms of log-odds, the LRA equation can be transformed to odds by exponentiation: With respect to odds, the influence of each predictor is multiplicative. Thus, for each one unit increase in X j, the predicted odds is increased by a factor of exp(b j ) If X is decreased by one unit, the multiplicative factor is exp(-b j ). Note that exp(c) = e c. Similarly, if all predictors are set equal to 0, the predicted odds are exp(a). Finally, the results can be expressed in terms of probabilities by use of the logistic function. With estimates substituted the equation above becomes: Although this form of the model may seem simplest because it results in predicted probabilities for Y = 1, there is no simple interpretation for the logistic regression coefficients in this form. However, for the case of a single predictor, X, a plot can be generated with X represented by the abscissa (i.e., horizontal axis) and P(Y X) represented by the ordinate (i.e., vertical axis). The resulting curve is S-shaped and is illustrated in one of the exemplary analyses presented below. 4. Illustrative Examples The database used for purposes of illustration is provided with SPSS and is based on 50 premature infants, some of whom did not survive infancy. The SPSS setup for this database, including the raw data, is contained in Appendix B. The outcome variable is dichotomous with "1" representing survival of the infant and "0" representing death of the infant. For the first exemplary analysis, a single independent variable, weight at birth in kilograms, was utilized. For the second analysis, three additional predictors were entered as described below. (1) Single Predictor Example Using birth weight as the predictor, the logistic regression equation for log-odds in favor of survival is estimated to be (the SPSS LOGISTIC REGRESSION printout for this block is provided in Appendix C): To estimate odds per se, the equation is exponentiated: file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (3 of 9) [4/23/ :31:29 AM]

4 Finally, the probability of survival is obtained by applying the logistic transformation: In the database, birth weight, in kilograms, ranged from 1.03 to 3.64 (i.e., from 2.27 to 8.02 pounds). Thus, for the lowest weight recorded, 1.03, the log-odds in favor of survival are , the odds in favor of survival are.1640 and probability of survival is.14. At the other extreme, for birth weight of 3.64 kilograms, the log-odds in favor of survival are , the odds in favor of survival are and the probability of survival is.94. The relation between birth weight and survival of infants is displayed in Figure 1. This graph was constructed by systematically varying birth weight from 1 to 4 kilograms (shown on the abscissa) and calculating the estimated probability of survival (shown on the ordinate). Note from reading the graph that infants with a birth weight of slightly over 2 kilograms (actually, 2.07) have an estimated odds of 1:1 in favor of survival (i.e., for π is estimated to equal.5 for a birth weight of 2.07 kilograms). The estimated logistic regression coefficient for birth weight is and the exponential of this value is e = This indicates that for a one kilogram increase in birth weight, the odds in favor of survival are estimated to be increased by a multiplicative factor of 5.7. The reported standard error for b is.5786 and statistical significance can be assessed by the Wald chi-squared statistic, (1.7406/.5786) 2 = 9.05, that, with 1 degree of freedom, is significant at conventional levels (the empirical two-tailed p-value is reported to be.0026 in Appendix C). Thus, this study supports the conclusion that birth weight is a useful predictor of infant survival. To aid in interpretation, a classification table can be constructed by predicting the survival or death of each infant based on whether or not the odds for survival are greater or less than 1.0, and comparing these predictions to the actual outcome for each infant. As shown in Appendix C, the percents of correct decisions are 56.5 for infants who survived, 74.1 for infants who died, and 66.0 overall. This overall result can be compared with a rate of 54.0% that would be obtained by simply predicting death as the outcome for every infant (i.e., since 27 of the 50 infants died, this is the prediction with the greater likelihood of being correct). (2) Multiple Predictor Example Building on the previous example, the following three predictors, in addition to birth weight, were included in the model: gestation age, sex of infant, and the APGAR score (a measure of infant well being). Using this set of four predictors, the logistic regression equation for the log-odds in favor of survival is estimated to be (note: the SPSS LOGISTIC REGRESSION printout for this block is provided in Appendix D): To estimate odds, the equation is exponentiated: Finally, the probability of survival is obtained by applying the logistic transformation: Unlike the case of a single predictor, there is no simple graphical display that can be used. However, log-odds, odds and probabilities can be calculated for various combinations of the predictors. In the present case, there is more interest in deciding whether or not the three predictors, in addition to birth weight, have improved the predictive efficiency of the model. The Wald chi-squared statistics, as taken from Appendix D, are non-significant for age, sex and APGAR (i.e., p-values of.23,.60 and.58, respectively), whereas the chi-squared value for weight is significant at the.05 level (i.e., p-value of.02). Thus, given that the other predictors remain in the model, removing weight as a predictor would result in significantly poorer predictive efficiency, although removing any of the other predictors does not have a significant impact. A second, global way of comparing the single and multiple predictor models is based on the improvement chi-squared value that is reported as based on 3 degrees of freedom This value is non-significant at conventional levels and, thus, we conclude that the predictive efficiency of the 4-predictor model is no greater than that of the 1-predictor model. It should be noted that the classification table results are exactly the same using four predictors as for using only a single predictor (e.g., 66.0% correct classifications overall). file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (4 of 9) [4/23/ :31:29 AM]

5 Suggested References Fienberg, S. E. (1980). The Analysis of Cross-Classified Categorical Data (Second Edition). Cambridge, MA: The MIT Press Fleiss, J. (1981). Statistical Methods for Rates and Proportions (Second Edition). New York: Wiley Hosmer, D. W. & Lemeshow, S. (1989). Applied Logistic Regression, New York: Wiley. Appendix A: The Meaning of Odds Odds represent the relative frequency with which different outcomes occur. Odds are sometimes expressed as a ratio of the form a:b. For example, odds of 3:1 in favor of the first outcome means that the first outcome occurs 3 times for each single occurrence of the second outcome. Similarly, odds of 5:2 means that the first outcome occurs 5 times for each 2 occurrences of the second outcome. Odds are directly related to probabilities and can be translated back and forth using these relations: Probability = a/(a+b) when odds are expressed as a:b; or Probability = Odds/(1 + Odds) when odds are expressed in decimal form (e.g., 3:1 becomes 3.0 and 5:2 becomes 2.5) Odds = Probability/(1 - Probability ) Some examples: The probability of rolling a 1 with a true die is 1/6 = The odds in favor of rolling a one are 1/6 5/6 =.20, or 1:5. Viewed the other way, the odds against rolling a 1 with a true die are 5:1, or 5.0. The odds in favor of not drawing a face card from a standard deck of playing cards (i.e., drawing a card other than a Jack, Queen, King or Ace) are 36:16 or 9:4. The corresponding probability is 9/(9+4) = 9/13 = Appendix B: Exemplary Database (SPSS Syntaz) TITLE 'RDS: Respiratory Distress Syndrome'. DATA LIST /CASEID 1-2 SURVIVAL 4 TREATMNT 6 TIME 8-10(1) WEIGHT 12-15(3) APGAR SEX 20 AGE PH 33-35(2) RESP 37. RECODE SURVIVAL (1=2) (0=1). VARIABLE LABELS SURVIVAL 'Infant Survival' TREATMNT 'Treatment Administered' TIME 'Time to Spontaneous Respiration' WEIGHT 'Birthweight in Kilograms' APGAR 'APGAR Score' SEX 'Sex of Infant' PH 'PH Level' RESP 'Respiratory Therapy'. MISSING VALUE RESP(9). VALUE LABELS TREATMNT 1 'Tham' 0 'Sodium Bicarbonate' /SEX 1 'Male' 0 'Female' /RESP 1 'Yes' 0 'No' 9 'No Answer' /SURVIVAL 2 'Survive' 1 'Die'. BEGIN DATA file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (5 of 9) [4/23/ :31:29 AM]

6 END DATA. Appendix C: Exemplary SPSS Analysis for a Single Predictor Total number of cases: 50 (Unweighted) Number of selected cases: 50 Number of unselected cases: 0 Number of selected cases: 50 Number rejected because of missing data: 0 Number of cases included in the analysis: 50 Dependent Variable Encoding: Original Internal Value Value Dependent Variable.. SURVIVE Beginning Block Number 0. Initial Log Likelihood Function -2 Log Likelihood * Constant is included in the model. Beginning Block Number 1. Method: Enter Variable(s) Entered on Step Number 1.. WEIGHT Estimation terminated at iteration number 3 because file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (6 of 9) [4/23/ :31:29 AM]

7 Log Likelihood decreased by less than.01 percent. -2 Log Likelihood Goodness of Fit Cox & Snell - R^2.214 Nagelkerke - R^2.286 Chi-Square df Significance Chi-Square df Significance Model Block Step Variables in the Equation Variable B S.E. Wald df Sig R Exp(B) WEIGHT Constant Beginning Block Number 2. Method: Enter Variable(s) Entered on Step Number 1.. GESTAGE APGAR SEX Estimation terminated at iteration number 4 because Log Likelihood decreased by less than.01 percent. -2 Log Likelihood Goodness of Fit Cox & Snell - R^2.239 Nagelkerke - R^2.319 Chi-Square df Significance Chi-Square df Significance Model Block Step Appendix D: Exemplary SPSS Analysis for Multiple Predictors file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (7 of 9) [4/23/ :31:29 AM]

8 Variables in the Equation Variable B S.E. Wald df Sig R Exp(B) WEIGHT GESTAGE APGAR SEX Constant file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (8 of 9) [4/23/ :31:29 AM]

9 Figure 1 file:///c /Webpage/LRA/LOGISTIC REGRESSION ANALYSIS.htm (9 of 9) [4/23/ :31:29 AM]

### Overview Classes. 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7)

Overview Classes 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7) 2-4 Loglinear models (8) 5-4 15-17 hrs; 5B02 Building and

### 11. Analysis of Case-control Studies Logistic Regression

Research methods II 113 11. Analysis of Case-control Studies Logistic Regression This chapter builds upon and further develops the concepts and strategies described in Ch.6 of Mother and Child Health:

### Multinomial and Ordinal Logistic Regression

Multinomial and Ordinal Logistic Regression ME104: Linear Regression Analysis Kenneth Benoit August 22, 2012 Regression with categorical dependent variables When the dependent variable is categorical,

### Binary Logistic Regression

Binary Logistic Regression Main Effects Model Logistic regression will accept quantitative, binary or categorical predictors and will code the latter two in various ways. Here s a simple model including

### Ordinal Regression. Chapter

Ordinal Regression Chapter 4 Many variables of interest are ordinal. That is, you can rank the values, but the real distance between categories is unknown. Diseases are graded on scales from least severe

### Multiple logistic regression analysis of cigarette use among high school students

Multiple logistic regression analysis of cigarette use among high school students ABSTRACT Joseph Adwere-Boamah Alliant International University A binary logistic regression analysis was performed to predict

### The Probit Link Function in Generalized Linear Models for Data Mining Applications

Journal of Modern Applied Statistical Methods Copyright 2013 JMASM, Inc. May 2013, Vol. 12, No. 1, 164-169 1538 9472/13/\$95.00 The Probit Link Function in Generalized Linear Models for Data Mining Applications

### Multivariate Logistic Regression

1 Multivariate Logistic Regression As in univariate logistic regression, let π(x) represent the probability of an event that depends on p covariates or independent variables. Then, using an inv.logit formulation

### Two Correlated Proportions (McNemar Test)

Chapter 50 Two Correlated Proportions (Mcemar Test) Introduction This procedure computes confidence intervals and hypothesis tests for the comparison of the marginal frequencies of two factors (each with

: Table of Contents... 1 Overview of Model... 1 Dispersion... 2 Parameterization... 3 Sigma-Restricted Model... 3 Overparameterized Model... 4 Reference Coding... 4 Model Summary (Summary Tab)... 5 Summary

### The first three steps in a logistic regression analysis with examples in IBM SPSS. Steve Simon P.Mean Consulting www.pmean.com

The first three steps in a logistic regression analysis with examples in IBM SPSS. Steve Simon P.Mean Consulting www.pmean.com 2. Why do I offer this webinar for free? I offer free statistics webinars

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

### Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.)

Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.) Logistic regression generalizes methods for 2-way tables Adds capability studying several predictors, but Limited to

### Statistics in Retail Finance. Chapter 2: Statistical models of default

Statistics in Retail Finance 1 Overview > We consider how to build statistical models of default, or delinquency, and how such models are traditionally used for credit application scoring and decision

### II. DISTRIBUTIONS distribution normal distribution. standard scores

Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,

### Generalized Linear Models

Generalized Linear Models We have previously worked with regression models where the response variable is quantitative and normally distributed. Now we turn our attention to two types of models where the

### Review of Fundamental Mathematics

Review of Fundamental Mathematics As explained in the Preface and in Chapter 1 of your textbook, managerial economics applies microeconomic theory to business decision making. The decision-making tools

### Logit Models for Binary Data

Chapter 3 Logit Models for Binary Data We now turn our attention to regression models for dichotomous data, including logistic regression and probit analysis. These models are appropriate when the response

### Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study)

Cairo University Faculty of Economics and Political Science Statistics Department English Section Students' Opinion about Universities: The Faculty of Economics and Political Science (Case Study) Prepared

### Statistics 305: Introduction to Biostatistical Methods for Health Sciences

Statistics 305: Introduction to Biostatistical Methods for Health Sciences Modelling the Log Odds Logistic Regression (Chap 20) Instructor: Liangliang Wang Statistics and Actuarial Science, Simon Fraser

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### This unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.

Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course

### Chapter Seven. Multiple regression An introduction to multiple regression Performing a multiple regression on SPSS

Chapter Seven Multiple regression An introduction to multiple regression Performing a multiple regression on SPSS Section : An introduction to multiple regression WHAT IS MULTIPLE REGRESSION? Multiple

### Introduction to Quantitative Methods

Introduction to Quantitative Methods October 15, 2009 Contents 1 Definition of Key Terms 2 2 Descriptive Statistics 3 2.1 Frequency Tables......................... 4 2.2 Measures of Central Tendencies.................

### INTRODUCTORY STATISTICS

INTRODUCTORY STATISTICS FIFTH EDITION Thomas H. Wonnacott University of Western Ontario Ronald J. Wonnacott University of Western Ontario WILEY JOHN WILEY & SONS New York Chichester Brisbane Toronto Singapore

### Logistic Regression. http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests

Logistic Regression http://faculty.chass.ncsu.edu/garson/pa765/logistic.htm#sigtests Overview Binary (or binomial) logistic regression is a form of regression which is used when the dependent is a dichotomy

### Simple linear regression

Simple linear regression Introduction Simple linear regression is a statistical method for obtaining a formula to predict values of one variable from another where there is a causal relationship between

### HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

### Module 4 - Multiple Logistic Regression

Module 4 - Multiple Logistic Regression Objectives Understand the principles and theory underlying logistic regression Understand proportions, probabilities, odds, odds ratios, logits and exponents Be

### SAS Software to Fit the Generalized Linear Model

SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling

3.2 LOGARITHMIC FUNCTIONS AND THEIR GRAPHS Copyright Cengage Learning. All rights reserved. What You Should Learn Recognize and evaluate logarithmic functions with base a. Graph logarithmic functions.

### A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution

A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September

### III. INTRODUCTION TO LOGISTIC REGRESSION. a) Example: APACHE II Score and Mortality in Sepsis

III. INTRODUCTION TO LOGISTIC REGRESSION 1. Simple Logistic Regression a) Example: APACHE II Score and Mortality in Sepsis The following figure shows 30 day mortality in a sample of septic patients as

### 13. Poisson Regression Analysis

136 Poisson Regression Analysis 13. Poisson Regression Analysis We have so far considered situations where the outcome variable is numeric and Normally distributed, or binary. In clinical work one often

### It is important to bear in mind that one of the first three subscripts is redundant since k = i -j +3.

IDENTIFICATION AND ESTIMATION OF AGE, PERIOD AND COHORT EFFECTS IN THE ANALYSIS OF DISCRETE ARCHIVAL DATA Stephen E. Fienberg, University of Minnesota William M. Mason, University of Michigan 1. INTRODUCTION

### Fairfield Public Schools

Mathematics Fairfield Public Schools AP Statistics AP Statistics BOE Approved 04/08/2014 1 AP STATISTICS Critical Areas of Focus AP Statistics is a rigorous course that offers advanced students an opportunity

### Pennsylvania System of School Assessment

Pennsylvania System of School Assessment The Assessment Anchors, as defined by the Eligible Content, are organized into cohesive blueprints, each structured with a common labeling system that can be read

### CHAPTER TWELVE TABLES, CHARTS, AND GRAPHS

TABLES, CHARTS, AND GRAPHS / 75 CHAPTER TWELVE TABLES, CHARTS, AND GRAPHS Tables, charts, and graphs are frequently used in statistics to visually communicate data. Such illustrations are also a frequent

### Statistics. Measurement. Scales of Measurement 7/18/2012

Statistics Measurement Measurement is defined as a set of rules for assigning numbers to represent objects, traits, attributes, or behaviors A variableis something that varies (eye color), a constant does

### SAMPLE SIZE TABLES FOR LOGISTIC REGRESSION

STATISTICS IN MEDICINE, VOL. 8, 795-802 (1989) SAMPLE SIZE TABLES FOR LOGISTIC REGRESSION F. Y. HSIEH* Department of Epidemiology and Social Medicine, Albert Einstein College of Medicine, Bronx, N Y 10461,

### VI. Introduction to Logistic Regression

VI. Introduction to Logistic Regression We turn our attention now to the topic of modeling a categorical outcome as a function of (possibly) several factors. The framework of generalized linear models

### Bivariate Statistics Session 2: Measuring Associations Chi-Square Test

Bivariate Statistics Session 2: Measuring Associations Chi-Square Test Features Of The Chi-Square Statistic The chi-square test is non-parametric. That is, it makes no assumptions about the distribution

### I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

Beckman HLM Reading Group: Questions, Answers and Examples Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Linear Algebra Slide 1 of

### The Chi-Square Test. STAT E-50 Introduction to Statistics

STAT -50 Introduction to Statistics The Chi-Square Test The Chi-square test is a nonparametric test that is used to compare experimental results with theoretical models. That is, we will be comparing observed

### SUMAN DUVVURU STAT 567 PROJECT REPORT

SUMAN DUVVURU STAT 567 PROJECT REPORT SURVIVAL ANALYSIS OF HEROIN ADDICTS Background and introduction: Current illicit drug use among teens is continuing to increase in many countries around the world.

### Test Positive True Positive False Positive. Test Negative False Negative True Negative. Figure 5-1: 2 x 2 Contingency Table

ANALYSIS OF DISCRT VARIABLS / 5 CHAPTR FIV ANALYSIS OF DISCRT VARIABLS Discrete variables are those which can only assume certain fixed values. xamples include outcome variables with results such as live

### Longitudinal Meta-analysis

Quality & Quantity 38: 381 389, 2004. 2004 Kluwer Academic Publishers. Printed in the Netherlands. 381 Longitudinal Meta-analysis CORA J. M. MAAS, JOOP J. HOX and GERTY J. L. M. LENSVELT-MULDERS Department

### Equations. #1-10 Solve for the variable. Inequalities. 1. Solve the inequality: 2 5 7. 2. Solve the inequality: 4 0

College Algebra Review Problems for Final Exam Equations #1-10 Solve for the variable 1. 2 1 4 = 0 6. 2 8 7 2. 2 5 3 7. = 3. 3 9 4 21 8. 3 6 9 18 4. 6 27 0 9. 1 + log 3 4 5. 10. 19 0 Inequalities 1. Solve

### Statistics in Retail Finance. Chapter 6: Behavioural models

Statistics in Retail Finance 1 Overview > So far we have focussed mainly on application scorecards. In this chapter we shall look at behavioural models. We shall cover the following topics:- Behavioural

### Big Ideas in Mathematics

Big Ideas in Mathematics which are important to all mathematics learning. (Adapted from the NCTM Curriculum Focal Points, 2006) The Mathematics Big Ideas are organized using the PA Mathematics Standards

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Algebra 1 Course Information

Course Information Course Description: Students will study patterns, relations, and functions, and focus on the use of mathematical models to understand and analyze quantitative relationships. Through

### Chapter 3 RANDOM VARIATE GENERATION

Chapter 3 RANDOM VARIATE GENERATION In order to do a Monte Carlo simulation either by hand or by computer, techniques must be developed for generating values of random variables having known distributions.

### Weight of Evidence Module

Formula Guide The purpose of the Weight of Evidence (WoE) module is to provide flexible tools to recode the values in continuous and categorical predictor variables into discrete categories automatically,

### CREDIT SCORING MODEL APPLICATIONS:

Örebro University Örebro University School of Business Master in Applied Statistics Thomas Laitila Sune Karlsson May, 2014 CREDIT SCORING MODEL APPLICATIONS: TESTING MULTINOMIAL TARGETS Gabriela De Rossi

### Logistic Regression 1. y log( ) logit( y) 1 y = = +

Written by: Robin Beaumont e-mail: robin@organplayers.co.uk Date Thursday, 05 July 2012 Version: 3 Logistic Regression model Can be converted to a probability dependent variable = outcome/event log odds

### Chapter 5 Analysis of variance SPSS Analysis of variance

Chapter 5 Analysis of variance SPSS Analysis of variance Data file used: gss.sav How to get there: Analyze Compare Means One-way ANOVA To test the null hypothesis that several population means are equal,

### Module 5: Introduction to Multilevel Modelling SPSS Practicals Chris Charlton 1 Centre for Multilevel Modelling

Module 5: Introduction to Multilevel Modelling SPSS Practicals Chris Charlton 1 Centre for Multilevel Modelling Pre-requisites Modules 1-4 Contents P5.1 Comparing Groups using Multilevel Modelling... 4

### Lecture 19: Conditional Logistic Regression

Lecture 19: Conditional Logistic Regression Dipankar Bandyopadhyay, Ph.D. BMTRY 711: Analysis of Categorical Data Spring 2011 Division of Biostatistics and Epidemiology Medical University of South Carolina

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### Regression Analysis: A Complete Example

Regression Analysis: A Complete Example This section works out an example that includes all the topics we have discussed so far in this chapter. A complete example of regression analysis. PhotoDisc, Inc./Getty

### Calculating the Probability of Returning a Loan with Binary Probability Models

Calculating the Probability of Returning a Loan with Binary Probability Models Associate Professor PhD Julian VASILEV (e-mail: vasilev@ue-varna.bg) Varna University of Economics, Bulgaria ABSTRACT The

### Statistical estimation using confidence intervals

0894PP_ch06 15/3/02 11:02 am Page 135 6 Statistical estimation using confidence intervals In Chapter 2, the concept of the central nature and variability of data and the methods by which these two phenomena

### Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@

Developing Risk Adjustment Techniques Using the SAS@ System for Assessing Health Care Quality in the lmsystem@ Yanchun Xu, Andrius Kubilius Joint Commission on Accreditation of Healthcare Organizations,

### CHAPTER 12 EXAMPLES: MONTE CARLO SIMULATION STUDIES

Examples: Monte Carlo Simulation Studies CHAPTER 12 EXAMPLES: MONTE CARLO SIMULATION STUDIES Monte Carlo simulation studies are often used for methodological investigations of the performance of statistical

### ELEMENTARY STATISTICS

ELEMENTARY STATISTICS Study Guide Dr. Shinemin Lin Table of Contents 1. Introduction to Statistics. Descriptive Statistics 3. Probabilities and Standard Normal Distribution 4. Estimates and Sample Sizes

### Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS)

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) April 30, 2008 Abstract A randomized Mode Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the

### Tips for surviving the analysis of survival data. Philip Twumasi-Ankrah, PhD

Tips for surviving the analysis of survival data Philip Twumasi-Ankrah, PhD Big picture In medical research and many other areas of research, we often confront continuous, ordinal or dichotomous outcomes

Using Your TI-83/84 Calculator: Linear Correlation and Regression Elementary Statistics Dr. Laura Schultz This handout describes how to use your calculator for various linear correlation and regression

### Poisson Models for Count Data

Chapter 4 Poisson Models for Count Data In this chapter we study log-linear models for count data under the assumption of a Poisson error structure. These models have many applications, not only to the

### UNDERSTANDING THE DEPENDENT-SAMPLES t TEST

UNDERSTANDING THE DEPENDENT-SAMPLES t TEST A dependent-samples t test (a.k.a. matched or paired-samples, matched-pairs, samples, or subjects, simple repeated-measures or within-groups, or correlated groups)

### Simple Linear Regression, Scatterplots, and Bivariate Correlation

1 Simple Linear Regression, Scatterplots, and Bivariate Correlation This section covers procedures for testing the association between two continuous variables using the SPSS Regression and Correlate analyses.

### with functions, expressions and equations which follow in units 3 and 4.

Grade 8 Overview View unit yearlong overview here The unit design was created in line with the areas of focus for grade 8 Mathematics as identified by the Common Core State Standards and the PARCC Model

### Logit and Probit. Brad Jones 1. April 21, 2009. University of California, Davis. Bradford S. Jones, UC-Davis, Dept. of Political Science

Logit and Probit Brad 1 1 Department of Political Science University of California, Davis April 21, 2009 Logit, redux Logit resolves the functional form problem (in terms of the response function in the

### Elements of statistics (MATH0487-1)

Elements of statistics (MATH0487-1) Prof. Dr. Dr. K. Van Steen University of Liège, Belgium December 10, 2012 Introduction to Statistics Basic Probability Revisited Sampling Exploratory Data Analysis -

### X X X a) perfect linear correlation b) no correlation c) positive correlation (r = 1) (r = 0) (0 < r < 1)

CORRELATION AND REGRESSION / 47 CHAPTER EIGHT CORRELATION AND REGRESSION Correlation and regression are statistical methods that are commonly used in the medical literature to compare two or more variables.

### Current Standard: Mathematical Concepts and Applications Shape, Space, and Measurement- Primary

Shape, Space, and Measurement- Primary A student shall apply concepts of shape, space, and measurement to solve problems involving two- and three-dimensional shapes by demonstrating an understanding of:

### Research Methods & Experimental Design

Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and

### Regression Modeling Strategies

Frank E. Harrell, Jr. Regression Modeling Strategies With Applications to Linear Models, Logistic Regression, and Survival Analysis With 141 Figures Springer Contents Preface Typographical Conventions

### ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS

DATABASE MARKETING Fall 2015, max 24 credits Dead line 15.10. ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS PART A Gains chart with excel Prepare a gains chart from the data in \\work\courses\e\27\e20100\ass4b.xls.

### Gender Effects in the Alaska Juvenile Justice System

Gender Effects in the Alaska Juvenile Justice System Report to the Justice and Statistics Research Association by André Rosay Justice Center University of Alaska Anchorage JC 0306.05 October 2003 Gender

Lecture 16: Generalized Additive Models Regression III: Advanced Methods Bill Jacoby Michigan State University http://polisci.msu.edu/jacoby/icpsr/regress3 Goals of the Lecture Introduce Additive Models

### Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression

Unit 31 A Hypothesis Test about Correlation and Slope in a Simple Linear Regression Objectives: To perform a hypothesis test concerning the slope of a least squares line To recognize that testing for a

### Regression 3: Logistic Regression

Regression 3: Logistic Regression Marco Baroni Practical Statistics in R Outline Logistic regression Logistic regression in R Outline Logistic regression Introduction The model Looking at and comparing

### Modelling spousal mortality dependence: evidence of heterogeneities and implications

1/23 Modelling spousal mortality dependence: evidence of heterogeneities and implications Yang Lu Scor and Aix-Marseille School of Economics Lyon, September 2015 2/23 INTRODUCTION 3/23 Motivation It has

### Credit Risk Analysis Using Logistic Regression Modeling

Credit Risk Analysis Using Logistic Regression Modeling Introduction A loan officer at a bank wants to be able to identify characteristics that are indicative of people who are likely to default on loans,

### This chapter will demonstrate how to perform multiple linear regression with IBM SPSS

CHAPTER 7B Multiple Regression: Statistical Methods Using IBM SPSS This chapter will demonstrate how to perform multiple linear regression with IBM SPSS first using the standard method and then using the

### Introduction to Fixed Effects Methods

Introduction to Fixed Effects Methods 1 1.1 The Promise of Fixed Effects for Nonexperimental Research... 1 1.2 The Paired-Comparisons t-test as a Fixed Effects Method... 2 1.3 Costs and Benefits of Fixed

### Means, standard deviations and. and standard errors

CHAPTER 4 Means, standard deviations and standard errors 4.1 Introduction Change of units 4.2 Mean, median and mode Coefficient of variation 4.3 Measures of variation 4.4 Calculating the mean and standard

### The Dummy s Guide to Data Analysis Using SPSS

The Dummy s Guide to Data Analysis Using SPSS Mathematics 57 Scripps College Amy Gamble April, 2001 Amy Gamble 4/30/01 All Rights Rerserved TABLE OF CONTENTS PAGE Helpful Hints for All Tests...1 Tests

### DATA INTERPRETATION AND STATISTICS

PholC60 September 001 DATA INTERPRETATION AND STATISTICS Books A easy and systematic introductory text is Essentials of Medical Statistics by Betty Kirkwood, published by Blackwell at about 14. DESCRIPTIVE

### UNDERSTANDING THE TWO-WAY ANOVA

UNDERSTANDING THE e have seen how the one-way ANOVA can be used to compare two or more sample means in studies involving a single independent variable. This can be extended to two independent variables

### Section 14 Simple Linear Regression: Introduction to Least Squares Regression

Slide 1 Section 14 Simple Linear Regression: Introduction to Least Squares Regression There are several different measures of statistical association used for understanding the quantitative relationship

### 1. The parameters to be estimated in the simple linear regression model Y=α+βx+ε ε~n(0,σ) are: a) α, β, σ b) α, β, ε c) a, b, s d) ε, 0, σ

STA 3024 Practice Problems Exam 2 NOTE: These are just Practice Problems. This is NOT meant to look just like the test, and it is NOT the only thing that you should study. Make sure you know all the material

### Descriptive Statistics and Measurement Scales

Descriptive Statistics 1 Descriptive Statistics and Measurement Scales Descriptive statistics are used to describe the basic features of the data in a study. They provide simple summaries about the sample

### The Influence of Sale Promotion Factors on Purchase Decisions: A Case Study of Portable PCs in Thailand

2011 International Conference on Financial Management and Economics IPEDR vol.11 (2011) (2011) IACSIT Press, Singapore The Influence of Sale Promotion Factors on Purchase Decisions: A Case Study of Portable