Multiple Regression. SPSS output. Multiple Regression. α, β 1, β 2, β 3,..., β q in the model can all be estimated by least square estimators:

Transcription

1 Multple Regresson Relatng a response (dependent, nput) y to a set of explanatory (ndependent, output, predctor) varables x, x,, x. A technue for modelng the relatonshp between one response varable wth several predctor varables. y = μ y x, x, x,..., x = α Determnstc component +... Random component Multple Regresson : y α β mnmze = x e = [ y ( α + β x β x )] α, β, β, β,..., β n the model can all be estmated by least suare estmators: α, ˆ β, ˆ β, ˆ β,..., ˆ ˆ β The Least-Suare Regresson Euaton: y = ˆ α + ˆ β x + ˆ β x + ˆ β x +... ˆ + ˆ βx Study weght (y) usng age (x ) and heght (x ). Weght 0 00 Weght 0 00 Data: (months), heght (nches), weght (pounds) were recorded for a group of school chldren : 0 y = α 50 Scatter plo above show that both age and heght are lnearly related to weght. wth weght y, age x, and heght x 4 SPSS output Summary Adjusted Std. Error of R R Suare R Suare the Estmate.794 a a. Predctors:,, t of determnaton: the percentage of varablty n the response varable (Weght) that can be descrbed by predctor varables (, ) through the model. 5 Regresson Resdual Total a. Predctors:,, b. Dependent Varable: Weght ANOVA b Sum of Suares df Mean Suare F Sg a Test for sgnfcance of the model: : s nsgnfcant (β s are all zeros). H a : s sgnfcant (Some β s are not zeros). 6

2 estmaton: SPSS output Unstandard Tes for Regresson : α = 0 vs. H a : α : β = 0 vs. H a : β : β = 0 vs. H a : β Collnearty * statstcs: If the VIF (Varance Inflaton Factor) s greater than 0 there s problem of Multcollnearty. (Some sad VIF needs to be less than 4.) 7 Unstandard Least suare regresson euaton: yˆ = x x The average weght of chldren 44 months old and whose heght s 55 nches would be: (44) +.09(55) = lbs (estmated by the model) 8 How to nterpret α, β and β? : where y = α + β x + β x y: Weght, x :, x : α s the constant or the y-ntercept n the model. It s the average response when both predctor varables are 0. β s the rate of change of expected (average) weght per unt change of age adjusted for the heght varable. β s the rate of change of expected (average) weght per unt change of heght adjusted for the age varable. Other possble models: ( y: Weght, x:, x: ) y = α + β x y = α + β x Interacton term Wth nteracton term (Non-addtve): y = α + β x + β x + β x x y = α + β x + β x x y = α + β x + β x x 9 0 t Estmaton wth Interacton Between and : y = α x INTAG_HT wth weght y, age x, and heght x Unstandard E E Hgh VIF mples very serous collnearty. Interacton should not be used n the model. Unstandard For boys: Is there a serous collnearty? Wrte the weght predcton euaton usng age and heght as predctor varables. Fnd the average weght for boys that are 44 months old and 55 nches tall.

3 Unstandard For grls: Is there a serous collnearty? Wrte the weght predcton euaton usng age and heght as predctor varables. Fnd the average weght for boys that are 44 months old and 55 nches tall. Indcator Varables - are bnary varables that take only two possble values, 0 and, and can be use for ncludng categorcal varables n the model. Weght Male: Female: 0 Male Female Group Statstcs Std. Error N Mean Std. Devaton Mean One Bnary Independent Varable : (A model that models two ndependent samples stuaton wth eual varances condton.) y = α + β x Two ndependent samples t-test can be modeled wth smple lnear regresson model SPSS output for two ndependent samples t-test for comparng the mean weght between male and female. Levene's Test for Eualty of Varances Independent Samples Test t-test for Eualty of Means where y : Weght, x : (x = 0 for female, x = for male) When x = 0: y = α When x = : y = α + β The dfference of the means of the two categores s β. 5 Mean Std. Error F Sg. t df Sg. (-taled) Dfference Dfference Weght Eual varances assumed Eual varances not assumed SPSS output for lnear regresson wth gender as predctor Unstandard L and as Predctor Varables : y = α wth y weght, x ( x = 0 female, x age, and x = male) gender Unstandard Weght 0 00 and are both sgnfcant varables for predctng weght. Male There s sgnfcant dfference n average weght between genders f adjusted for age varable. Female

4 ,, & as Predctors : wth y = α y x x weght age heght x gender ( x = 0 female, x = male) W e g h t Male 9 Female 0 Unstandard varable becomes nsgnfcant wth and varables n the model. When comparng the dfference n average wegh between genders, and adjusted for age and heght varables, the dfference s statstcally nsgnfcant. How to nclude a categorcal varable n the model? The proper way to nclude a categorcal varable s to use ndcator varables. For havng a categorcal varable wth k categores, one should set up k ndcator varables. Race varable: Whte =, Black =, Hspanc =. - ndcator varables wll be needed. Common Mstake: Use of the nternally coded values of a categorcal explanatory varable drectly n lnear regresson modelng calculaton. Race : Whte =, Black =, Hspanc =. Number of hours of exercse per week Use of ndcator varables x and x for Race varable x = represen Whte, otherwse x = 0, x = represen Black, otherwse x = 0, x = 0 and x = 0 represen Hspanc. : y = α + β x + β x + β x Body Fat Percentage Number of hours of exercse per week Race : y = α + β x + β x + β x Body Fat Percentage Race Interpretaton of the model: Race: Whte x = and x = 0, y = α + β + β x Race: Black x = 0 and x =, y = α + β + β x Race: Hspanc x = 0 and x = 0, y = α + β x 4 4

5 Suppose that the least suares regresson euaton for the model above s y = 0 +. x x +. x. Estmate the avg. body fat for a whte person exercse 0 hours per week: 0 +. x +. x 0.0 =. Study female lfe expectancy usng percentage of urbanzaton and brth rate Estmate the avg. body fat for a black person exercse 0 hours per week: 0 +. x 0 +. x.0 = 0. Estmate the avg. body fat for a hspanc person exercse 0 hours per week: 0 +. x 0 +. x 0.0 = 8.9 Female lfe expectancy Female lfe expectancy Brths per 000 populaton, Percent urban, y lfe expectancy, x : y = α brth rate, x Summary Adjusted Std. Error of R R Suare R Suare the Estmate.904 a a. Predctors:, Brths per 000 populaton, 99, Percent urban, 99 percent urban Regresson Resdual Total ANOVA b Sum of Suares df Mean Suare F Sg a a. Predctors:, Brths per 000 populaton, 99, Percent urban, 99 b. Dependent Varable: Female lfe expectancy 99 Test for sgnfcance of the model: t of determnaton: the percentage of varablty n the response varable (female lfe expectancy) that can be descrbed by predctor varables (brth rate, percentage of urbanzaton) through the model. 7 : s nsgnfcant (β s are all zeros). H a : s sgnfcant (Some β s are not zeros). 8 estmaton: (SPSS output) Brths per 000 populaton, 99 Percent urban, 99 a. Dependent Varable: Female lfe expectancy 99 Tes for Regresson : α = 0 v.s. H a : α : β = 0 v.s. H a : β : β = 0 v.s. H a : β Unstandard Collnearty * statstcs:if the VIF (Varance Inflaton Factor) s greater than 0 there s multcollnearty problem. (Some sad VIF needs to be less than 4.) 9 Least suare regresson euaton for estmatng average response value yˆ = x x The average female lfe expectancy for the countres whose brth rate per 000 s 0 and whose percentage of urbanzaton s would be (0) () =

6 Use of regresson analyss Descrpton (model, system, relaton): Relaton between lfe expectancy & brth rate, GDP, Relaton between salary & rank, years of servce, Control: Ded too young, underpad, overpad, Predcton: Lfe expectancy, salary for new comers, future salary, Varable screenng (mportant factors): Sgnfcant factors for lfe expectancy, Sgnfcant factors for salary. Constructon of regresson models μ. Hypothesze the form of the model for Selectng predctor varables. y x, x, x,..., x Decdng functonal form of the regresson euaton. Defnng scope of the model (desgn range).. Collect the sample data (observatons, expermen).. Use sample estmate unknown parameters n the model. 4. Understand the dstrbuton of the random error. 5. dagnostcs, resdual analyss. 6. Apply the model n decson makng. 7. Revew the model wth new data. 6