Multiple Regression. SPSS output. Multiple Regression Multiple Regression Model:

Transcription

1 Weght Multple Regresson Multple Regresson Relatng a response (dependent, nput) y to a set of eplanatory (ndependent, output, predctor) varables,,,. A technue for modelng the relatonshp between one response varable wth several predctor varables. y y,,,..., Determnstc component... Random component Multple Regresson : y... mnmze e [ y (... )],,,,..., n the model can all be estmated by least suare estmators:, ˆ, ˆ, ˆ,..., ˆ ˆ The Least-Suare Regresson Euaton: y ˆ ˆ ˆ ˆ... ˆ ˆ Study weght (y) usng age ( ) and heght ( ) Data: (months), heght (nches), weght (pounds) were recorded for a group of school chldren : 0 Scatter plo above show that both age and heght are lnearly related to weght. y 50 wth weght y, age, and heght 4 SPSS output Summary Adjusted Std. Error of R R Suare R Suare the Estmate.794 a a. Predctors:,, Coeffcent of determnaton: the percentage of varablty n the response varable (Weght) that can be descrbed by predctor varables (, ) through the model. 5 Regresson Resdual a. Predctors:,, b. Dependent Varable: Weght ANOVA b Sum of Suares df Mean Suare F Sg a Test for sgnfcance of the model: H 0 : s nsgnfcant ( s are all zeros). H a : s sgnfcant (Some s are not zeros). 6

2 Multple Regresson estmaton: SPSS output Tes for Regresson Coeffcen H 0 : = 0 vs. H a : 0 H 0 : = 0 vs. H a : 0 H 0 : = 0 vs. H a : 0 Collnearty * statstcs: If the VIF (Varance Inflaton Factor) s greater than 0 there s problem of Multcollnearty. (Some sad VIF needs to be less than 4.) Least suare regresson euaton: yˆ The average weght of chldren 44 months old and whose heght s 55 nches would be: (44) +.09(55) = lbs (estmated by the model) 8 How to nterpret, and? : y = + where y: Weght, :, : s the constant or the y-ntercept n the model. It s the average response when both predctor varables are 0. s the rate of change of epected (average) weght per unt change of age adjusted for the heght varable. s the rate of change of epected (average) weght per unt change of heght adjusted for the age varable. Other possble models: ( y: Weght, :, : ) y = + + y = + + Interacton term Wth nteracton term (Non-addtve): y = y = y = Coeffcent Estmaton wth Interacton Between and : INTAG_HT y wth weght y, age, and heght E E Hgh VIF mples very serous collnearty. Interacton should not be used n the model. For boys: Is there a serous collnearty? Wrte the weght predcton euaton usng age and heght as predctor varables. Fnd the average weght for boys that are 44 months old and 55 nches tall.

3 Weght Multple Regresson For grls: Is there a serous collnearty? Wrte the weght predcton euaton usng age and heght as predctor varables. Fnd the average weght for boys that are 44 months old and 55 nches tall. Indcator Varables - are bnary varables that take only two possble values, 0 and, and can be use for ncludng categorcal varables n the model. Weght Male: Female: 0 Male Female Group Statstcs Std. Error N Mean Std. Devaton Mean One Bnary Independent Varable : (A model that models two ndependent samples stuaton wth eual varances condton.) y = + Two ndependent samples t-test can be modeled wth smple lnear regresson model SPSS output for two ndependent samples t-test for comparng the mean weght between male and female. Levene's Test f or Eualty of Varances Independent Samples Test t-test for Eualty of Means where y : Weght, : ( = 0 for female, = for male) When = 0: y = When = : y = The dfference of the means of the two categores s. 5 Mean Std. Error F Sg. t df Sg. (-taled) Dff erence Dff erence Weght Eual varances assumed Eual varances not assumed SPSS output for lnear regresson wth gender as predctor L and as Predctor Varables : y wth y weght, ( 0 female, age, and male) gender and are both sgnfcant varables for predctng weght. Male There s sgnfcant dfference n average weght between genders f adjusted for age varable. Female

4 Multple Regresson,, & as Predctors : wth y y weght age heght gender ( 0 female, male) W e g h t Male 9 Female varable becomes nsgnfcant wth and varables n the model. When comparng the dfference n average wegh between genders, and adjusted for age and heght varables, the dfference s statstcally nsgnfcant. How to nclude a categorcal varable n the model? The proper way to nclude a categorcal varable s to use ndcator varables. For havng a categorcal varable wth k categores, one should set up k ndcator varables. Race varable: Whte =, Black =, Hspanc =. - ndcator varables wll be needed. Common Mstake: Use of the nternally coded values of a categorcal eplanatory varable drectly n lnear regresson modelng calculaton. Race : Whte =, Black =, Hspanc =. Number of hours of eercse per week Use of ndcator varables and for Race varable = represen Whte, otherwse = 0, = represen Black, otherwse = 0, = 0 and = 0 represen Hspanc. : y = Body Fat Percentage Number of hours of eercse per week Race : y = Body Fat Percentage Race Interpretaton of the model: Race: Whte = and = 0, y = Race: Black = 0 and =, y = Race: Hspanc = 0 and = 0, y = 4 4

5 Female lfe epectancy 99 Multple Regresson Suppose that the least suares regresson euaton for the model above s y 0... Estmate the avg. body fat for a whte person eercse 0 hours per week: =. Study female lfe epectancy usng percentage of urbanzaton and brth rate Estmate the avg. body fat for a black person eercse 0 hours per week: = 0. Estmate the avg. body fat for a hspanc person eercse 0 hours per week: = Brths per 000 populaton, Percent urban, y lfe epectancy, : y brth rate, Summary Adjusted Std. Error of R R Suare R Suare the Estmate.904 a a. Predctors:, Brths per 000 populaton, 99, Percent urban, 99 percent urban Regresson Resdual ANOVA b Sum of Suares df Mean Suare F Sg a a. Predctors:, Brths per 000 populaton, 99, Percent urban, 99 b. Dependent Varable: Female lfe epectancy 99 Test for sgnfcance of the model: Coeffcent of determnaton: the percentage of varablty n the response varable (female lfe epectancy) that can be descrbed by predctor varables (brth rate, percentage of urbanzaton) through the model. 7 H 0 : s nsgnfcant ( s are all zeros). H a : s sgnfcant (Some s are not zeros). 8 estmaton: (SPSS output) Brths per 000 populaton, 99 Percent urban, 99 a. Dependent Varable: Female lfe epectancy 99 Tes for Regresson Coeffcen H 0 : = 0 v.s. H a : 0 H 0 : = 0 v.s. H a : 0 H 0 : = 0 v.s. H a : Collnearty * statstcs:if the VIF (Varance Inflaton Factor) s greater than 0 there s multcollnearty problem. (Some sad VIF needs to be less than 4.) 9 Least suare regresson euaton for estmatng average response value yˆ The average female lfe epectancy for the countres whose brth rate per 000 s 0 and whose percentage of urbanzaton s would be (0) () =

6 Multple Regresson Female Lfe Epectancy Multple Scatter Plot Before Transformaton Female lfe epectan Response varable: Female lfe epectancy Eplana varables: Brth Rate, Urbanzaton, Phones, Doctors, and GDP. Brths per 000 popu Percent urban, 99 Phones per 00 peopl Whch varables are sgnfcant factors to female lfe epectancy n the model? Doc tors per 0,000 p GDP per c apta Multple Scatter Plot After ln() Transformaton on Phones, Doctors, GDP Female lfe epectan Summary b Adjusted R Std. Error of R R Suare Suare the Estmate Durbn-Waon.94 a a. Predctors:, Natural log of GDP, Percent urban, 99, Brths per 000 populaton, 99, Natural log of doctors per 0000, per 00 people b. Dependent Varable: Female lfe epectancy 99 Brths per 000 popu Percent urban, 99 Natural log of phone Natural log of docto Natural log of GDP ANOVA b Sum of Suares df Mean Suare F Sg. Regresson a Resdual a. Predctors:, Natural log of GDP, Percent urban, 99, Brths per 000 populaton, 99, Natural log of doctors per 0000, per 00 people b. Dependent Varable: Female lfe epectancy 99 4 Multcollnearty Stepwse Selecton z ed Coeffcent Coeffcen s Brths per populaton, 99 Percent urban, 99.97E per 00 people Natural log of doctors per 0000 ANOVA d Sum of Suares df Mean Suare F Sg. Regresson a Resdual Regresson b Resdual Regresson c Resdual Natural log of GDP a. Dependent Varable: Female lfe epectancy 99 Tolerance measures the strength of the lnear relaton between the ndependent varables.it s better to be a. Predctors:, per 00 people Predctors:, per 00 people, Brths per 000 populaton, 99 hgher than 0.. VIF s the recprocal of Tolerance. 5 6 b. c. Predctors:, per 00 people, Brths per 000 populaton, 99, Natural log of doctors per 0000 d. Dependent Varable: Fem ale lfe epectancy 99 6

7 Multple Regresson What are the varables that are sgnfcantly related to the female s lfe epectancy? per 00 people per 00 people Brths per 000 populaton, 99 per 00 people Brths per 000 populaton, 99 Natural log of doctors per 0000 Coeffcen B Std. Error a. Dependent Varable: Female lfe epectancy 99 z ed Coeffcent s Beta Use of regresson analyss Descrpton (model, system, relaton): Relaton between lfe epectancy & brth rate, GDP, Relaton between salary & rank, years of servce, Control: Ded too young, underpad, overpad, Predcton: Lfe epectancy, salary for new comers, future salary, Varable screenng (mportant factors): Sgnfcant factors for lfe epectancy, Sgnfcant factors for salary. 8 Constructon of regresson models. Hypothesze the form of the model for y,,,..., Selectng predctor varables. Decdng functonal form of the regresson euaton. Defnng scope of the model (desgn range).. Collect the sample data (observatons, epermen).. Use sample estmate unknown parameters n the model. 4. Understand the dstrbuton of the random error. 5. dagnostcs, resdual analyss. 6. Apply the model n decson makng. 7. Revew the model wth new data. 9 What s lnear model? Eample of a lnear model: y = y = y = y = y = 0 + ln() + y = 0 + e + s lnear n terms of parameters. 7