A SAS/IML Macro for Goodnessof-Fit Testing in Logistic Regression Models with Sparse Data

Transcription

1 A SAS/IML Macro for Goodnessof-Ft Testng n Logstc Regresson Models wth Sparse Data Olver Kuss Insttute of Medcal Epdemology, Bometry and Informatcs, Halle/Saale, Germany

2 Overvew Introducton Goodness-of-Ft Testng %GOFLOGIT Concluson SAS s a regstered trademark or trademark of SAS Insttute Inc. n the USA and other countres. ndcates USA regstraton. Other brand and product names are regstered trademarks or Trademarks of ther respectve companes.

3 Introducton (I): The Logstc Regresson Model Logstc Regresson s the standard analyzng tool for bnary responses Reasons: Interpretaton, Prognoss, Software Logstc Regresson n SAS : Use LOGISTIC, GENMOD, CATMOD, PROBIT procedure

4 Introducton (II): Assessng Goodness-of-Ft Resdual Analyss: Check ndvdual observatons ( and Opton n statement) Goodness-of-Ft Testng: Combne evdence on lack-of-ft n a sngle number and apply statstcal test

5 Introducton (III): Notaton N grouped observatons (y,x ), y response, y ~bnomal(m, ), x vector of covarates (1, x 1,..., x p ) Model equaton: logt( )=x syntax:

6 Pearson statstc Devance Statstcal test: Compare X and D to a -dstrbuton Goodness-of-Ft Testng (I): Standard Tests ( ) + = = N m y m y m y y D π π ˆ ln ˆ ln 1 ( ) ( ) = = N m m y X 1 ˆ 1 ˆ ˆ π π π χ N p 1

7 Goodness-of-Ft Testng (II): Standard Tests and Sparse Data Valdty of the χ N p 1 - dstrbuton reles on the assumpton of large m!!! Unrealstc wth contnuous covarates See what happens wth m 1: Pearson statstc X N Devance D N ˆ π = ˆ π ln = 1 1 ˆ π + ( ˆ π ) ln 1

8 Goodness-of-Ft Testng (III): Solutons n PROC LOGISTIC Group ndvdual observatons n new groups dependng on the estmated probablty from the model ( Hosmer-Lemeshow test) statement Opton Buld groups of ndvdual observatons on your own statement!" #$%$"&' $"&'(

9 Goodness-of-Ft Testng (IV): Solutons n PROC LOGISTIC However... The Hosmer-Lemeshow test depends heavly on the calculatng algorthm Buldng new groups on your own s knd of arbtrary and open for manpulaton

10 Goodness-of-Ft Testng (V): Solutons n the Lterature X and D are normally dstrbuted wth sparse data (Osus/Rojek, McCullagh ) Farrngton test X X O m π X McC ( 1 ˆ π ) ( ) ( y ) mπ ˆ 1 ˆ π N = + F X ˆ = 1

11 Goodness-of-Ft Testng (VI): Solutons n the Lterature IM test (Whte, Orme ) Compare two dfferent estmators of the nformaton matrx whch should gve equvalent results under a satsfactory model ft RSS test RSS = M ( y ) πˆ = 1 IM DIAG

12 Goodness-of-Ft Testng (VII): Whch Test works??? Lmted evdence n the lterature Hosmer et al. fnd: X and RSS perform best, but: McC dd not nclude all avalable tests only consdered m 1

13 Goodness-of-Ft Testng (VIII): Whch Test works??? Own smulaton study wth SAS/IML Results: X and D don t work X X F X McC O outperforms and IM outperforms DIAG RSS In every stuaton there s a test that outperforms the Hosmer-Lemeshow test

14 %GOFLOGIT (I) = SAS/IML Macro that performs the prescbed tests Syntax: )*+,$+*"' *+,$+*"'- '...$"&' $"&'..$+*"&'"/ $+*"&'"/( where y= varable wth number of observed events m= varable wth number of observed trals xlst= lst of covarates logstc= optonal runnng of

15 %GOFLOGIT (II): An Example Rsk factors for occupatonal hand eczema n hardressers, M=574 (340 events), N=334, 6 covarates Syntax: )*+,$+*"' *+,$+*"'- ' 0 "!-!%&&%!......$"&' $"&' 1%'1+!3 1+!3 %! 4%4'1 % '+ -&0"-!+&"&/%4'%! &3"4!+'%/'"+4!+'%/'"+4. $+*"&'"/(

16 %GOFLOGIT (III): Results Hosmer and Lemeshow Goodness-of-Ft Test Ch-Square DF Pr > ChSq Value p-value Standard Pearson Test Standard Devance Osus-Test McCullagh-Test Farrngton-Test IM-Test RSS-Test

17 %GOFLOGIT (IV): Results* * = after removal of two outlers Hosmer and Lemeshow Goodness-of-Ft Test Ch-Square DF Pr > ChSq Value p-value Standard Pearson Test Standard Devance Osus-Test McCullagh-Test Farrngton-Test IM-Test RSS-Test

18 Concluson Goodness-of-ft testng n logstc regresson s an mportant, but nontrval task Don t trust the standard methods f you have sparse data Use the %GOFLOGIT Macro nstead

19 About the Speaker Speaker Locaton of company Olver Kuss Magdeburger Str Halle/Saale, Germany Telephone Fax E-Mal Olver.Kuss@medzn.un-halle.de

20 Lterature Farrngton, C.P., (1996), On Assessng Goodness of Ft of Generalzed Lnear Models to Sparse Data, Journal of the Royal Statstcal Socety, B, 58, Hosmer, D.W., Hosmer, T., Le Cesse, S. and Lemeshow, S. (1997), A comparson of goodness-of-ft tests for the logstc regresson model, Statstcs n Medcne, 16, Hosmer, D.W. and Lemeshow, S. (1980), Goodness of ft tests for the multple logstc regresson model, Communcatons n Statstcs Theory and Methods, 9, McCullagh, P. (1985), On the Asymptotc Dstrbuton of Pearson s Statstc n Lnear Exponental-Famly Models, Internatonal Statstcal Revew, 53, Orme, C. (1988), The calculaton of the nformaton matrx test for bnary data models, The Manchester School, 54, Osus, G. and Rojek, D. (199), Normal Goodness-of-Ft Tests for Multnomal Models Wth Large Degrees of Freedom, Journal of the Amercan Statstcal Assocaton, 87, Whte, H. (198), Maxmum Lkelhood Estmaton of Msspecfed Models, Econometrca, 50, 1-5.