Retrospective Test for Loss Reserving Methods - Evidence from Auto Insurers

Transcription

1 Retrospective Test or Loss Reserving Methods - Evidence rom Auto Insurers eng Shi - Northern Illinois University joint work with Glenn Meyers - Insurance Services Oice CAS Annual Meeting, November 8, 010

2 Outline Introduction Loss reserving methods Sampling o NAIC Schedule Analysis or the industry Analysis or individual insurers Concluding remarks

3 Introduction A loss reserving model rom a upper triangle training data, one is interested in whether it is a good or bad predictive distribution. Standard error is commonly used measure o variability, does a small standard error mean a good predictive model? Hold-out observations are needed to answer the above question. For a run-o triangle o incremental paid losses, suppose we observe all the losses in the lower triangle hold-out sample, the retrospective test in this study is based on the ollowing well-know result: I X is a random variable with distribution F, then the transormation FX ollows a uniorm distribution on 0,1.

4 Introduction X : total reserve - Use a sample o independent insurers. - Test whether the percentiles o total reserves are rom uniorm 0,1. - Inorms us whether a predictive model is good or the whole industry. X : incremental paid losses in each cell o the lower triangle - Test or each single insurer. - Test whether the percentiles o incremental losses in the lower triangle hold-out sample are rom uniorm 0,1 - Inorms us whether a predictive model perorms well or a particular insurer

5 Loss reserving methods Three methods are considered: Mack chain ladder, bootstrap overdispersed oisson, Bayesian log-normal An industry benchmark: Chain-Ladder technique - Large literature on CL, see England and Verrall 00, Wüthrich and Merz Many stochastic models reproduce CL estimates, e.g. Mack 1993,1999, Renshaw and Verrall 1998, Verrall Modiications o CL, e.g. Barnnett and Zehnwirth 000 Mack CL: - Variability can be rom recursive relationship, see Mack Assume normality in the calculation o percentiles Bootstrap OD: - Resample residuals o GLM - Fit CL to pseudo data - Simulate incremental loss or each cell

6 A Bayesian Log-normal Model revious studies: Alba 00,006, Ntzouras and Dellaportas 00 Calendar year eect has been ignored We propose log Y µ ij ij ~ N µ, σ ij = αi + β j + t= i+ j, i, j = 1, N Y α - trend or accident year i β ij - normalized incremental paid lossor cell i,j - trend or development lag j - trend or calendar year t t i j We use accident year premium as exposure variable

7 A Bayesian Log-normal Model Dierent ways to speciy calendar year trend Calendar year trend introduce correlation due to calendar year eects The state space speciication could be used on accident year or development year trend We ocus on AR and RW speciications in the ollowing analysis 0, ~ 0, ~ Random Walk :, ~ 0, ~ Autoregressive Model: 0, ~ IIDSpeciication : 1 1 σ σ η η σ µ σ η η φ σ η η N N N N N t t t t t t t t t + = + =

8 A Bayesian Log-normal Model The likelihood unction can be derived as ollows We perorm the analysis using WinBUGS,,, where where we use log - normal speciication,,, then },,, { }and,,, { Let φ σ σ η η η β α φ σ σ σ β α η η y n n n n n n n i n j j i n i n j ij t j i = = = = = = = = = = = = y y y y

9 Sampling o NAIC Schedule

10 Sampling o NAIC Schedule Training data is rom 1997 schedule Accident year Hold-out sample is rom schedule o subsequent years e.g. actual paid losses or AY 1989 is rom 1998 schedule actual paid losses or AY 1990 is rom 1999 schedule actual paid losses or AY 1997 is rom 006 schedule Limit to group insurers or single entities Use data or personal auto and commercial auto or our analysis Check overlapping periods in training data and hold-out sample e.g. Training AY 1989 Hold-out AY 1989

11 Analysis or the industry Consider largest 50 insurers or personal and commercial auto lines Use net premiums written to measure size For each line o business: - derive the predictive distribution o total reserves or insurer i, say F i - calculate the percentile o the actual losses p i = F i loss i - repeat or all 50 irms Test i p i ollows uniorm 0,1 Implications: - i a predictive model perorms well, percentiles should be a realization rom uniorm 0,1 - an outcome that alls on the lower or higher percentile o the distribution does not suggest a bad model

12 Commercial Auto Consider Mack CL and bootstrap OD or top 50 insurers Compare point estimate o total reserve and prediction error 1 st igure compares point prediction that conirms two methods provide same estimates nd igure compares percentiles o actual losses, indicating a similar predictive distribution

13 Commercial Auto Next two slides present the percentiles p i i = 1,,50 o total reserves or the 50 insurers under dierent loss reserving methods Histogram and uniorm pp-plot are produced or our methods K-S test is used to test i p i ollows uniorm We observe: - again Mack CL and bootstrap OD provides similar results - pp-plots show both might have overitting problem - among state space modeling, AR1 speciication perorms better with a high p-value in the K-S test

14 Mack CL Bootstrap OD

15 LN - RW LN AR: p-value o K-S test is 0.43

16 ersonal Auto Repeat above analysis o total reserves or personal auto First we consider Mack CL and bootstrap OD using data rom largest 50 insurers Comparison o point prediction and percentile o actual losses conirms the close results rom the two chain ladder models

17 ersonal Auto As done or commercial auto, next two slides present the percentiles p i i = 1,,50 o total reserves or the 50 insurers under dierent loss reserving methods We exhibit both histogram and uniorm pp-plot, and K-S test is used to test i p i ollows uniorm We observe: - again Mack CL and bootstrap OD provides similar results - the perormance i worse than the commercial auto, since most realized outcomes lie on the lower percentile o the predictive distribution - Log-normal model does not suer like the above two, and a high p-value o the K-S test suggests the good perormance o the AR1 speciication

18 Mack CL Bootstrap OD

19 LN - RW LN AR: p-value o K-S test is 0.1

20 Analysis or individual insurers Consider individual insurers For illustrative purposes, we pick out insurers or each line Compare OD and LN-AR model Out o the two individual insurers or each line, we show that OD is better or one irm and LN model is better or the other one Though the analysis, we hope to explain why a certain method outperorms the other one

21 Commercial Auto Insurer A For insurer A, we derive the predictive distribution or each cell in the lower part o the triangle Then calculate the percentiles or actual incremental paid losses in the hold-out sample Uniorm pp-plots o percentiles with the p-value o K-S tests are shown in next slide LN model outperorms OD slightly We also compare mean error and mean absolute percentage error o the two methods over the 9 testing periods The result, to a great extent, agrees with K-S test

22 Commercial Auto Insurer A

23 Commercial Auto Insurer A In the next two slides, we analyze the predictive distributions rom the two methods 1st slide shows the predictive distributions or calendar year reserves - or early calendar years, LN provides wider distribution, as one moves to the bottom right o the lower triangle, LN provides narrow distribution - recall calendar year reserve is the sum o losses rom cells in the same diagonal nd slide shows the predictive distribution o each cell in calendar year CY=, that is calendar year or top right cells on the diagonal, LN provides narrower distribution, and or bottom let cells, LN provides wider distribution - LN provides higher volatility or early development year

24 redictive distribution or calendar year reserves

25 redictive distribution or each cell o irst calendar year

26 Commercial Auto Insurer A We look into the pattern o the training data We show time-series plot o incremental losses or each accident year and over development lag Let panel shows losses and right panel shows loss ratio We observe high volatility in early development lag that might explain the better perormance o LN model

27 Commercial Auto Insurer B We did similar analysis or insurer B and the results are summarized in the ollowing three slides For insurer B, OD outperorms LN model slightly Again we observe that the wider predictive distribution or early calendar years rom LN model is explained by the wider distribution or early development year

28 Commercial Auto Insurer B

29 redictive distribution or calendar year reserves

30 redictive distribution or each cell o irst calendar year

31 Commercial Auto Insurer B Figures below show time-series plot o incremental losses or each accident year and over development lag Let panel shows losses and right panel shows loss ratio Dierent with insurer A, there is less volatility in early development years LN model might underit the data

32 ersonal Auto Insurer A For insurer A, we derive the predictive distribution or each cell in the lower part o the triangle Then calculate the percentiles or actual incremental paid losses in the hold-out sample Uniorm pp-plots o percentiles with the p-value o K-S tests are shown in next slide LN model outperorms OD We also compare mean error and mean absolute percentage error o the two methods over the 9 testing periods For each testing period, LN perorms better than OD

33 ersonal Auto Insurer A

34 ersonal Auto Insurer A In the next two slides, we analyze the predictive distributions rom the two methods 1st slide shows the predictive distributions or cells with development year 10 - the predictive distribution or all accident year are similar - LN provides narrower distributions nd slide shows the predictive distribution or cells in accident year we want to see the eects over development lags - LN provides wider distribution or early development years and narrower distribution or later development years

35 redictive distributions or cells with development year 10

36 redictive distributions or cells with accident year 1997

37 ersonal Auto Insurer A Again look into the pattern o the training data We show time-series plot o incremental losses or each accident year and over development lag Let panel shows losses and right panel shows loss ratio Again the high volatility in early development lag that might explain the better perormance o LN model

38 ersonal Auto Insurer B We did similar analysis or insurer B and the results are summarized in the ollowing three slides For insurer B, OD outperorms LN model From the predictive distributions, we observe again LN provides wider distributions or early development years, while the distributions across accident years are similar under two methods

39 ersonal Auto Insurer B

40 redictive distributions or cells with development year 10

41 redictive distributions or cells with accident year 1997

42 ersonal Auto Insurer B Figures below show time-series plot o incremental losses or each accident year and over development lag Let panel shows losses and right panel shows loss ratio Dierent with insurer A, there is less volatility in early development years, especially or the loss ratio Thus LN model introduces more volatility and does not work well or this insurer

43 Concluding Remarks We use simple test to examine the perormance o loss reserving methods Our analysis is based on hold-out sample We ind the current industry standard over-estimate reserves or the industry We compare chain ladder and LN model on individual insurers Chain ladder ails in case o higher volatility Bayesian methods mitigates the potential overitting problem