Model Risk in Retirement Simulations

Transcription

1 January 2009 Model Risk in Retirement Simulations A Sensitivity Analysis Marlena I. lee Fi n a n c i a l a d v i s o r s o f t e n u s e Mo n t e Ca r l o s i m u l at i o n s to aid clients with retirement planning. The user provides inputs for the expected rate of return, return volatility, inflation, savings, and withdrawal rates, as well as the retirement timeline. Based on these assumptions, thousands of trials are simulated by generating normally distributed returns with a mean and standard deviation specified by the user. A distribution of outcomes is produced, and the user can assess the probability of successfully reaching stated retirement goals given the modeling assumptions. While providing a useful framework to help investors understand the key factors that impact retirement wealth, these simulations embed additional assumptions about the distribution of returns, aside from the mean and volatility parameters. Monte Carlo tools typically assume that returns are normally distributed, although historical returns tend to include more extreme results than would be predicted by a normal distribution. Additionally, returns generated in such simulations are assumed to be independent and identically distributed ( i.i.d. ). The i.i.d. assumption implies that means and standard deviations are constant through time, so that today s return distribution does not The helpful comments of Jim Davis, Inmoo Lee, Sunil Wahal, and Weston Wellington are gratefully acknowledged. This article contains the opinions of the author but not necessarily the opinions of Dimensional. All materials presented are compiled from sources believed to be reliable and current, but accuracy cannot be guaranteed. The material in this publication is provided solely as background information for registered investment advisors, institutional investors, and other sophisticated investors, and is not intended for public use. It should not be distributed to investors of products managed by Dimensional Fund Advisors Inc. or to potential investors. Dimensional Fund Advisors is an investment advisor registered with the Securities and Exchange Commission. Consider the investment objectives, risks, and charges and expenses of the Dimensional funds carefully before investing. For this and other information about the Dimensional funds, please read the prospectus carefully before investing. Prospectuses are available by calling Dimensional Fund Advisors Inc. collect at (310) ; on the internet at or, by mail, DFA Securities Inc., c/o Dimensional Fund Advisors Inc., 1299 Ocean Avenue, Santa Monica, CA Funds distributed by DFA Securities Inc.

2 2 Dimensional Fund Advisors depend in any way on past returns. It rules out time series patterns in returns such as mean reversion, momentum, and volatility clustering. This assumption is undermined by evidence in the academic literature that documents serial correlation in both returns and return volatility. The extent to which these modeling assumptions are inaccurate descriptions of reality constitutes a source of model risk. Results from Monte Carlo simulations only account for risk due to return volatility. However, uncertainty about the various modeling assumptions adds a layer of risk that is not incorporated into Monte Carlo output. This paper attempts to document the sensitivity of retirement simulations to these various modeling assumptions and highlights a major shortcoming of the Monte Carlo tool for use in retirement planning. The results show that although simulation results are fairly insensitive to certain assumptions about the return distribution, serial correlation in returns has the potential to materially impact long-term investment risk. Additionally, wealth outcomes are highly sensitive to assumptions about expected returns and savings and withdrawal rates. Because model risk in Monte Carlo simulations is potentially large, the tool should be used with caution. Retirement simulations can be useful to illustrate the broad impact of retirement choices, but they should not be relied upon to make accurate predictions of future wealth. Section 1 describes the framework for the Monte Carlo simulation and sets up the baseline case where returns are i.i.d. normal. In sections 2 to 4, the baseline simulation is compared to simulations where the i.i.d. normal assumption is relaxed to account for non-normality in returns, volatility clustering, and return predictability. The results indicate that in most cases, the i.i.d. normal assumption is fairly harmless. The distributionof-wealth outcomes when returns are simulated from historical returns are remarkably similar to the wealth distribution resulting from returns drawn from a normal distribution. Likewise, simulated returns with time-varying volatility also produces wealth outcomes that are similar to the constant volatility case. Incorporating long-horizon return predictability does have a noticeable impact on the uncertainty of future wealth, although the direction of the impact depends on how return predictability is specified. In section 5, I conduct additional sensitivity analysis with respect to the various user inputs, which include the expected return and the savings and withdrawal rates. Simulation results are highly sensitive to these user inputs, indicating that uncertainty about expected returns and savings and withdrawal rates is potentially a much bigger source of modeling risk than the i.i.d. normal assumption. Concluding remarks are made in section 6. Readers interested in the technical details regarding simulation procedures are referred to the appendix. 1. Baseline Simulation with I.I.D. Normal Returns The simulations in this paper assume there is an investor who opens an investment account at the age of 30 with an initial amount of $50,000. He saves $12,000 in the first year, making monthly deposits to the investment account, and increases the real

3 Model Risk in Retirement Simulations 3 amount saved by 5% each year until he retires at the age of 60. Starting at age 60, the investor makes monthly withdrawals from his investment account to fund $100,000 of real annual consumption. The withdrawals continue until the account is depleted of funds or until the investor reaches the age of 90. The savings and withdrawal rates are adjusted to reflect a constant 3% inflation rate. The growth of the portfolio is determined by independent draws of monthly nominal returns from a normal distribution with a monthly mean of 0.93% and standard deviation of 5.39%. These numbers are the average and standard deviation of monthly returns of the CRSP Value-Weighted Market Index (the market portfolio ) from January 1926 to July 2008, and they correspond to an annualized average return of 11.7% and standard deviation of 18.7%. Portfolios are simulated 5,000 times, resulting in a distribution of the value of the portfolio through time. To isolate the impact of the distributional assumptions on Monte Carlo results, the rest of the framework is kept fairly simple. Tax consequences from dividends or withdrawals as well as any fees associated with the management of the account are ignored. Additionally, the return distribution is kept the same over the entire investment horizon, and the mean and variance inputs are matched to the historical returns of a 100% equity portfolio invested in the market portfolio. Academic studies have mainly focused on departures from the i.i.d. normality assumption in stock return data. While bond returns may also deviate from an i.i.d. normal distribution, this paper focuses on relaxing the return distribution in ways that match the distribution of historical stock returns. 2. Non-Normality A convenient property of normal distributions is that they are two-parameter distributions. Only mean and variance values must be specified in order to define the shape of the entire distribution. In moving away from normality, one must consider additional parameters, including the skewness and kurtosis of the distribution. Skewness measures the degree of asymmetry of a distribution. Normal distributions are symmetric and have no skewness, meaning that the left side of the distribution is identical to the right side. Kurtosis measures how much weight lies in the tails of the distribution. Distributions with excess kurtosis have sharper peaks and fatter tails, making extreme events more likely. Exhibit 1 shows the distribution of historical monthly returns for the market portfolio from January 1926 to July Relative to a normal distribution, the historical return distribution displays excess kurtosis. If returns were normal, one-month returns in excess of 20% in absolute value would only have a 0.04% chance of occurring. Historically, they have occurred 1.01% of the time. Additionally, the summary statistics for returns in Table 1 show that the historical distribution has slight positive skewness. Scott and Horvath (1980) show that investors with typical preferences benefit from positive skewness but are harmed by excess kurtosis.

4 4 Dimensional Fund Advisors Exhibit 1 Distribution of Monthly Returns 25% 20% Frequency 15% 10% 5% 0% Monthly Return Distribution of historical monthly returns of the CRSP Value-Weighted Market Index from January 1926 to July 2008 relative to a normal distribution. How do the fat-tails and slight skewness of the return distribution impact an investor s total return? To try to answer this question, I randomly sample with replacement from the distribution of historical monthly returns. To illustrate this bootstrapping methodology, suppose there is a deck of cards, with one card for each month between January 1926 and July Monthly returns are simulated by drawing a card from the deck and recording the historical return in that month. The card is returned to the deck before another card is randomly selected. The process is repeated until 5,000 sixty-year samples of monthly returns are simulated. The bootstrapped returns will have the same excess kurtosis and slight skewness observed in the historical data, as shown by the return summary statistics in Table 1. By construction, expected returns and variance are identical to those in the baseline simulation, and returns are still assumed to be i.i.d. I summarize the Monte Carlo results in a number of ways. Summary statistics for retirement wealth and final wealth are reported in Table 1, while the full distributions are shown in Exhibit 2. Exhibit 3 displays the 90% confidence interval of wealth over time. These depictions of the wealth distribution show that bootstrapping returns from the historical distribution produces wealth outcomes that are very similar to those generated in the baseline simulation that assumes normality. The 90% confidence intervals of wealth over time for the two simulations are almost overlapping. Probability of portfolio survival, measured as the percentage of simulations with positive portfolio value at a given age, is depicted in Exhibit 4. Relative to the baseline

5 Model Risk in Retirement Simulations 5 Table 1 Summary Statistics Monthly Returns (%) Retirement Wealth ($MM) Final Wealth ($MM) Mean Std. Dev. Skewness Kurtosis Mean Std. Dev. Mean Std. Dev. Historical Returns CRSP Value-Weighted Market Index Simulated Returns I.I.D. Normal Bootstrap GARCH Return Predictability, CRSP Return Predictability, EAFE Summary statistics for historical and simulated monthly returns, retirement wealth, and final wealth. Historical returns are from the CRSP Value-Weighted Market Index from January 1926 to July Simulated returns are from 5,000 trials, each with 720 months of returns. simulation, the portfolios with returns drawn from the historical distribution have lower survival rates. However, this difference is small, typically less than 1.5 percentage points. For example, 79.7% of the i.i.d. normal portfolios survive to the age of 85, while the comparable number for the bootstrapped portfolios is 78.2%. These results indicate that accounting for the non-normality in monthly returns does not lead to dramatically different conclusions about the distribution of wealth and portfolio survival rates compared to simulations with normally distributed returns. 3. Volatility Clustering Time series return data exhibit volatility clustering, meaning that there are periods of relatively high variance and periods of relatively low variance. The top panel of Exhibit 5 displays monthly standard deviations of market returns, computed from daily returns. 1 In contrast to the assumption of constant volatility, historical stock return volatility is not constant through time and appears to be serially correlated. The ARCH/ GARCH line of research, beginning with Engle (1982) and Bollerslev (1986), directly models the variance to address volatility clustering. How does the serial correlation of volatility affect a long-term investor concerned about his wealth in retirement? To try to answer this question, I simulate not only the returns, 1. Following French et al. (1987), monthly standard deviations, σ mt, are computed from daily returns, N 2 r it, as σ mt = [ r it + 2 r it r i+1,t ] ½ t N t 1. i=1 i=1

6 6 Dimensional Fund Advisors Exhibit 2 Distribution of Retirement Wealth and Final Wealth Retirement Wealth Frequency 20% 18% 16% 14% 12% 10% 8% I.i.d. Normal Bootstrap GARCH Return Predictability, CRSP Return Predictability, EAFE 6% 4% 2% 0% > 7.0 Wealth ($ millions) Distribution of portfolio values at age 60 and 90 from Monte Carlo simulations with 5,000 trials under various return scenarios. Final Wealth Frequency 30% 25% 20% 15% I.i.d. Normal Bootstrap GARCH Return Predictability, CRSP Return Predictability, EAFE 10% 5% 0% 0 0 < x Wealth ($ millions) > 70 Distribution of portfolio values at age 60 and 90 from Monte Carlo simulations with 5,000 trials under various return scenarios.

7 Model Risk in Retirement Simulations 7 Exhibit 3 90% Confidence Interval of Wealth over Time $1,000,000,000 $100,000,000 $10,000,000 I.i.d. Normal Bootstrap GARCH Return Predictability, CRSP Return Predictability, EAFE Wealth $1,000,000 $100,000 $10,000 $1, Age The 5th and 95th percentiles of wealth resulting from 5,000 simulations under various return scenarios. Exhibit 4 Portfolio Survival Percent of Portfolios with Positive Value 100% 95% 90% 85% 80% 75% I.i.d. Normal Bootstrap GARCH Return Predictability, CRSP Return Predictability, EAFE 70% Age Distribution of portfolios with positive value at specified ages under various return scenarios. There are 5,000 simulated trials for each return scenario.

8 8 Dimensional Fund Advisors Exhibit 5 Actual and Predicted Monthly Return Volatility Monthly Standard Deviations from Historical Returns 30% 25% Standard Deviation 20% 15% 10% 5% 0% Year Monthly Standard Deviations Predicted with GARCH(1,1) 30% 25% Standard Deviation 20% 15% 10% 5% 0% Year Monthly return standard deviations are computed from daily returns of the CRSP Value Weighted Market Index from January 1926 to July Predicted monthly return standard deviations are estimated from a GARCH(1,1) model.

9 Model Risk in Retirement Simulations 9 but also the variances of returns. As in the baseline simulation, returns are drawn from normal distributions with constant means equal to the average monthly market return. However, unlike the baseline i.i.d. normal simulation that assumes constant volatility, this simulation allows the volatility of returns to be autocorrelated over time. Each trial employs a GARCH(1,1) model, with parameters estimated from monthly market returns, to simulate a time series of return variances that displays volatility clustering. The bottom panel of Exhibit 5 displays the predicted monthly standard deviations estimated with the GARCH model. Parameter estimates and additional details about the simulation procedure are provided in the Appendix. Exhibits 2 and 3 demonstrate how the distributions of wealth produced in the GARCH simulations are remarkably similar to those in the benchmark i.i.d. case. There is virtually no difference in the 90% confidence interval of wealth over time between the GARCH simulation and the baseline i.i.d. simulation, as shown in Exhibit 3. Exhibit 4 shows that although the GARCH portfolios have slightly lower survival rates than the baseline constant volatility portfolios, the differences are small. The biggest difference in survival rates occurs at the age of 90, where 76.3% of the baseline portfolios survive versus 74.7% of the GARCH portfolios. These results indicate that while the assumption of constant volatility is not a valid description of historical returns, it is a fairly harmless assumption because it does not substantially bias the distribution of wealth over time. 4.1 CRSP Data 4. Return Predictability Research has shown that historical stock prices in the US tend to mean-revert at long horizons (Fama and French 1988; Poterba and Summers 1988). This implies that periods of high realized returns tend to be followed by periods of low returns, and vice versa, thus violating the assumption that returns are independent through time with a constant expected value. One simple way to provide statistical evidence on this return predictability is to compute variance ratios. If prices follow a random walk, the variance of k-year log returns should equal k times the variance of annual log returns. If prices are mean-reverting, long-horizon returns are less volatile than would be implied by the random walk hypothesis; and the variance ratio, defined as the k-year variance divided by k times the one-year variance, would be less than 1. The variance ratios computed using monthly market portfolio returns for k = 2, 3, 4, 5, and 10 are displayed in Table 2. The values reported here are very similar to those reported in previous studies, with the variance ratio falling below 1 at horizons of three years of more. Although these variance ratios would seem to indicate mean reversion, there are a number of complications that make inference with variance ratios

10 10 Dimensional Fund Advisors Table 2 Variance Ratios CRSP Value-Weighted Market Index Annual Std. Dev. Variance Ratio at Return Horizon January 1926-July Baseline Simulation, i.i.d. Normal Returns 5,000 trials, each with 720 months Return Predictability, CRSP 5,000 trials, each with 720 months MSCI EAFE Index January 1926-July Serially Correlated Simulation, EAFE 5,000 trials, each with 720 months difficult. 2 Variance ratio tests suffer from low power; and under the null hypothesis of independent, identically distributed normal returns, the variance ratios computed using the monthly market returns do not statistically differ from 1.0 at the 10% level. Additionally, the average variance ratios from the baseline simulation, where returns are truly independent, also fall below 1.0. This is consistent with the simulations done in Jorion (2003), which find that sample variance ratios are biased downward. Despite this lack of power, declining variance ratios are often interpreted as evidence for mean reversion in asset prices. Over long investment horizons, mean reversion would tend to dampen investment risk. 3 The idea is that with a sufficiently long horizon, low returns tend to get cancelled out by high returns. To quantify the reduction of risk from the serial correlation of returns in the Monte Carlo framework, I simulate returns from normal distributions that incorporate predictability in expected returns. In any given month, the expected return is higher if the total return over the past one year is high, capturing short-term momentum effects; and the expected return is lower if the total return over the past four years is high, capturing long-term reversal effects. The parameters are estimated using monthly CRSP market returns, and the model ensures that the unconditional mean and 2. See Campbell, Lo, and MacKinlay (1997) for a discussion. 3. This is related to, but not the same as, time diversification, which says that investment risk decreases with the investment horizon. Mean reversion in returns can decrease risk relative to the case where returns follow a random walk. However, for the degree of mean reversion found in historical data, investment risk still increases as one extends the investment horizon.

11 Model Risk in Retirement Simulations 11 variance remain equal to those used in the baseline i.i.d. normal simulation. Additional details about the simulation procedure are provided in the Appendix. The average variance ratios for the simulated returns, provided in Table 2, have a similar pattern to those found in the historical CRSP data, albeit with somewhat lower variance ratios at the three- to five-year horizon. This suggests that there may be more mean reversion in the simulated returns than the actual returns, which could potentially exaggerate the impact of mean reversion on the Monte Carlo analysis. Exhibit 2 displays the distribution of retirement wealth and final wealth for the i.i.d. normal and serially correlated return scenarios. Although investment risk still increases as one extends the investment horizon, it is noticeably reduced relative to the baseline case where returns are independent. There is less uncertainty over wealth, meaning less probability mass in the tails, when returns are predictable in this fashion. This decrease in risk can also be seen in Exhibit 3 where the 90% confidence of wealth over time is narrowed relative to the previous simulations. Additionally, the probability of very bad outcomes is reduced, resulting in increased portfolio survival rates as shown in Exhibit 4. Return predictability increases portfolio survival at age 90 by almost 5% over the baseline i.i.d. normal simulation. 4.2 EAFE Data Because the evidence on mean reversion mainly comes from studies using US data, characterizing long horizon returns is complicated by a small sample size and by survivorship bias. With returns data starting in 1926, there are only about eight truly independent observations of ten-year returns. Furthermore, the US is the most successful capitalist system in the world. Studying only US data may lead one to underestimate long-horizon equity risk. Using an expanded international sample, Jorion (2003) finds that variance ratios are larger than one in thirteen out of thirty countries. The three-, five-, and ten-year variance ratios for the MSCI EAFE Index from January 1970 to July 2008, shown in Table 2, are 1.26, 1.13, and 0.89, respectively. With only 38 years of data, these estimates are far from precise. However, taking the variance ratios at face value, they suggest that there is more positive serial correlation in EAFE returns than in US returns. Even if the autocorrelations are not statistically significant, this does not preclude an assessment of the impact of this autocorrelation structure on Monte Carlo simulations. As in the previous analysis, I allow expected returns to vary through time, depending on the total returns in the past one and four years. The parameters of the model are adjusted to match the serial correlation structure of monthly EAFE returns, but with the same unconditional mean and unconditional variance of the monthly CRSP returns. When the model is calibrated using EAFE returns instead of CRSP returns, the positive relationship between expected returns and the past one-year returns is strengthened,

12 12 Dimensional Fund Advisors while the negative relationship between expected returns and the past four-year returns is weakened. In other words, momentum at the one-year horizon is stronger and reversals at the four-year horizon appear to be weaker in EAFE returns than in CRSP returns. The simulated returns have variance ratios similar to those computed using monthly EAFE returns, as shown in Table 2. Additional details about the simulation procedure are provided in the Appendix. As one can see in Exhibit 2, there is much more uncertainty over wealth in this simulation than in the baseline case. More portfolios end up in the tails of the distribution, meaning that there is greater probability for very good and very bad wealth outcomes. This increased investment risk is also illustrated by wider 90% confidence intervals for wealth, as shown in Exhibit 3. Exhibit 4 displays portfolio survival rates, which are reduced by as much as 5.5 percentage points relative to the i.i.d. normal simulation. Predictability in returns has the potential to distort the characterization of long-term investment risk in retirement simulations. However, long-term risk can be either overstated or understated, depending on how return predictability is specified. Given that there is not enough data to formulate a precise model of return predictability, any model of returns risks being an inaccurate description of future returns. To assume constant expected returns is probably a better approach than to attempt to specify a more complex model of returns. The extent to which this assumption may be inaccurate constitutes a source of model risk that one must keep in mind when interpreting simulation output. 5. Sensitivity to User Inputs The i.i.d. normal assumption is not the only source of modeling risk in a Monte Carlo simulation. The distribution of wealth in retirement also depends critically on the usersupplied input for the mean and variance of returns, savings rates, and withdrawal rates. The sensitivity of Monte Carlo simulations to these uncertain variables is a large shortcoming of the Monte Carlo tool, which is why the output from a single Monte Carlo run should not be treated as the final answer. This section demonstrates the sensitivity of simulation output to expected returns and to savings and withdrawal rates. 5.1 Expected Returns One of the critical inputs into the Monte Carlo simulations is the mean and variance of returns. How does one figure out what these values should be? One natural way would be to estimate these parameters from historical returns, as I have done in this study. However, there is no guarantee that the past is an indicator of future results. Many asset pricing models predict a much lower equity premium than that observed in historical returns, a discrepancy which has been labeled the equity premium puzzle. Fama and

13 Model Risk in Retirement Simulations 13 French (2002) estimate the equity premium to be 2.55% and 4.32% using dividend and earnings growth rates, respectively. These values are far lower than the historical average premium of 8%. 4 Given the uncertainty about the equity premium, a relevant question is how sensitive are Monte Carlo results to the expected return input? Exhibit 6 shows the distribution of wealth under different expected returns assumptions, ranging from 6.2% to 11.4% on an annual basis. With an annualized average one-month US Treasury bill rate of 3.7%, these expected return values correspond to annual equity premiums of 2.5% to 7.7%. In all expected return scenarios, returns are assumed to be distributed i.i.d. normal. Additional wealth summary statistics are provided in Table 3. The results show that wealth outcomes are highly sensitive to the expected return assumption. As the expected return increases, the distribution of wealth shifts left toward higher wealth values, and the left tail becomes longer. This can also be seen in Exhibit 7, which shows that the 90% confidence interval for wealth shifts up as one assumes higher expected returns. Survival rates, plotted in Exhibit 8, deteriorate rapidly as expected returns fall. Under the historical average of 11.7% annual returns, 76% of portfolios survive to age 90. With 10% or 8% annual returns, only 56% and 31% of portfolios, respectively, make it to age 90 with positive values. These changes in survival probabilities are much more drastic than in the previous simulations that relax i.i.d. normality but keep expected returns constant. Moreover, an 8% annual return, which provides a 4.4% premium over the historical one-month US Treasury bill rate, is entirely within the realm of possibility given the research of Fama and French (2002). Relative to the i.i.d. normality assumption, the assumption about the level of expected returns is a much greater source of modeling risk. 5.2 Savings and Withdrawal Rates Arguably, investors have more control over their savings and withdrawal rates than over the distribution of returns, but unforeseen personal circumstances or uncertain inflation levels will introduce some degree of uncertainty into these Monte Carlo inputs. The previous simulations all assume the investor initially saves $12,000 and increases real savings by 5% each subsequent year. In this section, I repeat the simulation under different assumptions about the savings growth rate, allowing it to range from 4% to 6% on a real, annual basis. An alternative interpretation is that the investor targets an 8% nominal increase in savings, but inflation ranges from 2% to 4%. 4. Fama and French (2002) find evidence that suggests that the difference between historical average returns and the expected return premium is due to a decline in discount rates, which produces a large and unexpected capital gain.

14 14 Dimensional Fund Advisors Exhibit 6 Distribution of Retirement Wealth and Final Wealth, by Expected Returns Retirement Wealth Percent Frequency 45% 40% 35% 30% 25% 20% Expected Return (monthly, annual) 0.50%, 6.17% 0.60%, 7.44% 0.70%, 8.73% 0.80%, 10.03% 0.90%, 11.35% 15% 10% 5% 0% > 7.0 Wealth ($ millions) Final Wealth Percent Frequency 90% 80% 70% 60% 50% 40% Expected Return (monthly, annual) 0.50%, 6.17% 0.60%, 7.44% 0.70%, 8.73% 0.80%, 10.03% 0.90%, 11.35% 30% 20% 10% 0% 0 0 < x > 65 Wealth ($ millions) Distribution of portfolio values at age 60 and 90 from Monte Carlo simulations with 5,000 trials. Monthly returns drawn as independent and identically distributed normal random variables with means ranging from 0.45% to 0.9% and with a standard deviation of 5.392%.

15 Model Risk in Retirement Simulations 15 Exhibit 7 90% Confidence Interval of Wealth over Time, by Expected Returns The 5th and 95th percentiles of wealth resulting from 5,000 simulations under various return scenarios. Exhibit 8 Portfolio Survival, by Expected Returns Percent of Portfolios with Positive Value Expected Return (monthly, annual) 0.50%, 6.17% 0.60%, 7.44% 0.70%, 8.73% 0.80%, 10.03% 0.90%, 11.35% Age Distribution of portfolios with positive value at specified ages under various expected return scenarios. There are 5,000 simulated trials for each return scenario.

16 16 Dimensional Fund Advisors Exhibit 9 Distribution of Retirement Wealth and Final Wealth, by Annual Growth of Savings Retirement Wealth Frequency 20% 18% 16% 14% 12% 10% 8% Annual Growth of Savings 4.0% 4.5% 5.0% 5.5% 6.0% 6% 4% 2% 0% > 10.0 Wealth ($ millions) Final Wealth Frequency 35% 30% 25% 20% 15% Annual Growth of Savings 4.0% 4.5% 5.0% 5.5% 6.0% 10% 5% 0% 0 0 < x Wealth ($ millions) > 95 Distribution of portfolio values at age 60 and 90 from Monte Carlo simulations with 5,000 trials. Monthly returns drawn as independent and identically distributed normal random variables.

17 Model Risk in Retirement Simulations 17 Wealth summary statistics are shown in Table 3, and the distribution of retirement and final wealth under the various savings growth rate scenarios is displayed in Exhibit 9. As expected, increasing savings reduces the likelihood of very low wealth outcomes and makes higher wealth outcomes slightly more probable. This is depicted in Exhibit 10 as an upward shift in the 90% confidence interval for wealth. Survival rates in Exhibit 11 also increase as savings rise, with 71% portfolio survival to age 90 when savings grows at 4% and 80% survival under a 6% savings growth assumption. Similar results are obtained when the withdrawal amount in retirement changes in $5,000 increments from $90,000 to $110,000 per year. Exhibit 12 shows that in these simulations, each additional $5,000 withdrawn per year reduces age 90 survival rates by about 2 percentage points. Table 3 Wealth Summary Statistics Simulation Inputs Retirement Wealth ($MM) Final Wealth ($MM) Return (%) Save (%) Withdraw ($K) Mean Std. Dev. Mean Std. Dev. Varying Expected Return Varying Growth of Savings Varying Withdrawal Rate Summary statistics for retirement wealth and final wealth under various assumptions for the expected return, growth rate of saving, and the level of annual withdrawals.

18 18 Dimensional Fund Advisors Exhibit 10 90% Confidence Interval of Wealth over Time, by Annual Growth of Savings $1,000,000,000 $100,000,000 $10,000,000 Annual Savings Growth Rate Wealth $1,000,000 $100,000 $10, Age 90 The 5th and 95th percentiles of wealth resulting from 5,000 simulations. Exhibit 11 Portfolio Survival, by Annual Growth of Savings Percent of Portfolios with Positive Wealth 100% 95% 90% 85% 80% 75% Annual Savings Growth Rate % Distribution of portfolios with positive value at specified ages under various savings growth rate scenarios. There are 5,000 simulated trials for each scenario. Age

19 Model Risk in Retirement Simulations 19 Exhibit 12 Portfolio Survival, by Annual Retirement Withdrawal Percent of Portfolios with Positive Wealth 100% 95% 90% 85% 80% 75% Annual Withdrawal in Retirement ($K) % Age Distribution of portfolios with positive value at specified ages under various withdrawal rate scenarios. There are 5,000 simulated trials for each scenario. 6. Conclusion Like all models, Monte Carlo simulations simplify the world, which in this case makes tractable the dynamic problem of retirement savings in an uncertain environment. Retirement simulations show how return volatility creates uncertainty about wealth over time. However, return volatility is only one source of risk. One must also be aware of additional sources of risk that stem from uncertainty about the modeling assumptions. Before making decisions based on any model, retirement simulations being just one example, it is prudent to understand how the various modeling assumptions could affect the conclusions. One of the assumptions embedded in most Monte Carlo simulations is that returns are i.i.d. (independent and identically distributed) normal. Analyses of historical monthly returns show that this assumption is violated in a number of ways. However, sensitivity analyses in this paper show that the non-normality and the volatility clustering observed in monthly historical returns are unlikely to contribute to drastically different results compared to the simple i.i.d. normal case. On the other hand, return predictability has potential to substantially skew the assessment of long-term investment risk relative to constant expected returns. Depending on how one specifies the correlation structure in returns, long-term investment risk can either increase or decrease. Unfor-

20 20 Dimensional Fund Advisors tunately, there is simply not enough data to measure long-run serial correlations in returns with any precision. The debate in the academic literature about the existence of mean reversion in returns is unlikely to be put to rest until several more decades of data unfold. With so much uncertainty about the long-run correlations of returns, and given evidence showing that predictability can bias Monte Carlo results in either direction, taking the simple approach and assuming constant returns is probably no worse than specifying a more complicated process for returns. One must simply keep in mind that this is a potential source of model risk. Additional sensitivity analysis reveals that users of retirement simulations need to be aware of the sensitivity of the results to user-supplied assumptions, which include the expected return, growth in savings, and withdrawal amounts. Inferences about long-run investment risk and survival rates are highly sensitive to the expected return assumption, and research by Fama and French (2002) suggests that using historical returns may lead one to overestimate the equity premium. For the simulations in this paper, using the historical average to estimate expected returns would lead one to conclude the portfolio has a 76% probability of survival to age 90. However, if the equity premium is actually closer to 3% to 4%, survival rates at age 90 are only around 17%-24%. Additionally, small changes in savings or withdrawal assumptions are shown to have noticeable impact on simulation results. While Monte Carlo simulations will continue to be a useful tool to help illustrate the impact of return uncertainty on retirement outcomes, users must be aware of its limitations. Monte Carlo projections are hypothetical, and real outcomes will be affected by any number of factors that are outside the model. While simulation results are a useful tool to demonstrate the broad impacts of key retirement decisions, they are unlikely to be accurate predictions of future results. They often do not take into account all factors that affect investment returns, such as fees and tax consequences; and as this paper shows, they are quite sensitive to modeling assumptions alone. GARCH Simulation Appendix In a GARCH(1,1) model, the variance of tomorrow s return depends on the variance of today s return and the squared realization of today s return. Monthly returns exhibiting time-varying volatility are simulated from a normal distribution with a mean equal to the average empirical monthly return of 0.929% and with conditional variance, h t, modeled as the following: (A1) h t = 6.37 x h t ε 2 t-1 (1.92 x 10-5 ) (0.018) (0.019) Parameter values are estimated using monthly return data from the CRSP Value-

21 Model Risk in Retirement Simulations 21 Weighted Market Index from January 1926 through July 2008, and standard errors are reported in parenthesis. To simulate returns from the estimated GARCH process in (A1), the initial variance, h 1, is set to equal the unconditional variance of monthly market portfolio returns, 5.392% 2. This value of h 1 is used to draw ε 1 from a normal distribution with a mean of zero and variance equal to h 1. The draws of h 1 and ε 1 are put in the GARCH updating formula (A1) to obtain a forecast of the variance of the second observation, h 2. That is the input to forecast h 3, and so forth, until an entire time series of variance forecasts is constructed. An additional constraint imposed on the simulated returns is that returns must be drawn from the same range as empirical returns. Without this constraint, about 0.1% of the simulated returns are either greater than the maximum return or less than the minimum return observed in the empirical data. In these instances, the return is discarded and redrawn. Simulating Serially Correlated Returns Monthly returns exhibiting serial correlation properties similar to the CRSP Value- Weighted Market Index are simulated from the following model: (A2) R * t,t+1 = R* t,t % R * t 12,t = R* t 12,t 11.73% R * t 48,t = R* t 48,t 55.86% R * t,t+1 = R* t 12,t R* t 48,t + ε t+1 (0.0106) (0.0028) ε t+1 ~ N (0, ) Here, R * is the demeaned m n period return from m to n. Coefficients are estimated m,n using monthly returns of the CRSP Value-Weighted Market Index from January 1926 to July The residual term, ε t, is simulated from a normal distribution with a mean of zero and with variance equal to the root mean square error of the regression. Newey- West standard errors are reported in parenthesis. Similarly, returns calibrated using monthly returns of the MSCI EAFE Index from January 1970 to July 2008 are simulated from the following model: (A3) R * t,t+1 = R* t,t % R * t 12,t = R* t 12,t 11.73% R * t 48,t = R* t 48,t 55.86%

22 22 Dimensional Fund Advisors R * t,t+1 = R* t 12,t R* t 48,t + ε t+1 (0.0121) (0.0031) ε t+1 ~ N (0, ) Drawing return observations from (A2) or (A3) requires at least four years of past observations, which are drawn from a normal distribution with mean and variance equal to the sample moments of market portfolio monthly returns. An additional 1,000 observations are simulated from (A2) or (A3) prior to the initial investment in the Monte Carlo scenario. References Bodie, Zvi On the risk of stocks in the long run. Financial Analysts Journal 51: Bollerslev, Tim Generalized autoregressive conditional heteroscedasticity. Journal of Econometrics 31: Campbell, John Y., Andrew W. Lo, and A. Craig MacKinlay The econometrics of financial markets. Princeton, NJ: Princeton University Press. Engle, Robert F Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica 50: Fama, Eugene F., and Kenneth R. French Permanent and temporary components of stock prices. Journal of Political Economy 96: The equity premium. Journal of Finance 57: French, Kenneth R., G. William Schwert, and Robert F. Stambaugh Expected stock returns and volatility. Journal of Financial Economics 19:3-29. Jorion, Philippe The long-term risks of global stock markets. Financial Management 32 (4): Mandelbrot, Benoit, Adlai Fisher, and Laurent Calvet A multifractal model of asset returns. Cowles Foundation discussion paper no. 1164, Yale University. Markowitz, Harry M., Portfolio selection: Efficient diversification of investments. New York: John Wiley and Sons.

23 Model Risk in Retirement Simulations 23 Poterba, James M., and Lawrence H. Summers Mean-reversion in stock prices: evidence and implications. Journal of Financial Economics 22: Scott, Robert C., and Philip A. Horvath On the direction of preference for moments of higher order than the variance. Journal of Finance 19: