Passive versus Active Fund Performance: Do Index Funds Have Skill?

Transcription

1 Passive versus Active Fund Performance: Do Index Funds Have Skill? Alan Crane, Kevin Crotty Jones Graduate School of Business, Rice University, Houston, TX 77005, U.S.A. Abstract We use the cross-section of index funds to assess the extent of skill in active mutual funds. First, we apply methods designed to disentangle skill and luck in performance evaluation to a set of traded funds that should not exhibit skill: index funds. Surprisingly, the tests imply that index fund skill exists, is persistent, and is found in similar proportion as in active funds. Since this seems implausible, we propose a new method to control for luck in active funds using the index fund cross-section as a zero-skill distribution. Using before-fee returns, we find strong evidence that inferior active funds exist, but little evidence that the top active funds are skilled. Second-order stochastic dominance tests indicate that the performance distribution of index funds dominates that of active funds. We thank Kerry Back, Jonathan Berk, Martijn Cremers, David De Angelis, Hitesh Doshi, Nishad Kapadia, Andy Koch, Sebastien Michenaud, Dermot Murphy, Barbara Ostdiek, James Weston, and seminar/conference participants at Rice University and the 2014 Lone Star Finance Conference for helpful discussions and comments. addresses: alan.d.crane@rice.edu (Alan Crane), kevin.p.crotty@rice.edu (Kevin Crotty) September 29, 2014

2 1. Introduction A fundamental issue in evaluating mutual fund skill is determining whether funds perform well due to skill or luck. In this paper, we apply methods used to separate skill from luck to a set of traded funds that by definition should not exhibit portfolio selection skill: index funds. Under these tests, one would conclude that index fund skill exists, is persistent, and is found in proportions similar to those found in active funds. This is surprising and disconcerting, because outperformance by index funds should be attributed to luck. 1 We propose a new method to control for luck in the cross-section of fund performance. To evaluate active fund skill, we use the cross-section of index funds as a distribution of lucky fund performance. We test whether the best (and worst) active funds are skilled (or unskilled) by comparing their before-fee performance to beforefee index fund performance using quantile regressions. We find strong evidence of unskilled active managers in the left tail and little evidence of skilled managers in the highest percentiles of performance. Using stochastic dominance tests, we also test whether the aggregate amount of skill or lack thereof in active funds warrants investing in active funds versus index funds. We conclude that index funds stochastically dominate active funds in a second-order sense, indicating that a risk-averse investor should prefer a draw from the index fund distribution rather than the active fund distribution. Comparison of active fund performance to index performance is intuitive and dates back at least to Malkiel (1995). However, our tests have only recently been feasible due to the substantial growth in the cross-section of passively managed funds. Over the last forty years, the number of passive index funds has grown from one to over 350 (Investment Company Institute, 2013). This growth has not been confined to S&P 500 funds; index funds now track many style indices. For example, S&P 1 Luck could be due to estimation error or to benchmark model mis-specification. Cremers, Petajisto and Zitzewitz (2013) find that benchmark indices exhibit non-zero alphas under standard benchmark models and propose alternative benchmark models. We find substantial dispersion in index fund performance even using their proposed benchmark models. 1

3 produces underlying indices for 12 different domestic equity value/growth and market capitalization combinations alone, not to mention a variety of sector-specific indices. 2 Recent work on disentangling skill and luck has focused on tests using the crosssectional distribution of active fund performance. The growth in the number of index funds allows us to examine how these methods perform in a cross-section of funds that should be unskilled with respect to portfolio choice. We employ four methods from the literature. First, we find that a large fraction of index funds outperform simulated zeroalpha distributions, similar to the results for active funds in Kosowski, Timmermann, Wermers and White (2006) and Fama and French (2010). They conclude that this outperformance is due to skill, so our results would suggest that index funds are also skilled. In fact, the index funds generally exceed the bootstrap distribution above the 40th percentile under a variety of benchmark models. Second, we estimate the proportion of unskilled, skilled, and zero-alpha managers for the sample of index funds using the false discovery rate methodology of Barras, Scaillet and Wermers (2010). Under a standard Fama-French-Carhart four-factor model, for example, we find that 16% of index funds are classified as skilled, compared to 10% of active funds. Our third test examines the persistence of performance, which has been used as a measure of skill (e.g., Carhart (1997)). We find that performance is persistent in index funds, even before fees. 3 On average, the likelihood of an index fund remaining in the same performance quintile is about 30% from one five-year period to the next; for active funds, the average is approximately 20%, which is what one would expect by chance. Finally, Berk and Green (2004) argue that the observed positive relationship between past fund performance and future fund flows is due to rational learning about 2 S&P regularly compares active fund performance to these indices in its S&P Indices Versus Active Funds (SPIVA R ) U.S. Scorecard (S&P Dow Jones Indices, 2013). 3 Elton, Gruber and Busse (2004) show significant persistence in S&P 500 index fund net returns. Their findings show that much of this persistence is driven by the fees. Our results indicate a significant amount of persistence in the broader index fund space, even for gross returns. 2

4 the skill of fund managers. This suggests that the flow-performance relationship should not exist absent skill. We find that the relationship is even stronger for index funds. For example, an increase in Fama-French-Carhart abnormal performance of 10 basis points (bps) per month is associated with increased flows of 3.5 bps of assets under management for index funds. This is significantly greater than the 1.9 bps increase in assets for active funds. Taken together, these four results would suggest that the index fund distribution has just as much skill as active funds, if not more. If one agrees that index funds have no portfolio selection skill, then our results imply that the the conclusions drawn about the extent of skill using existing methods are subject to benchmark misspecification or estimation error. This is consistent with the evidence documented by Cremers et al. (2013) that some underlying benchmark indices have positive alpha. 4 Our work builds on this finding by showing that inferences related to skill versus luck based on the distribution of performance are affected by this issue. An alternative explanation for our findings is that index funds truly possess a significant amount of skill, possibly due to operational skills such as managing trading costs or lending shares. This possibility does not have strong support in the literature. For example, Fama and French (2010) argue that effects of trading costs and lending revenues are small for passive funds. While these first results are troubling, they also suggest a straightforward alternative to account for luck in the spirit of both Kosowski et al. (2006) and Fama and French (2010). If the dispersion in the performance of index funds arises not due to skill, but to luck, we can use the distribution of index fund performance estimates as a benchmark distribution to assess the extent of skill for active management. In our second set of results, we test for differences between the index fund and active fund distributions. Overall, the results of these tests are surprising. We find little evidence that active funds display greater benchmark-adjusted performance than passive funds. 4 We see consistent results under a host of benchmark models, including those proposed by Cremers et al. (2013) to correct for the issue of alphas for benchmark indices. 3

5 Using quantile regressions, we find that the best index funds compare favorably to the top-performing active funds, even before fees. At the 99th percentile, there is no difference between the gross alphas of active and passive funds. We do find small differences in favor of active funds consistently at the 75th percentile of approximately 5 basis points per month. However, the largest differences occur in the performance of poorly performing funds, with index funds doing significantly better. At the 1st percentile, index funds outperform active funds by approximately 40 basis points per month. These results suggest that a random draw from the distribution of active funds carries a much larger downside compared to index funds while having at most a small advantage on the upside. While the quantile regressions are useful for comparing particular points in the distribution, they cannot speak to the aggregate performance differences between the two groups. To address this question, we test for secondorder stochastic dominance between the two distributions. We find that index funds dominate active funds. Compared to the underperformance by the worst active managers, the magnitude of any active fund skill would be insufficient to induce a risk-averse investor to choose an active fund rather than an index fund. Previous authors have shown certain fund characteristics are correlated with skill using portfolio sorts of mutual funds. For instance, Cremers and Petajisto (2009) document that funds whose holdings deviate substantially from those of their benchmarks outperform (Active Share), and Kacperczyk, Sialm and Zheng (2008) show that funds that outperform their reported holdings also have abnormal returns (Return Gap). If we restrict our active sample to the top quartile of Active Share or Return Gap, we still conclude that index funds dominate these most-active funds in a second-order sense. Outperformance due to luck could be due to estimation noise or model misspecification. Our results do point to some benchmark model mis-specification. In particular, for the subset of funds that are benchmarked to the S&P 500, we see little dispersion in index fund returns net of the benchmark, but a great deal of dispersion in active funds returns in excess of the S&P 500 return. This suggests our use of index funds as portfolios lacking any portfolio selection skill beyond that 4

6 of the underlying index is justified. Our study is the first to use the distribution of index fund returns to better understand the performance ability of active managers. Prior work studying active versus passive performance (e.g., Gruber (1996), Malkiel (1995), Ferri and Benke (2013)) has generally focused on average net returns to investors, which reflect both potential manager skill and the rent-sharing agreement between the investors and the fund. Most recently, Del Guercio and Reuter (2013) compare average active and passive net performance to study incentives induced by the fund s distribution channel for active managers to exert effort. Berk and van Binsbergen (2014b) use Vanguard index funds in a benchmark model to estimate gross dollar performance and conclude that skill is widespread in mutual fund managers. However, index funds exhibit significant dispersion under this performance measure as well, consistent with our interpretation that the performance of active funds is similar to that of index funds. 5 Unlike the prior literature, we focus on the entire distribution of performance rather than average effects. This focus on the distribution leads to our main insights that current tests for skill versus luck may be confounded by benchmark model misspecification and that the distribution of index funds can be used as a benchmark distribution to assess active fund skill. Most broadly, our results contribute to the vast literature on the skill of actively managed mutual funds. A large number of papers conclude that active managers are skilled, while other papers conclude the opposite. 6 Our results, using a new economic hurdle to assess skill, are consistent with the latter. Our paper also contributes to the literature on the appropriate benchmark for 5 We employ this measure in Section Examples of papers concluding at least some active skill include Grinblatt and Titman (1989), Grinblatt and Titman (1992), Grinblatt and Titman (1993), Daniel et al. (1997), Chen et al. (2000), Wermers (2000), Kosowski et al. (2006), Jiang et al. (2007), Kacperczyk et al. (2008), Cremers and Petajisto (2009), Cohen et al. (2010), Fama and French (2010), Barras et al. (2010), Berk and van Binsbergen (2014b), Cremers et al. (2013), Pastor et al. (2014), Jiang et al. (2014), Hunter et al. (2014), and Kacperczyk et al. (2014). Papers concluding no skill include Jensen (1968), Elton et al. (1993), Malkiel (1995), Gruber (1996), Carhart (1997), and Bollen and Busse (2001). 5

7 mutual fund performance studies. As noted by Fama and French (2010) and Berk and van Binsbergen (2014b), passive benchmark returns from standard models do not account for trading costs. Moreover, Cremers, Petajisto and Zitzewitz (2013) show that some underlying passive benchmarks exhibit alpha relative to standard models. Both Berk and van Binsbergen (2014b) and Cremers, Petajisto and Zitzewitz (2013) propose new benchmark models using multiple indices to address these concerns and find evidence of active manager skill. While we use their proposed benchmark models (as well as others), our approach differs in that we compare the distribution of index funds to that of active funds. This is important because index funds exhibit dispersion in benchmark-adjusted returns even using these alternative benchmark models. Issues concerning benchmark choice illustrate the joint hypothesis problem faced by many mutual fund performance studies. Under the assumption that modeladjusted returns are perfect measures of skill, one can test for skill in the cross-section of funds. But if benchmark models are imperfect, it is challenging to distinguish skilled management from imperfect performance measures. In our setting, even if the performance measures are imperfect, as long as these imperfections affect index and active funds to the same extent, we can overcome the standard joint hypothesis problem and test the null of no skilled management. The rest of paper is organized as follows. In Section 2, we describe our sample and benchmark models. Section 3 shows that index funds can appear skilled using methodologies to disentangle skill and luck in performance evaluation. In Section 4, we use the distribution of index fund performance to evaluate the extent of skill in active funds. Section 5 addresses potential concerns for our distributional comparisons by conditioning active funds on proxies for skill and explores the implications of our findings by examining only funds benchmarked or indexed to the S&P 500. Section 6 concludes. 6

8 2. Data and Benchmark Models 2.1. Sample Construction We use fund characteristics and monthly returns from the Center for Research in Security Prices (CRSP) Survivor-bias-free U.S. Mutual Fund Database. Although the Vanguard 500 fund was introduced in the mid-1970 s, the number of index funds was small for the next two decades. Thus, we start our sample in 1995 with 29 index funds. Our sample contains 240 passive funds in total. We merge these data to s12 holdings data from Thomson Reuters using the WRDS MF Links file. We require that a fund match to the holdings data in order to be included in the sample. To avoid double-counting observations for multiple share classes, we aggregate information across share classes, weighting by total net assets in each class and summing total net assets across classes. 7 To be included in our sample, funds must be at least 24 months old to avoid the incubation bias documented by Evans (2010). We also exclude funds whose average net fund assets are below $5 million in the sample. We focus on equity funds, requiring that on average over the sample, at least 90% and at most 105% of the fund s assets be invested in common stocks for a fund to be included in the sample. Many studies identify index funds as funds containing INDEX in the fund s name. However, a portion of the funds identified as index funds in this manner are flagged as Index Enhanced or Index-based Funds by CRSP, suggesting a potential active component to the fund s management. Because we treat index funds as a group of funds with no portfolio choice talent beyond the underlying index, we use a stricter definition of index funds, utilizing the CRSP index fund flag. This flag is only populated later in the sample, so we carry the earliest value back. Under our strict definition of index funds, we only identify funds with a value of D as index funds. This corresponds to Pure Index Funds in the CRSP manual. 8 We examine each fund included in the Index Fund sample to verify our classification. Exchange-traded funds (ETFs) are included in the sample. For our purposes, the 7 We exclude several fund-months with obvious reporting errors in returns. 8 Our conclusions are unchanged when using a broader, name-based definition of index funds. 7

9 differences between traditional index funds and ETFs are minor as both represent traded portfolios without portfolio choice skill. In Section 4, we test whether active funds exhibit performance superior to passive funds. To be conservative in dispersion of performance for the latter set, we restrict the sample to exclude sector, international, and emerging market funds. 9 To identify these, we parse the fund names from CRSP and manually identify words associated with these funds. We flag funds containing these words as sector, emerging market, or international funds. 10 If a fund is flagged in any month, we exclude it from the full sample. We also exclude sector funds based on Lipper codes provided by CRSP. 11 Finally, we manually look at all remaining index funds to ensure that the fund is not a sector fund. Table 1 reports summary statistics for our sample of index and active funds. The sample includes 2,153 distinct funds, 240 of which are passive index mutual funds or exchange-traded funds (ETFs). On average, the index funds in our sample are over twice as large as active funds, but are also younger. 12 This is consistent with the rapid increase in index funds over the last two decades. As expected, expenses are much lower for index funds. The average expense ratio is 47 basis points for index funds and 125 basis points for active funds Are Index Funds Passive? We assume that index funds are passive investments with no portfolio choice skill. To assess the validity of this assumption, we present evidence from holdings and returns that index funds are predominantly passive investment vehicles. In Table 1, turnover, as reported by CRSP, is much lower for index funds; the median index fund has a turnover of 24% compared to 74% for active funds. Another 9 We exclude these funds because they may have the potential to create dispersion in the index fund distribution due to concentrated holdings. Our results are robust to inclusion of these funds. 10 Some words may plausibly appear as part of the fund name (e.g., due to the fund family) or in ways that are clearly not related to a sector fund. We manually checked words where this is the case and did not flag funds that are clearly not sector funds. 11 A list of the words and Lipper codes is available upon request. 12 We control for these differences by examining dollar returns and t-statistics, respectively. 8

10 measure used to capture unobserved actions of funds is the Return Gap measure of Kacperczyk, Sialm and Zheng (2008), which compares the net investor return with the net holdings return. The net holdings return represents the return the fund would have experienced if it had kept holdings constant. Any differences in returns are due to interim trades by the fund. Table 1 shows that the cross-sectional average and median return gap measures are close to zero for both active and passive funds, but that the dispersion is about twice as large for active funds relative to passive funds, indicating that index funds are much less active funds. We also obtain benchmark information, Active Share, and tracking error volatility from Antti Petajisto s website constructed using the methodology outlined in Petajisto (2013). 13 These data are available through For the subset of funds in the Petajisto (2013) dataset matching our sample, we first confirm that index funds are holding the underlying index constituents at the same weights as in the index. Cremers and Petajisto (2009) develop a holdings-based measure, Active Share, to study how active a manager is. If index funds are substantially deviating from benchmark weights, the Active Share should deviate from zero. Positions orthogonal to the index would have an Active Share of one. Table 1 shows large differences in Active Shares across fund type. The median index fund has an Active Share of 0.02, indicating that these funds hold assets in proportions very close to those of the benchmark. On the other hand, the median active fund deviates widely from its benchmark, as evidenced by the median Active Share of 0.8. The Petajisto (2013) dataset also calculates annualized volatility of tracking error of daily returns. We again find large differences between active and passive funds on this dimension. The median index fund has tracking error volatility of 70 bps per year while that of the median active fund is 6.5%. On balance, index funds appear quite passive relative to actively managed funds, validating their use as funds with no portfolio choice skill beyond that of the underlying index. 13 Available at: 9

11 2.3. Benchmark models As measures of performance, we use both benchmark-adjusted returns and their t-statistics. The appropriate benchmark model is a matter of extensive debate in the mutual fund literature. 14 of benchmark models detailed in Appendix A. For completeness, we present results using a number We use various benchmark models to account for different levels of systematic risk-taking. We start by using the single market model of Jensen (1968) as well as the Fama-French-Carhart (FFC) four-factor model of Carhart (1997). 15 Recently, Berk and van Binsbergen (2014b) use an orthogonal basis of eleven Vanguard index funds as benchmark funds, which we use as our fourth benchmark model. We also use the seven-factor model (CPZ7) proposed by Cremers, Petajisto and Zitzewitz (2013). Finally, we adjust for time-varying risk-taking using the conditional fourfactor model of Ferson and Schadt (1996). In Section 4, we compare performance across the active and passive distributions. An advantage of our approach is that our results should be insensitive to shortcomings in the benchmark models provided any mis-specification is similar across the two sets of funds. For each model, we estimate loadings and time-series alphas for each fund according to the following model: r it r f t = α i + n j=1 β j i F j t + ɛ it (1) where r it is fund i s return in month t, r f t is the risk-free rate, and F j t is the excess return on benchmark return or factor j in period t. Standard errors are adjusted for heteroskedasticity. We report summary statistics of the full sample gross Fama-French-Carhart factor loadings in Table 1. The loadings are similar across the index and actively managed funds. Both groups have average market betas of approximately one and a slight tilt 14 See, for example, Cremers, Petajisto and Zitzewitz (2013) or Berk and van Binsbergen (2014b). 15 Factor returns are available at Ken French s website: library.html. 10

12 towards small firms. Neither group loads heavily on value or momentum strategies on average. The estimated α is our first measure of performance. An advantage of this measure is that it provides the economic magnitude of any abnormal performance, allowing us to gauge the economic value added by a fund. Due to different sample lengths or heterogeneous risk-taking by funds, an estimated α may not have attractive sampling properties. For these reasons, Kosowski et al. (2006) and Fama and French (2010) analyze the distribution of t-statistics associated with estimated alphas. Consequently, we use t(α) as our second measure of performance. We examine these measures using before fee (gross) returns. 16 Gross alphas (and t-statistics) allow us to ask the question of whether or not a fund exhibits sufficient skill to outperform the passive benchmark implied by the model (an alpha greater than zero) or sufficient skill to outperform alternative investments (an alpha greater than that of index funds) Skilled Index Funds? Index funds, by definition, should be devoid of portfolio selection skill. In this section, we assess the efficacy of tests designed to separate skill and luck in performance evaluation. The results in this section are essentially placebo tests of the current methodology Bootstrapping the cross-section of t(α) Recent work by Fama and French (2010) and Kosowski, Timmermann, Wermers and White (2006) uses bootstrap analysis to simulate distributions of skill measures under the null of no skill. These studies both recognize that the underlying cross-sectional distribution of fund returns is likely to be non-normal, and therefore 16 For some funds, CRSP does not report expenses monthly. For these funds, we carry forward the annually reported fees to subsequent monthly observations. 17 A study of net returns, on the other hand, addresses a different question: whether active managers have sufficient skill to cover the fees they charge to investors. However, this will capture, in part, the bargaining process between fund investors and managers (Berk and Green (2004)). To abstract from this confounding economic mechanism, we analyze gross performance. 11

13 inference based on standard critical values can be confounded. The idea is simply that there is a spread in alpha estimates due to noise in estimation and the statistical properties of the individual fund returns, even in the absence of true alpha (i.e., skill). Both studies simulate the null distribution of fund returns by sampling from actual fund returns net of estimated alphas. The studies differ on sample construction and bootstrap methodology, but both conclude that a small set of active funds possesses skill. How do index funds, a set of funds with no portfolio selection skill beyond the underlying benchmark, fare relative to the bootstrap distribution? The results provide insights into the effect of possible model mis-specification and the importance of non-portfolio selection activities such as trading costs and securities lending revenues. To assess this, we use the approach of Fama and French (2010). While our sample differs, our bootstrap methodology is the same. For each fund-month, we subtract a fund s estimated gross alpha from the fund s monthly gross return. This leaves us with a panel of monthly fund zero-alpha returns. From this data, we draw a bootstrap sample of months (with replacement) from the set of all months in our sample. If we draw a given month, we use all fund returns from that month to retain any cross-sectional correlation in monthly returns. For each bootstrap sample, we then calculate the time series alpha and t(α) for each fund. This provides us with a cross-sectional distribution of t(α) estimated from returns that by construction should have a true alpha of zero. We repeat this 10,000 times and average across the bootstrap samples at each point in the distribution of estimated t(α). We perform this analysis for zero-alpha distributions using each of the benchmark models described in Section 2. Figure 1 plots the bootstrapped and actual distributions of gross zero-alpha performance (in terms of t(α)) for index and active funds under the Fama-French- Carhart model. The bottom panel shows that the results for actively managed funds in our sample are consistent with Fama and French (2010). There is evidence active funds both underperform (below the 40th percentile) and outperform (above the 40th percentile) the bootstrapped zero-alpha distribution. However, the top panel of Figure 1 shows that index funds also generally outperform the no-skill distribution 12

14 above the 20th percentile. This may not be surprising given that the index funds face practical trading considerations (e.g., trading costs or equity lending fees). However, if the alpha due to portfolio selection is zero, then this would imply that lending fees significantly exceed trading costs, which seems unlikely. It is surprising that the index funds perform better compared to the bootstrapped sample over a large part of the distribution. We tabulate the percentiles of the bootstrapped and actual distributions of gross t(α) performance for index funds in Table 2. The tabulated results support the graphical evidence across all of the benchmark models. In particular, the empirical distribution of t(α) for index funds outperforms the bootstrap distribution above the 50th percentile for all models. As in Fama and French (2010), we also report the fraction of bootstrap runs in which a given percentile of the simulated distribution falls below the empirical percentile value. Fama and French (2010) discuss how this measure can be used for informal inference concerning the likelihood of observing the difference in performance between the simulated and actual data. In particular, if the bootstrapped and actual distributions are equal at a given percentile (i.e., no skill), the likelihood value should be 0.5. Interestingly, the likelihood statistic can be biased away from 0.5 for extreme percentiles even for distributions with no skill (i.e., alpha of zero). This bias is decreasing in the number of funds in the cross-section (not the number of bootstrap samples). This is a possible concern given the small size of the index fund crosssection. We use Monte Carlo simulations to establish appropriate critical values for the likelihood statistics under the null of no differences in the distributions (i.e., no skill). 18 Our estimated likelihoods for percentiles where the actual estimate exceeds the average bootstrap value (bold entries in Table 2) are generally well in excess of these critical values. Thus, the actual distribution of index fund performance significantly outperforms the zero-skill bootstrap distribution. Table 3 reports the same tests for our actively-managed sample. The results look quite similar to both Fama and French (2010) and the index funds. The ac- 18 Details of the Monte Carlo results are available on request. 13

15 tual percentiles are greater than the bootstrapped percentiles above the median. Kosowski, Timmermann, Wermers and White (2006) and Fama and French (2010) interpret these results as evidence of skill. A similar interpretation of the index fund results implies that some index funds are skilled as well. The other possibility is that the tests are confounded by errors in measuring performance through the use of benchmark models. As such, our comparison between the active and index space in Section 4 can potentially further our understanding of skill relative to a comparison of the bootstrapped distribution alone The proportion of skilled, zero-alpha, and unskilled funds In a similar vein to the bootstrap tests, Barras, Scaillet and Wermers (2010) use a variant of the false discovery rate (FDR) estimation developed by Storey (2002) to estimate the fractions of funds in the cross-section that are skilled, unskilled, and zero-alpha. The technique controls for false discoveries of mutual fund skill, i.e., mutual funds exhibiting significant alphas by luck alone. If one assumes that funds are drawn from one of three populations (skilled, unskilled, and zero-alpha), the cross-sectional distribution of t-statistics for risk-adjusted alphas will be a mixture distribution. The right tail of this mixture distribution will contain both skilled funds and lucky zero-alpha funds. Using a critical value for the t-statistics alone will falsely attribute skill to these lucky zero-alpha funds. The FDR technique uses the fact that the p-values for both skilled and unskilled funds will be located disproportionately close to zero, while the p-values for zeroalpha funds will be uniformly distributed from zero to one. Intuitively, the more the mass in the p-value distribution close to zero differs from the uniform level of the distribution close to one, the lower the proportion of zero-alpha funds in the crosssection. The additional mass on the left-side of the p-distribution is due to either skilled (positive alphas), unskilled (negative alpha) funds, or both. The fractions of the population containing these types of funds can be estimated using the sign of the t(α) for each fund. We estimate the proportions of unskilled, zero-alpha, and skilled funds (ˆπ, ˆπ 0, 14

16 ˆπ + ) in the index fund population using the Barras et al. (2010) methodology. 19 The results using gross returns are shown in Table 4. For purposes of comparison, we also report estimates for the active fund sample. Surprisingly, the estimated proportion of skilled funds (ˆπ + ) in the index fund population is at least as large as the estimated proportion of skilled funds in the active fund population in all benchmark models except the market model. Under a standard Fama-French-Carhart four-factor model, we find that 16% of index funds are classified as skilled, compared to 10% of active funds. The results provide little evidence that active funds consist of skilled funds in greater frequencies than the supposedly unskilled group of index funds. If anything, the evidence suggests a smaller fraction of active funds are skilled than in index funds, even before fees are considered Persistence of α Empirical studies of mutual fund performance often point to the (lack of) persistence of risk-adjusted performance as evidence of the (lack of) skill for managers (e.g., Carhart (1997)). In this section, we evaluate the persistence of performance for index funds and compare it to that of active funds. Specifically, we estimate Fama-French-Carhart alphas for half-decade subsamples (i.e., , , etc.) and sort funds into quintiles based on the alphas in each period. If relative performance persists, then transition matrices of the alpha quintiles should be disproportionately populated along the diagonals. If funds are truly skilled, we should expect persistence in the top quintile in particular. If there is no persistence and performance rankings are random from one period to the next, we should see uniform transition probabilities of 20% across the entire matrix. We present gross alpha transition matrices for index and active funds in Table λ [0, 1] denotes the threshold above which p-values are assumed to be generated from zeroalpha funds only (i.e., funds with alpha p-values greater than λ are comprised solely of zero-alpha funds). γ denotes the significance level used for determining the critical t-value used to estimate the fraction of lucky zero-alpha funds incorrectly identified as possessing skill (or lack of skill). We fix λ at 0.5 and γ at 0.35 to put active and index funds on equal footing, but our results are qualitatively unchanged if we follow the selection algorithms for λ and γ used by Barras et al. (2010). 15

17 Panels 5a and 5b show transitions from to , and Panels 5c and 5d show transitions from to For the first transition, there is weak evidence of persistence in skill for the top active funds; 24.5% of funds in the top quintile remain in that quintile. This is only slightly above the null of 20%. In contrast, for those index funds in the top quintile in the first period, over 40% remain in that quintile. Moving to Panels 5c and 5d, we see stronger evidence of persistence for index funds and less evidence for actively managed funds over the later time period. The transition probabilities for the active funds are generally close to the null of 20%. However, we see significant persistence along the entire diagonal for index funds. Across the two periods, the average diagonal transition probability for index funds is about 30% compared to about 20% for active funds. Overall, the results provide little evidence that the top-performing active funds possess skill in the form of persistent alpha. If anything, passive funds display more persistent performance, even before fees The Flow-Performance Relationship A number of papers have documented a positive relationship between net fund flows and lagged performance in active mutual funds (e.g., Sirri and Tufano (1998), Chevalier and Ellison (1997)). A leading explanation for this relationship is that investors rationally update their beliefs about manager skill based on past performance (Berk and Green (2004)). We assess the flow-performance relationship in the context of our unskilled group of index funds. By construction, investors should not attempt to update their beliefs about manager talent for these funds. If flow-performance is due to rational learning by investors about managers stock picking abilities, index fund flows should not be responsive to flows. We examine the relationship between net fund flows and lagged performance for both active and index funds. As is standard in the literature, we measure new money 20 Elton, Gruber and Busse (2004) show significant persistence in S&P 500 index fund net returns. They find that much of this persistence is driven by fees. Our results indicate a significant amount of persistence in the broader index fund space, even for gross returns. 16

18 growth as: F low it = T NA it T NA it 1 (1 + r it ) T NA it 1 where T NA it is the total net assets under management by fund i in month t. Flows are winsorized at the 1% level. Table 6 presents panel regressions of net fund flows on lagged returns and an interaction of lagged returns with an index fund indicator variable. We use gross excess returns and benchmark-adjusted returns, controlling for log total net assets and a fund s expense ratio. 21 Each regression contains year and fund fixed effects. As has been widely documented in the literature for active funds, new money growth is positively correlated with lagged fund performance measured using any performance measure. We show that this effect exists for a broad set of index funds as well. 22 The interaction term estimates are positive and statistically significant using all benchmark-adjusted returns. An increase in Fama-French-Carhart abnormal performance of 10 basis points (bps) per month is associated with increased flows of 3.5 bps of assets under management for index funds. This is significantly greater than the 1.9 bps increase in assets for active funds. In general, the results indicate that new money growth in index funds is more sensitive to past performance than it is in active funds. 23 This result is inconsistent with investors rationally updating about fund skill if index funds are assumed to have no skill. As such, we provide empirical evidence consistent with Choi, Laibson and Madrian (2010), who present experimental evidence that investors chase past performance even within S&P 500 funds if presented with differently framed information. On the other hand, the results could be consistent with investors rationally updating about which investment strategies outperform and 21 Using preferences revealed by mutual fund flows, Berk and van Binsbergen (2014a) argue that CAPM is closest to the true model. To be consistent with our previous tests, we use abnormal returns from each benchmark model. 22 Elton, Gruber and Busse (2004) show a flow-performance relationship between S&P 500 funds using post-expense performance, but do not relate this to active funds. 23 While index mutual funds and ETFs are used for sector exposure, we note that we have excluded sector-specific funds from our sample, so this does not drive the result. 17

19 which funds have the best execution. Nonetheless, it remains puzzling that index funds would be more responsive to past returns than active funds. 4. Controlling for Luck Using the Performance Distribution of Index Funds The surprising finding that index funds appear skilled leads us to propose a new method to disentangle skill from luck. In this section, we assess the extent of activelymanaged skill by using the cross-section of index fund performance as a distribution of performance measures under the null of no skill. These tests are similar in spirit to the bootstrap tests of Kosowski et al. (2006) and Fama and French (2010), but we use the traded index fund distribution rather than the bootstrapped distribution as our counterfactual, no-skill distribution. We assess differences between active and passive fund performance at various points of the distribution using quantile regressions. Finally, we use (second-order) stochastic dominance tests to assess whether the extent of any active fund skill (or lack thereof) in aggregate is sufficient to induce a riskaverse investor to choose an active fund rather than an index fund Cumulative distribution functions In Figure 2, we plot the cumulative distribution functions (CDFs) of gross alphas for index and active funds under the various benchmark models. We also display the returns in excess of the S&P 500 since this is a common benchmark used for equity fund performance in practice. We expect the priors of most readers to be that index funds have a narrower benchmark-adjusted return distribution, with a possible negative level shift relative to active funds for before-fee returns. Absent a level shift, a narrower distribution for index fund alphas corresponds to a CDF that is to the right (left) of the active CDF below (above) the median. We focus on gross returns to abstract away from the bargaining process between investors and fund managers. Panel 2a shows the surprising result that the CDF of fund returns in excess of the S&P 500 is remarkably similar for above-median funds. The largest differences in the distributions are in the left half of the distributions where index fund exhibit alphas much closer to zero than those of the 18

20 worst-performing active funds. When adjusted for systematic risk using benchmark models (Panels 2b-2f), the poorer performance of below-median active funds remains. The similar performance of the top-performing funds generally survives the benchmark model adjustments. The largest above-median differences appear in the 70th to 90th percentiles, where active funds have slightly higher alphas. We explicitly test the statistical and economic significance of these differences in Section 4.2. Figure 3 plots the CDFs of alpha t-statistics for index and active funds. discussed in Kosowski et al. (2006) and Fama and French (2010), t-statistics may be a better representation of skill because they adjust the alpha estimate for the residual variance a fund takes in order to earn that alpha (as well as for statistical issues related to differing sample size across funds). There are some differences here across benchmark models, but we again see very few differences between active and passive funds in the right tail. Even after adjusting for differences in precision of alpha estimates due to residual variance differences, the top-performing funds appear quite similar in the portion of the distribution where one would expect to find the most skilled funds. The outperformance of index funds below the median is weaker under the t-statistic measure, but the results suggest they are at least as good as the active funds Quantile Regressions To statistically test for differences in these distributions, we examine the distribution of alphas and t-statistics for each risk model and measure differences between index funds and actively managed funds at various points in the distribution. To do this, we use a quantile regression approach. Following Angrist and Pischke (2008), we examine distribution effects by finding the conditional quantile function that solves the following: Q τ (y i X i ) = arg min E[ρ τ (y i q(x i ))] (2) q(x) where ρ τ (µ) = (τ 1(µ 0))µ for quantile τ and y i is a risk-adjusted fund performance measure estimated from equation 1, either α i or t(α i ). We estimate q(x i ) as As 19

21 a linear function of covariates: q(x i ) = β 0 + β 1 Index i (3) where Index i takes the value of one if fund i is an index fund. Using this approach, we test the differences between the index fund performance and the actively managed fund performance at various points of their respective distributions. We analyze the 1st, 5th, 25th, 50th, 75th, 95th, and 99th percentiles of the benchmark-adjusted return distribution. Statistical significance is determined by calculating bootstrapped standard errors Gross alphas We again start by examining gross alphas. In Table 7, we present results from the quantile regressions of benchmark-adjusted returns, before fees, on an index fund indicator variable. Each panel of the table presents the active fund performance (the constant) and the difference in alpha distributions between index funds and active funds (the coefficient on Index i ) under each of the different benchmark models. Each column represents a different quantile of the distribution in ascending order. For example, the middle column, Q50, represents the median active fund s benchmarkadjusted return (Constant coefficient) and the difference in medians across the two distributions (Index coefficient). The results are consistent with the visual evidence provided by the CDFs. The simplest benchmark adjustment considers fund returns in excess of the S&P 500 return. As in the CDF plot, we see almost no difference in passive and active fund performance at or above the median. The median fund in our sample beats the S&P 500 by 11 bps per month before fees. Below the median, however, index funds statistically and economically out-perform corresponding active funds. It is worth noting that this simple benchmark adjustment gives active managers credit for well-known strategies that most studies generally do not consider skill (e.g., value or momentum). It is therefore striking that even under the model that should bias most in favor of active managers, we see no evidence that the best funds outperform the best passive alternatives. 20

22 For purposes of discussion, we focus generally on the results from the distribution of four-factor (Fama-French-Carhart) alphas. These results are presented in the third panel of Table 7. Median risk-adjusted performance for actively managed funds is approximately one basis point per month. At the median, there is no economic or statistical difference between index funds and active funds; the coefficient on the index fund indicator is zero. Even before fees, median risk-adjusted performance is similar across the two groups. The left tail of the distribution is where we observe the largest differences. Under the four-factor model, we estimate that for the very worst funds (Q01), index funds outperform active funds. While the estimated performance of active funds in this quantile is -78 basis points per month, the index funds are only losing half that amount. This estimate is large economically and is statistically significant at the 1% level. As we move along the distribution, the performance difference in favor of index funds gets smaller economically, but is still large relative to the active fund performance at that point in the distribution. At the 25th percentile, active funds lose 11 basis points per month, while index funds lose three basis points. Perhaps these differences in the left tail are not surprising. If one believes that index funds merely track passive portfolios, then we might expect active managers, either due to poor talent or bad luck, to do worse on the down side just by virtue of the fact that they are picking stocks. However, this would suggest that the active managers should then perform better in the right tail of the distribution. We see relatively little evidence of this. At the 99th percentile, there is no statistically significant difference between the active and the passive funds. Even if we lack power to detect skill in the tail of the distribution, the magnitude of the effect is economically insignificant relative to the estimated performance of funds in that part of the distribution (the point estimate of the difference is zero basis points). 24 This is surprising. Given the evidence of Kosowski et al. (2006), Fama and French (2010), and Barras, Scaillet and Wermers (2010), who find small but significant talent in the 24 It is worth noting that we have power to detect differences in the left tail of the distribution, so it is reasonable that we do not lack power on the right side. 21

23 best funds, it is remarkable that this talent does not seem to exceed the performance of the index funds with the largest estimated alphas. We do estimate some differences in the performance of index funds at the 95th and 75th percentiles. In a few models, the index funds perform statistically significantly worse than the active funds at the 95th percentile. However, the magnitudes are model dependent and in one case, the estimated difference even flips signs. At the 75th percentile, there is consistent evidence in favor of the active funds. However, the magnitudes of this effect are small at around four basis points a month. This advantage is roughly half the size of the underperformance of the active funds at the 25th percentile. Overall, this suggests that any small advantage the active managers may have in the right shoulder of the distribution is more than offset by their poor performance in the left side of the distribution Gross t(α) In Table 8, we turn our attention to the distribution of t-statistics. There are some subtle differences relative to the alphas. While we still see little compelling evidence to suggest that active managers outperform, the advantage that index funds have in the left tail of the distribution is reduced when performance is adjusted for estimation precision. This is primarily due to the fact that the poor performance of the worst index funds, while smaller in magnitude, is very precisely estimated. Therefore, adjusting for residual risk, the index funds advantage in the left tail is smaller. In the right half of the distribution, the magnitudes of the differences are small, and the magnitude and signs of the estimates differ depending on the benchmark model choice. Overall, it would be difficult to conclude that the actively managed funds perform better in the right tail of the distribution Gross Dollar Returns Berk and van Binsbergen (2014b) argue that benchmark-adjusted alphas are poor measures for mutual fund skill and that skill should be measured as the total dollar value extracted from the market by fund managers. Specifically, they calculate skill 22

24 as the time-series average of the dollar value of benchmark-adjusted performance: Skill i = 1 T i T i t=1 q it 1 α it where q it 1 is the lagged real assets under management and α it is the gross benchmarkadjusted performance of fund i in month t. Given the size disparity between active and passive funds, this measure could lead to different conclusions relative to our previous tests. We calculate Skill i for active and index funds using our set of benchmark models and report the results in Table 9. Our skill measures are denoted in 2013 dollars. The results are qualitatively the same as using time-series alphas to measure skill. The distribution of the dollar amount extracted from the market by active funds overlaps substantially with the performance distribution of index funds in the right tail while index funds outperform active funds in the left tail. 25 In our sample, the median active fund loses $27,000 per month to the market on a benchmark-adjusted basis (Fama-French-Carhart), before fees. The point estimate for the median index fund is slightly higher, a difference of $52,000 per month. In the tails, the distribution of dollar value returns is narrow for index funds, although the results are only statistically significant in the left tail where the differences are also economically large. This outperformance by index funds is significant at the 75th percentile and below Stochastic Dominance Tests Our quantile regression results are useful in assessing how percentiles of the index and active fund distributions compare, but assessing the aggregate amount of skill in the active space is challenging due to multiple comparison issues. To overcome these issues, we ask whether there exists enough skill (positive or negative) in the active 25 Berk and van Binsbergen (2014b) report the distribution of their value added measure for active and index funds in Table 7 of their paper. Consistent with the main results of our paper, their active and index fund distributions exhibit remarkably similar levels of dispersion from the 5th to 95th percentile. Unlike our sample, they find that index fund dollar returns are lower at the 1st and 99th percentiles, presumably due to different sample periods and filters. 23

25 fund space to warrant a risk-averse investor choosing an active fund over an index fund. We answer this question by testing the null that active funds stochastically dominate index funds and vice versa. We can easily reject the nulls that either distribution dominates the other in a first-order sense, so we focus our discussion on second-order stochastic dominance. Our tests of stochastic dominance follow the bootstrap-based test of Barrett and Donald (2003), which are based on Kolmogorov-Smirnov tests comparing the distributions at all points. The intuition of the test of the null that distribution G secondorder stochastically dominates distribution F is that a number of bootstrapped draws from F generates a distribution of CDFs that can be compared to the empirical distribution G. The deviations between F and G can be compared to those found between F and the bootstrapped CDFs to determine the likelihood of observing the deviations between F and G by chance. Table 10 displays p-values associated with the tests of stochastic dominance for the distributions of alpha and t(α). For alphas, we cannot reject the null that index funds dominate active funds, but we can strongly reject the null that active funds dominate index funds for all benchmark models. When using t(α), we again cannot reject the null that index funds dominate active funds for any benchmark model. For four of the six benchmarks, we can again reject the null that active funds dominate index funds. Economically, the results indicate that the magnitude of any active fund outperformance relative to index funds is insufficient to outweigh the active funds underperformance relative the worst index funds. A risk-averse investor facing a random draw from the distribution of active funds versus the distribution of index funds should prefer the index fund lottery based on benchmark-adjusted performance. 5. Should All Active Funds Be Compared To Index Funds? We have shown that the distribution of benchmark-adjusted performance of index funds is remarkably similar to that of the population of active funds. If an investor faces a random draw from index funds versus active funds, the results suggest index funds provide an investment opportunity set that is at least as good as active funds. 24

26 Some readers may argue that this is not a fair comparison. One concern may be that investors need not randomly choose between active funds. Researchers have identified fund characteristics associated with better ex-post performance, and perhaps these active funds should be compared to index funds. Another concern may be that the appropriate comparison should be within a given benchmark, e.g. S&P 500 funds, rather than across all funds. In this section, we address these potential concerns and their implications Index Funds vs. High Active Share/Return Gap Active Funds As discussed in the introduction, a strand of the mutual fund evaluation literature has focused on identifying characteristics of funds that are correlated with ex-post performance. In this section, we test how the performance of index funds compare to that of active funds selected based on these characteristics. We split the crosssection of active funds using two characteristics: Active Share (Cremers and Petajisto (2009)) and Return Gap (Kacperczyk, Sialm and Zheng (2008)). We compare the performance of active funds in the top quartile of these measures over our sample period to the performance of index funds. The results using the Fama-French-Carhart model are summarized in Figure 4. If these most actively-managed funds are all skilled, their performance distributions should be shifted to the right; that is, they should first-order stochastically dominate the index funds. The plots of gross alphas show that the distribution of the most active actively-managed funds is wider than that of index funds, but they do not dominate the index funds. While some of the most active funds exhibit higher alpha estimates in the right tail before fees, the most active funds also underperform in the left tail. If gross alphas are scaled by idiosyncratic risk (t-statistics), the index fund distribution is still remarkably similar to that of the most active activelymanaged funds, and active funds are indistinguishable from passive funds over most of the distribution. In terms of second-order stochastic dominance, the alphas of index funds dominate even the alphas of these most-active funds for either top quartile Active Share or Return Gap funds. The test of the null Index dominates Active has a p-value 25

27 of (0.549) for Active Share (Return Gap) while the test of the null Active dominates Index is strongly rejected with a p-value of (0.001) for Active Share (Return Gap). Active funds look most attractive under the t-statistic distributions. For Active Share, these results are inconclusive; we cannot reject the null of secondorder dominance in either direction. On the other hand, Active Funds dominate Index Funds using the t-statistic distributions for the active funds in the top quartile of Return Gap. We can reject the null of Index dominates Active with a p-value of While these results do not provide support that all high Active Share or Return Gap funds are skilled, they do suggest that a fraction of these funds outperform the index funds. These results lend credence to the use of cross-sectional sorts to help identify skill. Nonetheless, these methods are noisy enough that investors who base decisions on benchmark-adjusted returns should still prefer an investment in index funds S&P 500 Funds Another potential concern with our proposed skill tests utilizing the cross-section of index funds lies in the (standard) benchmark models employed. One could argue that the appropriate comparison should be within a given benchmark, e.g. S&P 500 funds, rather than across all funds. In this section, we analyze the impact of benchmark model mis-specification by testing how index funds compare to active funds when we restrict both sets to have the same benchmark. In particular, we examine the relative performance of active versus passive funds for the subset of funds that benchmark to the S&P 500. The advantage of looking at this sample is that, for this subset of funds, we ostensibly know the true benchmark. 26 Therefore, we are able to abstract from standard benchmark models and just examine returns in excess of the return on the underlying S&P 500 index. With this approach, we credit active managers for skill even if they outperform the index by tilting toward well-documented strategies (e.g., value, momentum). To classify an index fund as an 26 This assumes that active managers state the correct benchmark. Sensoy (2009) provides evidence that many active funds state benchmarks that do not match their actual style. 26

28 S&P 500 fund, we hand-checked the underlying index for all index funds. For active funds, we use the subset of actively managed funds benchmarked to the S&P 500 as identified by Cremers and Petajisto (2009). Table 11 presents the results of quantile regressions for these two alpha groups. In the first panel of Table 11, we see that gross alphas are consistent with what we believe to be the common prior regarding the performance of actively managed funds. The spread in alphas for the S&P 500 index funds is very small. Relative to the true benchmark, the index fund alphas are estimated at -1 basis point for 1st percentile funds to +3 basis points for the 99th percentile. This suggests that these index funds generally do what they are designed to do: track the S&P 500 index. This also confirms that the index funds, when compared to their true benchmark exhibit little in the way of skill. It also suggests that the dispersion in index fund alphas found in Section 4 is due to benchmark model mis-specification rather than operational skill. Some actively-managed S&P 500 funds earn much larger positive returns before fees. At the 99th percentile, the best index funds underperform the best active funds by about 55 basis points a month. However, it is also clear that the poor performing active funds are much worse than the index funds at the same points in the respective distributions. The poor performance is relatively symmetric compared to the difference between the best index funds and active funds. The worst index funds outperform the worst active S&P 500 funds by about 52 basis points. 27 At the median, active funds do better by only four basis points per month. On a t-statistic basis, the story is very different. Once we take into account the residual variance of gross excess returns, we see that skill of actively managed funds is strictly less than that of index funds, even in the tails. While active funds may be able to outperform the benchmark and the index funds, they do so by taking on substantial risk relative to the benchmark. The t-statistics are statistically significantly higher for index funds across all major points in the distribution. From 27 Despite the smaller sample size, we continue to report the same quantiles as in the full sample analysis for comparison. 27

29 the standpoint of the t-statistics, investors would be better off choosing index funds, even before fees. In terms of our stochastic dominance tests, S&P 500 index funds dominate S&P 500 active funds. We can reject the null that Active dominates Index (p=0.000 for either alphas or t(α)) but cannot reject the null that Index dominates Active (p=0.224 for alphas, p=0.878 for t(α)). These results indicate that any skill in the active fund space is more than outweighed by poorly-performing, unskilled funds. If one believes that index funds should exhibit no portfolio selection skill, then the dispersion in the performance of these funds under classic benchmark models suggests either model mis-specification or operational skill. For the S&P 500 index funds, it is clear that using the true benchmark eliminates model mis-specification. We see almost no dispersion in performance for these funds relative to the true benchmark. This suggests that the dispersion in index fund alphas found in Section 4 due to operational skill is in fact quite small, as it should show up in the S&P 500 tests as well. Unfortunately, it is well known that the stated benchmark does not necessarily correspond to the true benchmark for active funds, so it is difficult to interpret any dispersion of performance in active funds in these tests as evidence of skill. 6. Conclusion We revisit the widely studied, yet still debated, topic of whether active mutual fund managers are skilled by comparing the distributions of index and active fund performance. Using standard measures from the literature, we first document that one would conclude that the index fund distribution contains skilled funds, contradicting their role as unskilled, passive investments. These results motivate the use of the index fund performance distribution as a set of counterfactual, unskilled funds to assess the extent of skill in the active fund distribution. We find that the topperforming index funds exhibit performance similar to the top active funds. However, for below-median funds, passive investments generally outperform actively-managed funds. In addition, we show that index funds stochastically dominate active funds in benchmark-adjusted performance, indicating that unskilled active managers more 28

30 than offset any skilled active managers. On balance, we interpret our empirical findings as consistent with the view that actively-managed funds exhibit little portfolio selection skill. 29

31 Appendix A. Benchmark models 1. Market Model (Jensen, 1968) r it r f t = α i + β 1 MKT t + β 2 SMB t + β 3 HML t + ɛ it 2. Fama-French-Carhart (Fama and French, 1993; Carhart, 1997) r it r f t = α i + β 1 MKT t + β 2 SMB t + β 3 HML t + β 4 UMD t + ɛ it 3. Vanguard Basis (Berk and van Binsbergen, 2014b) 11 r it r f t = α i + β j i V j t j=1 + ɛ it 4. Cremers-Petajisto-Zitzewitz 7-factor (Cremers, Petajisto and Zitzewitz, 2013) r it r f t = α i + β 1 S5RF t + β 2 RMS5 t + β 3 R2RM t + β 4 S5V S5G t + β 5 RMV RMG t + β 6 R2V R2G t + β 7 UMD t + ɛ it 5. Conditional Four-Factor (Ferson and Schadt, 1996) r it r f t = α i + β 1 MKT t + β 2 SMB t + β 3 HML t + β 4 UMD t K + B i,j [z j,t 1 MKT t ] + ɛ it j=1 The factor returns and conditioning variables are defined as follows: 30

32 r it is fund i s return in month t MKT t is the excess return of the CRSP value-weighted market return in month t SMB t is the return of a portfolio long small-cap stocks and short large-cap stocks in month t HML t is the return of a portfolio long high book-to-market stocks and short low book-to-market stocks in month t UMD t is the month t return of a portfolio long past winners and short past losers based on lagged returns V j t is the excess return on Vanguard index j, orthogonalized to V n<j t. The Vanguard index funds are S&P 500 Index, Extended Market Index, Small- Cap Index, European Stock Index, Pacific Stock Index, Value Index, Balanced Index, Emerging Markets Stock Index, Mid-Cap Index, Small-Cap Growth Index, and Small-Cap Value Index. S5RF t is the excess return of the S&P 500 index in month t RMS5 t is the Russell Midcap minus S&P 500 R2RM t is the Russell 2000 minus Russell Midcap S5V S5G t is the S&P 500 Value minus S&P 500 Growth RMV RMG t is the Russell Midcap Value minus Russell Midcap Growth 31

33 R2V R2G t is the Russell 2000 Value minus Russell 200 Growth z j,t 1 is the t 1 deviation of public information variable j from its time series mean We use K = 4 conditioning variables: 1. The 1-month Treasury bill yield 2. The dividend yield of NYSE/AMEX firms over the previous 12 months 3. The term spread (10-yr Treasury - 3-month Treasury yield) 4. The default spread (Yield difference between Baa and Aaa corporate bonds). 32

34 References Angrist, J.D., Pischke, J.S., Mostly harmless econometrics: An empiricist s companion. Princeton University Press. Barras, L., Scaillet, O., Wermers, R., False discoveries in mutual fund performance: Measuring luck in estimated alphas. The Journal of Finance 65, Barrett, G.F., Donald, S.G., Consistent tests for stochastic dominance. Econometrica 71, Berk, J., van Binsbergen, J.H., 2014a. Assessing Asset Pricing Models Using Revealed Preference. Working Paper. Berk, J., van Binsbergen, J.H., 2014b. Measuring Skill in the Mutual Fund Industry. Working Paper. Berk, J., Green, R., Mutual fund flows and performance in rational markets. Journal of Political Economy 112, Bollen, N.P., Busse, J.A., On the timing ability of mutual fund managers. Journal of Finance 56, Carhart, M.M., On persistence in mutual fund performance. Journal of Finance 52, Chen, H.L., Jegadeesh, N., Wermers, R., The value of active mutual fund management: An examination of the stockholdings and trades of fund managers. Journal of Financial and Quantitative Analysis 35,

35 Chevalier, J., Ellison, G.D., Risk taking by mutual funds as a response to incentives. Journal of Political Economy 105. Choi, J.J., Laibson, D., Madrian, B.C., Why does the law of one price fail? an experiment on index mutual funds. Review of Financial Studies 23, Cohen, R., Polk, C., Silli, B., Best ideas. Working Paper. Cremers, K.J.M., Petajisto, A., How active is your fund manager? A new measure that predicts performance. The Review of Financial Studies 22, Cremers, M., Petajisto, A., Zitzewitz, E., Should benchmark indices have alpha? revisiting performance evaluation. Critical Finance Review 2, Daniel, K., Grinblatt, M., Titman, S., Wermers, R., Measuring mutual fund performance with characteristic-based benchmarks. The Journal of Finance 52, Del Guercio, D., Reuter, J., Mutual fund performance and the incentive to generate alpha. Journal of Finance, forthcoming. Elton, E.J., Gruber, M.J., Busse, J.A., Are investors rational? choices among index funds. The Journal of Finance 59, Elton, E.J., Gruber, M.J., Das, S., Hlavka, M., Efficiency with costly information: A reinterpretation of evidence from managed portfolios. Review of Financial Studies 6,

36 Evans, R.B., Mutual fund incubation. The Journal of Finance 65, Fama, E.F., French, K.R., Common risk factors in the returns on stocks and bonds. Journal of Financial Economics 33, Fama, E.F., French, K.R., Luck versus skill in the cross-section of mutual fund returns. The Journal of Finance 65, Ferri, R.A., Benke, A.C., A Case for Index Fund Portfolios. Technical Report. URL: Ferson, W.E., Schadt, R.W., Measuring fund strategy and performance in changing economic conditions. The Journal of Finance 51, Grinblatt, M., Titman, S., Mutual fund performance: An analysis of quarterly portfolio holdings. Journal of Business, Grinblatt, M., Titman, S., The persistence of mutual fund performance. The Journal of Finance 47, Grinblatt, M., Titman, S., Performance measurement without benchmarks: An examination of mutual fund returns. Journal of Business, Gruber, M.J., Another puzzle: The growth in actively managed mutual funds. The Journal of Finance 51, Hunter, D., Kandel, E., Kandel, S., Wermers, R., Mutual fund performance evaluation with active peer benchmarks. Journal of Financial Economics 112,

37 Investment Company Institute, Investment Company Fact Book. Technical Report. URL: factbook.pdf. Jensen, M.C., The performance of mutual funds in the period The Journal of Finance 23, Jiang, G.J., Yao, T., Yu, T., Do mutual funds time the market? evidence from portfolio holdings. Journal of Financial Economics 86, Jiang, H., Verbeek, M., Wang, Y., Information content when mutual funds deviate from benchmarks. Management Science, forthcoming. Kacperczyk, M., Sialm, C., Zheng, L., Unobserved actions of mutual funds. Review of Financial Studies 21, Kacperczyk, M., Van Nieuwerburgh, S., Veldkamp, L., Time-varying fund manager skill. The Journal of Finance, forthcoming. Kosowski, R., Timmermann, A., Wermers, R., White, H., Can mutual fund stars really pick stocks? new evidence from a bootstrap analysis. The Journal of Finance 61, Malkiel, B.G., Returns from investing in equity mutual funds 1971 to The Journal of Finance 50, Pastor, L., Stambaugh, R., Taylor, L., Scale and Skill in Active Funds. Working Paper. 36

38 Petajisto, A., Active share and mutual fund performance. Financial Analysts Journal 69, Sensoy, B.A., Performance evaluation and self-designated benchmark indexes in the mutual fund industry. Journal of Financial Economics 92, Sirri, E., Tufano, P., Costly search and mutual fund flows. Journal of Finance 53, S&P Dow Jones Indices, S&P Indices Versus Active Funds U.S. Scorecard (SPIVA R ) Year-End Technical Report. URL: Storey, J.D., A direct approach to false discovery rates. Journal of the Royal Statistical Society 64, Wermers, R., Mutual fund performance: An empirical decomposition into stock-picking talent, style, transactions costs, and expenses. The Journal of Finance 55,

39 Table 1: Summary Statistics This table presents summary statistics for our sample. The sample contains monthly fund observations from 1995 to Risk-loadings are estimated using the Fama-French-Carhart four factor model: r it r f t = α i + β 1 MKT t + β 2 SMB t + β 3 HML t + β 4 UMD t + ɛ it where r it is fund i s return in month t. Fund characteristics are winsorized at the 1/99% level and averaged over the time-series of each fund. The cross-sectional mean, median, and standard deviation of these time-series averages are reported. Active Share and Tracking Error Volatility are for the subsample matching to the Active Share data of Petajisto (2013). Mean Median SD Number of distinct mutual funds 2153 Number of active funds 1913 Number of passive funds 240 Number of fund-month observations Number of funds per month Number of index funds per month Index Funds TNA (total net assets) (millions) Age (years) Expense ratio (in %) Turnover ratio (in %) Return Gap (in %) Active Share SD(Tracking Error) (annualized %) MKT loading SMB loading HML loading UMD loading Active Funds TNA (total net assets) (millions) Age (years) Expense ratio (in %) Turnover ratio (in %) Return Gap (in %) Active Share SD(Tracking Error) (annualized %) MKT loading SMB loading HML loading UMD loading

40 Table 2: Percentiles of t(α) for Index Fund Actual and Bootstrapped Returns This table shows t(α) values at various percentiles for both the bootstrapped zero-alpha distribution and the actual index fund performance distribution. The bootstrap methodology follows Fama and French (2010). The Sim columns contain the average bootstrap value of t(α) at the various percentiles (averaged over 10,000 draws). The Act columns contain the empirical distribution. Percentiles in which the actual value exceeds the average bootstrap value are in bold font. The Lik column reports the fraction of bootstrap runs in which the bootstrapped percentile falls below the actual percentile. A value of 0.5 indicates the actual distribution is indistinguishable from the bootstrap distribution. We display the percentiles for each of the five benchmark models. CAPM Fama-French-Carhart Vanguard Cremers-Petajisto-Zitzewitz Ferson-Schadt Pct Sim Act Lik Sim Act Lik Sim Act Lik Sim Act Lik Sim Act Lik

41 Table 3: Percentiles of t(α) for Active Fund Actual and Bootstrapped Returns This table shows t(α) values at various percentiles for both the bootstrapped zero-alpha distribution and the actual active fund performance distribution. The bootstrap methodology follows Fama and French (2010). The Sim columns contain the average bootstrap value of t(α) at the various percentiles (averaged over 10,000 draws). The Act columns contain the empirical distribution. Percentiles in which the actual value exceeds the average bootstrap value are in bold font. The Lik column reports the fraction of bootstrap runs in which the bootstrapped percentile falls below the actual percentile. A value of 0.5 indicates the actual distribution is indistinguishable from the bootstrap distribution. We display the percentiles for each of the five benchmark models. CAPM Fama-French-Carhart Vanguard Cremers-Petajisto-Zitzewitz Ferson-Schadt Pct Sim Act Lik Sim Act Lik Sim Act Lik Sim Act Lik Sim Act Lik

42 Table 4: Proportion of Skilled, Unskilled, and Zero-Alpha Funds - Gross Alpha This table presents estimates of the proportions of unskilled, zero-alpha, and skilled funds (ˆπ, ˆπ 0, ˆπ + ) in the population of active and index funds using the false-discovery rate methodology of Barras, Scaillet and Wermers (2010). Results are presented across a number of benchmark models. The estimated proportions are truncated at zero if negative. Standard errors are calculated as in Barras, Scaillet and Wermers (2010) and are shown in parenthesis. λ [0, 1] denotes the threshold above which p-values are assumed to be generated from zero-alpha funds only (i.e., funds with alpha p-values greater than λ are comprised solely of zero-alpha funds). To ensure comparable estimates across active and index fund distributions, the threshold λ is fixed at 0.5, and the significance level (γ) is fixed at 0.35 (see Barras, Scaillet and Wermers (2010) for details). Fund Type Unskilled (ˆπ ) Zero-Alpha (ˆπ 0 ) Skilled (ˆπ + ) CAPM Index (0.029) (0.065) (0.049) Active (0.010) (0.023) (0.018) Fama-French-Carhart Index (0.031) (0.063) (0.048) Active (0.011) (0.023) (0.017) Vanguard Index (0.035) (0.063) (0.046) Active (0.017) (0.022) (0.012) Cremers-Petajisto-Zitzewitz 7-Factor Index (0.023) (0.063) (0.051) Active (0.016) (0.022) (0.012) Ferson-Schadt Index (0.029) (0.064) (0.049) Active (0.012) (0.023) (0.017) 41

43 Table 5: Persistence of Gross α This table presents transition matrices for alpha estimates under the Fama-French-Carhart model. Funds are sorted into quintiles based on their estimated gross alphas from each half-decade subsample. Each row shows the transition of a fund from the lagged quintile into quintiles in the current period. Panel headers indicate the current period. The top quintile (High) contains the top-performing funds; the worst-performing funds are in the Low quintile. Beige represents transition probabilities close to random (15% - 25%). Green represents above random persistence (>25%) and red represents less than random persistence (<15%). (a) Index Funds ( ) (b) Active Funds ( ) (c) Index Funds ( ) (d) Active Funds ( ) 42