ETF Specific Data Point ethodologies orningstar ethodology Paper December 31 2010 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or in part without the prior written consent of orningstar Inc. is prohibited. 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 1
Tracking Error Description Tracking Error is a measure of how closely an ETF follows its benchmark index. This data point seeks to isolate the ability of the portfolio managers to track the index ignoring the confounding effects of ETF liquidity on market prices. To do this the calculation measures deviations of the portfolio net asset value from the total return version of the underlying index. Lower tracking error shows better replication of the benchmark index by the ETF manager which implies the ETF will fit better in an asset allocation based upon index returns. any other data providers measure tracking error as the average deviation of fund returns from the benchmark index. orningstar decomposes that traditional measure into two separate data points: tracking error and estimated holding cost. Our tracking error calculation isolates the risky unpredictable deviations of performance from the index which are typically caused by sampling or otherwise incomplete replication of the benchmark portfolio. Our estimated holding cost captures the predictable drift from the total return version of the index due to both explicit and hidden costs and revenues such as swap contract spreads dividend withholding taxes and securities lending. Calculation Tracking Error is calculated by a linear regression of the ETF s daily NAV returns on the underlying index s daily total returns for the past six months. Tracking Error is the standard deviation of the error term in that regression. To calculate Tracking Error pull daily NAV returns for the ETF and its underlying total return index for the past six months (125 trading days) and arrange them into matrixes X and Y as follows: 1 1 X = 1 1 B B B B 1 Y = 1 Where B-t is the return on the underlying total return index t trading days ago and N-t is the return on the ETF s NAV t trading days ago. 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 2
Tracking Error (continued) Estimate 2x1 matrix β with the following calculation: β = [X T X] -1 X T Y Where [X T X] -1 is the inverse of the 2x2 square matrix created by multiplying the transpose of X by X. Then the 125x1 matrix of residualsε is calculated as: ε = Y - X β Tracking Error = σ ε 250 Where σ ε is the standard deviation of the 125 elements ofε and the multiplier annualizes the calculation for easier comparison with other statistical calculations. 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 3
Estimated Holding Costs Description Estimated Holding Cost is a measure of how well an ETF manager is performing relative to their benchmark index after all expenses both disclosed and undisclosed. This calculation seeks to measure the ability of portfolio managers to produce extra income while keeping costs down ignoring the confounding effects of ETF liquidity on market prices. To do this the calculation measures predictable returns of the portfolio net asset value relative to the underlying index. A greater estimated holding cost shows that a manager is doing a better job of producing ancillary income while finding the lowest cost ways to replicate their index. any other data providers measure tracking error as the average deviation of fund returns from the benchmark index. orningstar decomposes that traditional measure into two separate data points: tracking error and estimated holding cost. Our tracking error calculation isolates the risky unpredictable deviations of performance from the index which are typically caused by sampling or otherwise incomplete replication of the benchmark portfolio. Our estimated holding cost captures the predictable drift from the total return version of the index due to both explicit and hidden costs and revenues such as swap contract spreads dividend withholding taxes and securities lending. Calculation The Estimated Holding Cost calculation uses the daily NAV and total return benchmark returns from the Tracking Error calculation. 2 125 1 G i= N i B i Estimated Holding Cost = [ ] Where G is the Gearing atio of a given fund an operational data point giving the relative level of short or long exposure that a fund seeks relative to its benchmark. The multiplier annualizes the calculated value for easier comparison with other statistical calculations.
arket Impact Cost Description arket Impact Cost along with the bid-ask ratio and volume measures is a measure of the liquidity of an ETF. Less liquid securities are more thinly traded and a single large trade can move their prices considerably. orningstar s market impact measure is an estimate of the basis point change in an ETF s price caused by a $100000 trade. A lower market impact implies the ETF is more liquid but the actual size of price movements due to a single trade may vary considerably from this estimate depending on market activity. The market impact cost for an ETF incorporates the information of the bid-ask spread and even the depth of market order books by measuring the volatility of market prices around the true portfolio value. Our market impact calculation standardizes that volatility to a basis point value from a $100000 trade through the common assumption that trades in the ETF are sequentially independent and return variance caused by the trade increases in linear proportion with the dollar size of each trade. Calculation arket Impact Cost is calculated as a fraction of the daily volatility in an ETF s market price relative to its underlying value (as determined by the NAV). The calculation of the daily volatility is done by a linear regression of the ETF s daily market returns on the ETF s daily NAV returns for the past six months. First pull daily market returns for the ETF and its underlying index for the past six months (125 trading days) and arrange them into matrixes X and Y as follows: 1 1 X = 1 1 N N N N 1 Y = 1 Where N-t is the NAV return on the ETF t trading days ago and -t is the market return on the ETF t trading days ago. 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 5
arket Impact Cost (continued) Estimate 2x1 matrix β with the following calculation: β = [X T X] -1 X T Y Where [X T X] -1 is the inverse of the 2x2 square matrix created by multiplying the transpose of X by X. Then the 125x1 matrix of residuals ε is calculated as: ε = Y - X β The standard deviation of the 125 elements of ε is σ ε which gives the daily volatility of the ETF s market price relative to its underlying value. This measure needs to be standardized into an estimate of market impact from a single $100000 trade. To do so pull the daily trading volumes and closing market prices for the past six months (125 trading days) and calculate the average daily dollar volume as: V 1 = 1 d i V = i P 125 i Where V i is the share trading volume for the ETF i trading days ago P i is the closing market price for the ETF i trading days ago and V d is the average daily dollar trading volume of the ETF. With V d and σ ε calculated arket Impact (I) is: I =σ ε 100000 V d 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 6
Portfolio Concentration Description Portfolio Concentration measures the idiosyncratic (non-market) risk taken on by an ETF. Since an efficient market does not compensate higher idiosyncratic risk with higher returns a lower portfolio concentration is better for investors seeking to avoid security-specific and sector-specific risks. Investors looking to make a tactical call on a specific industry or sector might seek higher concentration as that suggests an ETF portfolio has higher exposure to sector-specific return factors. This measure only currently applies to U.S. equity international equity and fixed-income ETFs as those three broad asset classes have diversified return factors generally agreed upon in the academic finance literature. Equity ETFs use proxies for U.S. or global market size and value risk factors to separate diversified systemic portfolio risk from idiosyncratic risk. Fixed-income ETFs use proxies for market (aggregate) credit and duration risk factors to separate diversified systemic portfolio risk from idiosyncratic risk. Calculation Portfolio Concentration is calculated by a linear regression of the ETF s daily market returns on the daily returns of three risk factors calculated from orningstar indexes. Portfolio Concentration is the standard deviation of the error term in that regression. To calculate Portfolio Concentration pull daily returns for the past six months (125 trading days) for the ETF and the following indexes: orningstar US arket T orningstar Large Cap T orningstar Small Cap T orningstar US Value T and orningstar US Growth T. For the trading day t days ago let -t be the return on the ETF s market price that day B-t be the return on the orningstar US arket T index S-t be the return on the orningstar Small Cap T index minus the return on the orningstar Large Cap T index and V-t be the return on the orningstar US Value T index minus the return on the orningstar US Growth T index. 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 7
Portfolio Concentration (continued) Arrange these calculated values into matrixes X and Y as follows: 1 1 X = 1 1 B B B B 1 S S S S 1 V V V V 1 Y = 1 Estimate 4x1 matrix β with the following calculation: β = [X T X] -1 X T Y Where [X T X] -1 is the inverse of the 4x4 square matrix created by multiplying the transpose of X by X. Then the 125x1 matrix of residuals ε is calculated as: ε = Y - X β The standard deviation of the 125 elements of ε is σ ε the concentration risk for the ETF. To adjust this methodology for Fixed-Income ETFs: eplace orningstar US arket T index with BarCap US Aggregate Bond T eplace orningstar Large Cap T index with BarCap US Aggregate 1-3 Yr T eplace orningstar Small Cap T index with BarCap US Aggregate 10+ Yr T eplace orningstar US Value T index with BarCap US Corporate High Yield T eplace orningstar US Growth T index with BarCap US Government T To adjust this methodology for International Equity ETFs: eplace orningstar US arket T index with SCI AC World Ex USA II G eplace orningstar Large Cap T index with SCI AC World Ex USA G eplace orningstar Small Cap T index with SCI AC World Ex USA Small G eplace orningstar US Value T index with SCI AC World Ex USA Value G eplace orningstar US Growth T index with SCI AC World Ex USA Growth G 2010 orningstar Inc. All rights reserved. The information in this document is the property of orningstar Inc. eproduction or transcription by any means in whole or part without the prior written consent of orningstar Inc. is prohibited. 8