Quality evaluation of the odel-based forecasts of iplied volatility index Katarzyna Łęczycka 1 Abstract Influence of volatility on financial arket forecasts is very high. It appears as a specific factor which causes that financial instruents prices are highly changeable which causes probles with accuracy of their forecasts. When the arket is dynaically developing, regulation of its volatility forecasts and agnification of their accuracy have particular significance, even though calculating it is coplex and its values are hard to be interchangeably deterined. That is the reason why odel-based forecasts and especially their accuracy should be constantly odified and iproved to as precisely as possible. The ai of this paper is extended analysis of the iplied volatility VIX index volatility forecasts quality by exaining various errors of forecasts based on the GARCH class odel which allows to decide whether odel-based forecasts are sufficiently accurate. These forecasts are next copared with realised VVIX index quotes fro the appropriate period of tie. The VVIX is established as an estiator of volatility of VIX, because financial arket and odel-free iplied volatility appear to be the ost accurate representation of this unobservable variable. It is not only the proble with volatility itself. By estiating the appropriate volatility estiator for VIX index, the result is an unobservable variable of unobservable basic instruent iplied volatility index. The chosen periods consider both cal and fluctuating periods, arket tendencies (analysis of quasi-stable ups or downs) and their interdependencies. Keywords: quality evaluation of forecasts, GARCH odeling, iplied volatility indices JEL Classification: C53, G17 1. Introduction The econoetric easures that allow to deterine volatility level of financial instruents are very iportant in financial analysis. It is necessary to be aware of possible changes in trends of the instruents prices, both for long-ter and iediate strategies, portfolio and single instruents. The faster decision ust be taken, the greater proble is with calculating volatility forecasts before taking the risk. Properly evaluated quality of forecasts allows to react ore efficiently. Process of coputing and evaluating data paraeters ought to be uch faster than it had been before, ainly because of rising frequency of recorded transactions and character of trading itself. This highly accelerates process of trading by shortening periods between recorded transactions to intradaily data, which are fundaentally irregularly spaced. 1 Corresponding author: Wroclaw University of Econoics, Financial Investents and Risk Manageent Departent, Koandorska 118/10, 53-345 Wrocław, Poland, e-ail: katarzyna.leczycka@ue.wroc.pl. 136
This ultra-high frequency data, where spaces between recorded transactions were finally shortened to only few nanoseconds, is liited only by the nuber of copleted transactions per given period of tie (Engle, 000). Traders are obligated to ake decisions faster and while they are creating their strategies in such conditions. The adapted ethods ust be precise and elastic to allow changing strategies ore frequently. There is also proble with volatility because of its abiguous character as a easure. Volatility cannot be precisely and interchangeably deterined as other observable variables, which causes that selecting appropriate estiator is necessary. These are only few reasons why iportance of analyzing the quality of forecasts is bigger than it had been before and is still rising. The ai of this paper is to conclude whether the conditional variances of the VIX index (coonly used abbreviation of the Volatility Index which is calculating on the Chicago Board Options Exchange) calculated using odel-based forecasts are sufficiently accurate as estiator in coparison with iplied volatility provided by odel-free VVIX index (the abbreviation of other Volatility Index which is also calculating on the CBOE and in which the basic instruent is the VIX). The latter approach is intuitively the ost appropriate estiator of volatility of VIX, because it is already the ost reliable indicator of arket signals. The observed interchangeable volatility values of VIX do not exist, therefore the VVIX index is assued for the needs of this paper as observed values of VIX volatility. Evaluating the quality of odel-based forecasts is given by coparison with the observed values of VVIX tie series for suitable period of tie. It could help to decide whether odelbased approach is sufficiently correct and whether relying on it instead of on iplied volatility is reasonable and adequate. The odel-based forecast will be represented by GARCH class odel. The quality of forecasts will be evaluated by interpreting the set of popular error types and easures outlined below in section. The paper proceeds as follows. Section outlines shortly the ost iportant eleents of different approaches which allow to easure volatility, both received fro odel-based forecasts and iplied volatility. It also shows the theoretical aspects about errors useful in evaluating forecasts quality. Section 3 presents data series used for this study and soe iportant data issues. Section 4 presents the epirical results and contains both single and coplex, synthetic interpretation of these results. At the end, section 5 provides the ain concluding rearks. 137
. Theoretical aspects of forecasting volatility Forecasting the ain trends of future volatility level and other iportant tendencies and changes in prices of financial instruents, such as derivatives, or in paraeters for instance as iplied volatility indices, nowadays sees to be necessary or in rational approach even obligatory. Mentioned process of receiving forecasts allows investors and analysts ore or less precisely decide, whether the risk of given investent should be taken or not..1 Measures of iplied volatility The instruents that easure volatility are financial instruents designed to track the value of iplied volatility of soe other derivative instruents, for instance, the CBOE Volatility Index (the VIX), which is coputed fro a weighted average of given iplied volatilities of various options that are quoted on the S&P 500 Index. Iplied volatility can be easured using different approaches. The odel-free iplied volatility presented in VIX index is for instance justifying validity of the VIX forula under the jup-diffusion process with stochastic instantaneous variance, denoted as a SVSCJ (Chien-Hung and Yueh-Neng, 010). The VVIX deterines a volatility of volatility by easuring the expected volatility of price of the VIX predicted in advance with 30-days. The ethod of calculation in VVIX index is the sae ethodology as the one which is used to calculating the values of VIX index, with the ain obvious difference - the basic instruent is the VIX. It is a value that is derived fro the price of a portfolio which consist of liquid VIX options that are at- and out-of-the-oney. This portfolio could be traded to handle the volatility risk of exposures to the VIX and to gain fro risk preiu which is a difference between the expected and observed volatility levels of VIX forward prices. The VVIX index could be described as a 30-day volatility, but in practice the VIX options with expiration date with 30 days forward are usually not available. The auxiliary VVIX-like values are in that case calculated fro these VIX options, that their expiration dates are the two dates before and after previously entioned 30-days period. Then the VVIX index is interpolated fro those additional nubers. The VVIX is calculated fro VIX options prices using the VIX forula presented below: K 1 i RT F Q( K ) 1 T e i i K T K (1) i 0 where σ is VVIX/100, T tie to expiration, F forward index level derived fro index option prices, K strike price, Ki strike price of ith option, ΔKi interval between strike prices, R risk-free rate to expiration, Q(Ki) - idpoint of bid-ask spread for each option. 138
It is also iportant that the value of VVIX index cannot be istaken with the expected volatility of the VIX itself, because these are two different approaches use two different ethods to various applications.. Model-based volatility forecasts and their quality easures The coplex and sophisticated ethods used in calculating iplied volatility in the VIX sees to be adequate and enough accurate to predict instruent's volatility relying only on the. In financial literature exists nuerous researches that concludes that iplied volatility yields superior forecasts of future volatility and in cobination of approaches often iplied volatility is preferred (Laoureux and Lastrapes, 1993; Pong et al., 004), but it is also exained, that S&P500 options arket cannot anticipate any unfailiar forward oveents in variability that could not be anticipated by the odel-based forecast, such as GARCH class odels (Becker et al., 007). The ain weakness of GARCH class odels is that the expected variance of returns is calculated as a polynoial of its historical values the past squared returns. On the other hand it is quite intuitive ultipurpose tool to research variability. Estiated conditional variance ht and residual of the odel εt could be received by adjusting odel to its epirical values (Piontek, 00): t hf, t k ht () where ht estiated conditional variance, εt estiated odel residual, (,, ) vector of estiated odel paraeters, hf, t+k forecast of conditional variance. In previous section, there was presented why quality of volatility forecasts is so iportant. The ain proble with easuring volatility is that this variable is unobservable, so to evaluate the quality of forecasts a coparison with values of assued volatility estiator is required. Because volatility in odel-based approach is deterined as conditional variance of returns, the natural consequence is to deterine squares of returns in approach using iplied volatility indices. This approach is not free of any coplications (Doan and Doan, 009). Errors of forecasts ex post forecast ^ y e is a difference between the value of the odel based in this paper received as a result of GARCH odel forecast and observed value of the forecasted variable y (Witkowska, 005), in this paper represented by the VVIX index, as on equation (3): 139
e ^ y y. (3) The set of the ost iportant easures that have applications in evaluating quality of forecasts, both in forecasts of conditional ean and of volatility, which are also called synthetic error easures ex post outlines as follows: Mean squared error: MSE 1 N ( ˆ ) N yt i y, (4) T i i1 Median squared error: Mean error: MedSE ediana ( y ˆ ) N T i yt i i, (5) 1 ME N 1 ( y ˆ T i yt i) N, (6) i1 Mean absolute percentage error: 1 y ˆ T iyt i. (7) N MAPE N i1 yti These errors allow to evaluate basically the quality of forecasts. Besides these entioned basic ones, there are any ore, that are soe kind of conversions and odifications of these presented above. For instance, the ean absolute error MAE where the only difference fro entioned above ME is that the eleents of subtractions are its absolute values, the root ean square error RMSE which is a root of previously entioned MSE and adjusted ean absolute percentage error (AMAPE) where the only difference fro the MAPE appears in its denoinator where the su consisted of was used. 3. The data set The paper shows a coparison of 3 different periods of the tie series presenting VIX index forecasts and suitable VVIX index observed quotes for 3 observed periods. The GARCH odels applied to calculate forecasts were fitted to 3 data sets. Each of this data sets consists of a characteristic data that represent appropriate type of data first one was the trend of the 140
VIX quotes quite aggressively decreasing in period fro 7 th October 008 to 4 th May 009, the second one was the trend apparently increasing in period fro 15 th May 008 to 17 th Noveber 008 and the last third period of relative static stabilization of the VIX quotes with possibly sall ean value of yields variance captured in period fro 18 th July 013 to nd January 014. The nuber of forecasts, which is equal to observed values of VVIX in each of these 3 periods was 55, which gives about 11 weeks (equal to less than 3 onths). Only the working days were considered. The tie series both of the VIX and the VVIX indices used for the calculations were downloaded fro the official website of CBOE, where the tie series of appropriate quotes are available and daily uploaded. 4. Analysis and interpretation The ain acroeconoic reason for decreasing trend in the end of 008 to May 009 could be that financial arkets started to receive stabilization after the shocks after crisis (the ore ruors and negative inforation such as crisis potential consequences, the higher level of volatility on the financial arkets). This could describes why the increasing tendency had place in the early 008, where the crisis was causing indirectly still the bigger and bigger volatility level. Nowadays, the volatility is uch lower, because of the lack of recent accuulation of negative inforation, so in given coparison period fro 013 to 014 appears as the ost stable. As it was declared in introduction (section 1), the ai of this paper is to evaluate quality of forecasts by coparing the results of VIX volatility calculated using conditional variances in GARCH class odel with the iplied volatility index observed for the sae period by analyzing the VVIX index. To calculate the VIX index odel-based volatility, after pursuing the appropriate coparison, the GARCH(p,q) odels that have been chosen was in all three cases the GARCH(1,1) for the ost stable period, the increasing period and the decreasing period. This were a GARCH odels with the Gaussian distribution. To select the best fitted odel, odels GARCH(1,1), GARCH(1,), GARCH(,), GARCH(,3) and GARCH(3,3) were copared with each other for each period. The ain reason why odel was chosen was the coparison of its p-value (the less p-value the better), standard error (the less standard error the better) and t-statistic. The received forecasts of VIX were copared with the VVIX in suitable periods to calculate the ex post forecast errors, which are presented in Table 1. 141
Forecast error Increasing period Decreasing period Stable period MSE 0.0084 0.0079 0.006 MedE 0.008 0.0001 0.005 ME 0.0088 0.000 0.001 RME 0.0094 0.0046 0.0461 MAPE 685.91% 117.68% 17.94% RMSE 43.55% 17.56% 8.30% AMAPE 65.93% 59.58% 49.30% MAE 0.013 0.009 0.0009 Table 1. Calculated errors of odel-based forecasts. The results of evaluating quality of forecasts proved that odel-based forecasts have no possibilities to equal to iplied volatility odels. The conditional variances are ainly based on historical dataset and do not reflect the other effects, because they are not designed in connection with the other paraeters, for instance expiration date of instruent or its arket price (on the other hand iplied volatility used in the VVIX index does). It was also quite siple to predict that divergence between the calculated values and observed ones will be high because both represent different ethodology of calculating unobserved paraeter the volatility. The odel-based forecasts are not recoended to be applied in high-frequency datasets and other instruents and strategies that deand taking the fast and efficient shortter decisions, always when the better and ore efficient equivalents are constantly available on arket. Iplied volatility is usually better fro ost of odel-based forecasts, such as GARCH class odels, as it was previously entioned in literature (Laoureux and Lastrapes, 1993; Pong et al., 004) in section. Despite the fact that recoendation of odel-based forecasts in predicting the future tendencies of volatility of volatility is quite negative, profound analysis of calculated errors both separately and considering the relations between the, could be efficient and valuable source of inforation about the reasons of differences and tendencies in copared datasets. Inforation received fro these forecast errors are not ultiplying the sae inforation. The odifications of forecast errors definitions essentially change their eaning. Each of the has its own different interpretation but the ost reliable is analyzing these paraeters together including their utual dependencies. For instance, the MSE, MAE and RMSE errors give quite siilar inforation about the efficiency of forecasts, but each of the react in different way for the untypical observations. The values of these 3 errors that are close to each other, which indicates that in forecasts series do not appear very untypical and extree observations. If MSE is saller than MAE, as it takes place in decreasing and stable periods, 14
it indicates that the single big-value errors appeared in these series. The values were quite siilar, but the MSE was uch saller than MAE, so the extree and untypical values have appeared, but they were in great inority. There are also uch saller ME than MAE values, what indicates that the underestiated and the overestiated values were quite equal in two periods - decreasing and stable ones, but in the increasing period, where both of errors are relative bigger to their absolute values, which sees that forecasts probably are systeatically underestiated (because the signs of these errors are positive). Conclusion The iplied volatility has ore inforation ipact with higher divergence of each paraeters than odel-based forecasts of volatility. It does not atter whether the odel is GARCH class or Stochastic Volatility, because the volatility that depends on ore than historical values and reflects the other iportant paraeters of basic instruents is a big advantage. There were also found that newly CBOE arkets are inforational efficient and both point and interval forecasts of iplied volatility are statistically significant (Konstandinidi et al., 008). Model-free iplied volatility captured in the VVIX index reflects better the financial arket factors as volatility of volatility estiator than odel-based ones. The other reason for existing disparity, is that because the forecasts easure the volatility of volatility and the VVIX index quotes are treated as observed values of realized volatility of the VIX index, because the volatility is unobservable variable and it could not be interchangeably deterined, but the assuption about appropriate estiator is required. The selected type of error is iportant in global interpretation of the received forecasts and each of these easures allows to receive quite different inforation about the forecasts as a series. It is also good source of inforation about utual dependencies that appear between the. Also the inforation could be achieved fro the sign of the error, its value (whether it is big or sall) and even fro its relation with the other paraeters (whether it is bigger or saller than rest of the easures). The siilarity of errors is purposeful to let to focus on the single differences that ay reflect on behavior of all dataset and its reactions for the single factors, but siultaneously these outwardly siilar easures do not only duplicate the sae inforation as it could see to appear. References Becker, R., Cleens, A. E., McClelland, A. (009). The jup coponent of S&P 500 volatility and the VIX index. Journal of Banking and Finance, 33, 1033-1038. 143
Becker, R., Cleens, A. E., White, S. I. (007). Does iplied volatility provide any inforation beyond that captured in odel-based volatility forecasts? Journal Banking & Finance, 31, 535-549. Chien-Hung, Ch., Yueh-Neng, L (010). Consistent odelling of S&P500 and VIX derivatives. Journal of Econoic Dynaic & Control, 34, 30-319. Doan, M., Doan, R. (009). Modelowanie zienności i ryzyka: etody ekonoetrii finansowej. Kraków: Oficyna Wolters Kluwer. Engle, R. (000). The econoetrics of ultra-high-frequency data. Econoetrica, 1(68), 1-. Konstandinidi, E., Skiadopoulos, G., Tzagkaraki, E. (008). Can the evolution of iplied volatility be forecasted? Evidence fro European and US iplied volatility indices. Journal of Banking and Finance, 3, 401-411. Laoureux, C. G., Lastrapes, W. D. (1993). Forecasting stock-return variance: Understanding of stochastic iplied volatilities. The Review of Financial Studies, 6, 93 36. Piontek, K. (00). Modelowanie i prognozowanie zienności instruentów finansowych. Wrocław. Pong, S., Shackleton, M. B., Taylor, S. J., Xu, X. (004). Forecasting currency volatility: A coparison of iplied volatilities and AR(FI)MA odels. Journal of Banking and Finance, 8, 541 563. Witkowska, D. (005). Podstawy ekonoetrii i teorii prognozowania. Kraków: Oficyna Ekonoiczna. 144