Maastricht University. High Frequency Trading Rules in Forex Markets - A Model Confidence Set Approach

Transcription

1 Maastricht University School of Business and Economics High Frequency Trading Rules in Forex Markets - A Model Confidence Set Approach Author: Raffael Danielli Supervisor: Dr. Sébastien Laurent July 1, 2013

2 Abstract This paper examines 366 different technical trading rules applied on six different exchange rates against the U.S. dollar using hourly data between January 2010 and December First, the Model Confidence Set technique by Hansen, Lunde and Nason (2010) is applied to find the data snooping free set of best models. Subsequently the test of Superior Predictive Ability is applied to select the best model for each currency (Hansen, 2005). The trading rules considered are moving averages, channel rules, filter rules and momentum rules, each with different parameters and applied on in-sample as well as out-of-sample periods. Taking into account realistic transaction costs we find no evidence of excess returns. Thus, the results are consistent with market efficiency. 1

3 1 Introduction If one sequentially flips a sufficiently large number of fair coins, then eventually one coin will emerge that always comes up head. The untrained eye could mistake this particular coin as special and maybe even bet money on it coming up head again in the next round. A statistician however knows that the past outcome was a result of pure randomness and the likelihood of this particular coin coming up head again in the next round will be 50 percent just like for all the other fair coins. In the stock market, things are not always this clear. Looking at financial data we have time series going back decades, but does the price of Google four week ago have an influence on the stock price of Google today? Or does the price of a grain future ten seconds ago have an influence on the price of it right now? Those are the questions technical analysts are trying to answer. Technical analysis tries to predict future price movements by extracting information from past data. Popular technical trading rules include moving averages, break out rules and momentum oscillators (Neely & Weller [6]). Academics tend to be sceptical when it comes to technical trading rules as markets are believed to be mostly efficient and therefore any attempt to achieve excess return is considered to be futile. Considerable efforts have been made by academics to find evidence whether technical trading profits exist but the results have been mixed at best. Research from the 1970s until the 1990s document long periods in which simple technical trading rules such as moving averages and filter rules produced substantial excess returns, questioning the weak form of the efficient market hypothesis. In later years, those findings have been challenged as being subjected to data snooping, publication bias and data mining, see for example Neely and Weller [8] or Park and Irwin [9]. Data snooping occurs when researches decide to test rules that are already proven to be profitable on part or all of the tested data. Data mining occurs when researchers test many rules and then base the overall inference on the most successful rules, ignoring any negative results. Publication bias is the tendency of journals to publish submissions with positive results than negative results. A way to reduce the problem of data snooping is the practice of dividing the whole sample into in- and out- of sample data and to only use the in-sample data to select rules that perform well. More recent research suggests that even after correcting for problems such as data snooping and data mining, relatively simple technical trading rules were genuinely profitable on major exchange rates from about 1975 to The profitability of those technical rules seemed to decline as they became widely publicized and eventually ceased to exist, while more complex and less known rules continued to be profitable. For a neat summary on this topic we refer to Neely and Weller [7]. This paper studies the profitability of technical trading rules applied on hourly data of six major foreign exchange markets. By applying Hansen s et al. [3] Model Confidence Set (MCS) approach we try to control for potential data snooping bias in the foreign exchange (FX) market. The MCS selects a set of models that contains the best performing model with a probability that is no less 2

4 than 1 α with α being the size of the test. An advantage of this method is that it acknowledges the possible limitations of the data. Applying the MCS test on informative data will result in a smaller number of models in the set, everything else being equal. Subsequently, we apply the test of Superior Predictive Ability (SPA) introduced by Hansen [4] which by itself is based on the Reality Check by White [11]. Using the SPA we test the null that the returns of the models in the MCS are not significantly better than the benchmark and continue to select the best model. Both methods rely on bootstrapping simulations to calculate the necessary p-values. The best model selected is then applied on out-of-sample data to test for the robustness of the returns. Using three years worth of hourly FX data and testing 366 different technical trading rules we find that the MCS test is not a suitable method to reliably select robust models. The MCS acts conservative and does not reduce the set of equally good models significantly, acknowledging the limitation of the underlying data and high volatility of the returns. The best rules selected by the SPA are significantly better than their respective benchmark in the past but fail to perform on out of sample data. The remainder of this paper is organized as follows. Section 2 explains the MCS method and the performance criteria we adopt to investigate trading-rule profitability in the FX market. Section 3 describes the data and the technical trading rules used whereas section 4 reports the main empirical results. Concluding remarks are contained in the final section. 2 The Model Confidence Set Approach When applying technical trading rules on past data researchers face the problem of data mining and data snooping. Data snooping occurs when the parameters of a technical trading rule are fitted on past data in such that the model only produces good outcomes on the past data. In other words, it was pure luck that the model performed well on that particular set of past data and it is unlikely that the same model will continue to work in the same way when applied on out-of-sample data. To mitigate the effect of data snooping, White [11] introduced the Reality Check method to test for the null that the best model is not significantly superior compared to a benchmark model which lay the foundation for future research. This paper applies the MCS method introduced by Hansen et al. [3], which builts on the ideas of White. The MCS is a random data-dependent set of models that includes the best forecasting model(s) with a certain confidence interval. All models are tested against each other so there is no need to specify a benchmark. This method will provide some confidence that the results are genuine and not a product of pure chance. An advantage of the MCS is that it acknowledges the limitation of data and has the power to weed out inferior models when the data contains sufficient information. We keep Hansen et al. [3] notation, and refer the reader to the original paper for a more detailed exposition. The MCS algorithm is based on the following three steps: 3

5 Step 0: Initially set M = M 0. Step 1: Test H 0 : E[d ij ] = 0 i, j M using equivalence test T max at level α where d ij measures the sample loss differential between model i and j using a loss function. Step 2: If H 0 is accepted we define ˆM 1 α = M, otherwise we use e M to eliminate an object from M and repeat the procedure beginning with Step 1. M 0 contains a finite number of objects that are indexed by i = 1,...m 0. The relative performance between two models is defined by d ij,t and the set of superior model models is defined as M = {i M 0 : E[d ij ] 0 j M 0 }. (1) The null hypothesis that is being tested is H 0 : E[d ij ] = 0 i, j M with M being a subset of M 0 against the alternative H A : E[d ij ] 0 for some i, j M. If the equivalence test rejects the null hypothesis, at least one model in the set M is inferior. Step 1 and step 2 are repeated until the remaining models in M equal ˆM 1 α, the 1 α model confidence set. Several possible test statistics can be used for the sequential testing of the null hypothesis. We follow the recommendation of Hansen et al [3] and choose: T max,m = max i M t i d with t i = i, var( ˆ d di = m 1 d j M ij and d ij = T 1 T t=1 d ij,t where m is i) the number of models in M. A bootstrap procedure is used to estimate the distribution of the test statistic which depend on unknown parameters. Results of the test are summarized by the MCS p-value p MCS. Model i ˆM α for α p MCS and i / ˆM α for α > p MCS. For example, the set of models that includes the best performing model with a probability no less than 95 percent is denoted ˆM 0.95 of which all models with p MCS values higher than 0.05 will belong to. 2.1 The Performance Criteria In order to use the MCS for testing technical trading rules it is necessary to define a loss function for each period. The most obvious choice is based on the excess return over the buy-and-hold strategy (Allen & Karjalainen [1]). The excess return for trading rule k at time t is defined as follows: r t,k = [log(p t ) log(p t 1 ) + i t i t )I t 1,k abs(i t 1,k I t 2,k )g (2) where P t is the closing price of the exchange rate on period t; i t and i t are foreign and domestic interest rate; I t,k represents the trading signal which is equal to 1 if rule k M signals buy and -1 if it signals sell and 0 if it signals out at time period t; g is the one way transaction cost. 4

6 The relative performance between two competing models i and j in terms of excess returns is then defined as E[d ij ] = T 1 T t=1 d ij,t = T 1 T t=1 (r t,i r t,j ) (3) The minus sign is added since d ij is defined in terms of losses where the lowest loss gives the best model. A more sophisticated evaluation measure might take into account the volatility of the returns, for example by including a term penalizing large daily losses or large overall draw downs. The MCS gives a set of equally good models in terms of forecast ability where the size of the set depends largely on the underlying data and the variance of the returns. However, an investor is more interested in a potential model that can outperform a natural benchmark significantly. Therefore we also included the SPA as a second performance criteria. The SPA compares one or several models with a benchmark and evaluates the forecasts using a pre-specified loss function similar to the MCS. The null hypothesis to be tested is that the best rule is no better than the benchmark, i.e. H 0 : min k=1,...,m E[R k ] 0 where the loss function is defined as E[R k ] = T 1 T t=1 R k = T 1 T t=1 (r t,k r t,0 ). (4) The returns of the benchmark are defined as r t,0, which is the buy and hold case. The investor buys the underlying currency against the U.S. dollar at time t = 1 and sells it at time T. A stationary bootstrap method is applied to the observed values of R k,t in order to extract the SPA p-values. The difference to the MCS method is that here the models are compared to a benchmark and the test reports the probability with which the returns are significantly higher than the benchmark returns (reject H0 SP A ). The MCS compares all models against each other and reports the probability with which the models in the set are likely to be equally good (cannot reject H0 MCS ). The SPA test differs from White s [11] reality check which also accounts for data snooping. For a more detailed review of the SPA method we refer to Hansen [4]. As a third performance criteria the Sharpe ratio is used. For the k-th trading rule the Sharpe ratio is defined as: S k = E(r k,t) var(rk,t ) (5) The best trading rule is selected as follows: first, the Sharpe ratio for all 366 different rules is calculated. Usually, a Sharpe ratio of 1 is considered as average and a Sharpe ratio of 2 as good. Hence, we assume that an investor is not interested in rules with a Sharpe ratio below 0.8 and remove those from the set. Second, the MCS test is applied to determine the set of equally good rules and account for data snooping bias. Third, on the set of models selected by the 5

7 MCS we apply the SPA test to find the rule that outperforms the benchmark the most and report it s p-value. Calculations are all done in OxMetrics using the additional software package MulCom. To keep the amount of computation time manageable we set the number of resamples for the bootstrap to 500 with a block length of 2 and α = 0.05 for the MCS. For the SPA the number of resamples are set at 500 and the smoothing parameter equal to Data and Technical Trading Rules 3.1 Data and Summary Statistics We collect minute data from the EBS, an electronic brokering system provider, which then was converted to hourly data. Data for the following currency prices relative to the U.S. dollar will be used: Austrian dollar (AUS), Euro (EUR), pound sterling (GBP), New Zeeland dollar (NZD), Swiss franc (CHF), and Japanese yen (JPY). The fields provided by the EBS data are date, time, open bid, high bid, low bid, close bid, open ask, high ask, low ask, close ask and total ticks. Our full sample of hourly exchange rates starts on January 3, 2010 and consists of hours, which equals to 3 trading years. The exchange rates (U.S. dollar price of foreign currency) needed to calculate the values of the technical trading rules are taken as the midpoint (average) between the bid and the ask of the closing price for each period. If a buy signal appears in period t then the speculator borrows U.S. dollars at the domestic interest rate and uses the proceeds to buy the foreign currency at the open of period t + 1. For interest rates, we obtain the LIBOR rates for each corresponding currency. Table 1 reports summary statistics of the continuous hourly log returns. The mean return rates show that on average the U.S. dollar appreciates against CHF, EUR and GBP, while depreciates against the other currencies over the sample period. The underlying returns are not normally distributed (Jarque- Bera rejects the null of normality) for all currencies. 3.2 Technical Trading Rules For our study we rely on four popular types of technical trading rules: filter rule (FR), moving average rule (MA), momentum rule (MR), and channel rule (CR). Those rules are among the most popular ones and have been investigated in various contexts by previous researchers such as (non exhaustive list) Brock, Lakonishok and LeBaron[2], Min and Wu [5], Neely and Weller [6] or Sullivan, Timmermann and White [10]. Previous research focuses on applying those rules on daily data whereas we are proceeding to analyse them on hourly data. Each trading rule has one or several parameters that are used to vary the timing of the trading signal. By altering the parameters of a trading rule, one can produce hundreds of trading systems, each producing slightly different trading signals. Intuitively, loading too many irrelevant parameters for each rule can reduce the test power due to data mining ( garbage-in-garbage-out principle). On the 6

8 Table 1: Descriptive statistics s t s t 1 AUD CHF EUR GBP JPY NZD Mean* Median* Max Min Std. Deviation Skewness Kurtosis Note: This table reports summary statistic for hourly changes in the logarithm exchange rates for the period between January 3, 2010 until December 31, 2012 with observations. other hand, using too few rules can cause bias in statistical inference. We try to strike a balance and select a fairly large variety of reasonable parameters that lie in the range used in previous literature. We assume that the speculator has the initial wealth to trade on margins and can take long and short positions in the foreign exchange market Filter Rule A filter rule generates a buy signal when the exchange rate has risen by more than b percent above its most recent low generated in the last n periods. It generates a sell signal when the exchange rate has fallen by more than b percentage from its most recent high generated in the last n periods. Thus, I t = 1 if P t z t (n)(1 + b), I t = 1 if P t x t (n)(1 b), I t = I t 1 otherwise. where I t is the indicator variable taking either value 1 for a long position or -1 for a short position. The exchange rate at period t is denoted as P t ; z t (n) is the most recent local minimum at time t that occurred in the last n periods and x t (n) is the most recent high at time t that happened in the last n periods. We denote these rules as F R(n, b) and take filter sizes b of , , 0.001, 0.002, 0.005, and 0.01 and period sizes n of 2, 4, 6, 8, 12, 18, 24, 36, 48, 120, 480, and 960 hours. This generates 6 12 = 72 different combinations Moving Average Rule A moving average rule generates a buy signal when the short moving average of past exchange rates crosses over the long moving average from below. Similarly, 7

9 it generates a sell signal when the short moving average crosses over the long moving average from above. We denote these rules by MA(S, L) where S stands for the number of periods in the short moving average and L stands for the number of periods in the short moving average. Thus, I t = 1 if S t L t, I t = 1 if S t < L t. The first L data points will be ignored for the analysis part as they are used to calculate the initial values of S and L, hence t {L,..., T }. We test the rule by using the parameters 1, 2, 4, 6, 8, 10, 12, 20, 24, 48, 120, 240, 480 and 960 for S and L respectively. This generates 14 i=1 i = 105 fast-slow combinations Momentum Rule The momentum rule generates a buy signal when the cumulative return of the exchange rate over the last n periods is positive. Similarly, it generates a sell signal when the cumulative return over the last n periods is negative. The position stays open for h hours after which it will be closed. Thus, t I t = 1 if i=t n P i P i 1 0, t I t = 1 if i=t n P i P i 1 < 0, I t = I t 1 for h periods, then 0. Where P i is the exchange rate at time i and t {n,..., T }. The period over which the cumulative return is calculated is denoted as n. We consider period sizes n of 2, 4, 6, 8, 10, 12, 18, 24, 36, 48, 120, 480, and 960 hours and leave the position open for h = 4, 8, 10, 12, 24, 48, 96, 120, and 480 hours. This generates 13 9 = 117 different combinations Channel Rule A channel rule generates a buy signal when the exchange rate exceeds the high of the last n periods by more than b percent. It generates a sell signal when the exchange rate falls below the low of the last n periods by more than b percent. Thus, I t = 1 if P t z t (n)(1 + b), I t = 1 if P t y t (n)(1 b), I t = I t 1 otherwise. Where z t (n) denotes the maximum exchange rate at time t over the last n periods; y t (n) denotes the minimum exchange rate at time t over the last b periods. We set n to be 2, 4, 6, 8, 12, 18, 24, 36, 48, 120, 480, and 960 periods and b to be , , 0.001, 0.002, 0.005, and This generates 12 6 = 72 different combinations. 8

10 Currency Best Trading Rule Table 2: Performance of the Best Trading Rule against U.S. dollar: Full Sample Number of Trades Mean Return Volatility Return Sharpe Ratio Sharpe > 0.8 MCS Set Size SPA p-value Panel A. Without Transaction Cost AUD MR(n = 120, h = 120) EUR MA(S = 20,L = 24) GBP MR(n = 4,h = 12) NZD FR(n = 24, b = ) CHF CR(n = 2, b = 0.005) JPY MR(n = 120,h = 480) Panel B. With One-way Transaction Cost 0.025% AUD FR(n = 8, b = 0.01) EUR CR(n = 4, b = 0.05) GBP FR(n = 120,b = 0.01) NZD CR(n = 4, b = 0.01) CHF CR(n = 2, b = 0.01) JPY CR(n = 18, b = 0.01) Note: This table reports the optimum trading rules for each currency over the full sample of periods. Panel A reports the results without transaction costs, panel B reports the results with 0.025% transaction costs per trade. The mean return and volatility return are annualized and in percentage. The Sharpe ratio is annualized. Sharpe > 0.8 gives the number of trading rules with a Sharpe ratio > 0.8 out of the universe of 366 rules. MCS set size is the number of relevant rules at a 0.05 alpha level in the set after applying the MCS test. SPA p-value is calculated by applying the SPA test on the MCS set and reported for the model with the best performance relative to the benchmark. The first 960 observations are omitted to calibrate the returns. 9

11 4 Empirical Results 4.1 Full Sample Results Panel A of Table 2 reports the performance of the best trading rules over the full sample period selected by the SPA. For the Australian dollar, the British pound and the Japanese yen the best rule selected is a momentum rule. Both, the Australian dollar and the Japanese yen select a momentum rule that adds up the returns over the last 120 hours to generate a signal and then keeps the position open for 120 and 480 hours respectively. The MR(4,12) selected for the British pound uses much shorter periods which results in the highest number of trades among all rules selected while generating only a yearly mean return of 7.7 percent, the lowest among the rules. The largest mean returns of 18.5 percent is generated by the MR(120,120) rule selected for the Australian dollar, which together with a volatility of 11.9 percent also generates the highest Sharpe ratio of among the rules selected. All the rules with a Sharpe ratio larger than 0.8 are also in the MCS at an alpha level of 5 percent, meaning the MCS contains the best model with a 95 percent certainty. The MCS contains the most models for the Swiss franc with 43 different models. The p-value shows that all rules are significant better than the benchmark 1 at a 10 percent level and all rules except for the British pound are also significant at the 5 percent level. It is premature to say that the rules selected are actually profitable as we ignored transaction costs. Real world transaction costs for trading currencies are generally very low though frequent trading can add up substantially. The momentum rule with n = 4 and h = 12 for the British pound generates 5066 trades. Applying transaction a costs of only percent per trade to this rule reduces the yearly mean return by 42.2 percentage points, resulting in a significant loss. This holds also true for the Euro which trades 2900 times using rule MA(20,24). For the other currencies the after-cost return per annum stays positive, though it reduces to 12.3 percent for the Australian dollar, 7.3 percent for the New Zeeland dollar, 13 percent for the Swiss franc and 2.7 percent for the Japanese yen after accounting for transaction costs of 798 trades. Applying transaction costs reduces mean returns significantly and shows that possibly rules that trade less frequently will be able to generate on average higher returns. Therefore we calculated the excess returns for each of the 366 trading rules again, this time by applying a transaction cost of percent per trade. The results are reported in panel B of Table 2. The number of rules with a Sharpe ratio larger than 0.8 is reduced to only 8 models for the Swiss franc and only 1 model in case of the British pound. Applying the MCS does not remove any more models. Applying the SPA on those remaining models then gives the best ones in term of excess return. SPA p-values show that for the British pound with rule FR(120, 0.01) and New Zeeland dollar with rule CR(4, 0.01) we cannot reject the null hypothesis of no excess return at the 10 percent 1 The benchmark is going long the underlying currency for the whole sample period 10

12 level. For the other we can reject the null at the 10 percent level and for the Australian dollar and the Swiss franc we can reject the null at the 5 percent level. Yearly mean returns are in the same range as before, with no transaction costs applied, though now the optimum rules selected trade much less frequently. In case of the Swiss franc the SPA select a channel rule which only trades 9 times over the whole sample period, reducing the power of statistical tests. Overall, the rules selected when applying costs are all different than in panel A, which shows that the cost filter has some effect on the overall performance of the trading rules. 4.2 Sub-sample Results The results above seem to suggest that technical analysis is useful in forecasting the direction of the exchange rate changes. Exchange rates can be subjected to long swings and a model that worked in one period might not any more in another. A good way to check the robustness of results is to conduct a subsample analysis. The sample period is split into two roughly equally long parts. For each currency in each sub-sample, we select the best trading rule out of the universe of the same 366 rules used for the full sample results. We pre-select the best models by only considering those with a Sharpe ratio of more than 0.8 and apply the MCS procedure to find the set of models that include the best model while accounting for data snooping bias. On the remaining rules the SPA test is applied and the best model is reported. Table 3 reports the performance of the best trading rule for each of the six exchange rates tested in the two sub-samples. Panel A reports the results for the first sub-sample and it can be observed that only for the Euro with MA(20,24) and the New Zeeland dollar with FR(24,0.0001) the rules are consistent with the full sample rules reported in Panel A of Table 2. The mean returns found in Panel A of Table 3 are all stronger than the corresponding ones for the full sample while the volatility stays about the same. The SPA test shows that the best rules are all significantly better than the benchmark at the 10 percent level except for the British pound. The evidence for the second sub-sample in Panel B is still strong in terms of p-value. For all currencies except the Euro the best rules are significantly better in terms of excess returns than the benchmark at the 5 percent level. However, the best trading rules are now all different than for the first sub-sample. This could mean that a break in the structure of the exchange rate happened between the first and second sub-sample which is reflected in the use of different models working best for each sample. Again, the MCS does not reduce the number of rules after only taking into account those with a Sharpe ratio larger than 0.8. In fact, the MCS test never decreased the number of rules in the set of best models, instead it always marked them as significant at a 5 percent level. 2 If the volatility of the results reported 2 We considered the possibility of selection bias by only considering rules with a Sharpe 11

13 Currency Best Trading Rule Table 3: Performance of the Best Trading Rule against U.S. dollar: Sub-Periods Number of Trades Mean Return Volatility Return Sharpe Ratio Sharpe > 0.8 MCS Set Size SPA p-value Panel A. Sub-sample hours AUD FR(n = 8, b= 0.01) EUR MA(S = 20, L = 24) GBP FR(n = 2, b = 0.01) NZD FR(n = 24, b = ) CHF MA(S = 6, L = 240) JPY MR( n = 8, h = 8) Panel B. Sub-sample hours AUD MR(n = 12, h = 96) EUR CR(n = 4, b = 0.005) GBP MA(S = 4, L = 10) NZD CR(n = 4, b = 0.005) CHF MR(n = 10, h = 8) JPY FR(n = 4, b = 0.01) Note: This table reports the optimum trading rules for each currency over two different sub-samples without transaction costs. Panel A reports the results over the periods , panel B reports the results over the periods The mean return and volatility return are annualized and in percentage. The Sharpe ratio is annualized. Sharpe > 0.8 gives the number of trading rules with a Sharpe ratio > 0.8 out of the universe of 366 rules. MCS set size is the number of relevant rules at a 0.05 alpha level in the set after applying the MCS test. SPA p-value is calculated by applying the SPA test on the MCS set and reported for the model with the best performance relative to the benchmark. The first 960 observations are omitted to calibrate the returns. 12

14 Table 4: Out-of-Sample Performance of the Best Rule Selected from the Sample Period In-Sample Out-of-Sample Currency Best Trading Rule Number of Trades Mean Return Number of Trades Sharpe Ratio Mean Return SPA p-value AUD FR(n = 8, b = 0.01) EUR MA(S = 20, L = 24) GBP FR(n = 2, b = 0.01) NZD FR(n = 24, b = ) CHF MA(S = 6, L = 240) JPY MR(n = 8, h = 8) Note: This table reports the out of sample performance during the periods of the optimum trading rules found in the sample from periods. The mean return and are annualized and in percentage. The Sharpe ratio is annualized. The first 960 observations are omitted to calibrate the returns. by the loss function are fairly high the MCS approach might be unable to differentiate any further. The volatility of the returns are indeed usually above 10 percent. It could also hint towards a problem with limited data which is acknowledged by the MCS by being unable to reduce the set of rules further. 4.3 Out-of-Sample Results To check for the robustness of the profitability of technical trading rules we run an out-of-sample test, which is also another way of purging the effects of data-snooping. We use the best technical trading rules found for the in-sample period ranging from 0 to Then those rules are applied to the out-ofsample period which ranges from period to Naturally, the MCS is not reported as only one rule is considered for each exchange rate and the SPA is only evaluated on one single rule for each exchange rate. For easy comparison the in-sample results are reported alongside the out-of-sample results. Table 4 reports the mean excess returns and the p-values for the out-ofsample test. We find that for each currency the mean excess return decrease compared to the first half of the sample and even become slightly negative for the British pound and the Japanese yen. Worse, the best trading rules for the in-sample period do not outperfrom their respective benchmark significantly except for the Japanese yen. The yearly profit generated by the momentum rule with parameters MR(8,8) for the Japanese yen is slightly negative with -0.5 percent but still significantly better than the benchmark of going long the Japanese yen against the U.S. dollar during the same time period. ratio larger than 0.8 and therefore applied the MCS test directly on the universe of 366 rules. The results remain unchanged: the MCS always contained the full set of 366 rules at an alpha level of 0.05 as well as at 0.10 for all exchange rates 13

15 5 Conclusion In this paper, the MCS is applied on a set of models to determine a data snooping free subset of rules. Subsequently, the robustness of the best rule is tested in an out of sample test. In all cases, the MCS was unable to reduce the number of models in the set. A possible explanation is that the volatility of the returns was too high for the MCS to define a subset of rules significantly better than the rest of the models. Another explanation is that the underlying data was not informative enough and more data points are needed. For each sub-sample period the best trading rule for each exchange rate is different. The profitability of trading rules is much less pronounced in the second half of the sample period. This points towards a structural change of the underlying exchange rate from one sample period to the next. It also shows that technical trading rules are in general less adapt in the second half due to changing markets behaviour. An explanation could be that markets adapt and exploit profitable trading rules as they become known. In most cases the best trading rule was significantly better than their respective benchmark at a 10 percent level for each sub-period. Out-of-sample returns of the best trading rules are significantly reduced and, except for the Japanese yen, not significantly better than their respective benchmark at the 10 percent level. Taking the best trading rule from the insample period and simply applying it on the market is therefore not advisable and it shows that data snooping continues to be a prevailing problem. Overall, technical trading rule profitability becomes much weaker over time, due to possible structural changes. Selecting the best trading rule today based on past data does not guarantee to outperform the benchmark in the future. 14

16 References [1] Allen, F. & Karjalainen, R. (1998). Using Genetic Algorithms to find Technical Trading Rules. Journal of Financial Economics 51, [2] Brock, W., Lakonishok, J. & LeBaron B. (1992). Simple Technical Trading Rules and the Stochastic Properties of Stock Returns. Journal of Finance 47, [3] Hansen, P. R., Lunde, A. & Nason, J. M. (2010). The Model Confidence Set Econometrica, 79, [4] Hansen, P.R. (2005). Test for Superior Predictive Ability. Journal of Business and Economic Statistics. [5] Min, Q. & Wu, Y. (2006). Technical Trading-Rule Profitability, Data Snooping, and Reality Check: Evidence from the Foreign Exchange Market. Journal of Money, Credit, and Banking 38, [6] Neely, C. J. & Weller, P. A. (2011). Lessons from the Evolution of Foreign Exchange Trading Strategies. Federal Reserve Bank of St. Louis, Working Paper. [7] Neely, C. J. & Weller, P. A. (2011). Technical Analysis in the Foreign Exchange Market. Federal Reserve Bank of St. Louis, Working Paper. [8] Neely, C. J. & Weller, P. A. (2003). Intraday technical trading in the foreign exchange market. Journal of International Money and Finance 22, [9] Park, C. & Irwin, S. H. (2005). The Profitability of Technical Trading Rules in US Futures Markets: A Data Snooping Free Test. University of Illinois at Urbana-Champaign. [10] Sullivan, R., Timmermann, A. & White, H. (1999). Data-Snooping, Technical Trading Rule Performance, and the Bootstrap. The Journal of Finance 54, [11] White, H. (2000). A Reality Check for Data Snooping. Econometrica 68,