Detrending Moving Average Algorithm: from finance to genomics and disordered materials Anna Carbone Physics Department Politecnico di Torino www.polito.it/noiselab September 27, 2009
Publications 1. Second-order moving average and scaling of long-range correlated series E. Alessio, A. Carbone, G. Castelli, and V. Frappietro, Eur. Phys. Jour. B 27, 197 (2002). 2. Scaling of long-range correlated noisy signals: application to financial markets A. Carbone and G. Castelli, Proc. of SPIE, 407, 5114 (2003). 3. Analysis of the clusters formed by the moving average of a long-range correlated stochastic series A. Carbone, G. Castelli and H. E. Stanley, Phys. Rev. E 69, 026105 (2004). 4. Time-Dependent Hurst Exponent in Financial Time Series A. Carbone, G. Castelli and H. E. Stanley, Physica A 344, 267 (2004). 5. Directed self-organized critical patterns emerging from fractional Brownian paths A. Carbone and H. E. Stanley, Physica A 340, 544 (2004). 6. Spatio-temporal complexity in the clusters generated by fractional Brownian paths A. Carbone and H.E. Stanley, Proc. of SPIE 5471, 1 (2004). 7. Quantifying signals with power-law correlations L. Xu, P. Ch. Ivanov, C. Zhi, K. Hu, A. Carbone, H. E. Stanley, Phys. Rev. E 71, 051101 (2005). 8. Scaling properties and entropy of long range correlated series A. Carbone and H.E. Stanley, Physica A 384, 21 (2007). 9. Tails and Ties A. Carbone, G. Kaniadakis, and A.M. Scarfone, Eur. Phys. J. B 57, 121-125 (2007). 10. Where do we stand on econophysics? A. Carbone, G. Kaniadakis and A.M. Scarfone, Physica A 382, xi-xiv (2007). 11. Detrending Moving Average (DMA) Algorithm: a closed form approximation of the scaling law S. Arianos and A. Carbone, Physica A 382, 9 (2007). 12. Algorithm to estimate the Hurst exponent of high-dimensional fractals A. Carbone, Phys. Rev. E 76, 056703 (2007). 13. Cross-correlation of long-range correlated series S. Arianos and A. Carbone, J. Stat. Mech: Theory and Experiment P03037, (2009).
The Efficient Market Hypothesis...and Its Critics A Random Walk Down Wall Street First edition published in the 70 s by Burton Malkiel on the wave of the academic success of the influential survey on the Efficient Market Hypothesis (EMH) by Eugene Fama (1970). A Non-Random Walk Down Wall Street First edition published in 1999 by Andrew Lo and Craig MacKinley
A Moving Average Walk... Down Wall Street
Surveys on the use of Technical Analysis by Finance Professionals 1 2 3 Horizons min % max% intraday 50% 100% 1 week 50% 80% 1 month 40% 70% 3 months 30% 50% 6 months 20% 30% 1 year 2% 25% 1 Charts, Noise and Fundamentals in the London Foreign Exchange Market. H.L. Allen and M.P. Taylor, The Economic Journal, 100, 49-59 (1990) 2 Technical trading rule profitability and foreign exchange intervention, B. LeBaron, Journal of International Economics 49, 125 (1999) 3 The Obstinate Passion of foreign Exchange Professional: Technical Analysis. Lukas Menkhoff and Mark P. Taylor, Journal of Economic Literature 45, 4 (2007)
Fractional Brownian or Levy Walk H > 0.5 positive correlation (persistence) H = 0.5 Random walk H < 0.5 negative correlation (anti-persistence)
Detrending Moving Average Algorithm (DMA) 20 ~ y n (i) σ 2 DMA = 1 N N i=n [ ] 2 y(i) ỹ n (i) \L y(i) FOXVWHU ỹ n (i) = 1 n n y(i k) k=0 σ 2 DMA n2h 10 1000 i c (j) L i c (j+1) τ j =i c (j+1)-i c (j) 2680
Detrending Moving Average Algorithm (DMA) σ 2 DMA n2h The log-log plot is a straight-line whose slope can be used to calculate the Hurst exponent H of the time series.
Detrending Moving Average Algorithm (DMA) Self-similarity of a time series x(t) σdma 2 is generalized variance for nonstationary signals. It can be derived from the auto-correlation function C xx (t, τ), which measures the self-similarity of a signal: σ xx (t, τ) [x(t) x(t)][x (t + τ) x (t + τ)] By taking τ = 0 and x = x(t), x n (i) = 1 n n k=0 x(i k) the autocovariance C xx (t, τ) reduces to the function σdma 2 4. 4 S. Arianos and A. Carbone Physica A 382, 9 (2007)
DAX (Deutscher AktienindeX) Prices (a) P(t) Log Returns (b) r(t) = lnp(t + t ) lnp(t) Volatility σ T (t) 2 = 1 T T 1 t=1 [r(t) r(t) T ] 2 (c) with T = 300min (one half trading day) (d) with T = 660min (one trading day)
FIB30 futures: PRICES The FIB30 is a future contract on the MIB30 index, which considers the thirty firms with higher capitalization and trading (the top 30 blue-chip index) of the MIBTEL (www.borsaitaliana.it) (Since 2004 it is named S&P MIB index).
FIB30 futures: VOLATILITIES 10-3 10-4 T=660min (1 trading day) T=6600min (10 trading days) σ DMA 10-5 10-6 10-7 Fract. Brown. Motion H=0.6 10 0 10 1 10 2 10 3 10 4 n
Cross-Similarity of two stochastic series x(t) and y(t) In many cases, the coupling between two systems (time series) might be of interest. A measure of cross-correlation can be implemented 5 by defining: σ xy (t, τ) [x(t) x(t)][y (t + τ) ỹ (t + τ)] Now it must be τ 0. Moreover x = x(t) and y = y(t) x n (i) = 1 n n k=0 x(i k) and ỹ n(i) = 1 n n k=0 y(i k). 5 S. Arianos and A.Carbone, J. Stat. Mech.: Theory and Experiment P03037, (2009).
Cross-Similarity of return and volatility σ 2 DMA nh 1+H 2 The log-log plot is a straight-line. The slope is given by the sum of the Hurst exponents H 1 and H 2 of the two time series. 6 6 S. Arianos and A.Carbone, J. Stat. Mech: Theory and Experiment P03037, (2009).
Leverage effect: DAX DMA Cross-correlation as a function of the lag τ for the return and volatility of the DAX series. 7 7 S. Arianos and A.Carbone, J. Stat. Mech: Theory and Experiment P03037, (2009).
Scaling law of the moving average clusters 8 9 n=200 n=600 y(t) y(t) n=1000 area n=2000 t duration t Anna Carbone 8 A.Carbone, Physics Department G.Castelli Politecnico anddih.e. TorinoStanley, www.polito.it/noiselab Detrending Phys. Moving Rev. Average E, 69, Algorithm: 026105 from (2004) finance to genom
Scaling law of the cluster lengths l 10 5 (a) 10 4 2.0 1.8 1.6 (a) 1.4 10 3 1.2 l 10 2 ψ l 1.0 0.8 0.6 10 1 H 0.4 0.2 10 0 10 0 10 1 10 2 10 3 τ 0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 H l τ ψ l ψ l = 1
Scaling law of the cluster areas s 10 6 10 5 (b) 1.9 (b) 2.0 10 4 1.8 s 10 3 10 2 10 1 ψ s 1.7 1.6 1.5 1.4 10 0 1.3 10-1 10-2 H 1.2 1.1 1+H 10-3 10 0 10 1 10 2 10 3 τ 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 H s τ ψs ψ s = 1 + H
Scaling law of the pdf of the cluster length and duration 10 0 2.0 10-1 H=0.30 1.9 1.8 β=2-h 10-2 1.7 1.6 ) P(τ) 10-3 β 1.5 1.4 1x10-4 1.3 1x10-5 (a) 10-6 10 0 10 1 10 2 10 3 τ 1.2 1.1 (a) 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 H P(τ, n) τ β F (τ, n) with β = 2 H the fractal dimension of the time series and F (τ, n) defined as: F(τ, n) = e τ/τ.
Scaling law of the pdf of the cluster area 10 0 2.0 1.9 1.8 γ=2/(1+h) 10-1 1.7 1.6 P(s) 10-2 10-3 (b) H=0.80 H=0.20 H=0.50 10 0 10 1 10 2 10 3 s γγγ 1.5 1.4 1.3 1.2 1.1 (b) 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 H with γ = 2/(1 + H) and: P(s, n) s γ F (s, n) F(s, n) = e s/s.
Self-organized criticality of the moving average clusters t addition of particles/energy y j t y addition of particles/energy Brownian path fractional Brownian path j+1 Brownian path cluster lifetime j+2 smoothed fractional Brownian path cluster diameter l dissipation of particles/energies dissipation of particles/energy Moving Average Clusters SOC Clusters l τ ψ l ψ l = 1 ψ l = 1 s τ ψ s ψ s = 1 + H ψ s = 3/2 P(l, n) l α α = 2 H α = 3/2 P(τ, n) τ β β = 2 H β = 3/2 P(s, n) s γ γ = 2/(1 + H) γ = 4/3
Entropy of long-range correlated time series 10 1 l n n=500 A n=1000 n=2000 n=3000 S(l,n) A' A'' D log l n=10000 0 0 1000 2000 3000 4000 5000 6000 l S(l, n) µ(l,n) P(l, n) log P(l, n). S(l, n) = S 0 + D log l + l n. 10 A. Carbone and H.E. Stanley Physica A 385, 21 (2007)
Genome Heterogeneity 0 5 10 15 20 25 30 0 5 10 15 20 25 30 G A T C C T T G A A G C G C C C C C A A G G G C A T C T T C T 1. The sequence of the nucleotide bases ATGC is mapped to a numeric sequence. 2. If the base is a purine (A,G) is mapped to +1, otherwise if the base is a pyrimidine (C,T) is mapped to 1. 3. The sequence of +1 and 1 steps is summed and a random walk y(x) is obtained.
Genome Heterogeneity % A, C, G, T % A, C, G, T % A, C, G, T 50 CHR1 40 n=2 A 30 T 20 C 10 G 0 0 10 20 30 40 50 50 CHR1 n=4 40 A 30 T 20 C 10 G 0 0 10 20 30 40 50 50 CHR1 40 n=10 A 30 T 20 C 10 G 0 0 10 20 30 40 50 length (bp)
Genome Heterogeneity % A, C, G, T % A, C, G, T % A, C, G, T 50 CHR14 A 40 n=2 T 30 20 10 C 0 G 0 10 20 30 40 50 50 CHR14 n=4 A T 40 30 20 10 C 0 G 0 10 20 30 40 50 50 CHR14 40 A n=10 T 30 20 10 C G 0 0 10 20 30 40 50 length (bp)
DMA for high-dimensional fractals (e.g. selfsimilar networks, rough surfaces) 11 Fractal surfaces 0.5 f(i 1,i 2 ) H=0.5 0.5 f(i 1,i 2 ) H=0.1 0.0 0.0-0.5-0.5-1.0 (b) 50 100 150 200 i 1 250 250 100 150200 i 50 2-1.0 (a) 50 100 150 200 i 1 250 250 100 150200 i 50 2 11 A. Carbone Phys. Rev. E 76, 056703 (2007)
DMA for high-dimensional fractals (e.g. selfsimilar networks, rough surfaces) 12 Moving Average Surface for the fractal surface with H = 0.5. 0.5 ~ f(i 1,i 2 ) n 1 x n 2 =15 x15 0.5 ~ f(i 1,i 2 ) n 1 x n 2 = 25 x 25 0.0 0.0-0.5-0.5-1.0 (a) 50 100 150 i 1 200 250 150 200250 100 i 50 2-1.0 (b) 50 100 150 i 1 200 250 150 200250 100 i 50 2 12 A. Carbone Phys. Rev. E 76, 056703 (2007)
DMA for high-dimensional fractals (e.g. selfsimilar networks, rough surfaces) 13 DMA 10 5 10 4 10 3 10 2 10 1 10 0 (c) N 1 x N 2 =4096 x 4096 10 1 10 2 10 3 10 4 s The plot of σ 2 DMA vs. S = (n 2 1 + n2 2 ) is a straight line. It corresponds to the scaling relation: i.e.: σ 2 DMA [ (n 2 1 + n2 2 ) ] 2H σ 2 DMA S2H 13 A. Carbone Phys. Rev. E 76, 056703 (2007)
THANK YOU!