An Online Portfolio Selection Algorithm with Regret Logarithmic in Price Variation

Transcription

1 An Online Porfolio Selecion Algorihm wih Regre Logarihmic in Price Variaion Elad Hazan echnion - Israel Ins. of ech. Haifa, Israel ehazan@ie.echnion.ac.il Sayen Kale IBM.J. Wason Research Cener P.O. Box 28, Yorkown Heighs, NY 0598 sckale@us.ibm.com Absrac We presen a novel efficien algorihm for porfolio selecion which heoreically aains wo desirable properies:. Wors-case guaranee: he algorihm is universal in he sense ha i asympoically performs almos as well as he bes consan rebalanced porfolio deermined in hindsigh from he realized marke prices. Furhermore, i aains he ighes known bounds on he regre, or he log-wealh difference relaive o he bes consan rebalanced porfolio. We prove ha he regre of algorihm is bounded by O(log Q), where Q is he quadraic variaion of he sock prices. his is he firs improvemen upon Cover s [Cov9] seminal work ha aains a regre bound of O(log ), where is he number of rading ieraions. 2. Average-case guaranee: in he Geomeric Brownian Moion (GBM) model of sock prices, our algorihm aains igher regre bounds, which are provably impossible in he wors-case. Hence, when he GBM model is a good approximaion of he behavior of marke, he new algorihm has an advanage over previous ones, albei reaining wors-case guaranees. We derive his algorihm as a special case of a novel and more general mehod for online convex opimizaion wih exp-concave loss funcions. Inroducion A widely used model in mahemaical finance for sock prices is he Geomeric Brownian Moion (GBM). his model has been successfully applied o sudy and price financial insrumens, and is he basis of much invesing in pracice. However, while i An exended absrac of his work appeared in [HK09]

2 is ofen an accepable approximaion, he GBM model is no always valid empirically. his moivaes a wors-case approach o invesing, called universal porfolio managemen, where he objecive is o maximize wealh relaive o he wealh earned by he bes fixed porfolio in hindsigh. In his paper we ie he wo approaches, and design an invesmen sraegy which is universal in he wors-case, and ye capable of significanly improved performance when he marke is closely approximaed by he GBM model. Average-case Invesing: Much of mahemaical finance heory is devoed o he modeling of sock prices and devising invesmen sraegies ha maximize wealh gain (while conrolling risk). ypically, such invesmen sraegies involve esimaing fiing a parameric model of sock prices o observed daa. Such sraegies are geared o he average-case marke (in he formal compuer science sense), and are naurally suscepible o drasic deviaions from he model, as winessed in he recen sock marke crash. Even so, empirically he Geomeric Brownian Moion (GBM) ([Osb59, Bac00]) has enjoyed grea predicive success and every year significan invesmens are made assuming his model. One illusraion of is wide accepabiliy is ha Black and Scholes [BS73] used his same model in heir Nobel prize winning work on pricing opions on socks. Wors-case Invesing: he fragiliy of average-case models in he face of rare bu dramaic deviaions led Cover [Cov9] o ake a wors-case approach o invesing in socks. he performance of an online invesmen algorihm for arbirary sequences of sock price reurns is measured wih respec o he bes CRP (consan rebalanced porfolio, see [Cov9]) in hindsigh. A universal porfolio selecion algorihm is one ha obains sublinear (in he number of rading periods ) regre, which is he difference in he logarihms of he final wealhs obained by he wo. Cover [Cov9] gave he firs universal porfolio selecion algorihm wih regre bounded by O(log ). here has been much follow-up work afer Cover s seminal work, such as [HSSW96, MF93, KV03, BK99, HAK07], which focused on eiher obaining alernae universal algorihms or improving he efficiency of Cover s algorihm. However, he bes regre bound is sill O(log ). his dependence of he regre on he number of rading periods is no enirely saisfacory for wo main reasons. Firs, a priori i is no clear why he online algorihm should have high regre (growing wih he number of ieraions) in an unchanging environmen. As an exreme example, consider a seing wih wo socks where one has an upward drif of % daily, whereas he second sock remains a he same price. One would expec o figure ou his paern quickly and focus on he firs sock, hus aaining a consan fracion of he wealh of he bes CRP in he long run, i.e. consan regre, unlike he wors-case bound of O(log ). he second problem arises from rading frequency. Suppose we need o inves over a fixed period of ime, say a year. rading more frequenly poenially leads o higher wealh gain, by capializing on shor erm sock movemens. However, increasing rading frequency increases, and hus one may expec more regre. he problem is acually even worse: since we measure regre as a difference of logarihms of he final wealhs, a regre bound of O(log ) implies a polynomial in facor raio beween he final wealhs. In realiy, however, experimens [AHKS06] show ha some known 2

3 online algorihms acually improve wih increasing rading frequency. Bridging Wors-case and Average-case Invesing: Boh hese issues are resolved if one can show ha he regre of a good online algorihm depends on oal variaion in he sequence of sock reurns, raher han purely on he number of ieraions. If he sock reurn sequence has low variaion, we expec our algorihm o be able o perform beer. If we rade more frequenly, hen he per ieraion variaion should go down correspondingly, so he oal variaion says he same. We analyze a porfolio selecion algorihm and prove ha is regre is bounded by O(log Q), where Q (formally defined in Secion.2) is he sum of squared deviaions of he reurns from heir mean. Since Q (afer appropriae normalizaion), we improve over previous regre bounds and reain he wors-case robusness. Furhermore, in an average-case model such as GBM, he variaion can be ied very nicely o he volailiy parameer, which explains he experimenal observaion he regre doesn increase wih increasing rading frequency. Our algorihm is efficien, and is implemenaion requires consan ime per ieraion (independen of he number of game ieraions).. New echniques and Comparison o Relaed Work Cesa-Bianchi, Mansour and Solz [CBMS07] iniiaed work on relaing wors case regre o he variaion in he daa for he relaed learning problem of predicion from exper advice, and conjecured ha he opimal regre bounds should depend on he observed variaion of he cos sequence. Recenly, his conjecured was proved and regre bounds of Õ( Q) were obained in he full informaion and bandi linear opimizaion seings [HK0, HK], where Q is he variaion in he cos sequence. In his paper we give an exponenial improvemen in regre, viz. O(log Q), for he case of online exp-concave opimizaion, which includes porfolio selecion as a special case. Anoher approach o connecing wors-case o average-case invesing was aken by Jamshidian [Jam92] and Cross and Barron [CB03]. hey considered a model of coninuous rading, where here are rading inervals, and in each he online invesor chooses a fixed porfolio which is rebalanced k imes wih k. hey prove familiar regre bounds of O(log ) (independen of k) in his model w.r.. he bes fixed porfolio which is rebalanced k imes. In his model our algorihm aains he igher regre bounds of O(log Q), alhough our algorihm has more flexibiliy. Furhermore heir algorihms, being exensions of Cover s algorihm, may require exponenial ime in general 2. Our bounds of O(log Q) regre require compleely differen echniques compared o he Õ( Q) regre bounds of [HK0, HK]. hese previous bounds are based on firs-order gradien descen mehods which are oo weak o obain O(log Q) regre. Insead we have o use he second-order Newon sep ideas based on [HAK07] (in paricular, he Hessian of he cos funcions). 2 Cross and Barron give an efficien implemenaion for some ineresing special cases, under assumpions on he variaion in reurns and bounds on he magniude of he reurns, and assuming k. A ruly efficien implemenaion of heir algorihm can probably be obained using he echniques of Kalai and Vempala. 3

4 he second-order echniques of [HAK07] are, however, no sensiive enough o obain O(log Q) bounds. his is because progress was measured in erms of he disance beween successive porfolios in he usual Euclidean norm, which is insensiive o variaion in he cos sequence. In his paper, we inroduce a differen analysis echnique, based on analyzing he disance beween successive predicions using norms ha keep changing from ieraion o ieraion and are acually sensiive o he variaion. A key echnical sep in he analysis is a lemma (Lemma 5) which bounds he sum of differences of successive Cesaro means of a sequence of vecors by he logarihm of is variaion. his lemma, which may be useful in oher conexs when variaion bounds on he regre are desired, is proved using he Kahn-Karush-ucker condiions, and also improves he regre bounds in previous papers..2 he model and saemen of resuls Porfolio managemen. In he universal porfolio managemen model [Cov9], an online invesor ieraively disribues her wealh over n asses before observing he change in asse price. In each ieraion =, 2,... he invesor commis o an n- dimensional disribuion of her wealh, x n = { i x i =, x 0}. She hen observes a price relaives vecor r R n +, where r (i) is he raio beween he closing price of he i h asse on rading period and he opening price. In he h rading period, he wealh of he invesor changes by a facor of (r x ). he overall change in wealh is hus (r x ). Since in a ypical marke wealh grows a an exponenial rae, we measure performance by he exponenial growh rae, which is log (r x ) = log(r x ). A consan rebalanced porfolio (CRP) is an invesmen sraegy which rebalances he wealh in every ieraion o keep a fixed disribuion. hus, for a CRP x n, he change in wealh is (r x). he regre of he invesor is defined o be he difference beween he exponenial growh rae of her invesmen sraegy and ha of he bes CRP sraegy in hindsigh, i.e. Regre := max log(r x ) log(r x ) x n Noe ha he regre doesn change if we scale all he reurns in any paricular period by he same amoun. So we assume w.l.o.g. ha in all periods, max i r (i) =. We assume ha here is known parameer r > 0, such ha for all periods, min,i r (i) r. We call r he marke variabiliy parameer. his is he only resricion we pu on he sock price reurns; hey could be chosen adversarially as long as hey respec he marke variabiliy bound. Online convex opimizaion. In he online convex opimizaion problem [Zin03], which generalizes universal porfolio managemen, he decision space is a closed, bounded, convex se K R n, and we are sequenially given a series of convex cos 3 funcions f : K R for =, 2,.... he algorihm ieraively produces a poin x K in every round, wihou knowledge of f (bu using he pas sequence of cos 3 Noe he difference from he porfolio selecion problem: here we have convex cos funcions, raher han concave payoff funcions. he porfolio selecion problem is obained by using log as he cos funcion. 4

5 funcions), and incurs he cos f (x ). he regre a ime is defined o be Regre := = f (x ) min x K f (x). = In his paper, we resric our aenion o convex cos funcions which can be wrien as f (x) = g(v x) for some univariae convex funcion g and a parameer vecor v R n (for example, in he porfolio managemen problem, K = n, f (x) = log(r x), g = log, and v = r ). hus, he cos funcions are paramerized by he vecors v, v 2,..., v. Our bounds will be expressed as a funcion of he quadraic variabiliy of he parameer vecors v, v 2,..., v, defined as Q(v,..., v ) := min µ v µ 2. his expression is minimized a µ = = v, and hus he quadraic variaion is jus imes he sample variance of he sequence of vecors {v,..., v }. Noe however ha he sequence can be generaed adversarially raher han by some sochasic process. We shall refer o his as simply Q if he vecors are clear from he conex. Main heorem. In he seup of he online convex opimizaion problem above, we have he following algorihmic resul: heorem. Le he cos funcions be of he form f (x) = g(v x). Assume ha here are parameers R, D, a, b > 0 such ha he following condiions hold:. for all, v R, 2. for all x K, we have x D, 3. for all x K, and for all, eiher g (v x) [0, a] or g (v x) [ a, 0], and 4. for all x K, and for all, g (v x) b. hen here is an algorihm ha guaranees he following regre bound: Regre = O((a 2 n/b) log( + bq + br 2 ) + ard log( + Q/R 2 ) + D 2 ). Now we apply heorem o he porfolio selecion problem. Firs, we esimae he relevan parameers. We have r n since all r (i), hus R = n. For any x n, x, so D =. g (v x) = (v, and hus x) g (v x) [ r, 0], so a = r. Finally, g (v x) = (v x), so b =. Applying heorem we ge he 2 following corollary: Corollary 2. For he porfolio selecion problem over n asses, here is an algorihm ha aains he following regre bound: ( n ) Regre = O r 2 log(q + n). = 5

6 2 Bounding he Regre by he Observed Variaion in Reurns 2. Preliminaries All marices are assumed be real symmeric marices in R n n, where n is he number of socks. We use he noaion A B o say ha A B is posiive semidefinie. We require he noion of a norm of a vecor x induced by a posiive definie marix M, defined as x M = x Mx. he following simple generalizaion of he Cauchy- Schwarz inequaliy is used in he analysis: x, y R n : x y x M y M. We denoe by A he deerminan of a marix A, and by A B = r(ab) = ij A ijb ij. As we are concerned wih logarihmic regre bounds, poenial funcions which behave like harmonic series come ino play. A generalizaion of harmonic series o high dimensions is he vecor-harmonic series, which is a series of quadraic forms ha can be expressed as (here A 0 is a posiive definie marix, and v, v 2,... are vecors in R n ): v (A + v v ) v, v 2 (A + v v + v 2 v 2 ) v 2,..., v (A + τ= v τ v τ ) v,... he following lemma is from [HAK07] (and proven in appendix A for compleeness): Lemma 3. For a vecor harmonic series given by an iniial marix A and vecors v, v 2,..., v, we have v (A + [ τ= v τ vτ ) A + ] τ= v log v τ vτ. A = he reader can noe ha in one dimension, if all vecors v = and A =, hen he series above reduces exacly o he regular harmonic series whose sum is bounded, of course, by log( + ). Henceforh we will denoe by he summaion =. 2.2 Algorihm and analysis We analyze he following algorihm and prove ha i aains logarihmic regre wih respec o he observed variaion (raher han number of ieraions). he algorihm follows he generic algorihmic scheme of Follow-he-Regularized-Leader (FRL) wih squared Euclidean regularizaion. Algorihm Exp-Concave-FL. In ieraion, use he poin x defined as: x arg min x n ( ) f τ (x) + 2 x 2 τ= Noe he mahemaical program which he algorihm solves is convex, and can be solved in ime polynomial in he dimension and number of ieraions. he running () 6

7 ime, however, for solving his convex program can be quie high. In secion 4, for he specific problem of porfolio selecion, where f (x) = log(r x), we give a faser implemenaion whose per ieraion running ime is independen of he number of ieraions. We now proceed o prove he heorem. Proof. [heorem ] Firs, we noe ha he algorihm is running a Follow-he-leader procedure on he cos funcions f 0, f, f 2,... where f 0 (x) = 2 x 2 is a ficiious period 0 cos funcion. In oher words, in each ieraion, i chooses he poin ha would have minimized he oal cos under all he observed funcions so far (and, addiionally, a ficiious iniial cos funcion f 0 ). his poin is referred o as he leader in ha round. he firs sep in analyzing such an algorihm is o use a sabiliy lemma from [KV05], which bounds he regre of any Follow-he-leader algorihm by he difference in coss (under f ) of he curren predicion x and he nex one x +, plus an addiional error erm which comes from he regularizaion. hus, we have (recall he noaion = ) Regre f (x ) f (x + ) + 2 ( x 2 x 0 2 ) f (x ) (x x + ) + 2 D2 = g (v x )[v (x x + )] + 2 D2 (2) he second inequaliy is because f is convex. he las equaliy follows because f (x ) = g (x v )v. Now, we need a handle on x x +. For his, define F = τ=0 f τ, and noe ha x minimizes F over K. Consider he difference in he gradiens of F + evaluaed a x + and x : F + (x + ) F + (x ) = = = = f τ (x + ) f τ (x ) τ=0 [g (v τ x + ) g (v τ x )]v τ + (x + x ) τ= [ g (v τ ζτ ) (x + x )]v τ + (x + x ) τ= g (v τ ζτ )v τ vτ (x + x ) + (x + x ). τ= (3) (4) Equaion 3 follows by applying he aylor expansion of he (muli-variae) funcion g (v τ x) a poin x, for some poin ζ τ on he line segmen joining x and x +. he equaion (4) follows from he observaion ha g (v τ x) = g (v τ x)v τ. 7

8 Define A = τ= g (v τ ζ τ )v τ v τ +I, where I is he ideniy marix, and x = x + x. hen equaion (4) can be re-wrien as: F + (x + ) F (x ) g (v x )v = A x. (5) Now, since x minimizes he convex funcion F over he convex se K, a sandard inequaliy of convex opimizaion (see [BV04]) saes ha for any poin y K, we have F (x ) (y x ) 0. hus, for y = x +, we ge ha F (x ) (x + x ) 0. Similarly, we ge ha F + (x + ) (x x + ) 0. Puing hese wo inequaliies ogeher, we ge ha ( F + (x + ) F (x )) x 0. (6) hus, using he expression for A x from (5) we have x 2 A = A x x = ( F + (x + ) F (x ) g (v x )v ) x g (v x )[v (x x + )] (from (6)) (7) Assume ha g (v x) [ a, 0] for all x K and all. he oher case is handled similarly. Inequaliy (7) implies ha g (v x ) and v (x x + ) have he same sign. hus, we can upper bound g (v x )[v (x x + )] a(v x ). (8) Define ṽ = v µ, µ = + τ= v τ. hen, we have v x = ṽ x + x (µ µ ) x µ + x + µ, (9) Now, define ρ = ρ(v,..., v ) = = µ + µ. hen we bound x (µ µ ) x µ + x + µ =2 =2 x µ µ + x µ + x + µ =2 Dρ + 2DR. (0) We will bound ρ momenarily. For now, we urn o bounding he firs erm of (9) using he Cauchy-Schwarz generalizaion as follows: ṽ x ṽ A x A. () By he usual Cauchy-Schwarz inequaliy, ṽ A x A ṽ 2 A ṽ 2 A x 2 A a(v x ). 8

9 from (7) and (8). We conclude, using (9), (0) and (), ha a(v x ) a ṽ 2 a(v A x ) + adρ + 2aDR. his implies (using he AM-GM inequaliy applied o he firs erm on he RHS) ha a(v x ) a 2 ṽ 2 + 2aDρ + 4aDR. A Plugging his ino he regre bound (2) we obain, via (8), Regre a 2 ṽ 2 A + 2aDρ + 4aDR + 2 D2. he proof is compleed by he following wo lemmas (Lemmas 4 and 5) which bound he RHS. he firs erm is a vecor harmonic series, and he second erm can be bounded by a (regular) harmonic series. Lemma 4. ṽ 2 5n A b log [ + bq + br 2]. Proof. We have A = τ= g (v τ ζτ )v τ vτ + I. Since g (v ζτ ) b, we have A I + b τ= v τ vτ. Using he fac ha ṽ = v µ and µ = + τ v τ we ge ha ṽ τ ṽτ = τ= = Since ( + s= (v s v τ )(v s v τ ) s + s + τ s τ s ) ( (τ + ) 2 2 v s vs + s + s + + s= τ=s s = we ge ha +2 s+ x 2 dx s + + τ=s τ=s s= r<s (τ + ) 2 + s τ=s ) (τ + ) 2 [v r vs + v s vr ]. x 2 dx = s +, (2) (τ + ) 2 s 2 +. (3) Since (v r ± v s )(v r ± v s ) 0, we have (v r vr + v s vs ) ±(v r vs + v s vr ), and so ( ) s + + (τ + ) 2 [v r vs + v s vr ] s + + (τ + ) 2 [v rvr + v s vs ] τ=s τ=s ( s 2 + ) [v r vr + v s vs ], 9

10 by (3). Also by (2), we have ( + τ=s (τ+) 2 2 s+ ) + s + 2 s+, so we have ṽ τ ṽτ τ= ( v s vs + s 2 + ) [v r vr + v s vs ] s= s= r<s ( 3 v s vs + s 2 + ) v r vr s= s= r<s ( 3 v s vs + v s vs r 2 + ) 3 5 s= v s vs + s= v s vs. s= s= s= r=s+ ( + ) v s vs s Le à = I + b τ ṽτ ṽτ. Noe ha he inequaliy above shows ha à 5A. hus, using Lemma 3, we ge ṽ 2 = ṽ A A ṽ 5 [ bṽ ] à [ [ ] bṽ ] 5 b b log Ã. (4) Ã0 o bound he laer quaniy noe ha Ã0 = I =, and ha à = I + b ṽ ṽ ( + b ṽ 2 2) n = ( + b Q) n where Q = ṽ 2 = v µ 2. Lemma 6, shows ha Q Q + R 2. his implies ha à ( + bq + br 2 ) n and he proof is compleed by subsiuing his bound ino (4). Lemma 5. ρ(v,..., v ) 4R[log( + Q/R 2 ) + 2]. Proof. Define, for τ = 0,, 2,...,, he vecor u τ = v τ µ, where µ = = v. Le v 0 = 0 and u 0 = µ. We have u 2 = = v µ 2 = Q. = 0

11 We have = + µ + µ = + 2 τ=0 + = + 2 ( + ) 2 τ=0 Summing up over all ieraions, ( ρ = µ + µ ( + ) 2 = u 0 u 0 + = = v τ + u τ + τ=0 v τ τ=0 u τ τ=0 u τ + + u +. τ=0 ) u τ + + u + ( + ) 2 + u 2 u = 4R[log( + Q/R 2 ) + 2]. he second inequaliy follows because τ= inequaliy uses he following facs: (τ+) 2 ( + τ= ) (τ + ) 2 x= x dx = 2. he las. Since v 0 = 0, u 0 = µ, and hence u 0 = µ R since for all, v R. 2. Noe u = v µ v + µ 2R. Applying Lemma 7 below wih x = u /2R for =, 2,...,, and using he fac ha = x2 = = u 2 /4R 2 Q/R 2, we ge ha = 2 u 4R(log( + Q/R 2 ) + ). Lemma 6. Q Q + R 2. Proof. Consider he Be-he-Leader (BL) algorihm played on he sequence of cos funcions c (x) = v x 2, for = 0,, 2,...,, wih v 0 = 0, when he convex domain is he ball of radius R. he BL algorihm is as follows. On round, his algorihm chooses he poin ha minimizes τ=0 c (x) over he domain. I is easy o see ha his poin is exacly + τ=0 v = µ. hus, he cos of he algorihm is τ=0 v µ 2 = Q, since µ 0 = v 0 = 0, and he firs period cos is hus 0. he bes fixed poin in hindsigh is µ. hus, he cos of he bes fixed poin in hindsigh, µ, is τ=0 v µ 2 = Q + µ 2. Kalai and Vempala [KV05] prove ha he BL algorihm incurs 0 regre, i.e. Q Q + µ 2 Q + R 2.

12 Lemma 7. Suppose ha 0 x and x2 Q. hen = x log( + Q) +. Proof. By he Lemma 8 below, he values of x ha maximize = x / mus have he following srucure: here is a k such ha for all k, we have x =, and for any index > k, we have x k+ /x (/k)/(/), which implies ha x k/. We firs noe ha k Q, since Q k = x2 = k. Now, we can bound he value as follows: = x k = + =k+ k 2 log(k + ) + k k = log( + Q) +. Lemma 8. Le a a 2... a n > 0. hen he opimal soluion of { max a i x i : 0 x i and } x 2 i Q i i has he following properies: x x 2... x n, and for any pair of indices i, j, wih i < j, eiher x i =, x i = 0 or x i /x j a i /a j. Proof. he fac ha in he opimal soluion x x 2... x n is obvious, since oherwise we could permue he x i s o be in decreasing order and increase he value. he second fac follows by he Karush-Kuhn-ucker (KK) opimaliy condiions, which imply he exisence of consans µ, λ,..., λ n, ρ,..., ρ n for which he opimal vecor x saisfies (here e i is he i h sandard basis vecor, and a = (a, a 2,..., a n )): a + 2µx + i (λ i + ρ i )e i = 0 By KK heory, complemenary slackness implies ha he consans λ i, ρ i are equal o zero for all indices of he soluion which saisfy x i / {0, }. For hese coordinaes, he KK equaion is a i 2µx i = 0, which implies he lemma. 3 Implicaions in he Geomeric Brownian Moion Model We begin wih a brief descripion of he model. he model assumes ha socks can be raded coninuously, and ha a any ime, he fracional change in he sock price 2

13 wihin an infiniesimal ime inerval is normally disribued, wih mean and variance proporional o he lengh of he inerval. he randomness is due o many infiniesimal rades ha jar he price, much like paricles in a physical medium are jarred abou by oher paricles, leading o he classical Brownian moion. Formally, he model is parameerized by wo quaniies, he drif µ, which is he long erm rend of he sock prices, and volailiy σ, which characerizes deviaions from he long erm rend. he parameer σ is ypically specified as annualized volailiy, i.e. he sandard deviaion of he sock s logarihmic reurns in one year. hus, a rading inerval of [0, ] specifies year. he model posulaes ha he sock price a ime, S, follows a geomeric Brownian moion wih drif µ and volailiy σ: ds = µs d + σs dw, where W is a coninuous-ime sochasic process known as he Wiener process or simply Brownian moion. he Wiener process is characerized by hree facs:. W 0 = 0, 2. W is almos surely coninuous, and 3. for any wo disjoin ime inervals [s, ] and [s 2, 2 ], he random variables W W s and W 2 W s2 are independen zero mean Gaussian random variables wih variance s and 2 s 2 respecively. Using Iō s lemma (see, for example, [KS04]), i can be shown ha he sock price a ime is given by S = S 0 exp((µ σ 2 /2) + σw ). (5) Now, we consider a siuaion where we have n socks in he GBM model. Le µ = (µ, µ 2,..., µ n ) be he vecor of drifs, and σ = (σ, σ 2,..., σ n ) be he vecor of (annualized) volailiies. Suppose we rade for one year. We now sudy he effec of rading frequency on he quadraic variaion of he sock price reurns. For his, assume ha he year-long rading inerval is sub-divided ino equally sized inervals of lengh /, and we rade a he end of each such inerval. Le r = (r (), r (2),..., r (n)) be he vecor of sock reurns in he h rading period. We assume ha is large enough, which is aken o mean ha i is larger han µ(i), σ(i), ( µ(i) σ(i) )2 for any i. hen using he facs of he Wiener process saed above, we can prove he following lemma, which shows ha he expeced quadraic variaion is he essenially he same regardless of rading frequency, while is variance decreases wih rading frequency. Lemma 9. In he seup of rading n socks in he GBM model over one year wih rading periods, where µ(i), σ(i), i, here is a vecor v such ha [ E = r v 2] σ 2 ( + O( )) and [ VAR = r v 2] 3 σ 4 ( + O( )), regardless of how he socks are correlaed. 3

14 Proof. For every sock i, i follows from he GBM equaion for he sock prices (S = S 0 exp((µ σ 2 /2) + σw ) ) ha is reurn r (i) in period is given by ) ) r (i) = exp ((µ(i) σ(i)2 2 + σ(i)x (i), where X (i) N (0, ). hus, for any given sock i, he reurns r (i), r 2 (i),..., r (i) are i.i.d. log-normal random variables, wih parameers: ( ) µ(i) r (i) ln N σ(i)2 2, σ(i)2 Recall ha for a log-normal random variable X ln N (µ, σ 2 ) he mean and variance are given by E[X] = e µ+σ2 /2 and VAR[X] = (e σ2 )e 2µ+σ2. Define he vecor v as v(i) = E[r (i)] = e µ(i)/. hen, assuming ha > µ(i), σ(i), we have using he exponenial approximaion e x + x + x 2 for x : [ E (r (i) v(i)) 2] = VAR(r (i)) = (e σ(i)2 / )e 2µ(i)/ ( σ(i)2 + O( )) e2µ(i)/ 2 σ(i)2 ( ( + O )) Where he las equaliy uses he aylor approximaion of he exponenial and he fac ha 2µ(i). Summing up over all socks i and all periods, and using lineariy of expecaion, we ge he firs par of he Lemma: [ E = r v 2] = = i= n E[(r (i) v(i)) 2 ] σ 2 ( + O( )) As for he bound on he variance, we firs bound he quaniy E[(r (i) v(i)) 4 ], using he fac ha if X ln N (µ, σ 2 ), hen X a ln N (aµ, a 2 σ 2 ). We denoe µ = µ(i) σ(i)2 2, σ2 = σ(i)2. E[(r (i) v(i)) 4 ] = E[r (i) 4 4r (i) 3 v(i) + 6r (i) 2 v(i) 2 4r (i)v(i) 3 + v(i) 4 ] = e 4 µ+8 σ2 4e 3 µ+ 9 2 σ2 e µ+ σ2 /2 + 6e 2 µ+2 σ2 e 2 µ+ σ2 4e µ+ σ2 /2 e 3 µ+ 3 2 σ2 + e 4 µ+2 σ2 = e 4 µ+8 σ2 4e 4 µ+5 σ2 + 6e 4 µ+3 σ2 4e 4 µ+2 σ2 + e 4 µ+2 σ2 = e 4 µ (e 8 σ2 4e 5 σ2 + 6e 3 σ2 3e 2 σ2 ) = (e σ2 ) 2 e 4 µ (e 6 σ2 + 2e 5 σ2 + 3e 4 σ2 3e 2 σ2 ) ( σ(i)4 2 + O( 3 )) e4 µ (e 6 σ 2 + 2e 5 σ2 + 3e 4 σ2 3e 2 σ2 ) = σ(i)4 2 (3 + O( 3σ(i)4 )) = 2 ( + O( )) 4

15 Above he fifh equaliy follows from he polynomial facorizaion: x 8 4x 5 + 6x 3 3x 2 = (x ) 2 (x 6 + 2x 5 + 3x 4 3x 2 ) hus for all socks i and ime periods, we have: VAR [ (r (i) v(i)) 2] E[(r (i) v(i)) 4 ] 3σ(i)4 2 ( + O( )) (6) Now, we use he following inequaliy which follows from he Cauchy-Schwarz inequaliy: for any random variables X, X 2,..., X n : Hence, using inequaliy (6), we ge: VAR[ n i= X i] ( n i= VAR[Xi ]) 2. VAR[ r v 2 ] ( n i= VAR[(r (i) v(i)) 2 ]) 2 3 σ 4 2 ( + O( )) Finally, using he fac ha in he GBM model, he reurns are independen beween periods we ge [ ] VAR r v 2 3 σ 4 ( + O( )). = Applying his bound in our algorihm, we obain he following regre bound from Corollary 2. heorem 0. In he seup of Lemma 9, for any δ > 0, wih probabiliy a leas δ, we have Regre = O(n log(( + 3 δ ) σ 2 + n)). Proof. Firs, we show ha he marke variabiliy parameer, r, is Ω() wih high probabiliy. Fix any sock i. he reurn r (i) for sock i a ime period is a log-normal random variable: r (i) = exp(n (i)) where N (i) N ((µ(i) σ(i)2 2 ), σ(i)2 ). Using sandard ail bounds for normal variables (which is given in Lemma 4 in he Appendix for compleeness), we have [ Pr N (i) (µ(i) σ(i)2 2 ) > σ(i) ] 2 log(2n/δ) < e log(2n/δ) = δ 2n. hus, assuming µ(i), σ(i), wih probabiliy a leas N (i) = O( σ(i) log(n/δ)). δ 2n, we have Since r (i) = exp(n (i)), we conclude ha wih probabiliy a leas r (i) = O( σ(i) log(n/δ)). δ 2n, we have 5

16 Applying a union bound over all socks and all periods, we conclude ha wih probabiliy a leas δ σ 2, he marke variabiliy parameer r is a leas O( log(n/δ)) > 0.5, if > Ω(σ 2 log(n/δ)). Now we bound he oal variaion. Applying Chebyshev s inequaliy o he random variable = r v 2 and using Lemma 9, we have [ Pr = r v 2 > ( + 3 δ ) σ 2] < 3 σ 4 ( + O( )) 9 σ 4 δ < δ 2. he resul now follows by a union bound applied o our regre bound of Corollary 2. heorem 0 shows ha one expecs o achieve consan regre independen of he rading frequency, as long as he oal rading period is fixed. his resul is only useful if increasing rading frequency improves he performance of he bes consan rebalanced porfolio. Indeed, his has been observed empirically (see e.g. [AHKS06] and Secion 5). o obain a heoreical jusificaion for increasing rading frequency, we consider an example where we have wo socks ha follow independen Black-Scholes models wih he same drifs, bu differen volailiies σ, σ 2. he same drif assumpion is necessary because in he long run, he bes CRP is he one ha pus all is wealh on he sock wih he greaer drif. We normalize he drifs o be equal o 0, his doesn change he performance in any qualiaive manner. Since he drif is 0, he expeced reurn of eiher sock in any rading period is ; and since he reurns in each period are independen, he expeced final change in wealh, which is he produc of he reurns, is also. hus, in expecaion, any CRP (indeed, any porfolio selecion sraegy) has overall reurn. We herefore urn o a differen crierion for selecing a CRP. he risk of an invesmen sraegy is measured by he variance of is payoff; hus, if differen invesmen sraegies have he same expeced payoff, hen he one o choose is he one wih minimum variance. We herefore choose he CRP wih he leas variance. Lemma. In he seup where we rade wo socks wih zero drif and volailiies σ, σ 2, he variance of he minimum variance CRP decreases as he rading frequency increases. Proof. o compue he minimum variance CRP, we firs compue he variance of he CRP (p, p): [ ] [ ] VAR (pr () + ( p)r (2)) = E (pr () + ( p)r (2)) 2 = E[(pr () + ( p)r (2)) 2 ]. he second equaliy above follows from he independence of he randomness in each period. hus, he minimum variance CRP is obained by minimizing E[(pr () + ( 6

17 p)r (2)) 2 ] over he range p [0, ]. We now noe ha in he Black-Scholes model, r () = e X where X N ( σ2 and hus his is minimized a p = 2, σ2 E[(pr () + ( p)r (2)) 2 ] ), and r (2) = e X2 where X 2 N ( σ2 2 = E[p 2 e 2X + 2p( p)e X+X2 + ( p) 2 e 2X2 ] = p 2 e σ 2 + 2p( p) + ( p) 2 e σ2 2. exp( σ2 2 ) exp( σ2 )+exp( σ2 2 ) 2 hus, he variance of he final payoff becomes [, and a his poin, is value is ] exp( σ2 +σ2 2 ). exp( σ2 ) + exp( σ2 2 ) 2 his is a decreasing funcion of. o prove his, by direc compuaion we have [ ] d exp( σ2 +σ2 2 ) d exp( σ2 ) + exp( σ2 2 ) 2 = [ exp( σ2 +σ2 2 ) exp( σ2 ) + exp( σ2 2 ) 2 2 (exp( σ ) )2 exp( σ2 2 < 0. ) σ2 2 ] log [ ] exp( σ2 +σ2 2 ) exp( σ2 ) + exp( σ2 2 ) 2 (exp( σ2 2 2 ) ) 2 exp( σ2 (exp( σ2 ) + exp( σ2 2 ) 2) 2 ) σ2 2 2, σ2 2 ), exp( σ2 +σ2 2 ) exp( σ2 )+exp( σ2 2 ) 2 he inequaliy above uses he fac ha exp( σ2 +σ2 2 ) > exp( σ2 ) + exp( σ2 2 ) 2 which can be verified by using he power series expansion of exp(x). hus, increasing he rading frequency decreases he variance of he minimum variance CRP, which implies ha i ges less risky o rade more frequenly; in oher words, he more frequenly we rade, he more likely he payoff will be close o he expeced value. On he oher hand, as we show in heorem 0, he regre does no change even if we rade more ofen; hus, one expecs o see improving performance of our algorihm as he rading frequency increases.. 4 Faser Implemenaion In his secion we describe a more efficien algorihm compared o he one from he main body of he paper. he regre bound deerioraes slighly, hough i is sill logarihmic in he oal quadraic variaion. he algorihm is based on he online Newon 7

18 mehod, inroduced in [HAK07], and is described in he following figure. For simpliciy, we focus on he porfolio managemen problem, alhough i is likely ha similar ideas can work for general loss funcions of he ype we consider in his paper. Algorihm Faser quadraic-variaion universal algorihm for = o do ( Use x arg min f x n τ= τ (x) + 2 ). x 2 Receive reurn vecor r. Le f (x) = log(r x ) r (x x) (r x ) + r(r (x x))2 8(r x ). 2 end for he basic idea is o bound he cos funcions by a paraboloid approximaion in he FRL algorihm. he paraboloid approximaion only increases he regre, bu since i is a simple quadraic funcion, he running ime of he he FRL algorihm is improved grealy. All we need o do is opimize he sum of quadraic cos funcions, which has a compac represenaion (unlike he sum of log funcions), over he n-dimensional simplex. his opimizaion can be carried ou in ime O(n 3.5 ) using inerior poin mehods (assuming real number operaions can be carried ou in O() ime). Using observaions made in [HAK07] i is possible o furher speed up he algorihm and aain a running ime proporional o O(n 3 ). We have he following regre bound for he algorihm: heorem 2. For he porfolio selecion problem, he regre of algorihm is bounded by ( n ) Regre = O r 3 log(q + n). Proof. We firs describe he paraboloid approximaion o he cos funcions ha we use in he algorihm insead of acual cos funcions. his approximaion, based on a more general lemma from [HAK07], has he following propery, for all x and y in he simplex and any reurn vecor v wih coordinaes in [r, ]: log(v x) log(v y) v (x y) (v y) + r(v (x y))2 8(v y) 2. hus for any, f (x ) = log(r x ), and for any x n, f (x) log(r x). hus, if x is he bes CRP in hindsigh, we have he following bound on he regre of algorihm : Regre = log(r x ) log(r x ) f (x ) f (x ). he RHS above is bounded by he regre of he algorihm assuming ha he cos funcions are f. We herefore proceed o bound his regre. of he algorihm wih cos funcions f. 8

19 he cos funcions f can be wrien in erms of he univariae funcions r g (y) = 8(r x ) 2 y2 ( + r ) y + r r x log(r x ) as f (x) = g (r x ). Now we noe ha even hough he saemen of he main heorem assumes ha he cos funcions can be wrien in erms of a single univariae funcion g for all, he proof of he heorem is flexible enough o handle differen funcions g for differen, as long as condiions 3. and 4. in he main heorem on he firs and second derivaives of g hold uniformly wih he same consans a and b for all funcions g. Furhermore, he proof only requires he bound a on he magniude of he firs derivaives a he poins x which he algorihm produces. hus, we can now esimae he a and b parameers for he g funcions as follows: g (r x ) = (r [ x ) r, 0], so we choose a = r. For any porfolio x n, and g r (r x) = 4(r x) r 2 4, hus we choose b = r 4. he regre bound is now obained via he bound of he main heorem. 5 Experimens We esed he performance of he bes CRP and Algorihm Exp-Concave-FL as well as is regre on sock marke daa. he following graph in Figure was generaed using real NYSE quoes for he 000 rading days from 200 and 2005 obained form Yahoo! finance. We randomly chose weny S&P500 socks, and compued he performance of he bes CRP and Algorihm Exp-Concave-FL on he relevan rading period, varying he rading frequency from daily o every 50 rading days (only ineger periods which divide 000 were esed). he regre, i.e difference beween he log wealh of boh mehods is also depiced. For various choices of socks, he fac ha he regre remains prey much consan was consisen, as prediced by he heoreical argumens in he paper. he change in performance of he bes CRP wih respec o rading frequency was no conclusive, a imes agreeing wih heory and a imes no. In one sense, however, his change does agree wih heory: in he previous work of [AHKS06], Figure 6 depics he performance of several online porfolio managemen as varies wih he rading period. hese more comprehensive experimens, which measure he average APY (annual percenage yield) in an experimen of sampling a se of socks from he S&P 500 (he average is aken over he differen samples of random socks from S&P 500). hese experimens clearly show ha on an average, he performance of many online algorihms, as well as ha of he bes CRP, improves wih rading frequency. 6 Conclusions We have presened an efficien algorihm for regre minimizaion wih exp-concave loss funcions whose regre sricly improves upon he sae of he ar. For he problem of porfolio selecion, he regre is bounded in erms of he observed variaion in sock reurns raher han he number of ieraions. his is he firs heoreical improvemen 9

20 BCRP Algorihm regre Figure : Bes CRP and Algorihm Exp-Concave-FL vs. rading Period. he x-axis denoes he rading period in days, and he y-axis is he log-wealh facor (i.e. a real number). in regre bounds for universal porfolio selecion algorihms since he work of Cover [Cov9]. We show how his fac implies ha in he sandard Geomeric Brownian Moion model for sock prices he regre does no increase wih rading frequency, hence giving he firs universal porfolio selecion algorihm whose performance improves when he underlying asses are close o GBM. his serves as a bridge beween universal porfolio heory and sochasic porfolio heory. Open quesions: I remains an inriguing open quesion o improve he dependence of he regre in porfolio selecion in erms of oher imporan parameers aside from he quadraic variabiliy: our dependence on he number of socks in he porfolio (previously denoed n) is linear. In conras [HSSW96] obain a logarihmic dependence in his parameer. Our regre bounds also depend on he marke variabiliy parameer (denoed r), whereas Cover s original algorihm does no have his dependence a all. Is i possible o obain a O(log Q) reger bound, via an efficien algorihm, ha behaves beer wih respec o n, r? References [AHKS06] Ami Agarwal, Elad Hazan, Sayen Kale, and Rober E. Schapire. Algorihms for porfolio managemen based on he newon mehod. In Proceedings of he 23rd inernaional conference on Machine learning, ICML 06, pages 9 6, New York, NY, USA, ACM. [Bac00] Louis Bachelier. héorie de la spéculaion. Annales Scienifiques de l École Normale Supérieure, 3(7):2 86,

21 [BK99] [BS73] Avrim Blum and Adam Kalai. Universal porfolios wih and wihou ransacion coss. Machine Learning, 35(3):93 205, 999. Fischer Black and Myron Scholes. he pricing of opions and corporae liabiliies. Journal of Poliical Economy, 8(3): , 973. [BV04] Sephen Boyd and Lieven Vandenberghe. Convex Opimizaion. Cambridge Universiy Press, New York, NY, USA, [CB03] Jason E Cross and Andrew R Barron. Efficien universal porfolios for pas dependen arge classes. Mahemaical Finance, 3(2): , [CBMS07] Nicolò Cesa-Bianchi, Yishay Mansour, and Gilles Solz. Improved second-order bounds for predicion wih exper advice. Machine Learning, 66(2-3):32 352, [Cov9] homas Cover. Universal porfolios. Mahemaical Finance, : 9, 99. [HAK07] [HK09] [HK0] [HK] Elad Hazan, Ami Agarwal, and Sayen Kale. Logarihmic regre algorihms for online convex opimizaion. Mach. Learn., 69(2-3):69 92, December Elad Hazan and Sayen Kale. On sochasic and wors-case models for invesing. In Y. Bengio, D. Schuurmans, J. Laffery, C. K. I. Williams, and A. Culoa, ediors, Advances in Neural Informaion Processing Sysems 22, pages Elad Hazan and Sayen Kale. Exracing cerainy from uncerainy: regre bounded by variaion in coss. Machine Learning, 80:65 88, 200. Elad Hazan and Sayen Kale. Beer algorihms for benign bandis. Journal of Machine Learning Research, 2:287 3, 20. [HSSW96] David P. Helmbold, Rober E. Schapire, Yoram Singer, and Manfred K. Warmuh. On-line porfolio selecion using muliplicaive updaes. In ICML, pages , 996. [Jam92] [KS04] [KV03] [KV05] Farshid Jamshidian. Asympoically opimal porfolios. Mahemaical Finance, 2:3 50, 992. Ioannis Karazas and Seven E. Shreve. Brownian Moion and Sochasic Calculus. Springer Verlag, New York, NY, USA, Adam Kalai and Sanosh Vempala. Efficien algorihms for universal porfolios. Journal of Machine Learning Research, 3: , Adam Kalai and Sanosh Vempala. Efficien algorihms for online decision problems. Journal of Compuer and Sysem Sciences, 7(3):29 307,

22 [MF93] Neri Merhav and Meir Feder. Universal schemes for sequenial decision from individual daa sequences. IEEE ransacions on Informaion heory, 39: , 993. [Osb59] M. F. M. Osborne. Brownian moion in he sock marke. Operaions Research, 2:45 73, 959. [Zin03] Marin Zinkevich. Online convex programming and generalized infiniesimal gradien ascen. In om Fawce and Nina Mishra, ediors, ICML, pages AAAI Press, A Proof of he Vecor Harmonic Series Lemma he proof is based on he following fac, whose one-dimensional analogue is an easy consequence of he aylor expansion of he logarihm. Lemma 3. Le A B 0 be posiive definie marices. hen A (A B) log A B where A denoes he deerminan of marix A. Proof. For any posiive definie marix C, denoe by λ (C), λ 2 (C),..., λ n (C) is (posiive) eigenvalues. Denoe by r(c) he race of he marix, which is equal o he sum of he diagonal enries of C, and also o he sum of is eigenvalues. Noe ha for he marix produc A B = n i,j= A ijb ij defined earlier, we have A B = r(ab) (where AB is he sandard marix muliplicaion), since he race is 22

23 equal o he sum of he diagonal enries. herefore, A (A B) = r(a (A B)) = r(a /2 (A B)A /2 ) = r(i A /2 BA /2 ) n [ ] = λ i (A /2 BA /2 ) i= ( r(c) = n i= n λ i (C)) i= [ ] log λ i (A /2 BA /2 ) ( x log(x)) [ n ] = log λ i (A /2 BA /2 ) i= = log A /2 BA /2 [ ] A = log. B In he las equaliy we use he following facs abou he deerminan of marices: A = n i= λ i(a), AB = A B and A = A. We can now prove he vecor harmonic series Lemma 3 : Lemma 3. v (A + [ τ= v τ vτ ) A + ] τ= v log v τ vτ. A = Proof. Le A := A + τ= v τ v τ and denoe A 0 = A. Now by he lemma above = v A v = = A v v A (A A ) i = log log A A [ ] A. A 0 23

24 B ail bounds for Normal random variables he following sandard bound on he ail of he Normal disribuion is given here for compleeness. Lemma 4. Le X N(µ, σ 2 ), hen Pr[ X µ > x] σ ( ) x exp x2 2σ 2. Proof. Firs, consider he case µ = 0, σ =. hen by definiion Pr[X > x] = x 2π exp( 2 /2) d Since he exponen is a convex funcion, we can upper bound i by he firs derivaive a he poin = x and obain: x 2π exp( 2 /2) d x 2π exp( x 2 /2 x( x)) d here he new exponen is he linearizaion of 2 /2 a = x. hen pull ou facors which don depend on o ge exp(x 2 /2) 2π and doing ha las inegral gives he bound: x exp( x) d Pr[X > x] 2πx exp( x 2 /2) Hence, by symmery of he disribuion, we ge for X N(0, ): 2 Pr[ X > x] exp( x 2 /2) πx x exp( x2 /2) Nex, consider Y N(µ, σ 2 ) = µ + σn(0, ). Hence, [ Pr[ Y µ > x] = Pr X > x ] σ ) ( σ x exp x2 2σ 2. 24