Follow the Leader If You Can, Hedge If You Must


 Brent Clifton Fleming
 1 years ago
 Views:
Transcription
1 Journal of Machine Learning Research 15 (2014) Submied 1/13; Revised 1/14; Published 4/14 Follow he Leader If You Can, Hedge If You Mus Seven de Rooij VU Universiy and Universiy of Amserdam Science Park 904, P.O. Box 94323, 1090 GH Amserdam, he Neherlands Tim van Erven Déparemen de Mahémaiques Universié ParisSud, Orsay Cedex, France Peer D. Grünwald Wouer M. Koolen Leiden Universiy (Grünwald) and Cenrum Wiskunde & Informaica (Grünwald and Koolen) Science Park 123, P.O. Box 94079, 1090 GB Amserdam, he Neherlands Edior: Nicolò CesaBianchi Absrac FollowheLeader (FTL) is an inuiive sequenial predicion sraegy ha guaranees consan regre in he sochasic seing, bu has poor performance for worscase daa. Oher hedging sraegies have beer worscase guaranees bu may perform much worse han FTL if he daa are no maximally adversarial. We inroduce he FlipFlop algorihm, which is he firs mehod ha provably combines he bes of boh worlds. As a sepping sone for our analysis, we develop AdaHedge, which is a new way of dynamically uning he learning rae in Hedge wihou using he doubling rick. AdaHedge refines a mehod by CesaBianchi, Mansour, and Solz (2007), yielding improved worscase guaranees. By inerleaving AdaHedge and FTL, FlipFlop achieves regre wihin a consan facor of he FTL regre, wihou sacrificing AdaHedge s worscase guaranees. AdaHedge and FlipFlop do no need o know he range of he losses in advance; moreover, unlike earlier mehods, boh have he inuiive propery ha he issued weighs are invarian under rescaling and ranslaion of he losses. The losses are also allowed o be negaive, in which case hey may be inerpreed as gains. Keywords: advice Hedge, learning rae, mixabiliy, online learning, predicion wih exper 1. Inroducion We consider sequenial predicion in he general framework of Decision Theoreic Online Learning (DTOL) or he Hedge seing (Freund and Schapire, 1997), which is a varian of predicion wih exper advice (Lilesone and Warmuh, 1994; Vovk, 1998; CesaBianchi and Lugosi, 2006). Our goal is o develop a sequenial predicion algorihm ha performs well no only on adversarial daa, which is he scenario mos sudies worry abou, bu also when he daa are easy, as is ofen he case in pracice. Specifically, wih adversarial daa, he worscase regre (defined below) for any algorihm is Ω( T ), where T is he number of predicions o be made. Algorihms such as Hedge, which have been designed o achieve his lower bound, ypically coninue o suffer regre of order T, even for easy daa, where c 2014 Seven de Rooij, Tim van Erven, Peer D. Grünwald and Wouer M. Koolen.
2 De Rooij, Van Erven, Grünwald and Koolen he regre of he more inuiive bu less robus FollowheLeader (FTL) algorihm (also defined below) is bounded. Here, we presen he firs algorihm which, up o consan facors, provably achieves boh he regre lower bound in he wors case, and a regre no exceeding ha of FTL. Below, we firs describe he Hedge seing. Then we inroduce FTL, discuss sophisicaed versions of Hedge from he lieraure, and give an overview of he resuls and conens of his paper. 1.1 Overview In he Hedge seing, predicion proceeds in rounds. A he sar of each round = 1, 2,..., a learner has o decide on a weigh vecor w = (w,1,..., w,k ) R K over K expers. Each weigh w,k is required o be nonnegaive, and he sum of he weighs should be 1. Naure hen reveals a Kdimensional vecor conaining he losses of he expers l = (l,1,..., l,k ) R K. Learner s loss is he do produc h = w l, which can be inerpreed as he expeced loss if Learner uses a mixed sraegy and chooses exper k wih probabiliy w,k. We denoe aggregaes of perrial quaniies by heir capial leer, and vecors are in bold face. Thus, L,k = l 1,k l,k denoes he cumulaive loss of exper k afer rounds, and H = h h is Learner s cumulaive loss (he Hedge loss). Learner s performance is evaluaed in erms of her regre, which is he difference beween her cumulaive loss and he cumulaive loss of he bes exper: R = H L, where L = min k L,k. We will always analyse he regre afer an arbirary number of rounds T. We will omi he subscrip T for aggregae quaniies such as L T or R T wherever his does no cause confusion. A simple and inuiive sraegy for he Hedge seing is FollowheLeader (FTL), which pus all weigh on he exper(s) wih he smalles loss so far. More precisely, we will define he weighs w for FTL o be uniform on he se of leaders {k L 1,k = L 1 }, which is ofen jus a singleon. FTL works very well in many circumsances, for example in sochasic scenarios where he losses are independen and idenically disribued (i.i.d.). In paricular, he regre for FollowheLeader is bounded by he number of imes he leader is overaken by anoher exper (Lemma 10), which in he i.i.d. case almos surely happens only a finie number of imes (by he uniform law of large numbers), provided he mean loss of he bes exper is sricly smaller han he mean loss of he oher expers. As demonsraed by he experimens in Secion 5, many more sophisicaed algorihms can perform significanly worse han FTL. The problem wih FTL is ha i breaks down badly when he daa are anagonisic. For example, if one ou of wo expers incurs losses 1 2, 0, 1, 0,... while he oher incurs opposie losses 0, 1, 0, 1,..., he regre for FTL a ime T is abou T/2 (his scenario is furher discussed in Secion 5.1). This has promped he developmen of a muliude of alernaive algorihms ha provide beer worscase regre guaranees. The seminal sraegy for he learner is called Hedge (Freund and Schapire, 1997, 1999). Is performance crucially depends on a parameer η called he learning rae. Hedge can be inerpreed as a generalisaion of FTL, which is recovered in he limi for η. In many analyses, he learning rae is changed from infiniy o a lower value ha opimizes 1282
3 Follow he Leader If You Can, Hedge If You Mus some upper bound on he regre. Doing so requires precogniion of he number of rounds of he game, or of some propery of he daa such as he evenual loss of he bes exper L. Provided ha he relevan saisic is monoonically nondecreasing in (such as L ), a simple way o address his issue is he socalled doubling rick: seing a budge on he saisic, and resaring he algorihm wih a double budge when he budge is depleed (CesaBianchi and Lugosi, 2006; CesaBianchi e al., 1997; Hazan and Kale, 2008); η can hen be opimised for each individual block in erms of he budge. Beer bounds, bu harder analyses, are ypically obained if he learning rae is adjused each round based on previous observaions, see e.g. (CesaBianchi and Lugosi, 2006; Auer e al., 2002). The Hedge sraegy presened by CesaBianchi, Mansour, and Solz (2007) is a sophisicaed example of such adapive uning. The relevan algorihm, which we refer o as CBMS, is defined in (16) in Secion 4.2 of heir paper. To discuss is guaranees, we need he following noaion. Le l = min k l,k and l + = max k l,k denoe he smalles and larges loss in round, and le L = l l and L + = l l+ denoe he cumulaive minimum and maximum loss respecively. Furher le s = l + l denoe he loss range in rial and le S = max{s 1,..., s } denoe he larges loss range afer rials. Then, wihou prior knowledge of any propery of he daa, including T, S and L, he CBMS sraegy achieves regre bounded by 1 R CBMS 4 (L L )(L + ST L ) T ln K + lower order erms (1) (CesaBianchi e al., 2007, Corollary 3). Hence, in he wors case L = L + ST/2 and he bound is of order S T, bu when he loss of he bes exper L [L, L + ST ] is close o eiher boundary he guaranees are much sronger. The conribuions of his work are wofold: firs, in Secion 2, we develop AdaHedge, which is a refinemen of he CBMS sraegy. A (very) preliminary version of his sraegy was presened a NIPS (Van Erven e al., 2011). Like CMBS, AdaHedge is compleely parameerless and unes he learning rae in erms of a direc measure of pas performance. We derive an improved worscase bound of he following form. Again wihou any assumpions, we have R ah 2 S (L L )(L + L ) L + L ln K + lower order erms (2) (see Theorem 8). The parabola under he square roo is always smaller han or equal o is CMBS counerpar (since i is nondecreasing in L + and L + L +ST ); i expresses ha he regre is small if L [L, L + ] is close o eiher boundary. I is maximized in L a he midpoin beween L and L +, and in his case we recover he worscase bound of order S T. Like (1), he regre bound (2) is fundamenal, which means ha i is invarian under ranslaion of he losses and proporional o heir scale. Moreover, no only AdaHedge s regre bound is fundamenal: he weighs issued by he algorihm are hemselves invarian 1. As poined ou by a referee, i is widely known ha he leading consan of 4 can be improved o using echniques by Györfi and Oucsák (2007) ha are essenially equivalen o our Lemma 2 below; Gerchinoviz (2011, Remark 2.2) reduced i o approximaely AdaHedge allows a sligh furher reducion o
4 De Rooij, Van Erven, Grünwald and Koolen under ranslaion and scaling (see Secion 4). The CBMS algorihm and AdaHedge are insensiive o rials in which all expers suffer he same loss, a naural propery we call imelessness. An aracive feaure of he new bound (2) is ha i expresses his propery. A more deailed discussion appears below Theorem 8. Our second conribuion is o develop a second algorihm, called FlipFlop, ha reains he worscase bound (2) (up o a consan facor), bu has even beer guaranees for easy daa: is performance is never subsanially worse han ha of FollowheLeader. A firs glance, his may seem rivial o accomplish: simply ake boh FTL and AdaHedge, and combine he wo by using FTL or Hedge recursively. To see why such approaches do no work, suppose ha FTL achieves regre R fl, while AdaHedge achieves regre R ah. We would only be able o prove ha he regre of he combined sraegy compared o he bes original exper saisfies R c min{r fl, R ah } + G c, where G c is he worscase regre guaranee for he combinaion mehod, e.g. (1). In general, eiher R fl or R ah may be close o zero, while a he same ime he regre of he combinaion mehod, or a leas is bound G c, is proporional o T. Tha is, he overhead of he combinaion mehod will dominae he regre! The FlipFlop approach we describe in Secion 3 circumvens his by alernaing beween Following he Leader and using AdaHedge in a carefully specified way. For his sraegy we can guaranee R ff = O(min{R fl, G ah }), where G ah is he regre guaranee for AdaHedge; Theorem 15 provides a precise saemen. Thus, FlipFlop is he firs algorihm ha provably combines he benefis of Followhe Leader wih robus behaviour for anagonisic daa. A key concep in he design and analysis of our algorihms is wha we call he mixabiliy gap, inroduced in Secion 2.1. This quaniy also appears in earlier works, and seems o be of fundamenal imporance in boh he curren Hedge seing as well as in sochasic seings. We elaborae on his in Secion 6.2 where we provide he big picure underlying his research and we briefly indicae how i relaes o pracical work such as (Devaine e al., 2013). 1.2 Relaed Work As menioned, AdaHedge is a refinemen of he sraegy analysed by CesaBianchi e al. (2007), which is iself more sophisicaed han mos earlier approaches, wih wo noable excepions. Firs, Chaudhuri, Freund, and Hsu (2009) describe a sraegy called NormalHedge ha can efficienly compee wih he bes ɛquanile of expers; heir bound is incomparable wih he bounds for CBMS and for AdaHedge. Second, Hazan and Kale (2008) develop a sraegy called Variaion MW ha has especially low regre when he losses of he bes exper vary lile beween rounds. They show ha he regre of Variaion MW is of order VAR max T ln K, where VAR max T = max T s=1 ( ls,k 1 L,k ) 2 wih k he bes exper afer rounds. This bound dominaes our worscase resul (2) (up o a muliplicaive consan). As demonsraed by he experimens in Secion 5, heir mehod does no achieve he benefis of FTL, however. In Secion 5 we also discuss he performance of NormalHedge and Variaion MW compared o AdaHedge and FlipFlop. 1284
5 Follow he Leader If You Can, Hedge If You Mus Oher approaches o sequenial predicion include Defensive Forecasing (Vovk e al., 2005), and Following he Perurbed Leader (Kalai and Vempala, 2003). These radically differen approaches also allow compeing wih he bes ɛquanile, as shown by Chernov and Vovk (2010) and Huer and Poland (2005); he laer also consider nonuniform weighs on he expers. The safe MDL and safe Bayesian algorihms by Grünwald (2011, 2012) share he presen work s focus on he mixabiliy gap as a crucial par of he analysis, bu are concerned wih he sochasic seing where losses are no adversarial bu i.i.d. FlipFlop, safe MDL and safe Bayes can all be inerpreed as mehods ha aemp o choose a learning rae η ha keeps he mixabiliy gap small (or, equivalenly, ha keeps he Bayesian poserior or Hedge weighs concenraed ). 1.3 Ouline In he nex secion we presen and analyse AdaHedge and compare is worscase regre bound o exising resuls, in paricular he bound for CBMS. Then, in Secion 3, we build on AdaHedge o develop he FlipFlop sraegy. The analysis closely parallels ha of AdaHedge, bu wih exra complicaions a each of he seps. In Secion 4 we show ha boh algorihms have he propery ha heir behaviour does no change under ranslaion and scaling of he losses. We furher illusrae he relaionship beween he learning rae and he regre, and compare AdaHedge and FlipFlop o exising mehods, in experimens wih arificial daa in Secion 5. Finally, Secion 6 conains a discussion, wih ambiious suggesions for fuure work. 2. AdaHedge In his secion, we presen and analyse he AdaHedge sraegy. To inroduce our noaion and proof sraegy, we sar wih he simples possible analysis of vanilla Hedge, and hen move on o refine i for AdaHedge. 2.1 Basic Hedge Analysis for Consan Learning Rae Following Freund and Schapire (1997), we define he Hedge or exponenial weighs sraegy as he choice of weighs w,k = w 1,ke ηl 1,k Z, (3) where w 1 = (1/K,..., 1/K) is he uniform disribuion, Z = w 1 e ηl 1 is a normalizing consan, and η (0, ) is a parameer of he algorihm called he learning rae. If η = 1 and one imagines L 1,k o be he negaive loglikelihood of a sequence of observaions, hen w,k is he Bayesian poserior probabiliy of exper k and Z is he marginal likelihood of he observaions. Like in Bayesian inference, he weighs are updaed muliplicaively, i.e. w +1,k w,k e ηl,k. The loss incurred by Hedge in round is h = w l, he cumulaive Hedge loss is H = h h, and our goal is o obain a good bound on H T. To his end, i urns 1285
6 De Rooij, Van Erven, Grünwald and Koolen ou o be echnically convenien o approximae h by he mix loss m = 1 η ln(w e ηl ), (4) which accumulaes o M = m m. This approximaion is a sandard ool in he lieraure. For example, he mix loss m corresponds o he loss of Vovk s (1998; 2001) Aggregaing Pseudo Algorihm, and racking he evoluion of m is a crucial ingredien in he proof of Theorem 2.2 of CesaBianchi and Lugosi (2006). The definiions may be exended o η = by leing η end o. We hen find ha w becomes a uniform disribuion on he se of expers {k L 1,k = L 1 } ha have incurred smalles cumulaive loss before ime. Tha is, Hedge wih η = reduces o FollowheLeader, where in case of ies he weighs are disribued uniformly. The limiing value for he mix loss is m = L L 1. In our approximaion of he Hedge loss h by he mix loss m, we call he approximaion error δ = h m he mixabiliy gap. Bounding his quaniy is a sandard par of he analysis of Hedgeype algorihms (see, for example, Lemma 4 of CesaBianchi e al. 2007) and i also appears o be a fundamenal noion in sequenial predicion even when only socalled mixable losses are considered (Grünwald, 2011, 2012); see also Secion 6.2. We le = δ δ denoe he cumulaive mixabiliy gap, so ha he regre for Hedge may be decomposed as R = H L = M L +. (5) Here M L may be hough of as he regre under he mix loss and is he cumulaive approximaion error when approximaing he Hedge loss by he mix loss. Throughou he paper, our proof sraegy will be o analyse hese wo conribuions o he regre, M L and, separaely. The following lemma, which is proved in Appendix A, collecs a few basic properies of he mix loss: Lemma 1 (Mix Loss wih Consan Learning Rae) For any learning rae η (0, ] 1. l m h l +, so ha 0 δ s. 1 η (w 2. Cumulaive mix loss elescopes: M = ln 1 e ηl) for η <, L for η =. 3. Cumulaive mix loss approximaes he loss of he bes exper: L M L + ln K η. 4. The cumulaive mix loss M is nonincreasing in η. In order o obain a bound for Hedge, one can use he following wellknown bound on he mixabiliy gap, which is obained using Hoeffding s bound on he cumulan generaing funcion (CesaBianchi and Lugosi, 2006, Lemma A.1): δ η 8 s2, (6) 1286
7 Follow he Leader If You Can, Hedge If You Mus from which S 2 T η/8, where (as in he inroducion) S = max{s 1,..., s } is he maximum loss range in he firs rounds. Togeher wih he bound M L ln(k)/η from mix loss propery #3 his leads o R = (M L ) + ln K η + ηs2 T 8. (7) The bound is opimized for η = 8 ln(k)/(s 2 T ), which equalizes he wo erms. This leads o a bound on he regre of S T ln(k)/2, maching he lower bound on worscase regre from he exbook by CesaBianchi and Lugosi (2006, Secion 3.7). We can use his uned learning rae if he ime horizon T is known in advance. To deal wih he siuaion where T is unknown, eiher he doubling rick or a imevarying learning rae (see Lemma 2 below) can be used, a he cos of a worse consan facor in he leading erm of he regre bound. In he remainder of his secion, we inroduce a compleely parameerless algorihm called AdaHedge. We hen refine he seps of he analysis above o obain a beer regre bound. 2.2 AdaHedge Analysis In he previous secion, we spli he regre for Hedge ino wo pars: M L and, and we obained a bound for boh. The learning rae η was hen uned o equalise hese wo bounds. The main disincion beween AdaHedge and oher Hedge approaches is ha AdaHedge does no consider an upper bound on in order o obain his balance: insead i aims o equalize and ln(k)/η. As he cumulaive mixabiliy gap is nondecreasing in (by mix loss propery #1) and can be observed online, i is possible o adap he learning rae direcly based on. Perhaps he easies way o achieve his is by using he doubling rick: each subsequen block uses half he learning rae of he previous block, and a new block is sared as soon as he observed cumulaive mixabiliy gap exceeds he bound on he mix loss ln(k)/η, which ensures hese wo quaniies are equal a he end of each block. This is he approach aken in an earlier version of AdaHedge (Van Erven e al., 2011). However, we can achieve he same goal much more eleganly, by decreasing he learning rae wih ime according o η ah = ln K ah 1 (8) (where ah 0 = 0, so ha ηah 1 = ). Noe ha he AdaHedge learning rae does no involve he end ime T or any oher unobserved properies of he daa; all subsequen analysis is herefore valid for all T simulaneously. The definiions (3) and (4) of he weighs and he mix loss are modified o use his new learning rae: w,k ah = wah 1,k e ηah L 1,k w1 ah e ηah L 1 and m ah = 1 η ah ln(w ah e ηah l ), (9) wih w ah 1 = (1/K,..., 1/K) uniform. Noe ha he muliplicaive updae rule for he weighs no longer applies when he learning rae varies wih ; he las hree resuls of Lemma 1 are also no longer valid. Laer we will also consider oher algorihms o deermine 1287
8 De Rooij, Van Erven, Grünwald and Koolen Single round quaniies for rial : l Loss vecor l = min k l,k, l + = max k l,k Min and max loss s = l + l Loss range w alg h alg = e ηalg L 1 / k e ηalg L 1,k Weighs played = w alg l Hedge loss m alg = 1 η alg δ alg v alg = h alg ( ln m alg = Var k w alg w alg e ηalg l ) Mix loss Mixabiliy gap [l,k ] Loss variance Aggregae quaniies afer rounds: (The final ime T is omied from he subscrip where possible, e.g. L = L T ) L, L, L+, Halg, M alg, alg, V alg τ=1 of l τ, l τ, l + τ, h alg τ, m alg τ, δτ alg, vτ alg S = max{s 1,..., s } Maximum loss range L = min k L,k Cumulaive loss of he bes exper R alg = H alg L Regre Algorihms (he alg in he superscrip above): (η) Hedge wih fixed learning rae η ah AdaHedge, defined by (8) fl FollowheLeader (η fl = ) ff FlipFlop, defined by (16) Table 1: Noaion variable learning raes; o avoid confusion he considered algorihm is always specified in he superscrip in our noaion. See Table 1 for reference. From now on, AdaHedge will be defined as he Hedge algorihm wih learning rae defined by (8). For concreeness, a malab implemenaion appears in Figure 1. Our learning rae is similar o ha of CesaBianchi e al. (2007), bu i is less pessimisic as i is based on he mixabiliy gap iself raher han is bound, and as such may exploi easy sequences of losses more aggressively. Moreover our uning of he learning rae simplifies he analysis, leading o igher resuls; he essenial new echnical ingrediens appear as Lemmas 3, 5 and 7 below. We analyse he regre for AdaHedge like we did for a fixed learning rae in he previous secion: we again consider M ah L and ah separaely. This ime, boh legs of he analysis become slighly more involved. Luckily, a good bound can sill be obained wih only a small amoun of work. Firs we show ha he mix loss is bounded by he mix loss we would have incurred if we would have used he final learning rae ηt ah all along: Lemma 2 Le dec be any sraegy for choosing he learning rae such ha η 1 η 2... Then he cumulaive mix loss for dec does no exceed he cumulaive mix loss for he sraegy ha uses he las learning rae η T from he sar: M dec M (η T ). 1288
9 Follow he Leader If You Can, Hedge If You Mus % Reurns he losses of AdaHedge. % l(,k) is he loss of exper k a ime funcion h = adahedge(l) [T, K] = size(l); h = nan(t,1); L = zeros(1,k); Dela = 0; end for = 1:T ea = log(k)/dela; [w, Mprev] = mix(ea, L); h() = w * l(,:) ; L = L + l(,:); [~, M] = mix(ea, L); dela = max(0, h()(mmprev)); % max clips numeric Jensen violaion Dela = Dela + dela; end % Reurns he poserior weighs and mix loss % for learning rae ea and cumulaive loss % vecor L, avoiding numerical insabiliy. funcion [w, M] = mix(ea, L) mn = min(l); if (ea == Inf) % Limi behaviour: FTL w = L==mn; else w = exp(ea.* (Lmn)); end s = sum(w); w = w / s; M = mn  log(s/lengh(l))/ea; end Figure 1: Numerically robus malab implemenaion of AdaHedge This lemma was firs proved in is curren form by Kalnishkan and Vyugin (2005, Lemma 3), and an essenially equivalen bound was inroduced by Györfi and Oucsák (2007) in he proof of heir Lemma 1. Relaed echniques for dealing wih imevarying learning raes go back o Auer e al. (2002). Proof Using mix loss propery #4, we have M dec T = T =1 which was o be shown. m dec = T =1 ( M (η) M (η) ) 1 T =1 ( M (η) M (η ) 1) 1 = M (η T ) T, We can now show ha he wo conribuions o he regre are sill balanced. Lemma 3 The AdaHedge regre is R ah = M ah L + ah 2 ah. Proof ah As δ ah 0 for all (by mix loss propery #1), he cumulaive mixabiliy gap is nondecreasing. Consequenly, he AdaHedge learning rae η ah as defined in (8) is nonincreasing in. Thus Lemma 2 applies o M ah ; ogeher wih mix loss propery #3 and (8) his yields M ah M (ηah T ) L + ln K η ah T = L + ah T 1 L + ah T. Subsiuion ino he rivial decomposiion R ah = M ah L + ah yields he resul. The remaining ask is o esablish a bound on ah. As before, we sar wih a bound on he mixabiliy gap in a single round, bu raher han (6), we use Bernsein s bound on he mixabiliy gap in a single round o obain a resul ha is expressed in erms of he variance of he losses, v ah = Var k w ah [l,k ] = k wah,k (l,k h ah )
10 De Rooij, Van Erven, Grünwald and Koolen Lemma 4 (Bernsein s Bound) Le η = η alg (0, ) denoe he finie learning rae chosen for round by any algorihm alg. The mixabiliy gap δ alg saisfies Furher, v alg δ alg g(s η ) s v alg (l + halg )(h alg l ) s2 /4., where g(x) = ex x 1. (10) x Proof This is Bernsein s bound (CesaBianchi and Lugosi, 2006, Lemma A.5) on he cumulan generaing funcion, applied o he random variable (l,k l )/s [0, 1] wih k disribued according o w alg. Bernsein s bound is more sophisicaed han Hoeffding s bound (6), because i expresses ha he mixabiliy gap δ is small no only when η is small, bu also when all expers have approximaely he same loss, or when he weighs w are concenraed on a single exper. The nex sep is o use Bernsein s inequaliy o obain a bound on he cumulaive mixabiliy gap ah. In he analysis of CesaBianchi e al. (2007) his is achieved by firs applying Bernsein s bound for each individual round, and hen using a elescoping argumen o obain a bound on he sum. Wih our learning rae (8) i is convenien o reverse hese seps: we firs elescope, which can now be done wih equaliy, and subsequenly apply Bernsein s inequaliy in a sricer way. Lemma 5 AdaHedge s cumulaive mixabiliy gap saisfies ( ah ) 2 V ah ln K + ( 2 3 ln K + 1)S ah. Proof In his proof we will omi he superscrip ah. Using he definiion of he learning rae (8) and δ s (from mix loss propery #1), we ge 2 = T =1 = ( ) = ( ) ln K 2δ + δ 2 η ( ) ( 1 + δ ) = ( ) ln K 2δ + s δ 2 ln K η ( ) 2δ 1 + δ 2 δ η + S. (11) The inequaliies in his equaion replace a δ erm by S, which is of no concern: he resuling erm S adds a mos 2S o he regre bound. We will now show δ η 1 2 v s δ. (12) This supersedes he bound δ /η (e 2)v for η s 1 used by CesaBianchi e al. (2007). Even hough a firs sigh circular, he form (12) has wo major advanages. Firs, inclusion of he overhead 1 3 s δ will only affec smaller order erms of he regre, bu admis a reducion of he leading consan o he opimal facor 1 2. This gain direcly percolaes o our regre bounds below. Second, (12) holds for unbounded η, which simplifies uning considerably. 1290
11 Follow he Leader If You Can, Hedge If You Mus Firs noe ha (12) is clearly valid if η =. Assuming ha η is finie, we can obain his resul by rewriing Bernsein s bound (10) as follows: 1 2 v s δ 2g(s η ) = δ s f(s η )δ, where f(x) = ex 1 2 x2 x 1 η xe x x 2 x. Remains o show ha f(x) 1/3 for all x 0. Afer rearranging, we find his o be he case if (3 x)e x 1 2 x2 + 2x + 3. Taylor expansion of he lefhand side around zero reveals ha (3 x)e x = 1 2 x2 + 2x x3 ue u for some 0 u x, from which he resul follows. The proof is compleed by plugging (12) ino (11) and finally relaxing s S. Combinaion of hese resuls yields he following naural regre bound, analogous o Theorem 5 of CesaBianchi e al. (2007). Theorem 6 AdaHedge s regre is bounded by Proof Lemma 5 is of he form R ah 2 V ah ln K + S( 4 3 ln K + 2). wih a and b nonnegaive numbers. Solving for ah hen gives which by Lemma 3 implies ha ( ah ) 2 a + b ah, (13) ah 1 2 b b 2 + 4a 1 2 b ( b 2 + 4a) = a + b, R ah 2 a + 2b. Plugging in he values a = V ah ln K and b = S( 2 3 ln K + 1) from Lemma 5 complees he proof. This firs regre bound for AdaHedge is difficul o inerpre, because he cumulaive loss variance V ah depends on he acions of he AdaHedge sraegy iself (hrough he weighs w ah ). Below, we will derive a regre bound for AdaHedge ha depends only on he daa. However, AdaHedge has one imporan propery ha is capured by his firs resul ha is no longer expressed by he worscase bound we will derive below. Namely, if he daa are easy in he sense ha here is a clear bes exper, say k, hen he weighs played 1 as increases, hen he loss variance mus decrease: v ah 0. Thus, Theorem 6 suggess ha he AdaHedge regre may be bounded if he weighs concenrae on he bes exper sufficienly quickly. This indeed urns ou o be he case: we can prove ha he regre is bounded for he sochasic seing where he loss vecors l are independen, and E[L,k L,k ] = Ω( β ) for all k k and any β > 1/2. This is an imporan feaure of AdaHedge when i is used as a sandalone algorihm, and Van Erven e al. (2011) provide a proof for he previous version of he by AdaHedge will concenrae on ha exper. If w ah,k 1291
12 De Rooij, Van Erven, Grünwald and Koolen sraegy. See Secion 5.4 for an example of concenraion of he AdaHedge weighs. Here we will no pursue his furher, because he FollowheLeader sraegy also incurs bounded loss in ha case; we raher focus aenion on how o successfully compee wih FTL in Secion 3. We now proceed o derive a bound ha depends only on he daa, using an approach similar o he one aken by CesaBianchi e al. (2007). We firs bound he cumulaive loss variance as follows: Lemma 7 Assume L H. The cumulaive loss variance for AdaHedge saisfies V ah S (L+ L )(L L ) L + L + 2S. In he degenerae case L = L + he fracion reads 0/0, bu since we hen have V ah = 0, from here on we define he raio o be zero in ha case, which is also is limiing value. Proof We omi all ah superscrips. By Lemma 4 we have v (l + h )(h l ). Now T V = v (l + h )(h l ) S =1 1 T = ST (l + h )(h l ) s (l + h )(h l ) (l + h ) + (h l ) S (L+ H)(H L ), (14) L + L where he las inequaliy is an insance of Jensen s inequaliy applied o he funcion B defined on he domain x, y 0 by B(x, y) = xy x+y for xy > 0 and B(x, y) = 0 for xy = 0 o ensure coninuiy. To verify ha B is joinly concave, we will show ha he Hessian is negaive semidefinie on he inerior xy > 0. Concaviy on he whole domain hen follows from coninuiy. The Hessian, which urns ou o be he rank one marix 2 2 B(x, y) = (x + y) 3 ( ) ( ) y y, x x is negaive semidefinie since i is a negaive scaling of a posiive ouer produc. Subsequenly using H L (by assumpion) and H L + 2 (by Lemma 3) yields as desired. (L + H)(H L ) (L+ L )(L + 2 L ) (L+ L )(L L ) + 2 L + L L + L L + L This can be combined wih Lemmas 5 and 3 o obain our firs main resul: Theorem 8 (AdaHedge WorsCase Regre Bound) AdaHedge s regre is bounded by R ah 2 S (L+ L )(L L ) L + L ln K + S( 16 3 ln K + 2). (15) 1292
13 Follow he Leader If You Can, Hedge If You Mus Proof If H ah < L, hen R ah < 0 and he resul is clearly valid. Bu if H ah L, we can bound V ah using Lemma 7 and plug he resul ino Lemma 5 o ge an inequaliy of he form (13) wih a = S(L + L )(L L )/(L + L ) and b = S( 8 3 ln K + 1). Following he seps of he proof of Theorem 6 wih hese modified values for a and b we arrive a he desired resul. This bound has several useful properies: 1. I is always smaller han he CBMS bound (1), wih a leading consan ha has been reduced from he previously besknown value of 2.63 o 2. To see his, noe ha (15) increases o (1) if we replace L + by he upper bound L + ST. I can be subsanially sronger han (1) if he range of he losses s is highly variable. 2. The bound is fundamenal, a concep discussed in deail by CesaBianchi e al. (2007): i is invarian o ranslaions of he losses and proporional o heir scale. I is herefore valid for arbirary loss ranges, regardless of sign. In fac, no jus he bound, bu AdaHedge iself is fundamenal in his sense: see Secion 4 for a discussion and proof. 3. The regre is small when he bes exper eiher has a very low loss, or a very high loss. The laer is imporan if he algorihm is o be used for a scenario in which we are provided wih a sequence of gain vecors g raher han losses: we can ransform hese gains ino losses using l = g, and hen run AdaHedge. The bound hen implies ha we incur small regre if he bes exper has very small cumulaive gain relaive o he minimum gain. 4. The bound is no dependen on he number of rials bu only on he losses; i is a imeless bound as discussed below. 2.3 Wha are Timeless Bounds? All bounds presened for AdaHedge (and FlipFlop) are imeless. We call a regre bound imeless if i does no change under inserion of addiional rials where all expers are assigned he same loss. Inuiively, he predicion ask does no become more difficul if naure should inser sameloss rials. Since hese rials do nohing o differeniae beween he expers, hey can safely be ignored by he learner wihou affecing her regre; in fac, many Hedge sraegies, including Hedge wih a fixed learning rae, FTL, AdaHedge and CBMS already have he propery ha heir fuure behaviour does no change under such inserions: hey are robus agains such ime dilaion. If any sraegy does no have his propery by iself, i can easily be modified o ignore equalloss rials. I is easy o imagine pracical scenarios where his robusness propery would be imporan. For example, suppose you hire a number of expers who coninually monior he asses in your porfolio. Usually hey do no recommend any changes, bu occasionally, when hey see a rare opporuniy or receive suble warning signs, hey may urge you o rade, resuling in a poenially very large gain or loss. I seems only beneficial o poll he expers ofen, and here is no reason why he many resuling equalloss rials should complicae he learning ask. 1293
14 De Rooij, Van Erven, Grünwald and Koolen The oldes bounds for Hedge scale wih T or L, and are hus no imeless. From he resuls above we can obain fundamenal and imeless varians wih, for parameerless algorihms, he bes known leading consans (he firs iem below follows Corollary 1 of CesaBianchi e al. 2007): Corollary 9 The AdaHedge regre saisfies he following inequaliies: R ah T=1 s 2 ln K + S( 4 3 ln K + 2) (analogue of radiional T based bounds), R ah 2 S(L L ) ln K + S( 16 3 ln K + 2) (analogue of radiional L based bounds), R ah 2 S(L + L ) ln K + S( 16 3 ln K + 2) (symmeric bound, useful for gains). Proof We could ge a bound ha depends only on he loss ranges s by subsiuing he wors case L = (L + + L )/2 ino Theorem 8, bu a sharper resul is obained by plugging he inequaliy v s 2 /4 from Lemma 4 direcly ino Theorem 6. This yields he firs iem above. The oher wo inequaliies follow easily from Theorem 8. In he nex secion, we show how we can compee wih FTL while a he same ime mainaining all hese worscase guaranees up o a consan facor. 3. FlipFlop AdaHedge balances he cumulaive mixabiliy gap ah and he mix loss regre M ah L by reducing η ah as necessary. Bu, as we observed previously, if he daa are no hopelessly adversarial we migh no need o worry abou he mixabiliy gap: as Lemma 4 expresses, δ ah is also small if he variance v ah of he loss under he weighs w,k ah is small, which is he case if he weigh on he bes exper max k w,k ah becomes close o one. AdaHedge is able o exploi such a lucky scenario o an exen: as explained in he discussion ha follows Theorem 6, if he weigh of he bes exper goes o one quickly, AdaHedge will have a small cumulaive mixabiliy gap, and herefore, by Lemma 3, a small regre. This happens, for example, in he sochasic seing wih independen, idenically disribued losses, when a single exper has he smalles expeced loss. Similarly, in he experimen of Secion 5.4, he AdaHedge weighs concenrae sufficienly quickly for he regre o be bounded. There is he poenial for a nasy feedback loop, however. Suppose here are a small number of difficul early rials, during which he cumulaive mixabiliy gap increases relaively quickly. AdaHedge responds by reducing he learning rae (8), wih he effec ha he weighs on he expers become more uniform. As a consequence, he mixabiliy gap in fuure rials may be larger han wha i would have been if he learning rae had sayed high, leading o furher unnecessary reducions of he learning rae, and so on. The end resul may be ha AdaHedge behaves as if he daa are difficul and incurs subsanial regre, even in cases where he regre of Hedge wih a fixed high learning rae, or of FollowheLeader, is bounded! Precisely his phenomenon occurs in he experimen in Secion 5.2 below: AdaHedge s regre is close o he worscase bound, whereas FTL hardly incurs any regre a all. 1294
15 Follow he Leader If You Can, Hedge If You Mus I appears, hen, ha we mus eiher hope ha he daa are easy enough ha we can make he weighs concenrae quickly on a single exper, by no reducing he learning rae a all; or we fear he wors and reduce he learning rae as much as we need o be able o provide good guaranees. We canno really inerpolae beween hese wo exremes: an inermediae learning rae may no yield small regre in favourable cases and may a he same ime desroy any performance guaranees in he wors case. I is unclear a priori wheher we can ge away wih keeping he learning rae high, or ha i is wiser o play i safe using AdaHedge. The mos exreme case of keeping he learning rae high, is he limi as η ends o, for which Hedge reduces o FollowheLeader. In his secion we work ou a sraegy ha combines he advanages of FTL and AdaHedge: i reains AdaHedge s worscase guaranees up o a consan facor, bu is regre is also bounded by a consan imes he regre of FTL (Theorem 15). Perhaps surprisingly, his is no easy o achieve. To see why, imagine a scenario where he average loss of he bes exper is subsanial, whereas he regre of eiher FollowheLeader or AdaHedge, is small. Since our combinaion has o guaranee a similarly small regre, i has only a very limied margin for error. We canno, for example, simply combine he wo algorihms by recursively plugging hem ino Hedge wih a fixed learning rae, or ino AdaHedge: he performance guaranees we have for hose mehods of combinaion are oo weak. Even if boh FTL and AdaHedge yield small regre on he original problem, choosing he acions of FTL for some rounds and hose of AdaHedge for he oher rounds may fail if we do i naively, because he regre is no necessarily increasing, and we may end up picking each algorihm precisely in hose rounds where he oher one is beer. Luckily, alernaing beween he opimisic FTL sraegy and he worscaseproof Ada Hedge does urn ou o be possible if we do i in a careful way. In his secion we explain he appropriae sraegy, called FlipFlop (superscrip: ff ), and show ha i combines he desirable properies of boh FTL and AdaHedge. 3.1 Exploiing Easy Daa by Following he Leader We firs invesigae he poenial benefis of FTL over AdaHedge. Lemma 10 below idenifies he circumsances under which FTL will perform well, which is when he number of leader changes is small. I also shows ha he regre for FTL is equal o is cumulaive mixabiliy gap when FTL is inerpreed as a Hedge sraegy wih infinie learning rae. Lemma 10 Le c be an indicaor for a leader change a ime : define c = 1 if here exiss an exper k such ha L 1,k = L 1 while L,k L, and c = 0 oherwise. Le C = c c be he cumulaive number of leader changes. Then he FTL regre saisfies R fl = ( ) S C. Proof We have M ( ) = L by mix loss propery #3, and consequenly R fl = ( ) + M ( ) L = ( ). To bound ( ), noice ha, for any such ha c = 0, all leaders remained leaders and incurred idenical loss. I follows ha m ( ) = L L 1 = h( ) 1295 and hence δ ( ) = 0. By
16 De Rooij, Van Erven, Grünwald and Koolen bounding δ ( ) as required. S for all oher we obain ( ) = T =1 δ ( ) = : c =1 δ ( ) : c =1 S = S C, We see ha he regre for FTL is bounded by he number of leader changes. This quaniy is boh fundamenal and imeless. I is a naural measure of he difficuly of he problem, because i remains small whenever a single exper makes he bes predicions on average, even in he scenario described above, in which AdaHedge ges caugh in a feedback loop. One example where FTL ouperforms AdaHedge is when he losses for wo expers are (1, 0) on he firs round, and keep alernaing according o (1, 0), (0, 1), (1, 0),... for he remainder of he rounds. Then he FTL regre is only 1/2, whereas AdaHedge s performance is close o he worscase bound (because is weighs w ah converge o (1/2, 1/2), for which he bound (6) on he mixabiliy gap is igh). This scenario is illusraed furher in he experimens, Secion FlipFlop FlipFlop is a Hedge sraegy in he sense ha i uses exponenial weighs defined by (9), bu he learning rae η ff now alernaes beween infiniy, such ha he algorihm behaves like FTL, and he AdaHedge value, which decreases as a funcion of he mixabiliy gap accumulaed over he rounds where AdaHedge is used. In Definiion 11 below, we will specify he flip regime R, which is he subse of imes {1,..., } where we follow he leader by using an infinie learning rae, and he flop regime R = {1,..., } \ R, which is he se of imes where he learning rae is deermined by AdaHedge (mnemonic: he posiion of he bar refers o he value of he learning rae). We accumulae he mixabiliy gap, he mix loss and he variance for hese wo regimes separaely: = δ ff τ ; M = m ff τ ; (flip) τ R τ R = δ ff τ ; M = m ff τ ; V = vτ ff. (flop) τ R τ R τ R We also change he learning rae from is definiion for AdaHedge in (8) o he following, which differeniaes beween he wo regimes of he sraegy: η ff = { η flip if R, η flop if R, where η flip = η fl = and η flop = ln K. (16) 1 Like for AdaHedge, η flop = as long as 1 = 0, which now happens for all such ha R 1 =. Noe ha while he learning raes are defined separaely for he wo regimes, he exponenial weighs (9) of he expers are sill always deermined using he cumulaive losses L,k over all rounds. We also poin ou ha, for rounds R, he learning rae η ff = η flop is no equal o η ah, because i uses 1 insead of ah 1. For his reason, he 1296
17 Follow he Leader If You Can, Hedge If You Mus % Reurns he losses of FlipFlop % l(,k) is he loss of exper k a ime ; phi > 1 and alpha > 0 are parameers funcion h = flipflop(l, alpha, phi) [T, K] = size(l); h = nan(t,1); L = zeros(1,k); Dela = [0 0]; scale = [phi/alpha alpha]; regime = 1; % 1=FTL, 2=AH end for = 1:T if regime==1, ea = Inf; else ea = log(k)/dela(2); end [w, Mprev] = mix(ea, L); h() = w * l(,:) ; L = L + l(,:); [~, M] = mix(ea, L); dela = max(0, h()(mmprev)); Dela(regime) = Dela(regime) + dela; if Dela(regime) > scale(regime) * Dela(3regime) regime = 3regime; end end Figure 2: FlipFlop, wih new ingrediens in boldface FlipFlop regre may be eiher beer or worse han he AdaHedge regre; our resuls below only preserve he regre bound up o a consan facor. In conras, we do compee wih he acual regre of FTL. I remains o define he flip regime R and he flop regime R, which we will do by specifying he imes a which o swich from one o he oher. FlipFlop sars opimisically, wih an epoch of he flip regime, which means i follows he leader, unil becomes oo large compared o. A ha poin i swiches o an epoch of he flop regime, and keeps using η flop unil becomes oo large compared o. Then he process repeas wih he nex epochs of he flip and flop regimes. The regimes are deermined as follows: Definiion 11 (FlipFlop s Regimes) Le ϕ > 1 and α > 0 be parameers of he algorihm (uned below in Corollary 16). Then FlipFlop sars in he flip regime. If is he earlies ime since he sar of a flip epoch where > (ϕ/α), hen he ransiion o he subsequen flop epoch occurs beween rounds and + 1. (Recall ha during flip epochs increases in whereas is consan.) Vice versa, if is he earlies ime since he sar of a flop epoch where > α, hen he ransiion o he subsequen flip epoch occurs beween rounds and + 1. This complees he definiion of he FlipFlop sraegy. See Figure 2 for a malab implemenaion. The analysis proceeds much like he analysis for AdaHedge. We firs show ha, analogously o Lemma 3, he FlipFlop regre can be bounded in erms of he cumulaive mixabiliy gap; in fac, we can use he smalles cumulaive mixabiliy gap ha we encounered 1297
18 De Rooij, Van Erven, Grünwald and Koolen in eiher of he wo regimes, a he cos of slighly increased consan facors. This is he fundamenal building block in our FlipFlop analysis. We hen proceed o develop analogues of Lemmas 5 and 7, whose proofs do no have o be changed much o apply o FlipFlop. Finally, all hese resuls are combined o bound he regre of FlipFlop in Theorem 15, which, afer Theorem 8, is he second main resul of his paper. Lemma 12 (FlipFlop version of Lemma 3) The following wo bounds hold simulaneously for he regre of he FlipFlop sraegy wih parameers ϕ > 1 and α > 0: ( ) ( ) ϕα ϕ R ff ϕ 1 + 2α S ϕ ; (17) ( ϕ R ff ϕ 1 + ϕ ) α S. (18) Proof The regre can be decomposed as R ff = H ff L = + + M + M L. (19) Our firs sep will be o bound he mix loss M + M in erms of he mix loss M flop of he auxiliary sraegy ha uses η flop for all. As η flop is nonincreasing, we can hen apply Lemma 2 and mix loss propery #3 o furher bound M flop M (ηflop T ) L + ln K η flop = L + T 1 L +. (20) Le 0 = u 1 < u 2 <... < u b < T denoe he imes jus before he epochs of he flip regime begin, i.e. round u i + 1 is he firs round in he ih flip epoch. Similarly le 0 < v 1 <... < v b T denoe he imes jus before he epochs of he flop regime begin, where we arificially define v b = T if he algorihm is in he flip regime afer T rounds. These definiions ensure ha we always have u b < v b T. For he mix loss in he flop regime we have M = (M flop u 2 Mv flop 1 ) + (Mu flop 3 Mv flop 2 ) (Mu flop b Mv flop b 1 ) + (M flop Mv flop b ). (21) Le us emporarily wrie η = η flop o avoid double superscrips. For he flip regime, he properies in Lemma 1, ogeher wih he observaion ha η flop does no change during he flip regime, give M = = b i=1 b ( ) M v ( ) i M u ( ) i = ( M (ηv i ) v i M (ηv i ) u i i=1 ( ) Mv flop 1 Mu flop 1 + b i=1 ( M ( ) v i L u i ) + ln K ) b = η vi i=1 ( ) Mv flop 2 Mu flop b i=1 ( M (ηv i ) v i L u i ) ( Mv flop i Mu flop i + ln K η ui +1 ) ( ) Mv flop b Mu flop b + b ui. (22) i=1 From he definiion of he regime changes (Definiion 11), we know he value of ui very accuraely a he ime u i of a change from a flop o a flip regime: ui > α ui = α vi 1 > ϕ vi 1 = ϕ ui
19 Follow he Leader If You Can, Hedge If You Mus By unrolling from low o high i, we see ha b b ui ϕ 1 i ub ϕ 1 i ub = i=1 i=1 i=1 ϕ ϕ 1 u b. Adding up (21) and (22), we herefore find ha he oal mix loss is bounded by b M + M M flop + ui M flop + ϕ ( ) ϕ ϕ 1 u b L + ϕ 1 + 1, i=1 where he las inequaliy uses (20). Combinaion wih (19) yields R ff ( ϕ ϕ ) +. (23) Our nex goal is o relae and : by consrucion of he regimes, hey are always wihin a consan facor of each oher. Firs, suppose ha afer T rials we are in he bh epoch of he flip regime, ha is, we will behave like FTL in round T + 1. In his sae, we know from Definiion 11 ha is suck a he value ub ha promped he sar of he curren epoch. As he regime change happened afer u b, we have ub S α ub, so ha S α. A he same ime, we know ha is no large enough o rigger he nex regime change. From his we can deduce he following bounds: 1 α ( S) ϕ α. On he oher hand, if afer T rounds we are in he bh epoch of he flop regime, hen a similar reasoning yields In boh cases, i follows ha α ( S) α. ϕ < α + S; < ϕ α + S. The wo bounds of he lemma are obained by plugging firs one, hen he oher of hese bounds ino (23). The flop cumulaive mixabiliy gap is relaed, as before, o he variance of he losses. Lemma 13 (FlipFlop version of Lemma 5) The cumulaive mixabiliy gap for he flop regime is bounded by he cumulaive variance of he losses for he flop regime: 2 V ln K + ( 2 3 ln K + 1)S. (24) 1299
20 De Rooij, Van Erven, Grünwald and Koolen Proof The proof is analogous o he proof of Lemma 5, wih insead of ah, V insead of V ah, and using η = η flop = ln(k)/ 1 insead of η = η ah = ln(k)/ ah 1. Furhermore, we only need o sum over he rounds R in he flop regime, because does no change during he flip regime. As i is sraighforward o prove an analogue of Theorem 6 for FlipFlop by solving he quadraic inequaliy in (24), we proceed direcly owards esablishing an analogue of Theorem 8. The following lemma provides he equivalen of Lemma 7 for FlipFlop. I can probably be srenghened o improve he lower order erms; we provide he version ha is easies o prove. Lemma 14 (FlipFlop version of Lemma 7) Suppose H ff L. variance for FlipFlop wih parameers ϕ > 1 and α > 0 saisfies V S (L+ L )(L ( L ) ϕ + L + L ϕ 1 + ϕ ) α + 2 S + S 2. Proof The sum of variances saisfies V = R v ff T =1 v ff S (L+ H ff )(H ff L ) L + L, The cumulaive loss where he firs inequaliy simply includes he variances for FTL rounds (which are ofen all zero), and he second follows from he same reasoning as employed in (14). Subsequenly using L H ff (by assumpion) and, from Lemma 12, H ff L + γ, where γ denoes he righhand side of he bound (18), we find which was o be shown. V S (L+ L )(L + γ L ) S (L+ L )(L L ) + Sγ, L + L L + L Combining Lemmas 12, 13 and 14, we obain our second main resul: Theorem 15 (FlipFlop Regre Bound) The regre for FlipFlop wih doubling parameers ϕ > 1 and α > 0 simulaneously saisfies he wo bounds R ff where c 1 = R ff c 1 ( ϕα ϕ 1 + 2α + 1 ) R fl + S S (L+ L )(L L ) L + L ϕ ϕ 1 + ϕ α + 2. ( ϕ ϕ ), ( ln K + c 1 S (c ) ln K + ) ln K S, This shows ha, up o a muliplicaive facor in he regre, FlipFlop is always as good as he bes of FollowheLeader and AdaHedge s bound from Theorem 8. Of course, if 1300
I M F S T A F F D I S C U S S I O N N O T E
I M F S T A F F D I S C U S S I O N N O T E February 29, 2012 SDN/12/01 Two Targes, Two Insrumens: Moneary and Exchange Rae Policies in Emerging Marke Economies Jonahan D. Osry, Aish R. Ghosh, and Marcos
More informationImproved Techniques for Grid Mapping with RaoBlackwellized Particle Filters
1 Improved Techniques for Grid Mapping wih RaoBlackwellized Paricle Filers Giorgio Grisei Cyrill Sachniss Wolfram Burgard Universiy of Freiburg, Dep. of Compuer Science, GeorgesKöhlerAllee 79, D79110
More informationWhen Should Public Debt Be Reduced?
I M F S T A F F D I S C U S S I ON N O T E When Should Public Deb Be Reduced? Jonahan D. Osry, Aish R. Ghosh, and Raphael Espinoza June 2015 SDN/15/10 When Should Public Deb Be Reduced? Prepared by Jonahan
More informationBoard of Governors of the Federal Reserve System. International Finance Discussion Papers. Number 1003. July 2010
Board of Governors of he Federal Reserve Sysem Inernaional Finance Discussion Papers Number 3 July 2 Is There a Fiscal Free Lunch in a Liquidiy Trap? Chrisopher J. Erceg and Jesper Lindé NOTE: Inernaional
More informationKONSTANTĪNS BEŅKOVSKIS IS THERE A BANK LENDING CHANNEL OF MONETARY POLICY IN LATVIA? EVIDENCE FROM BANK LEVEL DATA
ISBN 9984 676 20 X KONSTANTĪNS BEŅKOVSKIS IS THERE A BANK LENDING CHANNEL OF MONETARY POLICY IN LATVIA? EVIDENCE FROM BANK LEVEL DATA 2008 WORKING PAPER Lavias Banka, 2008 This source is o be indicaed
More informationThe concept of potential output plays a
Wha Do We Know (And No Know) Abou Poenial Oupu? Susano Basu and John G. Fernald Poenial oupu is an imporan concep in economics. Policymakers ofen use a onesecor neoclassical model o hink abou longrun
More informationDoes Britain or the United States Have the Right Gasoline Tax?
Does Briain or he Unied Saes Have he Righ Gasoline Tax? Ian W.H. Parry and Kenneh A. Small March 2002 (rev. Sep. 2004) Discussion Paper 02 12 rev. Resources for he uure 1616 P Sree, NW Washingon, D.C.
More informationThe Macroeconomics of MediumTerm Aid ScalingUp Scenarios
WP//6 The Macroeconomics of MediumTerm Aid ScalingUp Scenarios Andrew Berg, Jan Goschalk, Rafael Porillo, and LuisFelipe Zanna 2 Inernaional Moneary Fund WP//6 IMF Working Paper Research Deparmen The
More informationExchange Rate PassThrough into Import Prices: A Macro or Micro Phenomenon? Abstract
Exchange Rae PassThrough ino Impor Prices: A Macro or Micro Phenomenon? Absrac Exchange rae regime opimaliy, as well as moneary policy effeciveness, depends on he ighness of he link beween exchange rae
More informationOUTOFBAG ESTIMATION. Leo Breiman* Statistics Department University of California Berkeley, CA. 94708 leo@stat.berkeley.edu
1 OUTOFBAG ESTIMATION Leo Breiman* Saisics Deparmen Universiy of California Berkeley, CA. 94708 leo@sa.berkeley.edu Absrac In bagging, predicors are consruced using boosrap samples from he raining se
More informationFIRST PASSAGE TIMES OF A JUMP DIFFUSION PROCESS
Adv. Appl. Prob. 35, 54 531 23 Prined in Norhern Ireland Applied Probabiliy Trus 23 FIRST PASSAGE TIMES OF A JUMP DIFFUSION PROCESS S. G. KOU, Columbia Universiy HUI WANG, Brown Universiy Absrac This paper
More informationWhy Have Economic Reforms in Mexico Not Generated Growth?*
Federal Reserve Bank of Minneapolis Research Deparmen Saff Repor 453 November 2010 Why Have Economic Reforms in Mexico No Generaed Growh?* Timohy J. Kehoe Universiy of Minnesoa, Federal Reserve Bank of
More informationWhich Archimedean Copula is the right one?
Which Archimedean is he righ one? CPA Mario R. Melchiori Universidad Nacional del Lioral Sana Fe  Argenina Third Version Sepember 2003 Published in he YieldCurve.com ejournal (www.yieldcurve.com), Ocober
More informationThe Simple Analytics of Helicopter Money: Why It Works Always
Vol. 8, 201428 Augus 21, 2014 hp://dx.doi.org/10.5018/economicsejournal.ja.201428 The Simple Analyics of Helicoper Money: Why I Works Always Willem H. Buier Absrac The auhor proides a rigorous analysis
More informationMining the Most Interesting Rules
Appears in Pro. of he Fifh ACM SIGKDD In l Conf. on Knowledge Disovery and Daa Mining, 145154, 1999. Mining he Mos Ineresing Rules Robero J. Bayardo Jr. IBM Almaden Researh Cener hp://www.almaden.ibm.om/s/people/bayardo/
More informationISSN 15183548. Working Paper Series
ISSN 583548 Working Paper Series Nonlinear Mechanisms of he Exchange Rae PassThrough: A Phillips curve model wih hreshold for Brazil Arnildo da Silva Correa and André Minella November, 006 ISSN 583548
More informationThE Papers 07/02. Do sunk exporting costs differ among markets? Evidence from Spanish manufacturing firms.
ThE Papers 07/02 Deparameno de Teoría e Hisoria Económica Universidad de Granada Do sunk exporing coss differ among markes? Evidence from Spanish manufacuring firms. Blanes Crisóbal, José Vicene. Universidad
More informationA DecisionTheoretic Generalization of OnLine Learning and an Application to Boosting*
journal of compuer and sysem scences 55, 119139 (1997) arcle no. SS971504 A Decsonheorec Generalzaon of OnLne Learnng and an Applcaon o Boosng* Yoav Freund and Rober E. Schapre  A6 Labs, 180 Park Avenue,
More informationHow to Use Expert Advice
NICOLÒ CESABIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, California
More informationUsing Kalman Filter to Extract and Test for Common Stochastic Trends 1
Usig Kalma Filer o Exrac ad Tes for Commo Sochasic Treds Yoosoo Chag 2, Bibo Jiag 3 ad Joo Y. Park 4 Absrac This paper cosiders a sae space model wih iegraed lae variables. The model provides a effecive
More informationThe effect of the increase in the monetary base on Japan s economy at zero interest rates: an empirical analysis 1
The effec of he icrease i he moeary base o Japa s ecoomy a zero ieres raes: a empirical aalysis 1 Takeshi Kimura, Hiroshi Kobayashi, Ju Muraaga ad Hiroshi Ugai, 2 Bak of Japa Absrac I his paper, we quaify
More informationAdvanced Fixed Income Callable Bonds Professor Anh Le
1 What are callable bonds? When you take out a fixed rate mortgage to buy a house, you usually have the option of pre paying the mortgage. The common term to use is to refinance. And people would refinance
More informationCompeting in the Dark: An Efficient Algorithm for Bandit Linear Optimization
Competing in the Dark: An Efficient Algorithm for Bandit Linear Optimization Jacob Abernethy Computer Science Division UC Berkeley jake@cs.berkeley.edu Elad Hazan IBM Almaden hazan@us.ibm.com Alexander
More informationVIPer12ADIP VIPer12AS
VIPer12ADIP VIPer12AS LOW POWER OFF LINE SMPS PRIMARY SWITCHER TYPICAL POWER CAPABILITY Mains ype SO8 DIP8 European (195265 Vac) 8 W 13 W US / Wide range (85265 Vac) 5 W 8 W n FIXED 60 KHZ SWITCHING
More informationTwo faces of active learning
Two faces of active learning Sanjoy Dasgupta dasgupta@cs.ucsd.edu Abstract An active learner has a collection of data points, each with a label that is initially hidden but can be obtained at some cost.
More informationA New Approach to Linear Filtering and Prediction Problems 1
R. E. KALMAN Research Insue for Advanced Sudy, Balmore, Md. A New Approach o Lnear Flerng and Predcon Problems The classcal flerng and predcon problem s reexamned usng he Bode Shannon represenaon of
More informationHow Boosting the Margin Can Also Boost Classifier Complexity
Lev Reyzin lev.reyzin@yale.edu Yale University, Department of Computer Science, 51 Prospect Street, New Haven, CT 652, USA Robert E. Schapire schapire@cs.princeton.edu Princeton University, Department
More informationON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION. A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT
ON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT A numerical study of the distribution of spacings between zeros
More informationRobust Deanonymization of Large Sparse Datasets
Robust Deanonymization of Large Sparse Datasets Arvind Narayanan and Vitaly Shmatikov The University of Texas at Austin Abstract We present a new class of statistical deanonymization attacks against highdimensional
More informationSubspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity
Subspace Pursuit for Compressive Sensing: Closing the Gap Between Performance and Complexity Wei Dai and Olgica Milenkovic Department of Electrical and Computer Engineering University of Illinois at UrbanaChampaign
More information