Learning Eigenvectors for Free

Transcription

1 Learning Eigenvecors for Free Wouer M Koolen Royal Holloway and CWI wouer@csrhulacuk Wojek Kołowski Cenrum Wiskunde & Informaica kolowsk@cwinl Manfred K Warmuh UC Sana Cruz manfred@cseucscedu Absrac We exend he classical problem of predicing a sequence of oucomes from a finie alphabe o he marix domain In his exension, he alphabe of n oucomes is replaced by he se of all dyads, ie ouer producs uu > where u is a vecor in R n of uni lengh Whereas in he classical case he goal is o learn (ie sequenially predic as well as) he bes mulinomial disribuion, in he marix case we desire o learn he densiy marix ha bes explains he observed sequence of dyads We show how popular online algorihms for learning a mulinomial disribuion can be exended o learn densiy marices Inuiively, learning he n parameers of a densiy marix is much harder han learning he n parameers of a mulinomial disribuion Compleely surprisingly, we prove ha he wors-case regres of cerain classical algorihms and heir marix generalizaions are idenical he reason is ha he wors-case sequence of dyads share a common eigensysem, ie he wors case regre is achieved in he classical case So hese marix algorihms learn he eigenvecors wihou any regre 1 Inroducion We consider he exension of he classical online problem of predicing oucomes from a finie alphabe o he marix domain In his exension, he alphabe of n oucomes is replaced by a se of all dyads, ie ouer producs uu > where u is a uni vecor in R n Whereas classically he goal is o learn as well as he bes mulinomial disribuion over oucomes, in he marix case we desire o learn he disribuion over dyads ha bes explains he sequence of dyads seen so far A disribuion on dyads is summarized as a densiy marix, ie a symmeric posiive-definie 1 marix of uni race Such marices are heavily used in quanum physics, where dyads represen saes We will show how popular online algorihms for learning mulinomials can be exended o learn densiy marices Considerable aenion has been placed recenly on generalizing algorihms for learning and opimizaion problems from probabiliy vecor parameers o densiy marices [17, 19] Efficien semidefinie programming algorihms have been devised [1] and beer approximaion algorihms for NP-hard problems have been obained [] by employing on-line algorihms ha updae a densiy marix parameer Also wo imporan quanum complexiy classes were shown o collapse based on hese algorihms [8] Even hough he marix generalizaion led o progress in many conexs, in he original domain of on-line learning, he regre bounds proven for he algorihms in he marix case are ofen he same as hose provable for he original classical finie alphabe case [17, 19] herefore i was posed as an open problem o deermine wheher his is jus a case of loose classical bound or wheher here ruly exiss a free marix lunch for some of hese algorihms [18] Such algorihms essenially would learn he eigensysem of he daa for free wihou incurring any addiional regre his is non-inuiive, since one would expec a marix o have n parameers and be much harder o learn han an n dimensional parameer vecor 1 We use posiive in he non-sric sense, and omi symmeric and definie Our marices are real-valued 1

2 for rial =1,,, do Algorihm predics wih probabiliy vecor! 1 Naure responds wih oucome x Algorihm incurs loss log! 1,x end for Probabiliy vecor predicion for rial =1,,, do Algorihm predics wih densiy marix W 1 Naure responds wih densiy marix X Algorihm incurs loss r X log(w 1) end for Densiy marix predicion Figure 1: Proocols In his paper we invesigae his frivolously named bu deep free marix lunch quesion in arguably he simples conex: learning a mulinomial disribuion In he classical case, here are n oucomes and a disribuion is paramerized by an n-dimensional probabiliy vecor!, where! i is he probabiliy of oucome i One can view he base vecors e i as he elemenary evens and he probabiliy vecor as a mixure of hese evens:! = P i! ie i We define a marix generalizaion of a mulinomial which is paramerized by a densiy marix W (posiive marix of uni race) Now he elemenary evens are dyads of he form uu >, where u is a uni vecor in R n Dyads are he represenaions of saes used in quanum physics [0] A densiy marix is a mixure of dyads Whereas probabiliy vecors represen uncerainy over n basis vecors, densiy marices can be viewed as represening uncerainy over infiniely many dyads in R n In he classical case, he algorihm predics a rial wih mulinomial! 1 Naure produces an oucome x {1,,n}, and he algorihm incurs loss log(! 1,x ) he mos common heurisic (aka he Laplace esimaor) chooses! 1,i proporional o 1 plus he number of previous rials in which oucome i was observed he on-line algorihms are evaluaed by heir wors-case regre over daa sequences, where he regre is he addiional loss of he algorihm over he oal loss of he bes probabiliy vecor chosen in hindsigh In his paper we develop he corresponding marix seing, where he algorihm predics wih a densiy marix W 1, Naure produces a dyad x x >, and he algorihm incurs loss x > log(w 1 )x Here log denoes he marix logarihm We are paricularly ineresed in how he regre changes when he algorihms are generalized o he marix case Surprisingly we can show ha for he Laplace as well as he Krichevsky-rofimov [10] esimaors he wors-case regre is he same in he marix case as i is in he classical case For he Las-Sep Minimax algorihm [16], we can prove he same regre bound for he marix case ha was proven for he classical case Why are we doing his? Mos machine learning algorihms deal wih vecor parameers he goal of his line of research is o develop mehods for handling marix parameers We are used o dealing wih probabiliy vecors Recenly a probabiliy calculus was developed for densiy marices [0] including various Bayes rules for updaing generalized condiionals he vecor problems are ypically reained as special cases of he marix problems, where he eigensysem is fixed and only he vecors of eigenvalues has o be learned We exhibi for he firs ime a basic fundamenal problem, for which he regre achievable in he marix case is no higher han he regre achievable in he original vecor seing Paper ouline Definiions and noaion are given in he nex secion, followed by proofs of he free marix lunch for he hree discussed algorihms in Secion 3 A he core of our proofs is a new echnical lemma for mixing quanum enropies We also discuss he minimax algorihm for mulinomials due o Sharkov, and corresponding minimax algorihm for densiy marices We provide srong experimenal evidence ha he free marix lunch holds for his algorihm as well o pu he resuls ino conex, we moivae and discuss our choice of he loss funcion, and compare i o several alernaives in Secion 4 More discussion and perspecive is provided in he Secion 5 Seup he proocol for he classical probabiliy vecor predicion problem and he new densiy marix predicion problem are displayed side-by-side in Figure 1 We explain he laer problem Learning proceeds in rials During rial he algorihm predics wih a densiy marix W 1 We use index 1 o indicae ha is based on he 1 previous oucomes hen naure responds wih an oucome

3 densiy marix X he discrepancy beween predicion and oucome is measured by he marix enropic loss `(W 1, X ) := r X log(w 1 ), (1) where log denoes marix logarihm When he oucome densiy marix X is a dyad x x >, hen his loss becomes x > log(w 1 )x, which is he simplified form of he enropic loss discussed in he inroducion Also if he predicion densiy marix is diagonal, ie i has he form W 1 = P i! 1,i e i e > i for some probabiliy vecor! 1, and he oucome X is an eigendyad e j e > j of he same eigensysem, hen his loss simplifies o he classical log loss: `(W 1, X )= log(! 1,j ) he above definiion is no he only way o promoe he log loss o he marix domain Ye, in Secion 4 we jusify his choice We aim o design algorihms wih low regre compared o he bes fixed densiy marix in hindsigh he loss of he bes fixed densiy marix can be expressed succincly in erms of he von Neumann enropy, which is defined for any densiy marix D as H(D) := r(d log D), and he sufficien saisic S = P =1 X P as follows: inf W =1 `(W, X )=H S For fixed daa X 1,,X, he regre of a sraegy ha issues predicion W afer observing X 1,,X is S `(W 1, X ) H, () =1 and he wors-case regre on rials is obained by aking sup X1,,X over () Our aim is o design sraegies for densiy marix predicion ha have low wors-case regre 3 Free Marix Lunches In his secion, we will show how four popular online algorihms for learning mulinomials can be exended o learning densiy marices We sar wih he simple Laplace esimaor, coninue wih is improved version known as he Krichevsky-rofimov esimaor, and also exend he less known Las Sep Minimax sraegy which has even less regre We will prove a version of he free marix lunch (FML) for all hree algorihms Finally we discuss he minimax algorihm for which we have experimenal evidence ha he free marix lunch holds as well 31 Laplace Afer observing classical daa wih sufficien saisic vecor = P q=1 e x q, classical Laplace predics wih he probabiliy vecor! := +n +1 consising of he normalized smoohed couns By analogy, afer observing marix daa wih sufficien saisic S = P q=1 X, marix Laplace predics wih he correspondingly smoohed marix W := S+I +n Classical Laplace is commonly moivaed as eiher he Bayes predicive disribuion wr he uniform prior or as a loss minimizaion wih virual oucomes [3] he laer moivaion can be lifed o he marix domain by adding n virual oucomes a I/n: ( ) X W = argmin n`(w, I/n)+ `(W, X q ) = S + I W dens ma + n (3) he wors-case regre of classical Laplace afer ieraions equals log +n 1 n 1 apple (n 1) log( +1) (see eg [6]) We now show ha in he marix case, no addiional regre is incurred heorem 1 (Laplace FML) he wors-case regres of classical and marix Laplace coincide Proof Le W denoe he bes densiy marix for he firs oucomes he regre () of marix Laplace can be bounded as follows: `(W 1, X ) =1 `(W, X ) apple =1 q=1 =1 `(W 1, X ) `(W, X ) (4) For any posiive marix wih eigendecomposiion A = P i i aia> i, log(a) := P i log( i) aia> i 3

4 Now consider each erm in he righ-hand sum separaely he h erm equals r X log S 1 + I log S 1+n = log r X log(s 1 + I) log S 1+n Noe ha he firs erm consiues he classical par of he per-round regre, while he second erm is he marix par he marix par is non-posiive since S 1 + I S, and he logarihm is a marix monoone operaion (ie A B implies log A log B) By omiing i, we obain an upper bound on he regre of marix Laplace, ha is igh: for any sequence of idenical dyads he marix par is zero and (4) holds wih equaliy since W = W for all apple he same upper bound is also me by classical Laplace on any sequence of idenical oucomes [6] We jus showed ha marix Laplace has he same wors-case regre as classical Laplace, albei marix Laplace learns a marix of n parameers whereas classical Laplace only learns n probabiliies No addiional regre is incurred for learning he eigenvecors Marix Laplace can updae W in O(n ) ime per rial he same will be rue for our nex algorihm 3 Krichevsky-rofimov (K) Classical and marix K smooh by adding 1 o each coun, ie! := +1/ +n/ and W := S+I/ +n/ he former can again be obained as he Bayes predicive disribuion wr Jeffreys prior, he laer as he soluion o he marix enropic loss minimizaion problem (3) wih n/ virual oucomes insead of n for Laplace he leading erm in he wors-case regre for classical K is he opimal 1 log( ) rae per parameer insead of he log( ) rae for Laplace More precisely, classical K s wors-case regre afer ( +n/) (1/) ieraions is known o be log ( +1/) + log (n/) apple n 1 log( + 1) + log( ) (see eg [6]) Again we show ha no addiional regre is incurred in he marix case heorem (K FML) he wors-case regres of classical and marix K coincide he proof uses he following key enropy decomposiion lemma (proven in Appendix A): Lemma 1 For posiive marices A, B wih A = P i i a i a > i he eigendecomposiion of A: a > i H(A + B) Ba i r(b) H A +r(b) a ia > i, =1 Proof of heorem We sar by elescoping he regre () of marix K as follows S 1 + X S 1 r X log(w 1 ) H +( 1)H (5) 1 We bound each erm separaely Le us denoe he eigendecomposiion of S 1 by S 1 = P n i s i s > i Noice ha since W 1 plays in he eigensysem of S 1, we have: X n r X log(w 1 ) = r X log(! 1,i ) s i s > i = Moreover, i follows from Lemma 1 ha: S 1 + X H n X s > S i X s i H s > i X s i log(! 1,i ) 1 + s i s > i aking his equaliy and inequaliy ino accoun, he h erm in (5) is bounded above by: := s > S 1 + s i s > i S 1 i X s i log(! 1,i ) H +( 1)H, (6) 1 which, in urn, is a mos: apple sup i S log(! 1,i ) H s i s > i +( 1)H S 1 1

5 In oher words he per-round regre increase is larges for one of he eigenvecors of he sufficien saisic S 1, ie for classical daa o ge an upper bound, maximize over S 0,,S 1 independenly, each wih he consrain ha r(s )= A paricular maximizer is S = e 1 e > 1, which is he sufficien saisic of he sequence of oucomes all equal o X = e 1 e > 1 For his sequence all bounding seps hold wih equaliy Hence he marix K regre is below he classical K regre he reverse is obvious 33 Las Sep Minimax he bounding echnique, developed using Lemma 1 and applied o K can be used o prove bounds for a much broader class of predicion sraegies he crucial par of he K proof was showing ha each erm in he elescoped regre (5) can be bounded above by as defined in (6), in which all marices share he same eigensysem, and which is hence equivalen o he corresponding classical expression he only propery of he predicion sraegy ha we used was ha i plays in he eigensysem of he pas sufficien saisic herefore, using he same line of argumen, we can show ha if for some classical predicion sraegy we can obain a meaningful regre bound by bounding each erm in he regre independenly, we can obain he same bound for he corresponding marix sraegy, ie is specral promoion In paricular, we can push his argumen o is limi by considering he algorihm designed o minimize in each ieraion his algorihm is known as Las Sep Minimax In fac, he Las Sep Minimax (LSM) principle is a general recipe for online predicion, which saes ha he algorihm should minimize he wors-case regre wih respec o he nex oucome [16] In oher words, i should ac as he minimax algorihm given ha he ime horizon is one ieraion ahead In he classical case for he mulinomial disribuion, afer observing daa wih sufficien saisic 1, classical LSM predics wih ( )! 1 := argmin max! x `(!,x ) {z } log(! 1,x ) X `(!,x q ) q=1 {z } H( ) = Pj exp H( 1+ei ) exp H( 1+ej ) e i (7) Classical LSM is analyzed in [16] for he Bernoulli (n =) case For our sraighforward generalizaion o he classical mulinomial case, he regre is bounded by n 1 ln( + 1) + 1 LSM is herefore slighly beer han K Applying he Las Sep Minimax principle o densiy predicion, we obain marix LSM which issues predicion: S W 1 := argmin max r X log(w ) H W X We show ha marix LSM learns he eigenvecors wihou addiional regre heorem 3 (LSM FML) he regres of classical and marix LSM are a mos n 1 ln( + 1) + 1 Proof We deermine he form of W 1 By Sion s minimax heorem [15]: apple min max S r X log(w ) H = max W X min E P r X log(w ) P W where P ranges over probabiliy disribuion on densiy marices X Plugging in he minimizer W = E P [X ], he righ hand side becomes: apple S max H E P [X ] E P H (8) P Now decompose S inside he maximum: S E P appleh E P " H S 1 as P n i s i s > i Using Lemma 1, we can bound he second expression s > S i X s ih 1 + s is > i # = s > i S E P [X ] s ih 1 + s is > i, 5

6 On he oher hand, we know ha he enropy does no decrease when we replace he argumen E P [X ] by is pinching (aka projecive measuremen) P n (u> i E P [X ]u i ) u i u > i wr any eigensysem u i [1] herefore, we have:! H E P [X ] apple H (s > i E P [X ]s i ) s i s > i = H(p), where he las enropy is a classical enropy and p is a vecor such ha p i = s > i E P [X ]s i Combining hose wo resuls ogeher, we have: S 1 + e i H E P [X ] E P appleh apple H(p) p i H Noe ha we have equaliy only when he disribuion P pus nonzero mass only on he eigenvecors s 1,,s n his means ha when p is fixed, we will maximize (8) by using a disribuion wih such a propery, ie P is resriced o he eigensysem of S 1 his, in urn, means ha W 1 = E P [X ] will play in he eigensysem of S 1 as well I follows ha W 1 is he classical LSM sraegy in he eigensysem of S 1, ie W 1 = P i! 1,i s i s > i, where! 1 are aken as in (7) he proof of he classical LSM guaranee is based on bounding he per-round regre increase: 1 + e x 1 := log(! 1,x ) H +( 1)H, 1 by choosing he wors case wr x and 1 Since, for marices, he wors case for he corresponding marix version of, see (6), is he diagonal case, he whole analysis immediaely goes hrough and we ge he same bound as for classical LSM Noe ha he bound for LSM is no igh, ie here exiss no daa sequence for which he bound is achieved herefore, he bound for marix LSM is also no igh his heorem is a weaker FML because i only relaes wors-case regre bounds We have verified experimenally ha he acual regres coincide in dimension n =for up o =5oucomes, using a grid of 30 dyads per rial, wih uniformly spaced (x > e 1 ) So we believe ha in fac Conjecure 1 (LSM FML) he wors-case regres of classical and marix LSM coincide o execue he LSM marix sraegy, we need o have he eigendecomposiion of he sufficien saisic For densiy marix daa X, we may need o recompue i each rial in (n 3 ) ime For dyadic daa x x > i can be incremenally updaed in O(n ) per rial wih mehods along he lines of [11] 34 Sharkov Fix horizon he minimax algorihm for mulinomials, due o Sharkov [14], minimizes he worscase regre inf sup inf sup `(! 1,x ) H (9)! 0 x 1! 1 x =1 Afer observing daa wih sufficien saisic and hence wih r := rounds remaining, classical Sharkov predics wih! r 1( + e i) X r + c! := e i where r( ) := exp H r( ) c 1,,c n c 1,,c n Pn c i =r (10) he so-called Sharkov sum r can be evaluaed in ime O nrlog(r) using a sraighforward exension of he mehod described in [9] for compuing (0), which is based on dynamic programming and Fas Fourier ransforms he regre of classical Sharkov equals log (0) n 1 log( ) log(n ) + 1 [6] his is again beer han Las Sep Minimax, which is in urn beer han K which dominaes Laplace 6

7 he minimax algorihm for densiy marices, called marix Sharkov, opimizes he wors-case regre inf sup inf sup W 0 X 1 W 1 X =1 S `(W 1, X ) H (11) o his end, afer observing daa wih sufficien saisic S, wih r rounds remaining, i predics wih W := argmin W sup `(W, X)+R r X 1 (S + X), where R r is he ail sequence of inf/sups of (11) of lengh r We now argue ha he FML holds for marix Sharkov Marix Sharkov is surprisingly difficul o analyze However, we provide a simplifying conjecure ha we verified experimenally A rigorous proof remains an open problem Our conjecure is ha Lemma 1 holds wih he enropy H replaced by he minimax regre ail R r : Conjecure For each ineger r, for each pair of posiive marices A and B R r (A + B) X i a > i Ba i r(b) R r A +r(b) a i a > i Noe ha his conjecure generalizes Lemma 1, which is reained as he case r =0 I follows from his conjecure, using he same argumen as for LSM, ha marix Sharkov predics in he eigensysem of S, ie wih W = P i!,i s i s > i, where! as in (10), and furhermore ha Conjecure 3 (Sharkov FML) he wors-case regres of classical and marix Sharkov coincide We have verified Conjecure 3 for he marix Bernoulli case (n =) up o =5oucomes, using a grid of 30 dyads per rial, wih uniformly spaced (x > e 1 ) hen assuming ha R r (S) = log( ( )), where are he eigenvalues of S, for each n from o 5 we drew 10 5 race pairs uniformly from [0, 10], hen drew marix pairs A and B uniformly a random wih hose races Conjecure always held Obaining he FML for he minimax algorihm is mahemaically challenging and of academic ineres bu of minor pracical relevance Firs, he ime horizon mus be specified in advance, so he minimax algorihm can no be used in a purely online fashion Secondly, he running ime is superlinear in he number of rounds remaining, while i is consan for he previous hree algorihms 4 Moivaion and Discussion of he Loss Funcion he marix enropic loss (1) ha we choose as our loss funcion has a coding inerpreaion and i is a proper scoring rule he laer seems o be a necessary condiion for he free marix lunch Quanum coding Classical log-loss forecasing can be moivaed from he poin of view of daa compression and variable-lengh coding [7] In informaion heory, he Kraf-McMillan inequaliy saes ha, ignoring rounding issues, for every uniquely decodable code wih a code lengh funcion, here is a probabiliy disribuion! such ha i = log! i for all symbols i =1,,n, and vice versa herefore, he log loss can be inerpreed as he code lengh assigned o he observed oucome Quanum informaion heory[13, 5] generalizes variable lengh coding o he quanum/densiy marix case Insead of messages composed of bis, he sender and he receiver exchange messages described by densiy marices, and he role analogous o he message lengh is now played by he dimension of he densiy marix Variable-lengh quanum coding requires he definiion of a code lengh operaor L, which is a posiive marix such ha for any densiy marix X, r(xl) gives he expeced dimension ( lengh ) of he message assigned o X he quanum version of Kraf s inequaliy saes ha, ignoring rounding issues, for every variable-lengh quanum code wih codelengh operaor L, here exiss a densiy marix W such ha L = log W herefore, he marix enropic loss can be inerpreed as he (expeced) code lengh of he observed oucome Proper score funcion In decision heory, he loss funcion `(!,x) assessing he qualiy of predicions is also referred o as a score funcion A score funcion is said o be proper, if for any disribuion p on oucomes, he expeced loss is minimized by predicing wih p iself, ie argmin! E x p [`(!,x)] = p Minimizaion of a proper score funcion leads o well-calibraed forecasing he log loss is known o be a proper score funcion [4] 7

8 We will say ha a marix loss funcion `(W, X) is proper if for any disribuion P on densiy marix oucomes, he expeced loss wih respec o P is minimized by predicing wih he mean oucome of P, ie argmin W E X P [`(W, X)] = E X P [X] he marix enropic loss (1) is proper, for E X P [ r(x log W )] = r E X P [X] log W is minimized a W = E X P [X] [1] herefore, minimizaion of he marix enropic loss leads o well-calibraed forecasing, as in he classical case A second generalizaion of he log loss o he marix domain used in quanum physics [1] is he log race loss `(W, X) := log r(xw) Noe ha here he race and he logarihm are exchanged compared o (1) he expression r(xw) plays an imporan role in quanum physics as he expeced value of a measuremen oucome, and for X = xx >, r(xx > W ) is inerpreed as a probabiliy However, log race loss is no proper he counerexample is sraighforward: if we ake P uniform on {x 1 x > 1, x x > }, hen he minimizer of he expeced log race loss is W / (x 1 + x )(x 1 + x ) >, which differs from E X P [X] = 1 (x 1x > 1 + x x > ) Also for log race loss we found an example (no presened) agains he FML for he minimax algorihm A hird generalizaion of he loss is `(W, X) := log r(x W ), where denoes he commuaive produc beween marices ha underlies he probabiliy calculus of [0] 3 his loss upper bounds he log race loss We don know wheher i is a proper scoring funcion However, i equals he marix enropic loss when X is a dyad Finally, anoher loss explored in he on-line learning communiy is he race loss `(W, X) := r(wx) his loss is no a proper scoring funcion (i behaves like he absolue loss in he vecor case) and we have an example ha shows ha here is no FML for he minimax algorihm in his case (no presened) In summary, for here o exis a FML, properness of he loss funcion seems o be required 5 Conclusion We showed ha he free marix lunch holds for he marix version of he K esimaor hus he conjecured free marix lunch [18] is realized Our paper raises many open quesions Perhaps he main one is wheher he free marix lunch holds for he marix minimax algorihm Also we would like o know wha properies of he loss funcion and algorihm cause he free marix lunch o occur From he examples given in his paper i is emping o believe ha you always ge a free marix lunch when upgrading any classical sufficien-saisics-based predicor o a marix version by jus playing his predicor in he eigensysem of he curren marix sufficien saisics However he following couner example shows ha a general reducion mus be more suble: Consider floored K, which predics wih!,i /b,i c +1/ For =5rials in dimension n =, he wors-case regre is 197 for he classical log loss and 199 for marix enropic loss A Proof of Lemma 1 We prove he following slighly sronger inequaliy for all 0 he lemma is he case =1 a > i f( ) := H(A + B) Ba i r(b) H(A + r(b)a ia > i ) 0 Since f(0) = 0, i suffices o show ha f 0 ( ) = log(d) I, f 0 ( ) = r B log(a + B) + a > i Ba i r a i a > i log A + r(b) a i a > i = r B log A + r(b)i r B log(a + B) Since r(b)i B, we have A + r(b)i A + B, and hence he marix monooniciy of he logarihm implies ha log A + r(b)i log(a + B), so ha f 0 ( ) 0 3 We can compue A B as he marix exponenial of he sum of marix logarihms of A and B 8

9 References [1] S Arora, E Hazan, and S Kale Fas algorihms for approximae semidefinie programming using he muliplicaive weighs updae mehod In FOCS, pages , 005 [] S Arora and S Kale A combinaorial, primal-dual approach o semidefinie programs In SOC, pages 7 36, 007 [3] K S Azoury and M K Warmuh Relaive loss bounds for on-line densiy esimaion wih he exponenial family of disribuions Machine Learning, 43(3):11 46, 001 [4] J M Bernardo and A F M Smih Bayesian heory Wiley, 1994 [5] K Bosroem and Felbinger Lossless quanum daa compression and variable-lengh coding Phys Rev A, 65(3):03313, 00 [6] N Cesa-Bianchi and G Lugosi Predicion, Learning, and Games Cambridge Universiy Press, New York, NY, USA, 006 [7] M Cover and J A homas Elemens of Informaion heory John Wiley & Sons, 1991 [8] R Jain, Z Ji, S Upadhyay, and J Warous QIP = PSPACE In Proceedings of he 4nd ACM Symposium on heory of Compuing, SOC, pages , 010 [9] P Konkanen and P Myllymäki A fas normalized maximum likelihood algorihm for mulinomial daa In Proceedings of he Nineeenh Inernaional Join Conference on Arificial Inelligence (IJCAI-05), pages , 005 [10] R E Krichevsky and V K rofimov he performance of universal encoding IEEE ransacions on Informaion heory, 7():199 07, Mar 1981 [11] J Kwok and H Zhao Incremenal eigen decomposiion In IN PROC ICANN, pages 70 73, 003 [1] M A Nielsen and I L Chuang Quanum Compuaion and Quanum Informaion Cambridge Universiy Press, 000 [13] B Schumacher and M D Wesmoreland Indeerminae-lengh quanum coding Phys Rev A, 64(4):04304, 001 [14] Y M Sharkov Universal sequenial coding of single messages Problems of Informaion ransmission, 3(3):3 17, 1987 [15] M Sion On general minimax heorems Pacific Jouronal of Mahemaics, 8(1): , 1958 [16] E akimoo and M Warmuh he las-sep minimax algorihm In Proceedings of he 13h Annual Conference on Compuaional Learning heory, pages , 000 [17] K suda, G Räsch, and M K Warmuh Marix exponeniaed gradien updaes for on-line learning and Bregman projecions Journal of Machine Learning Research, 6: , June 005 [18] M K Warmuh When is here a free marix lunch In Proc of he 0h Annual Conference on Learning heory (COL 07) Springer-Verlag, June 007 Open problem [19] M K Warmuh and D Kuzmin Online variance minimizaion In Proceedings of he 19h Annual Conference on Learning heory (COL 06), Pisburg, June 006 Springer-Verlag [0] M K Warmuh and D Kuzmin Bayesian generalized probabiliy calculus for densiy marices Journal of Machine Learning, 78(1-):63 101, January 010 9