E0 370 Saisical Learning Theory Lecure 0 (ov 7, 0 Online Learning from Expers: Weighed Majoriy and Hedge Lecurer: Shivani Agarwal Scribe: Saradha R Inroducion In his lecure, we will look a he problem of learning from muliple expers in an online fashion. There are finie number of expers, who give heir predicions ξ,..., ξ. The learning algorihm has o use he predicor values and come up wih an oucome ŷ. The oal number of misakes made by he algorihm is compared wih he performance of he bes exper in consideraion. Online Predicion from Expers A general online predicion problem proceeds as follows. Online (binary predicion using muliple expers For =,..., T : Receive exper predicors ξ (x,..., ξ (x {±} Predic ŷ {±} Receive rue label y {±} Incur loss l(y, ŷ. Halving Algorihm Here we assume ha he se of expers ha we consider has an exper which would give he correc label for all insances. In he halving algorihm, for every ieraion, only he consisen expers are reained. If a predicor makes a misake i will no more be conribuing in he predicion process. Halving Algorihm Iniiae weighs wi = i ] For =,..., T : Receive exper predicors ξ (x,..., ξ (x {±} Predic ŷ = sign( n j= w j.ξ j (majoriy voe Receive rue label y {±} Incur loss l(y, ŷ Updae:- Updae:- i... : Ifξi y hen i 0 else i wi
Online Learning from Expers: Weighed Majoriy and Hedge Thus he maximum number of misakes, or he sum of loss over any given sequence is bounded by he logarihm of number of predicors. i.e. L 0- S Halving] log.. Weighed Majoriy (WM Algorihm In he halving algorihm, when a predicor makes even one misake, i will no be able o conribue o he predicion in he successive ieraions. When we don have an exper ha would predic correcly for all samples, his would no be a suiable approach. The weighed majoriy algorihm works well in such siuaions. Here every predicor is assigned equal weigh, say, iniially. Laer as hey make binary predicions on insances, he weighs of he predicors are decreased using muliplicaive updae, when hey commi misakes. The rae a which he weighs are updaed is governed by he parameer. Weighed majoriy Algorihm Iniiae weighs wi = i ] For =,..., T : Receive exper predicors ξ (x,..., ξ (x {±} Predic ŷ = sign( n j= w j.ξ j Receive rue label y {±} Incur loss l(y, ŷ Updae:- If ŷ y i... i w i exp(.i(y ξ i (majoriy voe Theorem.. Le ξ,..., ξ {±} T. Le S = (y,..., y T {±} and le > 0. Then he oal number of misakes ( L 0- S W eighedm ajoriy(] +exp(. min L 0- S ξ i ] + i. +exp(. Proof. Denoe L 0- S W eighedmajoriy] = L For each rial on which here is a misake, we have + = wi. exp (.I(y ξ i. ( = w i. exp + w i ( i:y ξ i i:y =ξ i = exp.w maj + W min (3 For all misake rials, we have + Therefore summing over =,..., T gives exp.w maj + W min + exp (W maj W min (4 = + exp.(w maj + W min +exp. For oher rials, + W W T + W T = + exp.( (5 ( + exp L. (6
Online Learning from Expers: Weighed Majoriy and Hedge 3 L ln W ln W T + +exp (. (7 Finding he lower bound on ln W T + + = j= w T + j w T + j exp.li w i ( i. (8 L ln W.L i ln wi +e ( = ln +.L i +e ( (9 (0 for all w j > 0 j Thus we obain he resul..3 Weighed Majoriy: Coninuous Version (WMC We now see he coninuous version of weighed majoriy algorihm. The final predicion is a weighed average of he exper predicor values. Here ỹ = ŷ = y = 0, ] Weighed majoriy Algorihm :Coninuous Version (WMC Iniiae weighs wi = i ] For =,..., T : Receive exper predicors ξ (x,..., ξ (x 0, ] Predic ŷ = w i.ξ i 0, ] (Weighed Average w i Receive rue label y 0, ] Incur loss l abs (y, ŷ = y ŷ Updae:- i... i wi. exp. ξ i y Theorem.. Le ξ,..., ξ 0, ] T. Le S = (y,..., y T 0, ] and le > 0. Then he oal number of misakes S W MC(] (. min exp i L abs L abs S ξ i ] +.. exp Proof. Denoe L abs S W MC(] = L For each rial we have + = wi. exp. y ξ i. ( wi. ( exp y ξi ]. (
4 Online Learning from Expers: Weighed Majoriy and Hedge + wi ( exp w i y ξi (3 w i. ( exp w i y ξi (4 w i =. ( exp ŷ y ] (5 ] +. exp ( exp. ŷ y. (6 + exp ( exp. T = ŷ y ] = exp ( exp.l exp ( exp. ŷ y ] (7 (8 Finding he lower bound on ln W T + L ln W ln W T + (exp. (9 + exp.li w i ( i. (0 Thus we obain he resul L ln W T + +.L i ln wi (exp ( ln +.L i (exp. ( 3 Online Allocaion The problem of online allocaion occurs in scenarios where we need o allocae differen fracion of resources ino differen opions. The loss associaed wih every opion is available a he end of every ieraion. We would like o reduce he oal loss suffered for he paricular allocaion. The allocaion for he nex ieraion is hen revised, based on he oal loss suffered in he curren ieraion using muliplicaive updae. Hedge Algorihm( Iniiae weighs wi = i ] For =,..., T : Make allocaion p p = w ; w i Receive vecor of loses l = (l,..., l 0, ] Incur loss p.l = p i.l i Updae:- i... i wi. exp (l i
Online Learning from Expers: Weighed Majoriy and Hedge 5 Theorem 3.. Le l,..., l T 0, ] The cumulaive loss of he algorihm is LA] = T p.l If he loss of a paricular opion over he T ieraions is given by Then L T l i. LHedge(] (. min L exp i +.. i exp Proof. Denoe LHedge(] = L For each rial we have + = wi. exp.l i. (3 wi. ( exp. w i.l i. (4 w i + wi ( exp p.l ]. (5 + exp ( exp.p.l ] (6 ] exp ( exp.l (7 L ln W ln W T + (exp. (8 Finding he lower bound on ln W T + + exp.li w i ( i. (9 Thus we obain he resul L ln W T + +.L i ln wi (exp (30 ln +.L i (exp. (3 4 ex Lecure In he nex lecure, we will inroduce he idea of minimax regre, in an adversarial learning seing. References