Online Learning for Global Cost Functions


 Cory Clark
 3 years ago
 Views:
Transcription
1 Online Learning for Global Cost Functions Eyal EvenDar Google Research Robert Kleinberg Cornell University Shie Mannor echnion an McGill University Yishay Mansour Google Research an el Aviv University Abstract We consier an online learning setting where at each time step the ecision maker has to choose how to istribute the future loss between k alternatives, an then observes the loss of each alternative Motivate by loa balancing an job scheuling, we consier a global cost function over the losses incurre by each alternative, rather than a summation of the instantaneous losses as one traitionally in online learning Such global cost functions inclue the makespan the maximum over the alternatives an the L norm over the alternatives Base on approachability theory, we esign an algorithm that guarantees vanishing regret for this setting, where the regret is measure with respect to the best static ecision that selects the same istribution over alternatives at every time step For the special case of makespan cost we evise a simple an efficient algorithm In contrast, we show that for concave global cost functions, such as L norms for <, the worstcase average regret oes not vanish Introuction We consier online learning with a global cost function Namely, a ecision maker who is facing an arbitrary environment is trying to minimize a cost function that epens on the entire history of his choices hence we term it a global cost function o motivate the iscussion, consier a job scheuling problem with a job that can be istribute to multiple machines an each job incurs a ifferent loss per machine At each stage the ecision maker first selects a istribution over machines an only then observes the losses loas of each Supporte by NSF grants CCF an CCF , an Alfre P Sloan Founation Fellowship, an a Microsoft Research New Faculty Fellowship Supporte by Horev Fellowship an by the ISF uner grant his work was supporte in part by the IS Programme of the European Community, uner the PASCAL2 Network of Excellence, by a grant from the Israel Science Founation an by a grant from Unite StatesIsrael Binational Science Founation BSF his publication reflects the authors views only machine he loss per machine as up weighte by the istribution the ecisionmaker selecte he objective of the ecision maker is to have his makespan ie, the loss of the worst machine, or the infinity norm of the losses as small as possible he comparison class the ecision maker consiers is that of all static allocations an the ecision maker wants to minimize the ifference between the cost of the loss vector an the cost of the best static allocation he online learning or regret minimization framework have ha many successful applications in machine learning, game theory, an beyon; see [9] In spite of the avantages of this approach it suffers from some inherent limitations First, the size of the comparison class affects the magnitue of the regret although only in a logarithmic way Secon, the approach usually assumes that there is essentially no state of the system an that the ecision problem is the same at every stage see [4, 5, 6, 7, 0] for eviations from this stanar approach Finally, there is a tacit assumption that the objective function is aitive in the losses over the time steps his last assumption oes not hol in many natural problems such as job scheuling an loa balancing settings, for example While there are similarities in the motivation between online competitive algorithms [8] an the regret minimization setup, the moels have some significant ifferences First, the comparison class is substantially ifferent While competitive analysis allows the optimal scheuler to be arbitrary, regret minimization limits the size of the comparison class On the other han, the competitive analysis approach bouns only the ratio of the performances, while the regret minimization bouns the ifference between them Finally, there is also a very important ifference in the information moel In regret minimization the ecision maker first selects an alternative or a istribution over alternatives an only then observes the losses, while in the stanar online setup the algorithm first observes the loas losses an only then selects a machine alternative Our moel assumes a finite number of alternatives machines he information moel is similar to the regret minimization: the ecision maker first selects a istribution over machines alternatives an only then observes the losses loas We have a static comparison class, which inclues the algorithms that use the same istribution over the machines in every time step Our objective function is much in the spirit of the job scheuling, incluing objective functions such as makespan or L 2 norm We boun the iffer
2 ence between the performance of our online algorithm an the performance of the best static istribution with respect to a global cost function say makespan Note that as oppose to many regret minimization works, we consier not only the best alternative, but rather the best istribution over alternatives, so the comparison class in our setup is infinite Our contributions inclue the following: We present the online learning moel for a global cost his moel is natural in the online learning setup an it has not been stuie to the best of our knowlege We present an algorithm for global cost functions such as makespan, or any L norm, for > that minimizes the ifference between the cost of our online algorithm an the best static istribution in a rate of O k/, where is the number of time steps Our algorithm only requires that the objective function is convex an Lipschitz an that the optimal cost is a concave an Lipschitz function Our general algorithm guaranteeing a vanishing average regret is base on approachability theory In spite of its goo rates, the explicit computation of the regret minimizing strategy oes not seem to be easy For the important special case of makespan cost function we provie a specialize simple eterministic online algorithm, which is both computationally efficient an has a regret boun of Ologk/, where k is the number of ifferent alternatives Our algorithm is base on a recursive construction where we first solve the case of two alternatives an then provie a recursive construction for any number of alternatives Our recursive construction maintains the computational efficiency of the two alternative algorithm, an its regret epens only logarithmically on the number of alternatives We complement our algorithms with an impossibility result We show that one cannot hope to have a similar result for any global cost function Specifically, we show for a wie class of concave function incluing any L norm, for 0 < < that any online algorithm woul have Ω regret More specifically, we show that the ratio of the online cost to the best istribution cost is boune away from one Relate Work: Unfortunately, we can not give a comprehensive review of the large boy of research on regret minimization algorithm, an we refer the intereste reaer to [9] In the following we highlight a few more relevant research irections here has been an ongoing interest in extening the basic comparison class for the regret minimization algorithm, It is instructive to iscuss the ifference between minimizing an aitive loss as in the best experts an minimizing the makespan the alternative with the maximum loss Consier two alternatives an a sequence of ientical losses of 2, Minimizing the aitive loss woul always select alternative 2, an have cost Minimizing the makespan woul select a istribution /3, 2/3 an woul have cost 2/3 for example by introucing shifting experts [2] time selection functions [7] an wie range regret [6] Still, all those works assume that the loss is aitive between time steps A ifferent research irection has been to improve the computational complexity of the regret minimization algorithms, especially in the case that the comparison class is exponential in size General computationally efficient transformation where given by [5], where the cost function is linear an the optimization oracle can be compute in polynomial time, an extene by [4], to the case where we are given only an approximateoptimization oracle here has been a sequence of works establishing the connection between online competitive algorithms [8] an online learning algorithm [9] One issue is that online learning algorithms are stateless, while many of the problems aress in the competitive analysis literature have a state see, [4] For many problems one can use the online learning algorithms an guarantee a nearoptimal static solution, however a straightforwar application requires both exponential time an space Computationally efficient solutions have been given to specific problems incluing, paging [5], atastructures [6], routing [2, ], averserial MDPs [, 8] an bin packing [3] In contrast to our work, all the above works concentrate on the case where the global cost function is aitive between time steps 2 Moel We consier an online learning setup where a ecision maker has a finite set K {,, k} of k alternatives machines to choose from At each time step t, the algorithm A selects a istribution αt A K over the set of alternatives K, where K is the set of istributions over K Following that the aversary selects a vector of losses loas l t [0, ] k he average loss of alternative i is enote by L i t l ti he average loss of the online algorithm A on alternative i is L A i t αa t il t i an its average loss vector is L A LA,, LA k Let the loss vector of a static allocation α K be L α α L where x y xy,, xkyk he objective of the online algorithm A is to minimize a global cost function CL A In traitional online learning the global cost C is assume to be an aitive function, ie, CL A k i LA i he main focus of this work is to consier alternative cost functions which arise in many natural problems such as job scheuling More specifically we consier the following norms: the makespan, C L A maxk i LA i, an the norm cost, C L A k i LA i / for > 0 Note that for > the norm cost is convex while for < it is concave Our comparison class is the class of static allocations for α K We efine the optimal cost function C L as the minimum over α K of CL α We enote by αc L a minimizing α K an calle the optimal
3 static allocation, 2 ie, C L min α K CLα min Cα L α K Cα CL L We enote by C an C the global optimal cost function C of the norm cost an the makespan, respectively he regret of algorithm A at time is efine as, R A CL A C L Given a subset of the time steps B we enote by L B t B l t the average losses uring the time steps in B, an for an algorithm A we enote by L A B i t B αa t il t i its average loss uring B on alternative i, an by L A B L A B,, LA B k the average loss of A uring B 3 Preliminaries For the most part of the paper we will assume that the cost function, C, is a convex function while C is a concave function he following is immeiate from the efinition of the norms Lemma he norm global cost function C, for >, an the makespan global cost function C are both convex an Lipschitz functions he challenging part is to show that the optimal cost function C is a concave function In Appenix A we prove this for norm Lemma 20 an the makespan Lemma 2 Lemma 2 he norm global optimal cost function C, for >, an the makespan global optimal cost function C are both concave an Lipschitz functions In Appenix A we erive for both the norm an the makespan their optimal cost function an optimal static allocation Lemmas 9 an 22 Lemma 3 For the norm global optimal cost function we have α C /L i P k j /L j i K an C L P k j /L j optimal cost function we have αc an C l P k j /L j, an for the makespan global /L i P k i /L j i K We now prove that given any partition of the losses to subsets, if we boun the regret in each subset, then the sum of the subsets regrets will boun the regret of the online algorithm 2 When the minimum is not unique we assume it can be selecte accoring to some preefine orer Lemma 4 Suppose that C is convex an C is concave Let i be a partition of the time steps to m sets, an let ON be an online algorithm such that for each i we have CL ON i C L i R i hen m CL ON C L R i i Proof: From the assumption in the theorem we have that m i CLON i m i C L i m i R i Since i is a partition, L m i L i an L ON m i LON i Since C is concave, C L m i C L i, an since C is convex, CL ON m i CLON i Combining the three inequalities erive the theorem he importance of Lemma 4 is the following If we are able to partition the time steps, in retrospect, an boun the regret in each part of the partition, then the above theorem states that the overall regret is boune by the sum of the local regret Remark: Our moel is essentially fractional, where the ecision maker selects a istribution, an each alternative receives exactly its fraction accoring to the istribution However, in a ranomize moel we can simulate our fractional moel, an use its istributions to run a ranomize algorithm Using stanar concentration bouns, the aitional loss per action, ue to the ranomization, is at most O log k/, with high probability ogether with the fact that C is Lipschitz we woul obtain similar results 4 Low regret approachabilitybase algorithm In this section we use Blackwell s approachability theory to construct a strategy that has zero asymptotic regret Recall that accoring to approachability, one consiers a vectorvalue repeate game he main question that is consiere is if the ecision maker can guarantee that the average vectorvalue rewar can be mae asymptotically close in norm sense to some esire target set We provie the essential backgroun an relevant results from approachability theory in Appenix B his section starts with construction of the game followe by a efinition of the target set an the application of approachability theory to prove that the set is approachable We finally escribe explicitly a regret minimizing strategy base on approachability theory Consier a repeate game against Nature where the ecision maker chooses at every stage an action α K an Nature chooses an action l [0, ] k he vector value rewar obtaine by the ecision maker is a 2k imensional vector whose first k coorinates are α l an the last k coorinates are l Let us enote this immeiate vectorvalue rewar by m t α t l t, l t It easily follows that the average rewar ˆm t m t/ L A, L We are now reay to efine the target set in this game S {x, y R k R k : x, y 0; Cx C y} Note that S is a set in R 2k We will show below that S is approachable see Appenix B We first claim that S is convex
4 Lemma 5 Suppose that C is convex an C is concave hen S is convex Proof:Suppose that x i, y i S for i, 2 an fix β [0, ] we let x i an y i enote vectors in R k Since C is convex we have that Cβx + βx 2 βcx + βcx 2 Since x i, y i S for i, 2 we have that Cx i C y i for i, 2 So that βcx + βcx 2 βc y + βc y 2 Using the fact that C is concave we obtain that βc y + βc y 2 C βy + βy 2 Combining the above inequalities we conclue that Cβx + βx 2 C βy + βy 2, completing the proof We are going to use Blackwell s approachability theory We note that in stanar approachability it is usually assume that the set of available actions is finite In our game, both players can choose from compact action spaces Namely, the ecision maker s action is α K an Nature s action is l [0, ] k Let p K enote a probability istribution over α an let q [0, ] k enote a probability istribution over l See Appenix B for further iscussion We can efine the expecte vectorvalue rewar as: mp, q α K heorem 6 he set S is approachable l [0,] k p α q l α l, l Proof:Accoring to Blackwell s theorem for convex sets heorem 24 in Appenix B it suffices to prove that for every q [0, ] k there exists a p K such that mp, q S Inee, consier some q [0, ] k an let Lq enote the mean of l uner q that is Lq q l l, an let p be a singleton whose mass is centere at αclq hen mp, q αc Lq Lq, Lq an is in S by efinition Since S is approachable let us explain how to approach it in a constructive way We follow Blackwell s first approachability theorem specialize for convex sets heorem 23 Given the current vector value rewar ˆm t L A t, L t we first compute the irection into the set S For that, we nee to solve the following convex optimization problem: min x,y R k L A t, L t x, y 2 2 st Cx C y x 0 Accoring to Blackwell s approaching strategy see Appenix B, if the solution to 2 is 0, ie, ˆm t S the ecision maker can act arbitrarily at time t If not, then let u x, y ˆm t where x, y are the minimizers of 2, an let us write u u α, u l to enote the two k imensional components of u We are looking for a mixe action p K that guarantees that the expecte rewar at time t + is on the other sie of the supporting hyperplane at x, y We know that such a mixe action exists since S is approachable an convex heorem 23 he optimization problem we try to solve is therefore: sup inf p K q [0,] k sup p K l [0,] k sup α K max α K min α α l p α q l u α α l + u l l 3 p α u α α l + u l l 4 min u l [0,] k α α l + u l l 5 min u l [0,] k α α l + u l l, 6 where 3 is the problem that we are trying to solve in the lifte space; Eq 4 hols since the epenence in l is linear in q so that for any p there is a singleton q which obtains the minimum; Eq 5 hols since p only affects Eq 4 linearly through α so that any measure over K can be replace with its mean; an Eq 6 hols because of continuity an the compactness of the sets we optimize on Now, the optimization problem of Eq 6, is a maximization of a convex function or alternatively, the solution of a zerosum game with k actions for one player an 2 k for the other player an can be efficiently solve for a fixe k We summarize the above results in the following heorem heorem 7 he following strategy guarantees that the average regret goes to 0 At every stage t solve 2 2 If the solution is 0, play an arbitrary action 3 If the solution is more than 0, compute u x, y ˆm t where x, y are the minimizers of 2 an solve 6 Play a maximizing α Furthermore, the rate of convergence of the expecte regret is O k/ Proof:his is an approaching strategy to S he only missing part is to argue that approaching S implies minimizing the regret his is true by construction an since C an C are Lipschitz As for the rate of convergence for approachability: see the iscussion in Appenix B 5 Algorithms for Makespan In this section we present algorithms for the makespan global cost function We presnet algorithms, which are simple to implement, computationally efficient, an have an improve regret boun For k alternatives the average regret vanishes at the rate of Olog k /2 he construction of the algorithms is one in two parts he first part is to show an algorithm for the basic case of two alternatives Section 5 he secon part is a recursive construction, that buils an algorithm for 2 r alternatives from two algorithms for 2 r alternatives, an one algorithm for two alternatives he recursive construction essentially buils a complete binary tree, the main issue in the construction is to efine the losses that each algorithm observes Section 52
5 he DIFF Algorithm: At time t we have p p 2 /2, at time t 2 we have, p t+ p t + pt2lt2 ptlt, an p t 2 p t Figure : Algorithm for two alternatives 5 wo Alternatives he DIFF Algorithm: We enote the istribution of DIFF at time t by αt D p t, p t 2 Algorithm DIFF starts with p p 2 /2, an the upate of the istribution of DIFF is one as follows, p t+ p t + p t2l t 2 p t l t, an by efinition p t 2 p t Note that we are guarantee that p t [0, ], since p t p t We first provie some high level intuition Assume that in a certain subset of the time steps the algorithm uses approximately the same probabilities, say ρ, ρ an that the loa on the two alternative increase by exactly the same amount at time stpes his will imply that ρl L ρl 2, so it has to be the case that ρ 2 L +L 2 Note that the right han sie is the optimal static allocation for time steps Now, using Lemma 4, if we partition the time steps to such s, we will have low regret originating mainly from the fact that we are using approximately the same istribution ρ For the analysis sake, we present a sequence of moification to the input sequence Our objective is to guarantee a partition of the time steps to s with the above properties on the moifie sequence In aition the moifie sequence will ensure that the static optimal policy s performance is similar on both sequences First, we a O losses to get the DIFF algorithm to en in its initial state, where the ifference between the alternatives is 0 See Lemma 8 Secon, we consier a partition [0, ] action istribution to bins of size Θ /2 each We moify the sequence of losses to make sure that for each upate, DIFF s allocation istribution before an after the upate are in one bin, possibly by splitting a single upate to two upates See Lemma 0 he ith subsequence is the set of times where DIFF s action istribution is at the ith bin We show that the optimal action for each subsequence is close to the actions istribution performe in the bin recall that any two actions in the bins can iffer by at most Θ /2 See Lemma Since uring each the increase in the online loa on the alternatives is ientical, then DIFF action istribution is near optimal We complete the proof by summing over the subsequences See Lemma 2 an heorem 3 We woul abuse notation an given a sequence of losses l we let L D t l, i t τ p τ il τ i be the aggregate loss of alternative i, after t steps using the DIF F algorithm, an L D t l be the aggregate loss vector Let Λ t l L D t l, 2 L D t l, Equivalently, algorithm DIFF sets p t /2+ Λ t l/ /2 he proof will be one by a sequence of transformation of the sequences of losses l, where for each transformation we woul boun the effect on the online DIF F algorithm an the optimal static assignment α C l Given the sequence of losses l we exten the sequence by m aitional losses to a loss sequence ˆl such that the DIF F algorithm woul en with two perfectly balance losses, ie, Λ +m ˆl 0 Lemma 8 he following hols for ˆl: L D +k ˆl, L D +k ˆl, 2, 2 C L D +m ˆl C L D l, an 3 C ˆl C l + m where m 2 /2 Proof: Without loss of generality assume that Λ l > 0, then we a losses of the form, 0 After each such loss we have in the DIF F algorithm that p t+ p t his implies that for some m 2 /2 we have /2 p +m /2 If the last inequality is strict, then the last loss will be η, 0 for some η [0, ] such that p +m /2 As a by prouct of our proof, we obtain that the loss on the two alternatives is similar at time Corollary 9 Λ l 2 Consier a gri G which inclues 0 /2 points z i 0 /2 We say that the upate crosses the gri at point z i at time t if either p t < z i < p t or p t > z i > p t Let BIN i be [z i, z i+ ], an i be the set of time steps that either start or en in bin BIN i, ie, i {t : p t BIN i or p t BIN i } Each time step can be in at most two consecutive bins, since p t p t /2 We will moify the loss sequence ˆl to a loss sequence ˆb an an action istrbution sequence q such that in each bin, all the upates start an en insie the bin D will act accoring to q t at time t an will observe loss ˆb t We efine L D ˆb to be t ˆb t q t, t ˆb t 2q t 2 an for any subset of times steps, S, L D S ˆb t S ˆb t q t, t S ˆb t 2q t 2 Lemma 0 C L D ˆb C L D ˆl an C ˆb C ˆl Proof: If at time t we have that both p t an p t are containe in one bin, then we set ˆb 2t l t, p t an ˆb 2t 0, 0, p t Otherwise, p t an p t are containe in two consecutive bins, say BIN i an BIN i+ We a to ˆb two elements, ˆb 2t λl t, p t an ˆb 2t λl t, p t, such that p t +λp t 2l t 2 p t l t / /2 z i+ Since we replace each entry in ˆl by two entries in ˆb, this implies that the length of ˆb is at most 2 + /2 4 By efinition we have, L D ˆb, a L D +m ˆl, a an L A ˆb, a +m t ˆl t a L +m ˆl, a
6 Now we can efine a partition of the time steps in ˆb Let δ t w t 2q t 2 w t q t / Note that for upates which o not cut a bounary we have δ 2t p t+ p t, while for upates that cut the bounary we have δ 2t + δ 2t p t+ p t Let v t /2+ t i δ t We say that upate t in ˆb belongs to bin BIN i if v t, v t [z i, z i ] Note that by construction we have that every upate belongs to a unique bin Let i be all the time points in ˆb that belong to BIN i Note that t i δ t 0, ie the increase loa for the two alternatives in DIFF is the same Since { i } is a partition of the inexes of ˆb, by Lemma 4 we can lower boun the optimal cost by C L D 2 +m ˆb C ˆb 0 /2 i Our main goal is to boun the ifference C L D i ˆb C ˆb i C L D i ˆb C ˆb i Given the losses at time steps i of ˆb, we enote static optimal allocation, α C ˆb i, by q i, q i Lemma q i z i 30 /2 Proof: For the upates in i we have that t i δ t 0 Consier the function, fx t i x w t t i xw t 2 xl i + L i 2 L i 2 First, for qi we have that fq i 0 Secon, since for any t i we have that p t [z i, z i+2 ] then fz i t i δ t 0 fz i+2 herefore qi [z i, z i+2 ] Since, z i+2 z i 30 /2, we have that z i qi 30 /2 Lemma 2 0 /2 i C L D i ˆb C ˆb i 240 /2 Proof: By Lemma for any t i we have that p t q i 30 /2 his implies that, C L D i ˆb C ˆb i 30 /2 max{l i ˆb,, L i ˆb, 2} an therefore 0 /2 i C L D i ˆb C ˆb i / /2 i 30 / /2 max{l i ˆb,, L i ˆb, 2} he MULI Algorithm A K 2 r: Alternative set: K of size 2 r partitione to I an J, I J 2 r Proceures: A 2, A I 2 r, A J 2 r Action selection: Algorithm A I 2 r gives x t I Algorithm A J 2 r gives x t J Algorithm A 2 gives z t, z t Output α t I J where, for i I set α t i x t iz t, an for j J set α t j y t j z t Loss istribution: Given the loss l t Algorithm A I 2 r receives l t I Algorithm A J 2 r receives l t J Algorithm A 2 receives ω t I, ω t J, where ω t I 2 r+ i I l tix t i, an ω t J 2 r+ j J l tjy t j Figure 2: Algorithm for multiple alternatives heorem 3 For any loss sequence l, we have that C L D l C l 24 /2 Proof: Given the loss sequence l, we efine the loss sequence ˆl By Lemma 8 we have that C L D +m ˆl C L D l an C ˆl C l+ /2 By Lemma 0 we have that C ˆl C ˆb, an C L D +m ˆl C L D 2 +m ˆb From Lemma 2 we have that 0 /2 i C L D i ˆb C i 240 /2 By Lemma 4 an Lemma 22, we can boun the sum by the global function, hence we have that C L D 2 +m ˆb C ˆb 240 /2 herefore, C L D l C l 24 /2 52 Multiple alternatives In this section we show how to transform a low regret algorithm for two alternatives to one that can hanle multiple alternatives For the base case we assume that we are given an online algorithm A 2, such that for any loss sequence l,, l over two alternatives, guarantees that its average regret is at most α/ Using A 2 we will buil a sequence of algorithms A 2 r, such that for any loss sequence l,, l over 2 r alternatives, A 2 r guarantees that its average regret is at most Oα r/ Our basic step in the construction is to buil an algorithm A 2 r using two instances of A 2 r an one instance of A 2 A 2 r woul work as follows We partition the set of actions K to two subsets of size 2 r, enote by I an J; for each subset we create a copy of A 2 r which we enote by A I 2 r an A J 2 he thir instance A M r 2 woul receive as an input the average loss of A I 2 an A J r 2 Let us be more precise about the construction of A r 2 r At each time step t, A 2 r receives a istribution x t I from A I 2, a istribution y r t J from A J 2 an r
7 probability z t [0, ] from A M 2 he istribution of A 2 r at time t, α t I J, is efine as follows For actions i I we set α t i x t iz t, an for actions j J we set α t j y t j z t Given the action α t, A 2 r observes a loss vector l t It then provies A I 2 with the r loss vector l t i i I, A J 2 with the loss vector l r t j j J, an A M 2 with the loss vector ω t I, ω t J, where ω t I 2 r+ i I l tix t i an ω t J 2 r+ j J l tjy t j he tricky part in the analysis is to relate the sequences x t, y t an z t to the actual aggregate loss of the iniviual alternatives We will o this in two steps First we will boun the ifference between the average loss on the alternatives, an the optimal solution hen we will show that the losses on the various alternatives are approximately balance, bouning the ifference between the maximum an the minimum loss he input to algorithm A M 2 on its first alternative generate by A I 2 has an average loss r [ W I ω t I 2 ] r+ x t il t i, t t i I similarly, on the secon alternative it has an average loss W J ω t J 2 r+ y t jl t j t t j J First, we erive the following technical lemma its proof is in Appenix C Lemma 4 Assume that A M 2 hen W I J W I + W J is at hight r part of A I J 2 r + α a I J + rα L a he following claim, base on Corrollary 9, bouns the ifference between the average of all alternatives an the loss of a specific alternative Claim 5 For any alternative i K we have, W K L A 2 r i r Proof: We can view the algorithm A 2 r as generating a complete binary tree Consier a path from the root to a noe that represents alternative i K Let the sets of alternatives on the path be I 0,, I r, where I 0 K an I r {i} We are intereste in bouning W I 0 W I r Clearly, r W I 0 W I r W I i W I i i Since W I i 2 W I i + W I i I i, we have that W I i W I i 2 W I i W I i I i Since the alternative sets I i an I i I i are the alternatives of an A M 2 algorithm at epth i, by Corollary 9 we have that W I i W I i I i 2 his implies that W I i W I i herefore, W I 0 W I r r Now we can combine the two claims to the following theorem heorem 6 Suppose the global cost function is makespan For any set K of 2 r alternatives, an for any loss sequence l,, l, algorithm A 2 r will have regret at most O log K, ie, C L A 2 r C l 242 r Proof: By Lemma 4 we have that W K C l rα By Claim 5 we have that for any alternative i K, L A 2 r i W K r herefore, C L A 2 r C l 242 r, since α 24 by heorem 3 6 Lower Boun for NonConvex functions In this section we show a lower boun that hols for a large range of nonconvex global cost functions, efine as follows Definition 7 A function f is γ f strictly concave if for every x > 0: fx/3+f2x/3 fx γ f >, 2 lim x 0 xf x 0 3 f is nonnegative concave an increasing 4 lim x 0 fx 0 Note that any function fx x β for β <, is γ f strictly concave with γ f 2 3 β + 3 β heorem 8 Let CL A k i fla i, where f is γ f  strictly concave For any online algorithm A there is an input sequence such that CL A /C L > γ f he proof can be foun in Appenix D 7 Open Problems In this work we efine the setting of an online learning with a global cost function C For the case that C is convex an C is concave, we showe that there are online algorithms with vanishing regret On the other han, we showe that for certain concave functions C the regret is not vanishing Giving a complete characterization when oes a cost function C enable a vanishing regret is very challenging open problem In this section we outline some possible research irections to aress this problem First, if C is such that for some monotone function g, gc satisfy the same properties as norm: C is convex, C L min α C L α is concave, an α L is Lipschitz, the algorithm of Section 4 for C woul still lea C to vanishing regret For example, CL k i L i can be solve with that approach A more interesting case is when the properties are violate Obviously, in light of the lower boun of Section 6, attaining C in all cases is impossible he question is if there is some more relaxe goal that can be attaine As in [7], one can take the convex hull of the target set in Eq an still consier an approaching strategy It can be shown that by using the approachability strategy to the convex hull, the ecision maker minimizes the regret obtaine with respect to the function C C efine below ie, R CA CLA CC L Specifically: C C L sup L,L 2,,L m, β,β m, β j 0: P βj, P m j βj L j L m C β j αcl j L j j
8 It follows that C L C C L an C L C C L if C is convex an C concave since in that case the convex hull of S equals S In general, however, C C may be strictly larger than C he question if C C is the lowest cost that can be attaine against any loss sequence is left for future research A ifferent open problem is a computational one he approachabilitybase scheme requires projecting a point to a convex set Even if we settle for ɛ projection which woul lea to a vanishing regret up to an ɛ, the computational complexity oes not seem to scale better than /ɛ k It woul be interesting to unerstan if the complexity minimizing the regret for cost functions other than the makespan can have better epenence on ɛ an k In particular, it woul be interesting to evise regret minimization algorithms for norm cost functions References [] B Awerbuch an R Kleinberg Online linear optimization an aaptive routing J Comput Syst Sci, 74:97 4, 2008 [4] S M Kakae, A aumankalai, an K Ligett Playing games with approximation algorithms In SOC, pages , 2007 [5] A Kalai an S Vempala Efficient algorithms for online ecision problems Journal of Computer an System Sciences, 73:29 307, 2005 An earlier version appeare in COL 2003 [6] E Lehrer A wie range noregret theorem Games an Economic Behavior, 42:0 5, 2003 [7] S Mannor an N Shimkin he empirical Bayes envelope an regret minimization in competitive Markov ecision processes MOR, 282: , 2003 [8] JY Yu, S Mannor, an N Shimikin Markov ecision processes with arbitrary rewar processes Mathematics of Operations Research, 2009 [2] B Awerbuch an Y Mansour Aapting to a reliable network path In PODC, pages , 2003 [3] D Blackwell An analog of the minimax theorem for vector payoffs Pacific J Math, 6: 8, 956 [4] A Blum an C Burch Online learning an the metrical task system problem In COL, pages 45 53, 997 [5] A Blum, C Burch, an A Kalai Finelycompetitive paging In FOCS, pages , 999 [6] A Blum, S Chawla, an A Kalai Static optimality an ynamic searchoptimality in lists an trees Algorithmica, 363: , 2003 [7] A Blum an Y Mansour From external to internal regret Journal of Machine Learning Research, 8: , 2007 [8] A Boroin an R ElYaniv Online Computation an Competitive Analysis Cambrige University Press, 998 [9] N CesaBianchi an G Lugosi Preiction, Learning, an Games Cambrige University Press, New York, 2006 [0] D P e Farias an N Megio Combining expert avice in reactive environments Journal of the ACM, 535: , 2006 [] E EvenDar, S M Kakake, an Y Mansour Online markov ecision processes Mathematics of Operations Research, 2009 [2] Y Freun, R Schapire, Y Singer, an M Warmuth Using an combining preictors that specialize In SOC, pages , 997 [3] A Gyrgy, G Lugosi, an G Ottucsk Online sequential bin packing In COL, pages , 2008
9 A Properties of Norms We are now reay to exten the results to any norm Let us start by computing what is the optimal assignment an what is its cost Lemma 9 Consier the norm global cost function an a sequence of losses l,, l he optimal stationary istribution is given by α C l C l /L i P k j /L j optimal global cost function is given by k j /L j i K an the Proof: Recall that L,, L k are the average losses of the alternatives o compute the optimal norm static istribution, we nee to optimize the following cost function k / C l i αil i subject to the constraint that k i αi Using Lagrange multipliers we obtain that the optimum is obtaine for the following values α C li αi /L i k j /L j his implies that the cost of the optimal istribution is given by C L i αil i / /L i k j L /L j i i i L i k j /L j k j /L j k j /L j i / L i / / After computing C we are reay to prove its concavity Lemma 20 he function C l is concave Proof:In orer to show that C is concave we once again will prove that its Hessian is semiefinite negative We first compute the erivative an the partial erivatives of orer 2, we use for notation β > an γ x P k j /Lβ j thus C x γ x /β C xi β γ x /β βxi β+ k xi β+ γ x β + m 2 xm β 2 C xi xj 2 C 2 xi xi β+ β + γ x /β βxj β+ k 2 m xm β β + γ x β +2 xi β+ xj β+ β + xi β+2 γ x β + +xi β+ β + γ x β β + xi β+2 γ x β + +β + xi 2β+ γ x β +2 βxi β+ k 2 m xm β β + γ x β +2 xm β + xi 2β+ xi β+2 m β + γ x β +2 xi β+2 m,k i xm β Since all entries of the Hessian are factor of β+γ x β +2 which is positive we can eliminate it without effecting the semiefinte negative property aha i ji+ + 2xi β+ xj β+ a i a j a 2 i xi β+2 i m,k i i ji+ a 2 i xi β+2 xj β a 2 jxj β+2 xi β +2xi β+ xj β+ a i a j i ji+ a2 i xi 2 i ji+ xi β xj β a2 j xj 2 + 2a ia j xixj xi β xj β a i xi Lemma 2 he function C is concave xm β a j xj 2 0 Proof: In orer to o so we will show that the Hessian of C is semi efinite positive We start by computing the partial
10 erivatives of C C xi /xi 2 2C k j /xj2 xi xj 2/xi2 xj 2 2C, k j /xj3 xi 2 xi 3 xj 2 j i k j /xj3 Next we show that aha < 0 where is the Hessian an a is any vector o simplify the computation we eliminate the common factor 2/ k j /xj3 which is positive from all entries aha i ji+ i ji+ a2 i xi 2 + i ji+ 2a i a j xi 2 xj 2 xixj i j i 2a ia j xixj a2 j xj 2 ai xixj xi a 2 i xi 3 xj a 2 j 0 xj herefore, we showe that C is concave an we conclue the proof Consier the makespan metric which is the stanar metric for loa balancing an job scheuling tasks Lemma 22 Consier the makespan global cost function an a sequence of losses l,, l he optimal stationary istribution is given by αc /L l i P k j /L j i K an the optimal global cost function is given by Cl k j /L j B Approachability theory In this appenix we provie the basic results from approachability theory in the context of compact action sets We start with the efinition of the game Consier a repeate vectorvalue game where player chooses actions in A R a an player 2 chooses actions in B R b We assume both A an B are compact We further assume that there exists a continuous vectorvalue rewar function m : A B R k At every stage t of the game, player selects action a t A an player 2 selects action b t B as a result player obtains a vectorvalue rewar m t ma t, b t he objective of player is to make sure that the average vectorvalue rewar ˆm t t m t t τ converges to a target set R k More precisely, we say that a set is approachable if there exists a strategy for player such that for every ɛ > 0, there exists an integer N such that, for every opponent strategy: Pr ist ˆm t, ɛ for some n N < ɛ, where ist is the pointtoset Eucliean istance We further consier mixe actions for both player A mixe action for player is a probability istribution over A We enote such a istribution by A Similarly, we enote by B the set of probability istributions over B We consier the lifting of m from A B to A B by extening m in the natural way: for p A an q B we efine the expecte rewar as: mp, q p a q b ma, b a b A B he following is the celebrate Blackwell heorem for convex sets he proof is essentially ientical to heorem 75 of [9] an is therefore omitte heorem 23 A convex set V is approachable if an only if for every u R k with u 2 we have that: sup p inf q mp, q u inf x u, x V where is the stanar inner prouct in R k he following strategy can be shown to have a convergence rate of O k/t as in [9], page 230 At time t calculate s t arg min s V ˆm t s 2 If s t ˆm t act arbitrarily If s t ˆm t play p A that satisfies: inf q mp, q s t ˆm t s t s t ˆm t Such a p must exist or else the set V is not approachable accoring to heorem 23 he following theorem is a variant of Blackwell s original theorem for convex sets heorem 3 in [3] heorem 24 A close convex set V is approachable if an only if for every q B there exists p A such that mp, q V Proof:he proof is ientical to Blackwell s original proof, noticing that for every q the set {mp, q, p A} is a close convex set C Proof of Lemma 4 Let W W I an W 2 W J he first inequality follows from the fact that our assumption of the regret of A M 2 implies that max{w, W 2 } /W +/W 2 + α his also proves the base of the inuction for r For the proof let π i I L i, π 2 j J L j an π π π 2 Also, s i I k I\{i} L k, s 2 j J k J\{j} L k, an s s π 2 + s 2 π One can verify that the optimal cost opt i /L i equals to π/s
11 Let β r α From the inuctive assumption that we have W i π i /s i + β for i {, 2} he proof follows from the following + α /W + /W 2 π + /s +β π 2/s 2+β s π + s2 +s β π 2+s 2β + α + α π + s βπ 2 + s 2 β s π 2 + s 2 β + s 2 π + s β + α π π 2 + βs π 2 + s 2 π + β 2 s s 2 s π 2 + s 2 π + 2s 2 s β π + βs + β2 s s 2 s + 2s 2 s β π s + β s + βs s 2 s + 2s 2 s β + + α α opt + β + α opt + r α + α hus hλ is increasing an its minimum is obtaine at λ 0 herefore the cost function of algorithm, A, is at least f2δ/3 + fδ/3 while the optimum achieves fδ Hence CL A /C σ 2 γ f he case of β < /3 is similar to that of β > 2/3, just using σ 3 rather than σ 2 D Lower boun for a concave loss function Proof:[heorem 8] Consier the following three sequences of losses: sequence σ has t time steps each with losses δ, δ, 2 sequence σ 2 starts with sequence σ followe by t time steps each with losses, δ, 3 sequence σ 3 starts with sequence σ followe by t time steps each with losses δ, an δ will be etermine later he opt for all sequences σ, σ 2 an σ 3 has a loss of fδ Given an online algorithm A, after σ it has an average loss of βδ on alternative an βδ on alternative 2 herefore, its loss on σ is fβδ + f βδ If β [/3, 2/3] then the loss of A is at least f/3δ+f2/3δ compare to a loss of fδ of OP Hence CL A /C σ γ f If β > 2/3 then consier the performance of A on σ 2 On the first t time steps it behaves the same as for σ an at the remaining t time steps it splits the weight such that its average loss on those steps is λ for alternative an λδ for alternative 2 herefore its cost function is fβδ + λ/2 + f β + λδ/2 First, we note that for λ > /2, we have that CL A f/4 Since the cost of OP is fδ an as δ goes to 0 C σ 2 goes to zero, then for sufficiently small δ we have that CL A /C σ 2 γ f For λ /2 efine hλ fβδ + λ/2 + f β + λδ/2δ/2 First, we woul like to show that h λ > 0 for every λ [0, /2] Consier, h λ f βδ + λ/2/2 δ 2 f β + λδ/2 > 0 Since f is concave f is ecreasing an given our assumption that δf δ/2 goes to zero as δ goes to zero, then for small enough δ an λ /2 we can boun as follows: f βδ + λ/2 f 4δf δ/4 δf β + λδ/2
Online Learning for Global Cost Functions
Online Learning for Global Cost Functions Eyal EvenDar Google Research evendar@googlecom Robert Kleinberg Cornell University rdk@cscornelledu Yishay Mansour el Aviv University mansour@tauacil Shie Mannor
More informationA Comparison of Performance Measures for Online Algorithms
A Comparison of Performance Measures for Online Algorithms Joan Boyar 1, Sany Irani 2, an Kim S. Larsen 1 1 Department of Mathematics an Computer Science, University of Southern Denmark, Campusvej 55,
More informationOn Adaboost and Optimal Betting Strategies
On Aaboost an Optimal Betting Strategies Pasquale Malacaria 1 an Fabrizio Smerali 1 1 School of Electronic Engineering an Computer Science, Queen Mary University of Lonon, Lonon, UK Abstract We explore
More informationA Generalization of Sauer s Lemma to Classes of LargeMargin Functions
A Generalization of Sauer s Lemma to Classes of LargeMargin Functions Joel Ratsaby University College Lonon Gower Street, Lonon WC1E 6BT, Unite Kingom J.Ratsaby@cs.ucl.ac.uk, WWW home page: http://www.cs.ucl.ac.uk/staff/j.ratsaby/
More informationAdaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
More informationFactoring Dickson polynomials over finite fields
Factoring Dickson polynomials over finite fiels Manjul Bhargava Department of Mathematics, Princeton University. Princeton NJ 08544 manjul@math.princeton.eu Michael Zieve Department of Mathematics, University
More informationMath 230.01, Fall 2012: HW 1 Solutions
Math 3., Fall : HW Solutions Problem (p.9 #). Suppose a wor is picke at ranom from this sentence. Fin: a) the chance the wor has at least letters; SOLUTION: All wors are equally likely to be chosen. The
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS Contents 1. Moment generating functions 2. Sum of a ranom number of ranom variables 3. Transforms
More informationChapter 2 Review of Classical Action Principles
Chapter Review of Classical Action Principles This section grew out of lectures given by Schwinger at UCLA aroun 1974, which were substantially transforme into Chap. 8 of Classical Electroynamics (Schwinger
More informationSensor Network Localization from Local Connectivity : Performance Analysis for the MDSMAP Algorithm
Sensor Network Localization from Local Connectivity : Performance Analysis for the MDSMAP Algorithm Sewoong Oh an Anrea Montanari Electrical Engineering an Statistics Department Stanfor University, Stanfor,
More informationCh 10. Arithmetic Average Options and Asian Opitons
Ch 10. Arithmetic Average Options an Asian Opitons I. Asian Option an the Analytic Pricing Formula II. Binomial Tree Moel to Price Average Options III. Combination of Arithmetic Average an Reset Options
More informationSensitivity Analysis of Nonlinear Performance with Probability Distortion
Preprints of the 19th Worl Congress The International Feeration of Automatic Control Cape Town, South Africa. August 2429, 214 Sensitivity Analysis of Nonlinear Performance with Probability Distortion
More information10.2 Systems of Linear Equations: Matrices
SECTION 0.2 Systems of Linear Equations: Matrices 7 0.2 Systems of Linear Equations: Matrices OBJECTIVES Write the Augmente Matrix of a System of Linear Equations 2 Write the System from the Augmente Matrix
More informationJON HOLTAN. if P&C Insurance Ltd., Oslo, Norway ABSTRACT
OPTIMAL INSURANCE COVERAGE UNDER BONUSMALUS CONTRACTS BY JON HOLTAN if P&C Insurance Lt., Oslo, Norway ABSTRACT The paper analyses the questions: Shoul or shoul not an iniviual buy insurance? An if so,
More informationData Center Power System Reliability Beyond the 9 s: A Practical Approach
Data Center Power System Reliability Beyon the 9 s: A Practical Approach Bill Brown, P.E., Square D Critical Power Competency Center. Abstract Reliability has always been the focus of missioncritical
More informationWeb Appendices of Selling to Overcon dent Consumers
Web Appenices of Selling to Overcon ent Consumers Michael D. Grubb A Option Pricing Intuition This appenix provies aitional intuition base on option pricing for the result in Proposition 2. Consier the
More informationState of Louisiana Office of Information Technology. Change Management Plan
State of Louisiana Office of Information Technology Change Management Plan Table of Contents Change Management Overview Change Management Plan Key Consierations Organizational Transition Stages Change
More informationOptimal Energy Commitments with Storage and Intermittent Supply
Submitte to Operations Research manuscript OPRE200909406 Optimal Energy Commitments with Storage an Intermittent Supply Jae Ho Kim Department of Electrical Engineering, Princeton University, Princeton,
More informationLecture L253D Rigid Body Kinematics
J. Peraire, S. Winall 16.07 Dynamics Fall 2008 Version 2.0 Lecture L253D Rigi Boy Kinematics In this lecture, we consier the motion of a 3D rigi boy. We shall see that in the general threeimensional
More informationAn intertemporal model of the real exchange rate, stock market, and international debt dynamics: policy simulations
This page may be remove to conceal the ientities of the authors An intertemporal moel of the real exchange rate, stock market, an international ebt ynamics: policy simulations Saziye Gazioglu an W. Davi
More informationFirewall Design: Consistency, Completeness, and Compactness
C IS COS YS TE MS Firewall Design: Consistency, Completeness, an Compactness Mohame G. Goua an XiangYang Alex Liu Department of Computer Sciences The University of Texas at Austin Austin, Texas 787121188,
More informationDigital barrier option contract with exponential random time
IMA Journal of Applie Mathematics Avance Access publishe June 9, IMA Journal of Applie Mathematics ) Page of 9 oi:.93/imamat/hxs3 Digital barrier option contract with exponential ranom time Doobae Jun
More informationWeb Appendices to Selling to Overcon dent Consumers
Web Appenices to Selling to Overcon ent Consumers Michael D. Grubb MIT Sloan School of Management Cambrige, MA 02142 mgrubbmit.eu www.mit.eu/~mgrubb May 2, 2008 B Option Pricing Intuition This appenix
More informationMathematics Review for Economists
Mathematics Review for Economists by John E. Floy University of Toronto May 9, 2013 This ocument presents a review of very basic mathematics for use by stuents who plan to stuy economics in grauate school
More informationWeek 1: Introduction to Online Learning
Week 1: Introduction to Online Learning 1 Introduction This is written based on Prediction, Learning, and Games (ISBN: 2184189 / 2184189 CesaBianchi, Nicolo; Lugosi, Gabor 1.1 A Gentle Start Consider
More informationThe Quick Calculus Tutorial
The Quick Calculus Tutorial This text is a quick introuction into Calculus ieas an techniques. It is esigne to help you if you take the Calculus base course Physics 211 at the same time with Calculus I,
More informationOptimal Control Policy of a Production and Inventory System for multiproduct in Segmented Market
RATIO MATHEMATICA 25 (2013), 29 46 ISSN:15927415 Optimal Control Policy of a Prouction an Inventory System for multiprouct in Segmente Market Kuleep Chauhary, Yogener Singh, P. C. Jha Department of Operational
More informationWitt#5e: Generalizing integrality theorems for ghostwitt vectors [not completed, not proofread]
Witt vectors. Part 1 Michiel Hazewinkel Sienotes by Darij Grinberg Witt#5e: Generalizing integrality theorems for ghostwitt vectors [not complete, not proofrea In this note, we will generalize most of
More informationModelling and Resolving Software Dependencies
June 15, 2005 Abstract Many Linux istributions an other moern operating systems feature the explicit eclaration of (often complex) epenency relationships between the pieces of software
More information(We assume that x 2 IR n with n > m f g are twice continuously ierentiable functions with Lipschitz secon erivatives. The Lagrangian function `(x y) i
An Analysis of Newton's Metho for Equivalent Karush{Kuhn{Tucker Systems Lus N. Vicente January 25, 999 Abstract In this paper we analyze the application of Newton's metho to the solution of systems of
More informationHull, Chapter 11 + Sections 17.1 and 17.2 Additional reference: John Cox and Mark Rubinstein, Options Markets, Chapter 5
Binomial Moel Hull, Chapter 11 + ections 17.1 an 17.2 Aitional reference: John Cox an Mark Rubinstein, Options Markets, Chapter 5 1. OnePerio Binomial Moel Creating synthetic options (replicating options)
More informationBV has the bounded approximation property
The Journal of Geometric Analysis volume 15 (2005), number 1, pp. 17 [version, April 14, 2005] BV has the boune approximation property G. Alberti, M. Csörnyei, A. Pe lczyński, D. Preiss Abstract: We prove
More informationThe Online Set Cover Problem
The Online Set Cover Problem Noga Alon Baruch Awerbuch Yossi Azar Niv Buchbinder Joseph Seffi Naor ABSTRACT Let X = {, 2,..., n} be a ground set of n elements, and let S be a family of subsets of X, S
More information2r 1. Definition (Degree Measure). Let G be a rgraph of order n and average degree d. Let S V (G). The degree measure µ(s) of S is defined by,
Theorem Simple Containers Theorem) Let G be a simple, rgraph of average egree an of orer n Let 0 < δ < If is large enough, then there exists a collection of sets C PV G)) satisfying: i) for every inepenent
More informationLowComplexity and Distributed Energy Minimization in Multihop Wireless Networks
LowComplexity an Distribute Energy inimization in ultihop Wireless Networks Longbi Lin, Xiaojun Lin, an Ness B. Shroff Center for Wireless Systems an Applications (CWSA) School of Electrical an Computer
More informationTHE SCHEDULING OF MAINTENANCE SERVICE
THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single
More informationThe most common model to support workforce management of telephone call centers is
Designing a Call Center with Impatient Customers O. Garnett A. Manelbaum M. Reiman Davison Faculty of Inustrial Engineering an Management, Technion, Haifa 32000, Israel Davison Faculty of Inustrial Engineering
More informationFOURIER TRANSFORM TERENCE TAO
FOURIER TRANSFORM TERENCE TAO Very broaly speaking, the Fourier transform is a systematic way to ecompose generic functions into a superposition of symmetric functions. These symmetric functions are usually
More informationAND. Dimitri P. Bertsekas and Paul Tseng
SET INTERSECTION THEOREMS AND EXISTENCE OF OPTIMAL SOLUTIONS FOR CONVEX AND NONCONVEX OPTIMIZATION Dimitri P. Bertsekas an Paul Tseng Math. Programming Journal, 2007 NESTED SET SEQUENCE INTERSECTIONS Basic
More informationM147 Practice Problems for Exam 2
M47 Practice Problems for Exam Exam will cover sections 4., 4.4, 4.5, 4.6, 4.7, 4.8, 5., an 5.. Calculators will not be allowe on the exam. The first ten problems on the exam will be multiple choice. Work
More informationUsing Stein s Method to Show Poisson and Normal Limit Laws for Fringe Subtrees
AofA 2014, Paris, France DMTCS proc. (subm., by the authors, 1 12 Using Stein s Metho to Show Poisson an Normal Limit Laws for Fringe Subtrees Cecilia Holmgren 1 an Svante Janson 2 1 Department of Mathematics,
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationDifferentiability of Exponential Functions
Differentiability of Exponential Functions Philip M. Anselone an John W. Lee Philip Anselone (panselone@actionnet.net) receive his Ph.D. from Oregon State in 1957. After a few years at Johns Hopkins an
More informationMannheim curves in the threedimensional sphere
Mannheim curves in the threeimensional sphere anju Kahraman, Mehmet Öner Manisa Celal Bayar University, Faculty of Arts an Sciences, Mathematics Department, Muraiye Campus, 5, Muraiye, Manisa, urkey.
More informationMODELLING OF TWO STRATEGIES IN INVENTORY CONTROL SYSTEM WITH RANDOM LEAD TIME AND DEMAND
art I. robobabilystic Moels Computer Moelling an New echnologies 27 Vol. No. 23 ransport an elecommunication Institute omonosova iga V9 atvia MOEING OF WO AEGIE IN INVENOY CONO YEM WIH ANOM EA IME AN
More information1 HighDimensional Space
Contents HighDimensional Space. Properties of HighDimensional Space..................... 4. The HighDimensional Sphere......................... 5.. The Sphere an the Cube in Higher Dimensions...........
More informationGame Theoretic Modeling of Cooperation among Service Providers in Mobile Cloud Computing Environments
2012 IEEE Wireless Communications an Networking Conference: Services, Applications, an Business Game Theoretic Moeling of Cooperation among Service Proviers in Mobile Clou Computing Environments Dusit
More informationConsumer Referrals. Maria Arbatskaya and Hideo Konishi. October 28, 2014
Consumer Referrals Maria Arbatskaya an Hieo Konishi October 28, 2014 Abstract In many inustries, rms rewar their customers for making referrals. We analyze the optimal policy mix of price, avertising intensity,
More informationRisk Adjustment for Poker Players
Risk Ajustment for Poker Players William Chin DePaul University, Chicago, Illinois Marc Ingenoso Conger Asset Management LLC, Chicago, Illinois September, 2006 Introuction In this article we consier risk
More informationMSc. Econ: MATHEMATICAL STATISTICS, 1995 MAXIMUMLIKELIHOOD ESTIMATION
MAXIMUMLIKELIHOOD ESTIMATION The General Theory of ML Estimation In orer to erive an ML estimator, we are boun to make an assumption about the functional form of the istribution which generates the
More informationWhich Networks Are Least Susceptible to Cascading Failures?
Which Networks Are Least Susceptible to Cascaing Failures? Larry Blume Davi Easley Jon Kleinberg Robert Kleinberg Éva Taros July 011 Abstract. The resilience of networks to various types of failures is
More informationRisk Management for Derivatives
Risk Management or Derivatives he Greeks are coming the Greeks are coming! Managing risk is important to a large number o iniviuals an institutions he most unamental aspect o business is a process where
More informationAn Introduction to Eventtriggered and Selftriggered Control
An Introuction to Eventtriggere an Selftriggere Control W.P.M.H. Heemels K.H. Johansson P. Tabuaa Abstract Recent evelopments in computer an communication technologies have le to a new type of largescale
More informationPythagorean Triples Over Gaussian Integers
International Journal of Algebra, Vol. 6, 01, no., 5564 Pythagorean Triples Over Gaussian Integers Cheranoot Somboonkulavui 1 Department of Mathematics, Faculty of Science Chulalongkorn University Bangkok
More informationNotes from Week 1: Algorithms for sequential prediction
CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 2226 Jan 2007 1 Introduction In this course we will be looking
More informationCURRENCY OPTION PRICING II
Jones Grauate School Rice University Masa Watanabe INTERNATIONAL FINANCE MGMT 657 Calibrating the Binomial Tree to Volatility BlackScholes Moel for Currency Options Properties of the BS Moel Option Sensitivity
More informationMeasures of distance between samples: Euclidean
4 Chapter 4 Measures of istance between samples: Eucliean We will be talking a lot about istances in this book. The concept of istance between two samples or between two variables is funamental in multivariate
More information6. Metric spaces. In this section we review the basic facts about metric spaces. d : X X [0, )
6. Metric spaces In this section we review the basic facts about metric spaces. Definitions. A metric on a nonempty set X is a map with the following properties: d : X X [0, ) (i) If x, y X are points
More informationAnswers to the Practice Problems for Test 2
Answers to the Practice Problems for Test 2 Davi Murphy. Fin f (x) if it is known that x [f(2x)] = x2. By the chain rule, x [f(2x)] = f (2x) 2, so 2f (2x) = x 2. Hence f (2x) = x 2 /2, but the lefthan
More informationIntegral Regular Truncated Pyramids with Rectangular Bases
Integral Regular Truncate Pyramis with Rectangular Bases Konstantine Zelator Department of Mathematics 301 Thackeray Hall University of Pittsburgh Pittsburgh, PA 1560, U.S.A. Also: Konstantine Zelator
More informationDynamic Network Security Deployment Under Partial Information
Dynamic Network Security Deployment Uner Partial nformation nvite Paper) George Theoorakopoulos EPFL Lausanne, Switzerlan Email: george.theoorakopoulos @ epfl.ch John S. Baras University of Marylan College
More informationParameterized Algorithms for dhitting Set: the Weighted Case Henning Fernau. Univ. Trier, FB 4 Abteilung Informatik 54286 Trier, Germany
Parameterize Algorithms for Hitting Set: the Weighte Case Henning Fernau Trierer Forschungsberichte; Trier: Technical Reports Informatik / Mathematik No. 086, July 2008 Univ. Trier, FB 4 Abteilung Informatik
More informationOptimal Control Of Production Inventory Systems With Deteriorating Items And Dynamic Costs
Applie Mathematics ENotes, 8(2008), 194202 c ISSN 16072510 Available free at mirror sites of http://www.math.nthu.eu.tw/ amen/ Optimal Control Of Prouction Inventory Systems With Deteriorating Items
More informationLecture 3: Linear Programming Relaxations and Rounding
Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can
More informationf(x) = a x, h(5) = ( 1) 5 1 = 2 2 1
Exponential Functions an their Derivatives Exponential functions are functions of the form f(x) = a x, where a is a positive constant referre to as the base. The functions f(x) = x, g(x) = e x, an h(x)
More informationEnterprise Resource Planning
Enterprise Resource Planning MPC 6 th Eition Chapter 1a McGrawHill/Irwin Copyright 2011 by The McGrawHill Companies, Inc. All rights reserve. Enterprise Resource Planning A comprehensive software approach
More information17.3.1 Follow the Perturbed Leader
CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.
More information19.2. First Order Differential Equations. Introduction. Prerequisites. Learning Outcomes
First Orer Differential Equations 19.2 Introuction Separation of variables is a technique commonly use to solve first orer orinary ifferential equations. It is socalle because we rearrange the equation
More information1 Introduction. 2 Prediction with Expert Advice. Online Learning 9.520 Lecture 09
1 Introduction Most of the course is concerned with the batch learning problem. In this lecture, however, we look at a different model, called online. Let us first compare and contrast the two. In batch
More informationIdeal Class Group and Units
Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals
More informationCHAPTER 5 : CALCULUS
Dr Roger Ni (Queen Mary, University of Lonon)  5. CHAPTER 5 : CALCULUS Differentiation Introuction to Differentiation Calculus is a branch of mathematics which concerns itself with change. Irrespective
More informationYishay Mansour Google & Tel Aviv Univ. Many thanks for my coauthors: A. Blum, N. CesaBianchi, and G. Stoltz
Regret Minimization: Algorithms and Applications Yishay Mansour Google & Tel Aviv Univ. Many thanks for my coauthors: A. Blum, N. CesaBianchi, and G. Stoltz 2 Weather Forecast Sunny: Rainy: No meteorological
More informationMinimumEnergy Broadcast in AllWireless Networks: NPCompleteness and Distribution Issues
MinimumEnergy Broacast in AllWireless Networks: NPCompleteness an Distribution Issues Mario Čagal LCAEPFL CH05 Lausanne Switzerlan mario.cagal@epfl.ch JeanPierre Hubaux LCAEPFL CH05 Lausanne Switzerlan
More informationThe Goldberg Rao Algorithm for the Maximum Flow Problem
The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }
More informationSafety Stock or Excess Capacity: Tradeoffs under Supply Risk
Safety Stock or Excess Capacity: Traeoffs uner Supply Risk Aahaar Chaturvei Victor MartínezeAlbéniz IESE Business School, University of Navarra Av. Pearson, 08034 Barcelona, Spain achaturvei@iese.eu
More informationA New Evaluation Measure for Information Retrieval Systems
A New Evaluation Measure for Information Retrieval Systems Martin Mehlitz martin.mehlitz@ailabor.e Christian Bauckhage Deutsche Telekom Laboratories christian.bauckhage@telekom.e Jérôme Kunegis jerome.kunegis@ailabor.e
More informationImage compression predicated on recurrent iterated function systems **
1 Image compression preicate on recurrent iterate function systems ** W. Metzler a, *, C.H. Yun b, M. Barski a a Faculty of Mathematics University of Kassel, Kassel, F. R. Germany b Faculty of Mathematics
More informationLecture 17: Implicit differentiation
Lecture 7: Implicit ifferentiation Nathan Pflueger 8 October 203 Introuction Toay we iscuss a technique calle implicit ifferentiation, which provies a quicker an easier way to compute many erivatives we
More informationUnsteady Flow Visualization by Animating EvenlySpaced Streamlines
EUROGRAPHICS 2000 / M. Gross an F.R.A. Hopgoo Volume 19, (2000), Number 3 (Guest Eitors) Unsteay Flow Visualization by Animating EvenlySpace Bruno Jobar an Wilfri Lefer Université u Littoral Côte Opale,
More informationFAST JOINING AND REPAIRING OF SANDWICH MATERIALS WITH DETACHABLE MECHANICAL CONNECTION TECHNOLOGY
FAST JOINING AND REPAIRING OF SANDWICH MATERIALS WITH DETACHABLE MECHANICAL CONNECTION TECHNOLOGY Jörg Felhusen an Sivakumara K. Krishnamoorthy RWTH Aachen University, Chair an Insitute for Engineering
More informationNotes on tangents to parabolas
Notes on tangents to parabolas (These are notes for a talk I gave on 2007 March 30.) The point of this talk is not to publicize new results. The most recent material in it is the concept of Bézier curves,
More informationFairness in Routing and Load Balancing
Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria
More informationLecture 13: Differentiation Derivatives of Trigonometric Functions
Lecture 13: Differentiation Derivatives of Trigonometric Functions Derivatives of the Basic Trigonometric Functions Derivative of sin Derivative of cos Using the Chain Rule Derivative of tan Using the
More informationUCLA STAT 13 Introduction to Statistical Methods for the Life and Health Sciences. Chapter 9 Paired Data. Paired data. Paired data
UCLA STAT 3 Introuction to Statistical Methos for the Life an Health Sciences Instructor: Ivo Dinov, Asst. Prof. of Statistics an Neurology Chapter 9 Paire Data Teaching Assistants: Jacquelina Dacosta
More informationA Data Placement Strategy in Scientific Cloud Workflows
A Data Placement Strategy in Scientific Clou Workflows Dong Yuan, Yun Yang, Xiao Liu, Jinjun Chen Faculty of Information an Communication Technologies, Swinburne University of Technology Hawthorn, Melbourne,
More informationThe oneyear nonlife insurance risk
The oneyear nonlife insurance risk Ohlsson, Esbjörn & Lauzeningks, Jan Abstract With few exceptions, the literature on nonlife insurance reserve risk has been evote to the ultimo risk, the risk in the
More informationOption Pricing for Inventory Management and Control
Option Pricing for Inventory Management an Control Bryant Angelos, McKay Heasley, an Jeffrey Humpherys Abstract We explore the use of option contracts as a means of managing an controlling inventories
More informationarxiv:1309.1857v3 [grqc] 7 Mar 2014
Generalize holographic equipartition for FriemannRobertsonWalker universes WenYuan Ai, Hua Chen, XianRu Hu, an JianBo Deng Institute of Theoretical Physics, LanZhou University, Lanzhou 730000, P.
More informationLagrangian and Hamiltonian Mechanics
Lagrangian an Hamiltonian Mechanics D.G. Simpson, Ph.D. Department of Physical Sciences an Engineering Prince George s Community College December 5, 007 Introuction In this course we have been stuying
More informationThe Multiplicative Weights Update method
Chapter 2 The Multiplicative Weights Update method The Multiplicative Weights method is a simple idea which has been repeatedly discovered in fields as diverse as Machine Learning, Optimization, and Game
More informationProduct Differentiation for SoftwareasaService Providers
University of Augsburg Prof. Dr. Hans Ulrich Buhl Research Center Finance & Information Management Department of Information Systems Engineering & Financial Management Discussion Paper WI99 Prouct Differentiation
More informationISSN: 22773754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 12, June 2014
ISSN: 77754 ISO 900:008 Certifie International Journal of Engineering an Innovative echnology (IJEI) Volume, Issue, June 04 Manufacturing process with isruption uner Quaratic Deman for Deteriorating Inventory
More informationConvex analysis and profit/cost/support functions
CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m
More informationOpen World Face Recognition with Credibility and Confidence Measures
Open Worl Face Recognition with Creibility an Confience Measures Fayin Li an Harry Wechsler Department of Computer Science George Mason University Fairfax, VA 22030 {fli, wechsler}@cs.gmu.eu Abstract.
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More informationnparameter families of curves
1 nparameter families of curves For purposes of this iscussion, a curve will mean any equation involving x, y, an no other variables. Some examples of curves are x 2 + (y 3) 2 = 9 circle with raius 3,
More informationLecture 7: Approximation via Randomized Rounding
Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining
More informationError Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach
Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and
More informationIPA Derivatives for MaketoStock ProductionInventory Systems With Lost Sales
IPA Derivatives for MaketoStock ProuctionInventory Systems With Lost Sales Yao Zhao Benjamin Melame Rutgers University Rutgers Business School Newark an New Brunswick Department of MSIS 94 Rockafeller
More informationIf you have ever spoken with your grandparents about what their lives were like
CHAPTER 7 Economic Growth I: Capital Accumulation an Population Growth The question of growth is nothing new but a new isguise for an ageol issue, one which has always intrigue an preoccupie economics:
More information