Online Learning for Global Cost Functions

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Online Learning for Global Cost Functions"

Transcription

1 Online Learning for Global Cost Functions Eyal Even-Dar Google Research Robert Kleinberg Cornell University Shie Mannor echnion an McGill University Yishay Mansour Google Research an el Aviv University Abstract We consier an online learning setting where at each time step the ecision maker has to choose how to istribute the future loss between k alternatives, an then observes the loss of each alternative Motivate by loa balancing an job scheuling, we consier a global cost function over the losses incurre by each alternative, rather than a summation of the instantaneous losses as one traitionally in online learning Such global cost functions inclue the makespan the maximum over the alternatives an the L norm over the alternatives Base on approachability theory, we esign an algorithm that guarantees vanishing regret for this setting, where the regret is measure with respect to the best static ecision that selects the same istribution over alternatives at every time step For the special case of makespan cost we evise a simple an efficient algorithm In contrast, we show that for concave global cost functions, such as L norms for <, the worst-case average regret oes not vanish Introuction We consier online learning with a global cost function Namely, a ecision maker who is facing an arbitrary environment is trying to minimize a cost function that epens on the entire history of his choices hence we term it a global cost function o motivate the iscussion, consier a job scheuling problem with a job that can be istribute to multiple machines an each job incurs a ifferent loss per machine At each stage the ecision maker first selects a istribution over machines an only then observes the losses loas of each Supporte by NSF grants CCF an CCF , an Alfre P Sloan Founation Fellowship, an a Microsoft Research New Faculty Fellowship Supporte by Horev Fellowship an by the ISF uner grant his work was supporte in part by the IS Programme of the European Community, uner the PASCAL2 Network of Excellence, by a grant from the Israel Science Founation an by a grant from Unite States-Israel Binational Science Founation BSF his publication reflects the authors views only machine he loss per machine as up weighte by the istribution the ecision-maker selecte he objective of the ecision maker is to have his makespan ie, the loss of the worst machine, or the infinity norm of the losses as small as possible he comparison class the ecision maker consiers is that of all static allocations an the ecision maker wants to minimize the ifference between the cost of the loss vector an the cost of the best static allocation he online learning or regret minimization framework have ha many successful applications in machine learning, game theory, an beyon; see [9] In spite of the avantages of this approach it suffers from some inherent limitations First, the size of the comparison class affects the magnitue of the regret although only in a logarithmic way Secon, the approach usually assumes that there is essentially no state of the system an that the ecision problem is the same at every stage see [4, 5, 6, 7, 0] for eviations from this stanar approach Finally, there is a tacit assumption that the objective function is aitive in the losses over the time steps his last assumption oes not hol in many natural problems such as job scheuling an loa balancing settings, for example While there are similarities in the motivation between online competitive algorithms [8] an the regret minimization setup, the moels have some significant ifferences First, the comparison class is substantially ifferent While competitive analysis allows the optimal scheuler to be arbitrary, regret minimization limits the size of the comparison class On the other han, the competitive analysis approach bouns only the ratio of the performances, while the regret minimization bouns the ifference between them Finally, there is also a very important ifference in the information moel In regret minimization the ecision maker first selects an alternative or a istribution over alternatives an only then observes the losses, while in the stanar online setup the algorithm first observes the loas losses an only then selects a machine alternative Our moel assumes a finite number of alternatives machines he information moel is similar to the regret minimization: the ecision maker first selects a istribution over machines alternatives an only then observes the losses loas We have a static comparison class, which inclues the algorithms that use the same istribution over the machines in every time step Our objective function is much in the spirit of the job scheuling, incluing objective functions such as makespan or L 2 -norm We boun the iffer-

2 ence between the performance of our online algorithm an the performance of the best static istribution with respect to a global cost function say makespan Note that as oppose to many regret minimization works, we consier not only the best alternative, but rather the best istribution over alternatives, so the comparison class in our setup is infinite Our contributions inclue the following: We present the online learning moel for a global cost his moel is natural in the online learning setup an it has not been stuie to the best of our knowlege We present an algorithm for global cost functions such as makespan, or any L -norm, for > that minimizes the ifference between the cost of our online algorithm an the best static istribution in a rate of O k/, where is the number of time steps Our algorithm only requires that the objective function is convex an Lipschitz an that the optimal cost is a concave an Lipschitz function Our general algorithm guaranteeing a vanishing average regret is base on approachability theory In spite of its goo rates, the explicit computation of the regret minimizing strategy oes not seem to be easy For the important special case of makespan cost function we provie a specialize simple eterministic online algorithm, which is both computationally efficient an has a regret boun of Ologk/, where k is the number of ifferent alternatives Our algorithm is base on a recursive construction where we first solve the case of two alternatives an then provie a recursive construction for any number of alternatives Our recursive construction maintains the computational efficiency of the two alternative algorithm, an its regret epens only logarithmically on the number of alternatives We complement our algorithms with an impossibility result We show that one cannot hope to have a similar result for any global cost function Specifically, we show for a wie class of concave function incluing any L -norm, for 0 < < that any online algorithm woul have Ω regret More specifically, we show that the ratio of the online cost to the best istribution cost is boune away from one Relate Work: Unfortunately, we can not give a comprehensive review of the large boy of research on regret minimization algorithm, an we refer the intereste reaer to [9] In the following we highlight a few more relevant research irections here has been an ongoing interest in extening the basic comparison class for the regret minimization algorithm, It is instructive to iscuss the ifference between minimizing an aitive loss as in the best experts an minimizing the makespan the alternative with the maximum loss Consier two alternatives an a sequence of ientical losses of 2, Minimizing the aitive loss woul always select alternative 2, an have cost Minimizing the makespan woul select a istribution /3, 2/3 an woul have cost 2/3 for example by introucing shifting experts [2] time selection functions [7] an wie range regret [6] Still, all those works assume that the loss is aitive between time steps A ifferent research irection has been to improve the computational complexity of the regret minimization algorithms, especially in the case that the comparison class is exponential in size General computationally efficient transformation where given by [5], where the cost function is linear an the optimization oracle can be compute in polynomial time, an extene by [4], to the case where we are given only an approximate-optimization oracle here has been a sequence of works establishing the connection between online competitive algorithms [8] an online learning algorithm [9] One issue is that online learning algorithms are stateless, while many of the problems aress in the competitive analysis literature have a state see, [4] For many problems one can use the online learning algorithms an guarantee a near-optimal static solution, however a straightforwar application requires both exponential time an space Computationally efficient solutions have been given to specific problems incluing, paging [5], atastructures [6], routing [2, ], averserial MDPs [, 8] an bin packing [3] In contrast to our work, all the above works concentrate on the case where the global cost function is aitive between time steps 2 Moel We consier an online learning setup where a ecision maker has a finite set K {,, k} of k alternatives machines to choose from At each time step t, the algorithm A selects a istribution αt A K over the set of alternatives K, where K is the set of istributions over K Following that the aversary selects a vector of losses loas l t [0, ] k he average loss of alternative i is enote by L i t l ti he average loss of the online algorithm A on alternative i is L A i t αa t il t i an its average loss vector is L A LA,, LA k Let the loss vector of a static allocation α K be L α α L where x y xy,, xkyk he objective of the online algorithm A is to minimize a global cost function CL A In traitional online learning the global cost C is assume to be an aitive function, ie, CL A k i LA i he main focus of this work is to consier alternative cost functions which arise in many natural problems such as job scheuling More specifically we consier the following norms: the makespan, C L A maxk i LA i, an the -norm cost, C L A k i LA i / for > 0 Note that for > the -norm cost is convex while for < it is concave Our comparison class is the class of static allocations for α K We efine the optimal cost function C L as the minimum over α K of CL α We enote by αc L a minimizing α K an calle the optimal

3 static allocation, 2 ie, C L min α K CLα min Cα L α K Cα CL L We enote by C an C the global optimal cost function C of the -norm cost an the makespan, respectively he regret of algorithm A at time is efine as, R A CL A C L Given a subset of the time steps B we enote by L B t B l t the average losses uring the time steps in B, an for an algorithm A we enote by L A B i t B αa t il t i its average loss uring B on alternative i, an by L A B L A B,, LA B k the average loss of A uring B 3 Preliminaries For the most part of the paper we will assume that the cost function, C, is a convex function while C is a concave function he following is immeiate from the efinition of the norms Lemma he -norm global cost function C, for >, an the makespan global cost function C are both convex an Lipschitz functions he challenging part is to show that the optimal cost function C is a concave function In Appenix A we prove this for -norm Lemma 20 an the makespan Lemma 2 Lemma 2 he -norm global optimal cost function C, for >, an the makespan global optimal cost function C are both concave an Lipschitz functions In Appenix A we erive for both the -norm an the makespan their optimal cost function an optimal static allocation Lemmas 9 an 22 Lemma 3 For the -norm global optimal cost function we have α C /L i P k j /L j i K an C L P k j /L j optimal cost function we have αc an C l P k j /L j, an for the makespan global /L i P k i /L j i K We now prove that given any partition of the losses to subsets, if we boun the regret in each subset, then the sum of the subsets regrets will boun the regret of the online algorithm 2 When the minimum is not unique we assume it can be selecte accoring to some preefine orer Lemma 4 Suppose that C is convex an C is concave Let i be a partition of the time steps to m sets, an let ON be an online algorithm such that for each i we have CL ON i C L i R i hen m CL ON C L R i i Proof: From the assumption in the theorem we have that m i CLON i m i C L i m i R i Since i is a partition, L m i L i an L ON m i LON i Since C is concave, C L m i C L i, an since C is convex, CL ON m i CLON i Combining the three inequalities erive the theorem he importance of Lemma 4 is the following If we are able to partition the time steps, in retrospect, an boun the regret in each part of the partition, then the above theorem states that the overall regret is boune by the sum of the local regret Remark: Our moel is essentially fractional, where the ecision maker selects a istribution, an each alternative receives exactly its fraction accoring to the istribution However, in a ranomize moel we can simulate our fractional moel, an use its istributions to run a ranomize algorithm Using stanar concentration bouns, the aitional loss per action, ue to the ranomization, is at most O log k/, with high probability ogether with the fact that C is Lipschitz we woul obtain similar results 4 Low regret approachability-base algorithm In this section we use Blackwell s approachability theory to construct a strategy that has zero asymptotic regret Recall that accoring to approachability, one consiers a vectorvalue repeate game he main question that is consiere is if the ecision maker can guarantee that the average vector-value rewar can be mae asymptotically close in norm sense to some esire target set We provie the essential backgroun an relevant results from approachability theory in Appenix B his section starts with construction of the game followe by a efinition of the target set an the application of approachability theory to prove that the set is approachable We finally escribe explicitly a regret minimizing strategy base on approachability theory Consier a repeate game against Nature where the ecision maker chooses at every stage an action α K an Nature chooses an action l [0, ] k he vector value rewar obtaine by the ecision maker is a 2k imensional vector whose first k coorinates are α l an the last k coorinates are l Let us enote this immeiate vector-value rewar by m t α t l t, l t It easily follows that the average rewar ˆm t m t/ L A, L We are now reay to efine the target set in this game S {x, y R k R k : x, y 0; Cx C y} Note that S is a set in R 2k We will show below that S is approachable see Appenix B We first claim that S is convex

4 Lemma 5 Suppose that C is convex an C is concave hen S is convex Proof:Suppose that x i, y i S for i, 2 an fix β [0, ] we let x i an y i enote vectors in R k Since C is convex we have that Cβx + βx 2 βcx + βcx 2 Since x i, y i S for i, 2 we have that Cx i C y i for i, 2 So that βcx + βcx 2 βc y + βc y 2 Using the fact that C is concave we obtain that βc y + βc y 2 C βy + βy 2 Combining the above inequalities we conclue that Cβx + βx 2 C βy + βy 2, completing the proof We are going to use Blackwell s approachability theory We note that in stanar approachability it is usually assume that the set of available actions is finite In our game, both players can choose from compact action spaces Namely, the ecision maker s action is α K an Nature s action is l [0, ] k Let p K enote a probability istribution over α an let q [0, ] k enote a probability istribution over l See Appenix B for further iscussion We can efine the expecte vector-value rewar as: mp, q α K heorem 6 he set S is approachable l [0,] k p α q l α l, l Proof:Accoring to Blackwell s theorem for convex sets heorem 24 in Appenix B it suffices to prove that for every q [0, ] k there exists a p K such that mp, q S Inee, consier some q [0, ] k an let Lq enote the mean of l uner q that is Lq q l l, an let p be a singleton whose mass is centere at αclq hen mp, q αc Lq Lq, Lq an is in S by efinition Since S is approachable let us explain how to approach it in a constructive way We follow Blackwell s first approachability theorem specialize for convex sets heorem 23 Given the current vector value rewar ˆm t L A t, L t we first compute the irection into the set S For that, we nee to solve the following convex optimization problem: min x,y R k L A t, L t x, y 2 2 st Cx C y x 0 Accoring to Blackwell s approaching strategy see Appenix B, if the solution to 2 is 0, ie, ˆm t S the ecision maker can act arbitrarily at time t If not, then let u x, y ˆm t where x, y are the minimizers of 2, an let us write u u α, u l to enote the two k imensional components of u We are looking for a mixe action p K that guarantees that the expecte rewar at time t + is on the other sie of the supporting hyperplane at x, y We know that such a mixe action exists since S is approachable an convex heorem 23 he optimization problem we try to solve is therefore: sup inf p K q [0,] k sup p K l [0,] k sup α K max α K min α α l p α q l u α α l + u l l 3 p α u α α l + u l l 4 min u l [0,] k α α l + u l l 5 min u l [0,] k α α l + u l l, 6 where 3 is the problem that we are trying to solve in the lifte space; Eq 4 hols since the epenence in l is linear in q so that for any p there is a singleton q which obtains the minimum; Eq 5 hols since p only affects Eq 4 linearly through α so that any measure over K can be replace with its mean; an Eq 6 hols because of continuity an the compactness of the sets we optimize on Now, the optimization problem of Eq 6, is a maximization of a convex function or alternatively, the solution of a zero-sum game with k actions for one player an 2 k for the other player an can be efficiently solve for a fixe k We summarize the above results in the following heorem heorem 7 he following strategy guarantees that the average regret goes to 0 At every stage t solve 2 2 If the solution is 0, play an arbitrary action 3 If the solution is more than 0, compute u x, y ˆm t where x, y are the minimizers of 2 an solve 6 Play a maximizing α Furthermore, the rate of convergence of the expecte regret is O k/ Proof:his is an approaching strategy to S he only missing part is to argue that approaching S implies minimizing the regret his is true by construction an since C an C are Lipschitz As for the rate of convergence for approachability: see the iscussion in Appenix B 5 Algorithms for Makespan In this section we present algorithms for the makespan global cost function We presnet algorithms, which are simple to implement, computationally efficient, an have an improve regret boun For k alternatives the average regret vanishes at the rate of Olog k /2 he construction of the algorithms is one in two parts he first part is to show an algorithm for the basic case of two alternatives Section 5 he secon part is a recursive construction, that buils an algorithm for 2 r alternatives from two algorithms for 2 r alternatives, an one algorithm for two alternatives he recursive construction essentially buils a complete binary tree, the main issue in the construction is to efine the losses that each algorithm observes Section 52

5 he DIFF Algorithm: At time t we have p p 2 /2, at time t 2 we have, p t+ p t + pt2lt2 ptlt, an p t 2 p t Figure : Algorithm for two alternatives 5 wo Alternatives he DIFF Algorithm: We enote the istribution of DIFF at time t by αt D p t, p t 2 Algorithm DIFF starts with p p 2 /2, an the upate of the istribution of DIFF is one as follows, p t+ p t + p t2l t 2 p t l t, an by efinition p t 2 p t Note that we are guarantee that p t [0, ], since p t p t We first provie some high level intuition Assume that in a certain subset of the time steps the algorithm uses approximately the same probabilities, say ρ, ρ an that the loa on the two alternative increase by exactly the same amount at time stpes his will imply that ρl L ρl 2, so it has to be the case that ρ 2 L +L 2 Note that the right han sie is the optimal static allocation for time steps Now, using Lemma 4, if we partition the time steps to such s, we will have low regret originating mainly from the fact that we are using approximately the same istribution ρ For the analysis sake, we present a sequence of moification to the input sequence Our objective is to guarantee a partition of the time steps to s with the above properties on the moifie sequence In aition the moifie sequence will ensure that the static optimal policy s performance is similar on both sequences First, we a O losses to get the DIFF algorithm to en in its initial state, where the ifference between the alternatives is 0 See Lemma 8 Secon, we consier a partition [0, ] action istribution to bins of size Θ /2 each We moify the sequence of losses to make sure that for each upate, DIFF s allocation istribution before an after the upate are in one bin, possibly by splitting a single upate to two upates See Lemma 0 he ith sub-sequence is the set of times where DIFF s action istribution is at the ith bin We show that the optimal action for each sub-sequence is close to the actions istribution performe in the bin recall that any two actions in the bins can iffer by at most Θ /2 See Lemma Since uring each the increase in the online loa on the alternatives is ientical, then DIFF action istribution is near optimal We complete the proof by summing over the subsequences See Lemma 2 an heorem 3 We woul abuse notation an given a sequence of losses l we let L D t l, i t τ p τ il τ i be the aggregate loss of alternative i, after t steps using the DIF F algorithm, an L D t l be the aggregate loss vector Let Λ t l L D t l, 2 L D t l, Equivalently, algorithm DIFF sets p t /2+ Λ t l/ /2 he proof will be one by a sequence of transformation of the sequences of losses l, where for each transformation we woul boun the effect on the online DIF F algorithm an the optimal static assignment α C l Given the sequence of losses l we exten the sequence by m aitional losses to a loss sequence ˆl such that the DIF F algorithm woul en with two perfectly balance losses, ie, Λ +m ˆl 0 Lemma 8 he following hols for ˆl: L D +k ˆl, L D +k ˆl, 2, 2 C L D +m ˆl C L D l, an 3 C ˆl C l + m where m 2 /2 Proof: Without loss of generality assume that Λ l > 0, then we a losses of the form, 0 After each such loss we have in the DIF F algorithm that p t+ p t his implies that for some m 2 /2 we have /2 p +m /2 If the last inequality is strict, then the last loss will be η, 0 for some η [0, ] such that p +m /2 As a by prouct of our proof, we obtain that the loss on the two alternatives is similar at time Corollary 9 Λ l 2 Consier a gri G which inclues 0 /2 points z i 0 /2 We say that the upate crosses the gri at point z i at time t if either p t < z i < p t or p t > z i > p t Let BIN i be [z i, z i+ ], an i be the set of time steps that either start or en in bin BIN i, ie, i {t : p t BIN i or p t BIN i } Each time step can be in at most two consecutive bins, since p t p t /2 We will moify the loss sequence ˆl to a loss sequence ˆb an an action istrbution sequence q such that in each bin, all the upates start an en insie the bin D will act accoring to q t at time t an will observe loss ˆb t We efine L D ˆb to be t ˆb t q t, t ˆb t 2q t 2 an for any subset of times steps, S, L D S ˆb t S ˆb t q t, t S ˆb t 2q t 2 Lemma 0 C L D ˆb C L D ˆl an C ˆb C ˆl Proof: If at time t we have that both p t an p t are containe in one bin, then we set ˆb 2t l t, p t an ˆb 2t 0, 0, p t Otherwise, p t an p t are containe in two consecutive bins, say BIN i an BIN i+ We a to ˆb two elements, ˆb 2t λl t, p t an ˆb 2t λl t, p t, such that p t +λp t 2l t 2 p t l t / /2 z i+ Since we replace each entry in ˆl by two entries in ˆb, this implies that the length of ˆb is at most 2 + /2 4 By efinition we have, L D ˆb, a L D +m ˆl, a an L A ˆb, a +m t ˆl t a L +m ˆl, a

6 Now we can efine a partition of the time steps in ˆb Let δ t w t 2q t 2 w t q t / Note that for upates which o not cut a bounary we have δ 2t p t+ p t, while for upates that cut the bounary we have δ 2t + δ 2t p t+ p t Let v t /2+ t i δ t We say that upate t in ˆb belongs to bin BIN i if v t, v t [z i, z i ] Note that by construction we have that every upate belongs to a unique bin Let i be all the time points in ˆb that belong to BIN i Note that t i δ t 0, ie the increase loa for the two alternatives in DIFF is the same Since { i } is a partition of the inexes of ˆb, by Lemma 4 we can lower boun the optimal cost by C L D 2 +m ˆb C ˆb 0 /2 i Our main goal is to boun the ifference C L D i ˆb C ˆb i C L D i ˆb C ˆb i Given the losses at time steps i of ˆb, we enote static optimal allocation, α C ˆb i, by q i, q i Lemma q i z i 30 /2 Proof: For the upates in i we have that t i δ t 0 Consier the function, fx t i x w t t i xw t 2 xl i + L i 2 L i 2 First, for qi we have that fq i 0 Secon, since for any t i we have that p t [z i, z i+2 ] then fz i t i δ t 0 fz i+2 herefore qi [z i, z i+2 ] Since, z i+2 z i 30 /2, we have that z i qi 30 /2 Lemma 2 0 /2 i C L D i ˆb C ˆb i 240 /2 Proof: By Lemma for any t i we have that p t q i 30 /2 his implies that, C L D i ˆb C ˆb i 30 /2 max{l i ˆb,, L i ˆb, 2} an therefore 0 /2 i C L D i ˆb C ˆb i / /2 i 30 / /2 max{l i ˆb,, L i ˆb, 2} he MULI Algorithm A K 2 r: Alternative set: K of size 2 r partitione to I an J, I J 2 r Proceures: A 2, A I 2 r, A J 2 r Action selection: Algorithm A I 2 r gives x t I Algorithm A J 2 r gives x t J Algorithm A 2 gives z t, z t Output α t I J where, for i I set α t i x t iz t, an for j J set α t j y t j z t Loss istribution: Given the loss l t Algorithm A I 2 r receives l t I Algorithm A J 2 r receives l t J Algorithm A 2 receives ω t I, ω t J, where ω t I 2 r+ i I l tix t i, an ω t J 2 r+ j J l tjy t j Figure 2: Algorithm for multiple alternatives heorem 3 For any loss sequence l, we have that C L D l C l 24 /2 Proof: Given the loss sequence l, we efine the loss sequence ˆl By Lemma 8 we have that C L D +m ˆl C L D l an C ˆl C l+ /2 By Lemma 0 we have that C ˆl C ˆb, an C L D +m ˆl C L D 2 +m ˆb From Lemma 2 we have that 0 /2 i C L D i ˆb C i 240 /2 By Lemma 4 an Lemma 22, we can boun the sum by the global function, hence we have that C L D 2 +m ˆb C ˆb 240 /2 herefore, C L D l C l 24 /2 52 Multiple alternatives In this section we show how to transform a low regret algorithm for two alternatives to one that can hanle multiple alternatives For the base case we assume that we are given an online algorithm A 2, such that for any loss sequence l,, l over two alternatives, guarantees that its average regret is at most α/ Using A 2 we will buil a sequence of algorithms A 2 r, such that for any loss sequence l,, l over 2 r alternatives, A 2 r guarantees that its average regret is at most Oα r/ Our basic step in the construction is to buil an algorithm A 2 r using two instances of A 2 r an one instance of A 2 A 2 r woul work as follows We partition the set of actions K to two subsets of size 2 r, enote by I an J; for each subset we create a copy of A 2 r which we enote by A I 2 r an A J 2 he thir instance A M r 2 woul receive as an input the average loss of A I 2 an A J r 2 Let us be more precise about the construction of A r 2 r At each time step t, A 2 r receives a istribution x t I from A I 2, a istribution y r t J from A J 2 an r

7 probability z t [0, ] from A M 2 he istribution of A 2 r at time t, α t I J, is efine as follows For actions i I we set α t i x t iz t, an for actions j J we set α t j y t j z t Given the action α t, A 2 r observes a loss vector l t It then provies A I 2 with the r loss vector l t i i I, A J 2 with the loss vector l r t j j J, an A M 2 with the loss vector ω t I, ω t J, where ω t I 2 r+ i I l tix t i an ω t J 2 r+ j J l tjy t j he tricky part in the analysis is to relate the sequences x t, y t an z t to the actual aggregate loss of the iniviual alternatives We will o this in two steps First we will boun the ifference between the average loss on the alternatives, an the optimal solution hen we will show that the losses on the various alternatives are approximately balance, bouning the ifference between the maximum an the minimum loss he input to algorithm A M 2 on its first alternative generate by A I 2 has an average loss r [ W I ω t I 2 ] r+ x t il t i, t t i I similarly, on the secon alternative it has an average loss W J ω t J 2 r+ y t jl t j t t j J First, we erive the following technical lemma its proof is in Appenix C Lemma 4 Assume that A M 2 hen W I J W I + W J is at hight r part of A I J 2 r + α a I J + rα L a he following claim, base on Corrollary 9, bouns the ifference between the average of all alternatives an the loss of a specific alternative Claim 5 For any alternative i K we have, W K L A 2 r i r Proof: We can view the algorithm A 2 r as generating a complete binary tree Consier a path from the root to a noe that represents alternative i K Let the sets of alternatives on the path be I 0,, I r, where I 0 K an I r {i} We are intereste in bouning W I 0 W I r Clearly, r W I 0 W I r W I i W I i i Since W I i 2 W I i + W I i I i, we have that W I i W I i 2 W I i W I i I i Since the alternative sets I i an I i I i are the alternatives of an A M 2 algorithm at epth i, by Corollary 9 we have that W I i W I i I i 2 his implies that W I i W I i herefore, W I 0 W I r r Now we can combine the two claims to the following theorem heorem 6 Suppose the global cost function is makespan For any set K of 2 r alternatives, an for any loss sequence l,, l, algorithm A 2 r will have regret at most O log K, ie, C L A 2 r C l 242 r Proof: By Lemma 4 we have that W K C l rα By Claim 5 we have that for any alternative i K, L A 2 r i W K r herefore, C L A 2 r C l 242 r, since α 24 by heorem 3 6 Lower Boun for Non-Convex functions In this section we show a lower boun that hols for a large range of non-convex global cost functions, efine as follows Definition 7 A function f is γ f -strictly concave if for every x > 0: fx/3+f2x/3 fx γ f >, 2 lim x 0 xf x 0 3 f is non-negative concave an increasing 4 lim x 0 fx 0 Note that any function fx x β for β <, is γ f -strictly concave with γ f 2 3 β + 3 β heorem 8 Let CL A k i fla i, where f is γ f - strictly concave For any online algorithm A there is an input sequence such that CL A /C L > γ f he proof can be foun in Appenix D 7 Open Problems In this work we efine the setting of an online learning with a global cost function C For the case that C is convex an C is concave, we showe that there are online algorithms with vanishing regret On the other han, we showe that for certain concave functions C the regret is not vanishing Giving a complete characterization when oes a cost function C enable a vanishing regret is very challenging open problem In this section we outline some possible research irections to aress this problem First, if C is such that for some monotone function g, gc satisfy the same properties as -norm: C is convex, C L min α C L α is concave, an α L is Lipschitz, the algorithm of Section 4 for C woul still lea C to vanishing regret For example, CL k i L i can be solve with that approach A more interesting case is when the properties are violate Obviously, in light of the lower boun of Section 6, attaining C in all cases is impossible he question is if there is some more relaxe goal that can be attaine As in [7], one can take the convex hull of the target set in Eq an still consier an approaching strategy It can be shown that by using the approachability strategy to the convex hull, the ecision maker minimizes the regret obtaine with respect to the function C C efine below ie, R CA CLA CC L Specifically: C C L sup L,L 2,,L m, β,β m, β j 0: P βj, P m j βj L j L m C β j αcl j L j j

8 It follows that C L C C L an C L C C L if C is convex an C concave since in that case the convex hull of S equals S In general, however, C C may be strictly larger than C he question if C C is the lowest cost that can be attaine against any loss sequence is left for future research A ifferent open problem is a computational one he approachability-base scheme requires projecting a point to a convex set Even if we settle for ɛ projection which woul lea to a vanishing regret up to an ɛ, the computational complexity oes not seem to scale better than /ɛ k It woul be interesting to unerstan if the complexity minimizing the regret for cost functions other than the makespan can have better epenence on ɛ an k In particular, it woul be interesting to evise regret minimization algorithms for -norm cost functions References [] B Awerbuch an R Kleinberg Online linear optimization an aaptive routing J Comput Syst Sci, 74:97 4, 2008 [4] S M Kakae, A auman-kalai, an K Ligett Playing games with approximation algorithms In SOC, pages , 2007 [5] A Kalai an S Vempala Efficient algorithms for online ecision problems Journal of Computer an System Sciences, 73:29 307, 2005 An earlier version appeare in COL 2003 [6] E Lehrer A wie range no-regret theorem Games an Economic Behavior, 42:0 5, 2003 [7] S Mannor an N Shimkin he empirical Bayes envelope an regret minimization in competitive Markov ecision processes MOR, 282: , 2003 [8] J-Y Yu, S Mannor, an N Shimikin Markov ecision processes with arbitrary rewar processes Mathematics of Operations Research, 2009 [2] B Awerbuch an Y Mansour Aapting to a reliable network path In PODC, pages , 2003 [3] D Blackwell An analog of the minimax theorem for vector payoffs Pacific J Math, 6: 8, 956 [4] A Blum an C Burch On-line learning an the metrical task system problem In COL, pages 45 53, 997 [5] A Blum, C Burch, an A Kalai Finely-competitive paging In FOCS, pages , 999 [6] A Blum, S Chawla, an A Kalai Static optimality an ynamic search-optimality in lists an trees Algorithmica, 363: , 2003 [7] A Blum an Y Mansour From external to internal regret Journal of Machine Learning Research, 8: , 2007 [8] A Boroin an R El-Yaniv Online Computation an Competitive Analysis Cambrige University Press, 998 [9] N Cesa-Bianchi an G Lugosi Preiction, Learning, an Games Cambrige University Press, New York, 2006 [0] D P e Farias an N Megio Combining expert avice in reactive environments Journal of the ACM, 535: , 2006 [] E Even-Dar, S M Kakake, an Y Mansour On-line markov ecision processes Mathematics of Operations Research, 2009 [2] Y Freun, R Schapire, Y Singer, an M Warmuth Using an combining preictors that specialize In SOC, pages , 997 [3] A Gyrgy, G Lugosi, an G Ottucsk On-line sequential bin packing In COL, pages , 2008

9 A Properties of Norms We are now reay to exten the results to any norm Let us start by computing what is the optimal assignment an what is its cost Lemma 9 Consier the -norm global cost function an a sequence of losses l,, l he optimal stationary istribution is given by α C l C l /L i P k j /L j optimal global cost function is given by k j /L j i K an the Proof: Recall that L,, L k are the average losses of the alternatives o compute the optimal norm static istribution, we nee to optimize the following cost function k / C l i αil i subject to the constraint that k i αi Using Lagrange multipliers we obtain that the optimum is obtaine for the following values α C li αi /L i k j /L j his implies that the cost of the optimal istribution is given by C L i αil i / /L i k j L /L j i i i L i k j /L j k j /L j k j /L j i / L i / / After computing C we are reay to prove its concavity Lemma 20 he function C l is concave Proof:In orer to show that C is concave we once again will prove that its Hessian is semiefinite negative We first compute the erivative an the partial erivatives of orer 2, we use for notation β > an γ x P k j /Lβ j thus C x γ x /β C xi β γ x /β βxi β+ k xi β+ γ x β + m 2 xm β 2 C xi xj 2 C 2 xi xi β+ β + γ x /β βxj β+ k 2 m xm β β + γ x β +2 xi β+ xj β+ β + xi β+2 γ x β + +xi β+ β + γ x β β + xi β+2 γ x β + +β + xi 2β+ γ x β +2 βxi β+ k 2 m xm β β + γ x β +2 xm β + xi 2β+ xi β+2 m β + γ x β +2 xi β+2 m,k i xm β Since all entries of the Hessian are factor of β+γ x β +2 which is positive we can eliminate it without effecting the semiefinte negative property aha i ji+ + 2xi β+ xj β+ a i a j a 2 i xi β+2 i m,k i i ji+ a 2 i xi β+2 xj β a 2 jxj β+2 xi β +2xi β+ xj β+ a i a j i ji+ a2 i xi 2 i ji+ xi β xj β a2 j xj 2 + 2a ia j xixj xi β xj β a i xi Lemma 2 he function C is concave xm β a j xj 2 0 Proof: In orer to o so we will show that the Hessian of C is semi efinite positive We start by computing the partial

10 erivatives of C C xi /xi 2 2C k j /xj2 xi xj 2/xi2 xj 2 2C, k j /xj3 xi 2 xi 3 xj 2 j i k j /xj3 Next we show that aha < 0 where is the Hessian an a is any vector o simplify the computation we eliminate the common factor 2/ k j /xj3 which is positive from all entries aha i ji+ i ji+ a2 i xi 2 + i ji+ 2a i a j xi 2 xj 2 xixj i j i 2a ia j xixj a2 j xj 2 ai xixj xi a 2 i xi 3 xj a 2 j 0 xj herefore, we showe that C is concave an we conclue the proof Consier the makespan metric which is the stanar metric for loa balancing an job scheuling tasks Lemma 22 Consier the makespan global cost function an a sequence of losses l,, l he optimal stationary istribution is given by αc /L l i P k j /L j i K an the optimal global cost function is given by Cl k j /L j B Approachability theory In this appenix we provie the basic results from approachability theory in the context of compact action sets We start with the efinition of the game Consier a repeate vectorvalue game where player chooses actions in A R a an player 2 chooses actions in B R b We assume both A an B are compact We further assume that there exists a continuous vector-value rewar function m : A B R k At every stage t of the game, player selects action a t A an player 2 selects action b t B as a result player obtains a vector-value rewar m t ma t, b t he objective of player is to make sure that the average vector-value rewar ˆm t t m t t τ converges to a target set R k More precisely, we say that a set is approachable if there exists a strategy for player such that for every ɛ > 0, there exists an integer N such that, for every opponent strategy: Pr ist ˆm t, ɛ for some n N < ɛ, where ist is the point-to-set Eucliean istance We further consier mixe actions for both player A mixe action for player is a probability istribution over A We enote such a istribution by A Similarly, we enote by B the set of probability istributions over B We consier the lifting of m from A B to A B by extening m in the natural way: for p A an q B we efine the expecte rewar as: mp, q p a q b ma, b a b A B he following is the celebrate Blackwell heorem for convex sets he proof is essentially ientical to heorem 75 of [9] an is therefore omitte heorem 23 A convex set V is approachable if an only if for every u R k with u 2 we have that: sup p inf q mp, q u inf x u, x V where is the stanar inner prouct in R k he following strategy can be shown to have a convergence rate of O k/t as in [9], page 230 At time t calculate s t arg min s V ˆm t s 2 If s t ˆm t act arbitrarily If s t ˆm t play p A that satisfies: inf q mp, q s t ˆm t s t s t ˆm t Such a p must exist or else the set V is not approachable accoring to heorem 23 he following theorem is a variant of Blackwell s original theorem for convex sets heorem 3 in [3] heorem 24 A close convex set V is approachable if an only if for every q B there exists p A such that mp, q V Proof:he proof is ientical to Blackwell s original proof, noticing that for every q the set {mp, q, p A} is a close convex set C Proof of Lemma 4 Let W W I an W 2 W J he first inequality follows from the fact that our assumption of the regret of A M 2 implies that max{w, W 2 } /W +/W 2 + α his also proves the base of the inuction for r For the proof let π i I L i, π 2 j J L j an π π π 2 Also, s i I k I\{i} L k, s 2 j J k J\{j} L k, an s s π 2 + s 2 π One can verify that the optimal cost opt i /L i equals to π/s

11 Let β r α From the inuctive assumption that we have W i π i /s i + β for i {, 2} he proof follows from the following + α /W + /W 2 π + /s +β π 2/s 2+β s π + s2 +s β π 2+s 2β + α + α π + s βπ 2 + s 2 β s π 2 + s 2 β + s 2 π + s β + α π π 2 + βs π 2 + s 2 π + β 2 s s 2 s π 2 + s 2 π + 2s 2 s β π + βs + β2 s s 2 s + 2s 2 s β π s + β s + βs s 2 s + 2s 2 s β + + α α opt + β + α opt + r α + α hus hλ is increasing an its minimum is obtaine at λ 0 herefore the cost function of algorithm, A, is at least f2δ/3 + fδ/3 while the optimum achieves fδ Hence CL A /C σ 2 γ f he case of β < /3 is similar to that of β > 2/3, just using σ 3 rather than σ 2 D Lower boun for a concave loss function Proof:[heorem 8] Consier the following three sequences of losses: sequence σ has t time steps each with losses δ, δ, 2 sequence σ 2 starts with sequence σ followe by t time steps each with losses, δ, 3 sequence σ 3 starts with sequence σ followe by t time steps each with losses δ, an δ will be etermine later he opt for all sequences σ, σ 2 an σ 3 has a loss of fδ Given an online algorithm A, after σ it has an average loss of βδ on alternative an βδ on alternative 2 herefore, its loss on σ is fβδ + f βδ If β [/3, 2/3] then the loss of A is at least f/3δ+f2/3δ compare to a loss of fδ of OP Hence CL A /C σ γ f If β > 2/3 then consier the performance of A on σ 2 On the first t time steps it behaves the same as for σ an at the remaining t time steps it splits the weight such that its average loss on those steps is λ for alternative an λδ for alternative 2 herefore its cost function is fβδ + λ/2 + f β + λδ/2 First, we note that for λ > /2, we have that CL A f/4 Since the cost of OP is fδ an as δ goes to 0 C σ 2 goes to zero, then for sufficiently small δ we have that CL A /C σ 2 γ f For λ /2 efine hλ fβδ + λ/2 + f β + λδ/2δ/2 First, we woul like to show that h λ > 0 for every λ [0, /2] Consier, h λ f βδ + λ/2/2 δ 2 f β + λδ/2 > 0 Since f is concave f is ecreasing an given our assumption that δf δ/2 goes to zero as δ goes to zero, then for small enough δ an λ /2 we can boun as follows: f βδ + λ/2 f 4δf δ/4 δf β + λδ/2

Online Learning for Global Cost Functions

Online Learning for Global Cost Functions Online Learning for Global Cost Functions Eyal Even-Dar Google Research evendar@googlecom Robert Kleinberg Cornell University rdk@cscornelledu Yishay Mansour el Aviv University mansour@tauacil Shie Mannor

More information

A Comparison of Performance Measures for Online Algorithms

A Comparison of Performance Measures for Online Algorithms A Comparison of Performance Measures for Online Algorithms Joan Boyar 1, Sany Irani 2, an Kim S. Larsen 1 1 Department of Mathematics an Computer Science, University of Southern Denmark, Campusvej 55,

More information

On Adaboost and Optimal Betting Strategies

On Adaboost and Optimal Betting Strategies On Aaboost an Optimal Betting Strategies Pasquale Malacaria 1 an Fabrizio Smerali 1 1 School of Electronic Engineering an Computer Science, Queen Mary University of Lonon, Lonon, UK Abstract We explore

More information

A Generalization of Sauer s Lemma to Classes of Large-Margin Functions

A Generalization of Sauer s Lemma to Classes of Large-Margin Functions A Generalization of Sauer s Lemma to Classes of Large-Margin Functions Joel Ratsaby University College Lonon Gower Street, Lonon WC1E 6BT, Unite Kingom J.Ratsaby@cs.ucl.ac.uk, WWW home page: http://www.cs.ucl.ac.uk/staff/j.ratsaby/

More information

Adaptive Online Gradient Descent

Adaptive Online Gradient Descent Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

More information

Factoring Dickson polynomials over finite fields

Factoring Dickson polynomials over finite fields Factoring Dickson polynomials over finite fiels Manjul Bhargava Department of Mathematics, Princeton University. Princeton NJ 08544 manjul@math.princeton.eu Michael Zieve Department of Mathematics, University

More information

Math 230.01, Fall 2012: HW 1 Solutions

Math 230.01, Fall 2012: HW 1 Solutions Math 3., Fall : HW Solutions Problem (p.9 #). Suppose a wor is picke at ranom from this sentence. Fin: a) the chance the wor has at least letters; SOLUTION: All wors are equally likely to be chosen. The

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS Contents 1. Moment generating functions 2. Sum of a ranom number of ranom variables 3. Transforms

More information

Chapter 2 Review of Classical Action Principles

Chapter 2 Review of Classical Action Principles Chapter Review of Classical Action Principles This section grew out of lectures given by Schwinger at UCLA aroun 1974, which were substantially transforme into Chap. 8 of Classical Electroynamics (Schwinger

More information

Sensor Network Localization from Local Connectivity : Performance Analysis for the MDS-MAP Algorithm

Sensor Network Localization from Local Connectivity : Performance Analysis for the MDS-MAP Algorithm Sensor Network Localization from Local Connectivity : Performance Analysis for the MDS-MAP Algorithm Sewoong Oh an Anrea Montanari Electrical Engineering an Statistics Department Stanfor University, Stanfor,

More information

Ch 10. Arithmetic Average Options and Asian Opitons

Ch 10. Arithmetic Average Options and Asian Opitons Ch 10. Arithmetic Average Options an Asian Opitons I. Asian Option an the Analytic Pricing Formula II. Binomial Tree Moel to Price Average Options III. Combination of Arithmetic Average an Reset Options

More information

Sensitivity Analysis of Non-linear Performance with Probability Distortion

Sensitivity Analysis of Non-linear Performance with Probability Distortion Preprints of the 19th Worl Congress The International Feeration of Automatic Control Cape Town, South Africa. August 24-29, 214 Sensitivity Analysis of Non-linear Performance with Probability Distortion

More information

10.2 Systems of Linear Equations: Matrices

10.2 Systems of Linear Equations: Matrices SECTION 0.2 Systems of Linear Equations: Matrices 7 0.2 Systems of Linear Equations: Matrices OBJECTIVES Write the Augmente Matrix of a System of Linear Equations 2 Write the System from the Augmente Matrix

More information

JON HOLTAN. if P&C Insurance Ltd., Oslo, Norway ABSTRACT

JON HOLTAN. if P&C Insurance Ltd., Oslo, Norway ABSTRACT OPTIMAL INSURANCE COVERAGE UNDER BONUS-MALUS CONTRACTS BY JON HOLTAN if P&C Insurance Lt., Oslo, Norway ABSTRACT The paper analyses the questions: Shoul or shoul not an iniviual buy insurance? An if so,

More information

Data Center Power System Reliability Beyond the 9 s: A Practical Approach

Data Center Power System Reliability Beyond the 9 s: A Practical Approach Data Center Power System Reliability Beyon the 9 s: A Practical Approach Bill Brown, P.E., Square D Critical Power Competency Center. Abstract Reliability has always been the focus of mission-critical

More information

Web Appendices of Selling to Overcon dent Consumers

Web Appendices of Selling to Overcon dent Consumers Web Appenices of Selling to Overcon ent Consumers Michael D. Grubb A Option Pricing Intuition This appenix provies aitional intuition base on option pricing for the result in Proposition 2. Consier the

More information

State of Louisiana Office of Information Technology. Change Management Plan

State of Louisiana Office of Information Technology. Change Management Plan State of Louisiana Office of Information Technology Change Management Plan Table of Contents Change Management Overview Change Management Plan Key Consierations Organizational Transition Stages Change

More information

Optimal Energy Commitments with Storage and Intermittent Supply

Optimal Energy Commitments with Storage and Intermittent Supply Submitte to Operations Research manuscript OPRE-2009-09-406 Optimal Energy Commitments with Storage an Intermittent Supply Jae Ho Kim Department of Electrical Engineering, Princeton University, Princeton,

More information

Lecture L25-3D Rigid Body Kinematics

Lecture L25-3D Rigid Body Kinematics J. Peraire, S. Winall 16.07 Dynamics Fall 2008 Version 2.0 Lecture L25-3D Rigi Boy Kinematics In this lecture, we consier the motion of a 3D rigi boy. We shall see that in the general three-imensional

More information

An intertemporal model of the real exchange rate, stock market, and international debt dynamics: policy simulations

An intertemporal model of the real exchange rate, stock market, and international debt dynamics: policy simulations This page may be remove to conceal the ientities of the authors An intertemporal moel of the real exchange rate, stock market, an international ebt ynamics: policy simulations Saziye Gazioglu an W. Davi

More information

Firewall Design: Consistency, Completeness, and Compactness

Firewall Design: Consistency, Completeness, and Compactness C IS COS YS TE MS Firewall Design: Consistency, Completeness, an Compactness Mohame G. Goua an Xiang-Yang Alex Liu Department of Computer Sciences The University of Texas at Austin Austin, Texas 78712-1188,

More information

Digital barrier option contract with exponential random time

Digital barrier option contract with exponential random time IMA Journal of Applie Mathematics Avance Access publishe June 9, IMA Journal of Applie Mathematics ) Page of 9 oi:.93/imamat/hxs3 Digital barrier option contract with exponential ranom time Doobae Jun

More information

Web Appendices to Selling to Overcon dent Consumers

Web Appendices to Selling to Overcon dent Consumers Web Appenices to Selling to Overcon ent Consumers Michael D. Grubb MIT Sloan School of Management Cambrige, MA 02142 mgrubbmit.eu www.mit.eu/~mgrubb May 2, 2008 B Option Pricing Intuition This appenix

More information

Mathematics Review for Economists

Mathematics Review for Economists Mathematics Review for Economists by John E. Floy University of Toronto May 9, 2013 This ocument presents a review of very basic mathematics for use by stuents who plan to stuy economics in grauate school

More information

Week 1: Introduction to Online Learning

Week 1: Introduction to Online Learning Week 1: Introduction to Online Learning 1 Introduction This is written based on Prediction, Learning, and Games (ISBN: 2184189 / -21-8418-9 Cesa-Bianchi, Nicolo; Lugosi, Gabor 1.1 A Gentle Start Consider

More information

The Quick Calculus Tutorial

The Quick Calculus Tutorial The Quick Calculus Tutorial This text is a quick introuction into Calculus ieas an techniques. It is esigne to help you if you take the Calculus base course Physics 211 at the same time with Calculus I,

More information

Optimal Control Policy of a Production and Inventory System for multi-product in Segmented Market

Optimal Control Policy of a Production and Inventory System for multi-product in Segmented Market RATIO MATHEMATICA 25 (2013), 29 46 ISSN:1592-7415 Optimal Control Policy of a Prouction an Inventory System for multi-prouct in Segmente Market Kuleep Chauhary, Yogener Singh, P. C. Jha Department of Operational

More information

Witt#5e: Generalizing integrality theorems for ghost-witt vectors [not completed, not proofread]

Witt#5e: Generalizing integrality theorems for ghost-witt vectors [not completed, not proofread] Witt vectors. Part 1 Michiel Hazewinkel Sienotes by Darij Grinberg Witt#5e: Generalizing integrality theorems for ghost-witt vectors [not complete, not proofrea In this note, we will generalize most of

More information

Modelling and Resolving Software Dependencies

Modelling and Resolving Software Dependencies June 15, 2005 Abstract Many Linux istributions an other moern operating systems feature the explicit eclaration of (often complex) epenency relationships between the pieces of software

More information

(We assume that x 2 IR n with n > m f g are twice continuously ierentiable functions with Lipschitz secon erivatives. The Lagrangian function `(x y) i

(We assume that x 2 IR n with n > m f g are twice continuously ierentiable functions with Lipschitz secon erivatives. The Lagrangian function `(x y) i An Analysis of Newton's Metho for Equivalent Karush{Kuhn{Tucker Systems Lus N. Vicente January 25, 999 Abstract In this paper we analyze the application of Newton's metho to the solution of systems of

More information

Hull, Chapter 11 + Sections 17.1 and 17.2 Additional reference: John Cox and Mark Rubinstein, Options Markets, Chapter 5

Hull, Chapter 11 + Sections 17.1 and 17.2 Additional reference: John Cox and Mark Rubinstein, Options Markets, Chapter 5 Binomial Moel Hull, Chapter 11 + ections 17.1 an 17.2 Aitional reference: John Cox an Mark Rubinstein, Options Markets, Chapter 5 1. One-Perio Binomial Moel Creating synthetic options (replicating options)

More information

BV has the bounded approximation property

BV has the bounded approximation property The Journal of Geometric Analysis volume 15 (2005), number 1, pp. 1-7 [version, April 14, 2005] BV has the boune approximation property G. Alberti, M. Csörnyei, A. Pe lczyński, D. Preiss Abstract: We prove

More information

The Online Set Cover Problem

The Online Set Cover Problem The Online Set Cover Problem Noga Alon Baruch Awerbuch Yossi Azar Niv Buchbinder Joseph Seffi Naor ABSTRACT Let X = {, 2,..., n} be a ground set of n elements, and let S be a family of subsets of X, S

More information

2r 1. Definition (Degree Measure). Let G be a r-graph of order n and average degree d. Let S V (G). The degree measure µ(s) of S is defined by,

2r 1. Definition (Degree Measure). Let G be a r-graph of order n and average degree d. Let S V (G). The degree measure µ(s) of S is defined by, Theorem Simple Containers Theorem) Let G be a simple, r-graph of average egree an of orer n Let 0 < δ < If is large enough, then there exists a collection of sets C PV G)) satisfying: i) for every inepenent

More information

Low-Complexity and Distributed Energy Minimization in Multi-hop Wireless Networks

Low-Complexity and Distributed Energy Minimization in Multi-hop Wireless Networks Low-Complexity an Distribute Energy inimization in ulti-hop Wireless Networks Longbi Lin, Xiaojun Lin, an Ness B. Shroff Center for Wireless Systems an Applications (CWSA) School of Electrical an Computer

More information

THE SCHEDULING OF MAINTENANCE SERVICE

THE SCHEDULING OF MAINTENANCE SERVICE THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single

More information

The most common model to support workforce management of telephone call centers is

The most common model to support workforce management of telephone call centers is Designing a Call Center with Impatient Customers O. Garnett A. Manelbaum M. Reiman Davison Faculty of Inustrial Engineering an Management, Technion, Haifa 32000, Israel Davison Faculty of Inustrial Engineering

More information

FOURIER TRANSFORM TERENCE TAO

FOURIER TRANSFORM TERENCE TAO FOURIER TRANSFORM TERENCE TAO Very broaly speaking, the Fourier transform is a systematic way to ecompose generic functions into a superposition of symmetric functions. These symmetric functions are usually

More information

AND. Dimitri P. Bertsekas and Paul Tseng

AND. Dimitri P. Bertsekas and Paul Tseng SET INTERSECTION THEOREMS AND EXISTENCE OF OPTIMAL SOLUTIONS FOR CONVEX AND NONCONVEX OPTIMIZATION Dimitri P. Bertsekas an Paul Tseng Math. Programming Journal, 2007 NESTED SET SEQUENCE INTERSECTIONS Basic

More information

M147 Practice Problems for Exam 2

M147 Practice Problems for Exam 2 M47 Practice Problems for Exam Exam will cover sections 4., 4.4, 4.5, 4.6, 4.7, 4.8, 5., an 5.. Calculators will not be allowe on the exam. The first ten problems on the exam will be multiple choice. Work

More information

Using Stein s Method to Show Poisson and Normal Limit Laws for Fringe Subtrees

Using Stein s Method to Show Poisson and Normal Limit Laws for Fringe Subtrees AofA 2014, Paris, France DMTCS proc. (subm., by the authors, 1 12 Using Stein s Metho to Show Poisson an Normal Limit Laws for Fringe Subtrees Cecilia Holmgren 1 an Svante Janson 2 1 Department of Mathematics,

More information

2.3 Convex Constrained Optimization Problems

2.3 Convex Constrained Optimization Problems 42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

More information

Differentiability of Exponential Functions

Differentiability of Exponential Functions Differentiability of Exponential Functions Philip M. Anselone an John W. Lee Philip Anselone (panselone@actionnet.net) receive his Ph.D. from Oregon State in 1957. After a few years at Johns Hopkins an

More information

Mannheim curves in the three-dimensional sphere

Mannheim curves in the three-dimensional sphere Mannheim curves in the three-imensional sphere anju Kahraman, Mehmet Öner Manisa Celal Bayar University, Faculty of Arts an Sciences, Mathematics Department, Muraiye Campus, 5, Muraiye, Manisa, urkey.

More information

MODELLING OF TWO STRATEGIES IN INVENTORY CONTROL SYSTEM WITH RANDOM LEAD TIME AND DEMAND

MODELLING OF TWO STRATEGIES IN INVENTORY CONTROL SYSTEM WITH RANDOM LEAD TIME AND DEMAND art I. robobabilystic Moels Computer Moelling an New echnologies 27 Vol. No. 2-3 ransport an elecommunication Institute omonosova iga V-9 atvia MOEING OF WO AEGIE IN INVENOY CONO YEM WIH ANOM EA IME AN

More information

1 High-Dimensional Space

1 High-Dimensional Space Contents High-Dimensional Space. Properties of High-Dimensional Space..................... 4. The High-Dimensional Sphere......................... 5.. The Sphere an the Cube in Higher Dimensions...........

More information

Game Theoretic Modeling of Cooperation among Service Providers in Mobile Cloud Computing Environments

Game Theoretic Modeling of Cooperation among Service Providers in Mobile Cloud Computing Environments 2012 IEEE Wireless Communications an Networking Conference: Services, Applications, an Business Game Theoretic Moeling of Cooperation among Service Proviers in Mobile Clou Computing Environments Dusit

More information

Consumer Referrals. Maria Arbatskaya and Hideo Konishi. October 28, 2014

Consumer Referrals. Maria Arbatskaya and Hideo Konishi. October 28, 2014 Consumer Referrals Maria Arbatskaya an Hieo Konishi October 28, 2014 Abstract In many inustries, rms rewar their customers for making referrals. We analyze the optimal policy mix of price, avertising intensity,

More information

Risk Adjustment for Poker Players

Risk Adjustment for Poker Players Risk Ajustment for Poker Players William Chin DePaul University, Chicago, Illinois Marc Ingenoso Conger Asset Management LLC, Chicago, Illinois September, 2006 Introuction In this article we consier risk

More information

MSc. Econ: MATHEMATICAL STATISTICS, 1995 MAXIMUM-LIKELIHOOD ESTIMATION

MSc. Econ: MATHEMATICAL STATISTICS, 1995 MAXIMUM-LIKELIHOOD ESTIMATION MAXIMUM-LIKELIHOOD ESTIMATION The General Theory of M-L Estimation In orer to erive an M-L estimator, we are boun to make an assumption about the functional form of the istribution which generates the

More information

Which Networks Are Least Susceptible to Cascading Failures?

Which Networks Are Least Susceptible to Cascading Failures? Which Networks Are Least Susceptible to Cascaing Failures? Larry Blume Davi Easley Jon Kleinberg Robert Kleinberg Éva Taros July 011 Abstract. The resilience of networks to various types of failures is

More information

Risk Management for Derivatives

Risk Management for Derivatives Risk Management or Derivatives he Greeks are coming the Greeks are coming! Managing risk is important to a large number o iniviuals an institutions he most unamental aspect o business is a process where

More information

An Introduction to Event-triggered and Self-triggered Control

An Introduction to Event-triggered and Self-triggered Control An Introuction to Event-triggere an Self-triggere Control W.P.M.H. Heemels K.H. Johansson P. Tabuaa Abstract Recent evelopments in computer an communication technologies have le to a new type of large-scale

More information

Pythagorean Triples Over Gaussian Integers

Pythagorean Triples Over Gaussian Integers International Journal of Algebra, Vol. 6, 01, no., 55-64 Pythagorean Triples Over Gaussian Integers Cheranoot Somboonkulavui 1 Department of Mathematics, Faculty of Science Chulalongkorn University Bangkok

More information

Notes from Week 1: Algorithms for sequential prediction

Notes from Week 1: Algorithms for sequential prediction CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking

More information

CURRENCY OPTION PRICING II

CURRENCY OPTION PRICING II Jones Grauate School Rice University Masa Watanabe INTERNATIONAL FINANCE MGMT 657 Calibrating the Binomial Tree to Volatility Black-Scholes Moel for Currency Options Properties of the BS Moel Option Sensitivity

More information

Measures of distance between samples: Euclidean

Measures of distance between samples: Euclidean 4- Chapter 4 Measures of istance between samples: Eucliean We will be talking a lot about istances in this book. The concept of istance between two samples or between two variables is funamental in multivariate

More information

6. Metric spaces. In this section we review the basic facts about metric spaces. d : X X [0, )

6. Metric spaces. In this section we review the basic facts about metric spaces. d : X X [0, ) 6. Metric spaces In this section we review the basic facts about metric spaces. Definitions. A metric on a non-empty set X is a map with the following properties: d : X X [0, ) (i) If x, y X are points

More information

Answers to the Practice Problems for Test 2

Answers to the Practice Problems for Test 2 Answers to the Practice Problems for Test 2 Davi Murphy. Fin f (x) if it is known that x [f(2x)] = x2. By the chain rule, x [f(2x)] = f (2x) 2, so 2f (2x) = x 2. Hence f (2x) = x 2 /2, but the lefthan

More information

Integral Regular Truncated Pyramids with Rectangular Bases

Integral Regular Truncated Pyramids with Rectangular Bases Integral Regular Truncate Pyramis with Rectangular Bases Konstantine Zelator Department of Mathematics 301 Thackeray Hall University of Pittsburgh Pittsburgh, PA 1560, U.S.A. Also: Konstantine Zelator

More information

Dynamic Network Security Deployment Under Partial Information

Dynamic Network Security Deployment Under Partial Information Dynamic Network Security Deployment Uner Partial nformation nvite Paper) George Theoorakopoulos EPFL Lausanne, Switzerlan Email: george.theoorakopoulos @ epfl.ch John S. Baras University of Marylan College

More information

Parameterized Algorithms for d-hitting Set: the Weighted Case Henning Fernau. Univ. Trier, FB 4 Abteilung Informatik 54286 Trier, Germany

Parameterized Algorithms for d-hitting Set: the Weighted Case Henning Fernau. Univ. Trier, FB 4 Abteilung Informatik 54286 Trier, Germany Parameterize Algorithms for -Hitting Set: the Weighte Case Henning Fernau Trierer Forschungsberichte; Trier: Technical Reports Informatik / Mathematik No. 08-6, July 2008 Univ. Trier, FB 4 Abteilung Informatik

More information

Optimal Control Of Production Inventory Systems With Deteriorating Items And Dynamic Costs

Optimal Control Of Production Inventory Systems With Deteriorating Items And Dynamic Costs Applie Mathematics E-Notes, 8(2008), 194-202 c ISSN 1607-2510 Available free at mirror sites of http://www.math.nthu.eu.tw/ amen/ Optimal Control Of Prouction Inventory Systems With Deteriorating Items

More information

Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

More information

f(x) = a x, h(5) = ( 1) 5 1 = 2 2 1

f(x) = a x, h(5) = ( 1) 5 1 = 2 2 1 Exponential Functions an their Derivatives Exponential functions are functions of the form f(x) = a x, where a is a positive constant referre to as the base. The functions f(x) = x, g(x) = e x, an h(x)

More information

Enterprise Resource Planning

Enterprise Resource Planning Enterprise Resource Planning MPC 6 th Eition Chapter 1a McGraw-Hill/Irwin Copyright 2011 by The McGraw-Hill Companies, Inc. All rights reserve. Enterprise Resource Planning A comprehensive software approach

More information

17.3.1 Follow the Perturbed Leader

17.3.1 Follow the Perturbed Leader CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.

More information

19.2. First Order Differential Equations. Introduction. Prerequisites. Learning Outcomes

19.2. First Order Differential Equations. Introduction. Prerequisites. Learning Outcomes First Orer Differential Equations 19.2 Introuction Separation of variables is a technique commonly use to solve first orer orinary ifferential equations. It is so-calle because we rearrange the equation

More information

1 Introduction. 2 Prediction with Expert Advice. Online Learning 9.520 Lecture 09

1 Introduction. 2 Prediction with Expert Advice. Online Learning 9.520 Lecture 09 1 Introduction Most of the course is concerned with the batch learning problem. In this lecture, however, we look at a different model, called online. Let us first compare and contrast the two. In batch

More information

Ideal Class Group and Units

Ideal Class Group and Units Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

More information

CHAPTER 5 : CALCULUS

CHAPTER 5 : CALCULUS Dr Roger Ni (Queen Mary, University of Lonon) - 5. CHAPTER 5 : CALCULUS Differentiation Introuction to Differentiation Calculus is a branch of mathematics which concerns itself with change. Irrespective

More information

Yishay Mansour Google & Tel Aviv Univ. Many thanks for my co-authors: A. Blum, N. Cesa-Bianchi, and G. Stoltz

Yishay Mansour Google & Tel Aviv Univ. Many thanks for my co-authors: A. Blum, N. Cesa-Bianchi, and G. Stoltz Regret Minimization: Algorithms and Applications Yishay Mansour Google & Tel Aviv Univ. Many thanks for my co-authors: A. Blum, N. Cesa-Bianchi, and G. Stoltz 2 Weather Forecast Sunny: Rainy: No meteorological

More information

Minimum-Energy Broadcast in All-Wireless Networks: NP-Completeness and Distribution Issues

Minimum-Energy Broadcast in All-Wireless Networks: NP-Completeness and Distribution Issues Minimum-Energy Broacast in All-Wireless Networks: NP-Completeness an Distribution Issues Mario Čagal LCA-EPFL CH-05 Lausanne Switzerlan mario.cagal@epfl.ch Jean-Pierre Hubaux LCA-EPFL CH-05 Lausanne Switzerlan

More information

The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

More information

Safety Stock or Excess Capacity: Trade-offs under Supply Risk

Safety Stock or Excess Capacity: Trade-offs under Supply Risk Safety Stock or Excess Capacity: Trae-offs uner Supply Risk Aahaar Chaturvei Victor Martínez-e-Albéniz IESE Business School, University of Navarra Av. Pearson, 08034 Barcelona, Spain achaturvei@iese.eu

More information

A New Evaluation Measure for Information Retrieval Systems

A New Evaluation Measure for Information Retrieval Systems A New Evaluation Measure for Information Retrieval Systems Martin Mehlitz martin.mehlitz@ai-labor.e Christian Bauckhage Deutsche Telekom Laboratories christian.bauckhage@telekom.e Jérôme Kunegis jerome.kunegis@ai-labor.e

More information

Image compression predicated on recurrent iterated function systems **

Image compression predicated on recurrent iterated function systems ** 1 Image compression preicate on recurrent iterate function systems ** W. Metzler a, *, C.H. Yun b, M. Barski a a Faculty of Mathematics University of Kassel, Kassel, F. R. Germany b Faculty of Mathematics

More information

Lecture 17: Implicit differentiation

Lecture 17: Implicit differentiation Lecture 7: Implicit ifferentiation Nathan Pflueger 8 October 203 Introuction Toay we iscuss a technique calle implicit ifferentiation, which provies a quicker an easier way to compute many erivatives we

More information

Unsteady Flow Visualization by Animating Evenly-Spaced Streamlines

Unsteady Flow Visualization by Animating Evenly-Spaced Streamlines EUROGRAPHICS 2000 / M. Gross an F.R.A. Hopgoo Volume 19, (2000), Number 3 (Guest Eitors) Unsteay Flow Visualization by Animating Evenly-Space Bruno Jobar an Wilfri Lefer Université u Littoral Côte Opale,

More information

FAST JOINING AND REPAIRING OF SANDWICH MATERIALS WITH DETACHABLE MECHANICAL CONNECTION TECHNOLOGY

FAST JOINING AND REPAIRING OF SANDWICH MATERIALS WITH DETACHABLE MECHANICAL CONNECTION TECHNOLOGY FAST JOINING AND REPAIRING OF SANDWICH MATERIALS WITH DETACHABLE MECHANICAL CONNECTION TECHNOLOGY Jörg Felhusen an Sivakumara K. Krishnamoorthy RWTH Aachen University, Chair an Insitute for Engineering

More information

Notes on tangents to parabolas

Notes on tangents to parabolas Notes on tangents to parabolas (These are notes for a talk I gave on 2007 March 30.) The point of this talk is not to publicize new results. The most recent material in it is the concept of Bézier curves,

More information

Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

More information

Lecture 13: Differentiation Derivatives of Trigonometric Functions

Lecture 13: Differentiation Derivatives of Trigonometric Functions Lecture 13: Differentiation Derivatives of Trigonometric Functions Derivatives of the Basic Trigonometric Functions Derivative of sin Derivative of cos Using the Chain Rule Derivative of tan Using the

More information

UCLA STAT 13 Introduction to Statistical Methods for the Life and Health Sciences. Chapter 9 Paired Data. Paired data. Paired data

UCLA STAT 13 Introduction to Statistical Methods for the Life and Health Sciences. Chapter 9 Paired Data. Paired data. Paired data UCLA STAT 3 Introuction to Statistical Methos for the Life an Health Sciences Instructor: Ivo Dinov, Asst. Prof. of Statistics an Neurology Chapter 9 Paire Data Teaching Assistants: Jacquelina Dacosta

More information

A Data Placement Strategy in Scientific Cloud Workflows

A Data Placement Strategy in Scientific Cloud Workflows A Data Placement Strategy in Scientific Clou Workflows Dong Yuan, Yun Yang, Xiao Liu, Jinjun Chen Faculty of Information an Communication Technologies, Swinburne University of Technology Hawthorn, Melbourne,

More information

The one-year non-life insurance risk

The one-year non-life insurance risk The one-year non-life insurance risk Ohlsson, Esbjörn & Lauzeningks, Jan Abstract With few exceptions, the literature on non-life insurance reserve risk has been evote to the ultimo risk, the risk in the

More information

Option Pricing for Inventory Management and Control

Option Pricing for Inventory Management and Control Option Pricing for Inventory Management an Control Bryant Angelos, McKay Heasley, an Jeffrey Humpherys Abstract We explore the use of option contracts as a means of managing an controlling inventories

More information

arxiv:1309.1857v3 [gr-qc] 7 Mar 2014

arxiv:1309.1857v3 [gr-qc] 7 Mar 2014 Generalize holographic equipartition for Friemann-Robertson-Walker universes Wen-Yuan Ai, Hua Chen, Xian-Ru Hu, an Jian-Bo Deng Institute of Theoretical Physics, LanZhou University, Lanzhou 730000, P.

More information

Lagrangian and Hamiltonian Mechanics

Lagrangian and Hamiltonian Mechanics Lagrangian an Hamiltonian Mechanics D.G. Simpson, Ph.D. Department of Physical Sciences an Engineering Prince George s Community College December 5, 007 Introuction In this course we have been stuying

More information

The Multiplicative Weights Update method

The Multiplicative Weights Update method Chapter 2 The Multiplicative Weights Update method The Multiplicative Weights method is a simple idea which has been repeatedly discovered in fields as diverse as Machine Learning, Optimization, and Game

More information

Product Differentiation for Software-as-a-Service Providers

Product Differentiation for Software-as-a-Service Providers University of Augsburg Prof. Dr. Hans Ulrich Buhl Research Center Finance & Information Management Department of Information Systems Engineering & Financial Management Discussion Paper WI-99 Prouct Differentiation

More information

ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 12, June 2014

ISSN: 2277-3754 ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 12, June 2014 ISSN: 77-754 ISO 900:008 Certifie International Journal of Engineering an Innovative echnology (IJEI) Volume, Issue, June 04 Manufacturing process with isruption uner Quaratic Deman for Deteriorating Inventory

More information

Convex analysis and profit/cost/support functions

Convex analysis and profit/cost/support functions CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

More information

Open World Face Recognition with Credibility and Confidence Measures

Open World Face Recognition with Credibility and Confidence Measures Open Worl Face Recognition with Creibility an Confience Measures Fayin Li an Harry Wechsler Department of Computer Science George Mason University Fairfax, VA 22030 {fli, wechsler}@cs.gmu.eu Abstract.

More information

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

More information

n-parameter families of curves

n-parameter families of curves 1 n-parameter families of curves For purposes of this iscussion, a curve will mean any equation involving x, y, an no other variables. Some examples of curves are x 2 + (y 3) 2 = 9 circle with raius 3,

More information

Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

More information

Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

More information

IPA Derivatives for Make-to-Stock Production-Inventory Systems With Lost Sales

IPA Derivatives for Make-to-Stock Production-Inventory Systems With Lost Sales IPA Derivatives for Make-to-Stock Prouction-Inventory Systems With Lost Sales Yao Zhao Benjamin Melame Rutgers University Rutgers Business School Newark an New Brunswick Department of MSIS 94 Rockafeller

More information

If you have ever spoken with your grandparents about what their lives were like

If you have ever spoken with your grandparents about what their lives were like CHAPTER 7 Economic Growth I: Capital Accumulation an Population Growth The question of growth is nothing new but a new isguise for an age-ol issue, one which has always intrigue an preoccupie economics:

More information