Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA Elad Hazan IBM Almaden Research Center 650 Harry Road San Jose, CA 950 Alexander Rakhlin Division of Computer Science UC Berkeley Berkeley, CA Abstract We study the rates of growth of the regret in online convex optimization First, we show that a simple extension of the algorithm of Hazan et al eliminates the need for a priori knowledge of the lower bound on the second derivatives of the observed functions We then provide an algorithm, Adaptive Online Gradient Descent, which interpolates between the results of Zinkevich for linear functions and of Hazan et al for strongly convex functions, achieving intermediate rates between T and log T Furthermore, we show strong optimality of the algorithm Finally, we provide an extension of our results to general norms Introduction The problem of online convex optimization can be formulated as a repeated game between a player and an adversary At round t, the player chooses an action x t from some convex subset K of R n, and then the adversary chooses a convex loss function f t The player aims to ensure that the total loss, T f tx t, is not much larger than the smallest total loss T f tx of any fixed action x The difference between the total loss and its optimal value for a fixed action is known as the regret, which we denote R T = f t x t min x K f t x Many problems of online prediction of individual sequences can be viewed as special cases of online convex optimization, including prediction with expert advice, sequential probability assignment, and sequential investment [] A central question in all these cases is how the regret grows with the number of rounds of the game Zinkevich [] considered the following gradient descent algorithm, with step size η t = Θ/ t Here, Π K v denotes the Euclidean projection of v on to the convex set K Corresponding author

2 Algorithm Online Gradient Descent OGD : Initialize x arbitrarily : for t = to T do 3: Predict x t, observe f t 4: Update x t+ = Π K x t η t+ f t x t 5: end for Zinkevich showed that the regret of this algorithm grows as T, where T is the number of rounds of the game This rate cannot be improved in general for arbitrary convex loss functions However, this is not the case if the loss functions are uniformly convex, for instance, if all f t have second derivative at least H > 0 Recently, Hazan et al [3] showed that in this case it is possible for the regret to grow only logarithmically with T, using the same algorithm but with the smaller step size η t = /Ht Increasing convexity makes online convex optimization easier The algorithm that achieves logarithmic regret must know in advance a lower bound on the convexity of the loss functions, since this bound is used to determine the step size It is natural to ask if this is essential: is there an algorithm that can adapt to the convexity of the loss functions and achieve the same regret rates in both cases Olog T for uniformly convex functions and O T for arbitrary convex functions? In this paper, we present an adaptive algorithm of this kind The key technique is regularization: We consider the online gradient descent OGD algorithm, but we add a uniformly convex function, the quadratic λ t x, to each loss function f t x This corresponds to shrinking the algorithm s actions x t towards the origin It leads to a regret bound of the form R T c λ t + pλ,, λ T The first term on the right hand side can be viewed as a bias term; it increases with λ t because the presence of the regularization might lead the algorithm away from the optimum The second term is a penalty for the flatness of the loss functions that becomes smaller as the regularization increases We show that choosing the regularization coefficient λ t so as to balance these two terms in the bound on the regret up to round t is nearly optimal in a strong sense Not only does this choice give the T and log T regret rates in the linear and uniformly convex cases, it leads to a kind of oracle inequality: The regret is no more than a constant factor times the bound on regret that would have been suffered if an oracle had provided in advance the sequence of regularization coefficients λ,, λ T that minimizes the final regret bound To state this result precisely, we introduce the following definitions Let K be a convex subset of R n and suppose that sup x K x D For simplicity, throughout the paper we assume that K is centered around 0, and, hence, D is the diameter of K Define a shorthand t = f t x t Let H t be the largest value such that for any x K, f t x f t x t + t x x t + H t x x t In particular, if f t H t I 0, then the above inequality is satisfied Furthermore, suppose t G t Define λ :t := t s= λ s and H :t := t s= H s Let H :0 = 0 Let us now state the Adaptive Online Gradient Descent algorithm as well as the theoretical guarantee for its performance Algorithm Adaptive Online Gradient Descent : Initialize x arbitrarily : for t = to T do 3: Predict x t, observe f t 4: Compute λ t = H :t + λ :t + 8 /3D H :t + λ :t 5: Compute η t+ = H :t + λ :t 6: Update x t+ = Π K x t η t+ f t x t + λ t x t 7: end for

3 Theorem The regret of Algorithm is bounded by R T 3 inf D λ :T + λ,,λ T G t + λ t D H :t + λ :t While Algorithm is stated with the squared Euclidean norm as a regularizer, we show that it is straightforward to generalize our technique to other regularization functions that are uniformly convex with respect to other norms This leads to adaptive versions of the mirror descent algorithm analyzed recently in [4, 5] Preliminary results The following theorem gives a regret bound for the OGD algorithm with a particular choice of step size The virtue of the theorem is that the step size can be set without knowledge of the uniform lower bound on H t, which is required in the original algorithm of [3] The proof is provided in Section 4 Theorem 4, where the result is extended to arbitrary norms Theorem Suppose we set η t+ = H :t Then the regret of OGD is bounded as In particular, loosening the bound, R T R T H :t max t t min t t s= H log T s Note that nothing prevents H t from being negative or zero, implying that the same algorithm gives logarithmic regret even when some of the functions are linear or concave, as long as the partial averages t t s= H s are positive and not too small The above result already provides an important extension to the log-regret algorithm of [3]: no prior knowledge on the uniform convexity of the functions is needed, and the bound is in terms of the observed sequence {H t } Yet, there is still a problem with the algorithm If H > 0 and H t = 0 for all t >, then t s= H s = H, resulting in a linear regret bound However, we know from [] that a O T bound can be obtained In the next section we provide an algorithm which interpolates between Olog T and O T bound on the regret depending on the curvature of the observed functions 3 Adaptive Regularization Suppose the environment plays a sequence of f t s with curvature H t 0 Instead of performing gradient descent on these functions, we step in the direction of the gradient of f t x = f t x + λ t x, where the regularization parameter λ t 0 is chosen appropriately at each step as a function of the curvature of the previous functions We remind the reader that K is assumed to be centered around the origin, for otherwise we would instead use x x 0 to shrink the actions x t towards the origin x 0 Applying Theorem, we obtain the following result Theorem 3 If the Online Gradient Descent algorithm is performed on the functions f t x = f t x + λ t x with η t+ = H :t + λ :t for any sequence of non-negative λ,, λ T, then R T D λ :T + G t + λ t D H :t + λ :t 3

4 Proof By Theorem applied to functions f t, f t x t + T λ t x t min f t x + x λ t x + G t + λ t D H :t + λ :t Indeed, it is easy to verify that condition for f t implies the corresponding statement with H t = H t +λ t for f t Furthermore, by linearity, the bound on the gradient of f t is G t +λ t x t G t +λ t D Define x T = arg min x f tx Then, dropping the x t terms and bounding x D, f t x t which proves the the theorem f t x + D λ :T + G t + λ t D H :t + λ :t, The following inequality is important in the rest of the analysis, as it allows us to remove the dependence on λ t from the numerator of the second sum at the expense of increased constants We have D λ :T + G t + λ t D H :t + λ :t D λ :T + 3 D λ :T + H :t + λ :t + λ t D H :t + λ :t + λ t H :t + λ :t, where the first inequality holds because a + b a + b for any a, b R It turns out that for appropriate choices of {λ t }, the above theorem recovers the O T bound on the regret for linear functions [] and the Olog T bound for strongly convex functions [3] Moreover, under specific assumptions on the sequence {H t }, we can define a sequence {λ t } which produces intermediate rates between log T and T These results are exhibited in corollaries at the end of this section Of course, it would be nice to be able to choose {λ t } adaptively without any restrictive assumptions on {H t } Somewhat surprisingly, such a choice can be made near-optimally by simple local balancing Observe that the upper bound of Eq consists of two sums: D T λ t and T H :t+λ :t The first sum increases in any particular λ t and the other decreases While the influence of the regularization parameters λ t on the first sum is trivial, the influence on the second sum is more involved as all terms for t t depend on λ t Nevertheless, it turns out that a simple choice of λ t is optimal to within a multiplicative factor of This is exhibited by the next lemma Lemma 3 Define C t H T {λ t } = H T λ λ T = λ :T +, H :t + λ :t C where C t 0 does not depend on λ t s If λ t satisfies λ t = t H :t+λ :t for t =,, T, then H T {λ t } inf H {λ t } 0 T {λ t } Proof We prove this by induction Let {λ t } be the optimal sequence of non-negative regularization coefficients The base of the induction is proved by considering two possibilities: either λ < λ or not In the first case, λ + C /H + λ = λ λ λ + C /H + λ The other case is proved similarly Now, suppose H T {λ t } H T {λ t } Consider two possibilities If λ :T < λ :T, then C t H T {λ t } = λ :T + = λ :T λ :T H T {λ t } H :t + λ :t 4

5 If, on the other hand, λ :T λ :T, then C t λ T + = H :T + λ :T H :T + λ :T H :T + λ :T Using the inductive assumption, we obtain H T {λ t } H T {λ t } C t C t λ T + C t H :T + λ :T The lemma above is the key to the proof of the near-optimal bounds for Algorithm Proof of Theorem By Eq and Lemma 3, R T 3 D λ :T + 6 inf λ,,λ T H :t + λ :t D λ :T + provided the λ t are chosen as solutions to inf λ,,λ T G t + λ t D H :t + λ :t 3D λ :T +, H :t + λ :t 3 G D t λ t = 3 H :t + λ :t + λ t It is easy to verify that λ t = H :t + λ :t + 8 /3D H :t + λ :t is the non-negative root of the above quadratic equation We note that division by zero in Algorithm occurs only if λ = H = G = 0 Without loss of generality, G 0, for otherwise x is minimizing f x and regret is negative on that round Hence, the algorithm has a bound on the performance which is 6 times the bound obtained by the best offline adaptive choice of regularization coefficients While the constant 6 might not be optimal, it can be shown that a constant strictly larger than one is unavoidable see previous footnote We also remark that if the diameter D is unknown, the regularization coefficients λ t can still be chosen by balancing as in Eq 3, except without the D term This choice of λ t, however, increases the bound on the regret suffered by Algorithm by a factor of OD Let us now consider some special cases and show that Theorem not only recovers the rate of increase of regret of [3] and [], but also provides intermediate rates For each of these special cases, we provide a sequence of {λ t } which achieves the desired rates Since Theorem guarantees that Algorithm is competitive with the best choice of the parameters, we conclude that Algorithm achieves the same rates Corollary 3 Suppose G t G for all t T Then for any sequence of convex functions {f t }, the bound on regret of Algorithm is O T Proof Let λ = T and λ t = 0 for < t T By Eq, D G t + λ t D λ :T + 3 H :t + λ :t D λ :T + 3 D T + H :t + λ :t G T = 3 T D + G Lemma 3 effectively describes an algorithm for an online problem with competitive ratio of In the full version of this paper we give a lower bound strictly larger than one on the competitive ratio achievable by any online algorithm for this problem 5

6 Hence, the regret of Algorithm can never increase faster than T We now consider the assumptions of [3] Corollary 3 Suppose H t H > 0 and < G for all t T Then the bound on regret of Algorithm is Olog T Proof Set λ t = 0 for all t It holds that R T T H :t T G th G H log T + The above proof also recovers the result of Theorem The following Corollary shows a spectrum of rates under assumptions on the curvature of functions Corollary 33 Suppose H t = t α and G t G for all t T If α = 0, then R T = Olog T If α > /, then R T = O T 3 If 0 < α /, then R T = OT α Proof The first two cases follow immediately from Corollaries 3 and 3 For the third case, let λ = T α and λ t = 0 for < t T Note that t s= H s t x=0 x + α dx = α t α α Hence, D G t + λ t D λ :T + 3 G H :t + λ :t D t λ :T + H :t + λ :t D T α + G α t α D T α + G α T α + O = OT α 4 Generalization to different norms The original online gradient descent OGD algorithm as analyzed by Zinkevich [] used the Euclidean distance of the current point from the optimum as a potential function The logarithmic regret bounds of [3] for strongly convex functions were also stated for the Euclidean norm, and such was the presentation above However, as observed by Shalev-Shwartz and Singer in [5], the proof technique of [3] extends to arbitrary norms As such, our results above for adaptive regularization carry on to the general setting, as we state below Our notation follows that of Gentile and Warmuth [6] Definition 4 A function g over a convex set K is called H-strongly convex with respect to a convex function h if x, y K gx gy + gy x y + H B hx, y Here B h x, y is the Bregman divergence with respect to the function h, defined as B h x, y = hx hy hy x y This notion of strong convexity generalizes the Euclidean notion: the function gx = x is strongly convex with respect to hx = x in this case B h x, y = x y More generally, the Bregman divergence can be thought of as a squared norm, not necessarily Euclidean, ie, B h x, y = x y Henceforth we also refer to the dual norm of a given norm, defined by y = sup x {y x} For the case of l p norms, we have y = y q where q satisfies p + q =, and by Hölder s inequality y x y x y + x this holds for norms other than l p as well 6

7 For simplicity, the reader may think of the functions g, h as convex and differentiable The following algorithm is a generalization of the OGD algorithm to general strongly convex functions see the derivation in [6] In this extended abstract we state the update rule implicitly, leaving the issues of efficient computation for the full version these issues are orthogonal to our discussion, and were addressed in [6] for a variety of functions h Algorithm 3 General-Norm Online Gradient Descent : Input: convex function h : Initialize x arbitrarily 3: for t = to T do 4: Predict x t, observe f t 5: Compute η t+ and let y t+ be such that hy t+ = hx t η t+ f t x t 6: Let x t+ = arg min x K B h x, y t+ be the projection of y t+ onto K 7: end for The methods of the previous sections can now be used to derive similar, dynamically optimal, bounds on the regret As a first step, let us generalize the bound of [3], as well as Theorem, to general norms: Theorem 4 Suppose that, for each t, f t is a H t -strongly convex function with respect to h, and let h be such that B h x, y x y for some norm Let f t x t G t for all t Applying the General-Norm Online Gradient Algorithm with η t+ = H :t, we have R T H :t Proof The proof follows [3], with the Bregman divergence replacing the Euclidean distance as a potential function By assumption on the functions f t, for any x K, f t x t f t x f t x t x t x H t B hx, x t By a well-known property of Bregman divergences see [6], it holds that for any vectors x, y, z, Combining both observations, x y hz hy = B h x, y B h x, z + B h y, z f t x t f t x f t x t x t x H t B h x, x t = η t+ hy t+ hx t x x t H t B h x, x t = [B h x, x t B h x, y t+ + B h x t, y t+ ] H t B h x, x t η t+ [B h x, x t B h x, x t+ + B h x t, y t+ ] H t B h x, x t, η t+ where the last inequality follows from the Pythagorean Theorem for Bregman divergences [6], as x t+ is the projection wrt the Bregman divergence of y t+ and x K is in the convex set Summing over all iterations and recalling that η t+ = R T = B h x, x t t= H :t, H t + B h x, x η t+ η t H + η η t+ B h x t, y t+ η t+ B h x t, y t+ 4 Since the set of points of nondifferentiability of convex functions has measure zero, convexity is the only property that we require Indeed, for nondifferentiable functions, the algorithm would choose a point x t, which is x t with the addition of a small random perturbation With probability one, the functions would be smooth at the perturbed point, and the perturbation could be made arbitrarily small so that the regret rate would not be affected 7

8 We proceed to bound B h x t, y t+ By definition of Bregman divergence, and the dual norm inequality stated before, B h x t, y t+ + B h y t+, x t = hx t hy t+ x t y t+ Thus, by our assumption B h x, y x y, we have = η t+ f t x t x t y t+ η t+ t + x t y t+ B h x t, y t+ η t+ t + x t y t+ B h y t+, x t η t+ t Plugging back into Eq 4 we get R T η t+ = H :t The generalization of our technique is now straightforward Let A = sup x K gx and B = sup x K gx The following algorithm is an analogue of Algorithm and Theorem 4 is the analogue of Theorem for general norms Algorithm 4 Adaptive General-Norm Online Gradient Descent : Initialize x arbitrarily Let gx be -strongly convex with respect to the convex function h : for t = to T do 3: Predict x t, observe f t 4: Compute λ t = H :t + λ :t + 8 /A + B H :t + λ :t 5: Compute η t+ = H :t + λ :t 6: 7: Let y t+ be such that hy t+ = hx t η t+ f t x t + λt gx t Let x t+ = arg min x K B h x, y t+ be the projection of y t+ onto K 8: end for Theorem 4 Suppose that each f t is a H t -strongly convex function with respect to h, and let g be a -strongly convex with respect h Let h be such that B h x, y x y for some norm Let f t x t G t The regret of Algorithm 4 is bounded by R T inf λ,,λ T A + B λ :T + G t + λ t B H :t + λ :t If the norm in the above theorem is the Euclidean norm and gx = x, we find that D = sup x K x = A = B and recover the results of Theorem References [] Nicolò Cesa-Bianchi and Gábor Lugosi Prediction, Learning, and Games Cambridge University Press, 006 [] Martin Zinkevich Online convex programming and generalized infinitesimal gradient ascent In ICML, pages , 003 [3] Elad Hazan, Adam Kalai, Satyen Kale, and Amit Agarwal Logarithmic regret algorithms for online convex optimization In COLT, pages , 006 [4] Shai Shalev-Shwartz and Yoram Singer Convex repeated games and fenchel duality In B Schölkopf, J Platt, and T Hoffman, editors, Advances in Neural Information Processing Systems 9 MIT Press, Cambridge, MA, 007 [5] Shai Shalev-Shwartz and Yoram Singer Logarithmic regret algorithms for strongly convex repeated games In Technical Report The Hebrew University, 007 [6] C Gentile and M K Warmuth Proving relative loss bounds for on-line learning algorithms using bregman divergences In COLT Tutorial, 000 8

### Online Convex Optimization

E0 370 Statistical Learning heory Lecture 19 Oct 22, 2013 Online Convex Optimization Lecturer: Shivani Agarwal Scribe: Aadirupa 1 Introduction In this lecture we shall look at a fairly general setting

### Optimal Strategies and Minimax Lower Bounds for Online Convex Games

Optimal Strategies and Minimax Lower Bounds for Online Convex Games Jacob Abernethy UC Berkeley jake@csberkeleyedu Alexander Rakhlin UC Berkeley rakhlin@csberkeleyedu Abstract A number of learning problems

### 1 Introduction. 2 Prediction with Expert Advice. Online Learning 9.520 Lecture 09

1 Introduction Most of the course is concerned with the batch learning problem. In this lecture, however, we look at a different model, called online. Let us first compare and contrast the two. In batch

CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.

### Mind the Duality Gap: Logarithmic regret algorithms for online optimization

Mind the Duality Gap: Logarithmic regret algorithms for online optimization Sham M. Kakade Toyota Technological Institute at Chicago sham@tti-c.org Shai Shalev-Shartz Toyota Technological Institute at

### Table 1: Summary of the settings and parameters employed by the additive PA algorithm for classification, regression, and uniclass.

Online Passive-Aggressive Algorithms Koby Crammer Ofer Dekel Shai Shalev-Shwartz Yoram Singer School of Computer Science & Engineering The Hebrew University, Jerusalem 91904, Israel {kobics,oferd,shais,singer}@cs.huji.ac.il

### Lecture Notes on Online Learning DRAFT

Lecture Notes on Online Learning DRAFT Alexander Rakhlin These lecture notes contain material presented in the Statistical Learning Theory course at UC Berkeley, Spring 08. Various parts of these notes

Amit Daniely Alon Gonen Shai Shalev-Shwartz The Hebrew University AMIT.DANIELY@MAIL.HUJI.AC.IL ALONGNN@CS.HUJI.AC.IL SHAIS@CS.HUJI.AC.IL Abstract Strongly adaptive algorithms are algorithms whose performance

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Online Learning. 1 Online Learning and Mistake Bounds for Finite Hypothesis Classes

Advanced Course in Machine Learning Spring 2011 Online Learning Lecturer: Shai Shalev-Shwartz Scribe: Shai Shalev-Shwartz In this lecture we describe a different model of learning which is called online

### Notes from Week 1: Algorithms for sequential prediction

CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### Week 1: Introduction to Online Learning

Week 1: Introduction to Online Learning 1 Introduction This is written based on Prediction, Learning, and Games (ISBN: 2184189 / -21-8418-9 Cesa-Bianchi, Nicolo; Lugosi, Gabor 1.1 A Gentle Start Consider

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### 10. Proximal point method

L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

Online Learning with Switching Costs and Other Adaptive Adversaries Nicolò Cesa-Bianchi Università degli Studi di Milano Italy Ofer Dekel Microsoft Research USA Ohad Shamir Microsoft Research and the Weizmann

### The p-norm generalization of the LMS algorithm for adaptive filtering

The p-norm generalization of the LMS algorithm for adaptive filtering Jyrki Kivinen University of Helsinki Manfred Warmuth University of California, Santa Cruz Babak Hassibi California Institute of Technology

### A Drifting-Games Analysis for Online Learning and Applications to Boosting

A Drifting-Games Analysis for Online Learning and Applications to Boosting Haipeng Luo Department of Computer Science Princeton University Princeton, NJ 08540 haipengl@cs.princeton.edu Robert E. Schapire

### Online Learning of Optimal Strategies in Unknown Environments

1 Online Learning of Optimal Strategies in Unknown Environments Santiago Paternain and Alejandro Ribeiro Abstract Define an environment as a set of convex constraint functions that vary arbitrarily over

### LINEAR PROGRAMMING WITH ONLINE LEARNING

LINEAR PROGRAMMING WITH ONLINE LEARNING TATSIANA LEVINA, YURI LEVIN, JEFF MCGILL, AND MIKHAIL NEDIAK SCHOOL OF BUSINESS, QUEEN S UNIVERSITY, 143 UNION ST., KINGSTON, ON, K7L 3N6, CANADA E-MAIL:{TLEVIN,YLEVIN,JMCGILL,MNEDIAK}@BUSINESS.QUEENSU.CA

### ALMOST COMMON PRIORS 1. INTRODUCTION

ALMOST COMMON PRIORS ZIV HELLMAN ABSTRACT. What happens when priors are not common? We introduce a measure for how far a type space is from having a common prior, which we term prior distance. If a type

### Stochastic Inventory Control

Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the

### Absolute Value Programming

Computational Optimization and Aplications,, 1 11 (2006) c 2006 Springer Verlag, Boston. Manufactured in The Netherlands. Absolute Value Programming O. L. MANGASARIAN olvi@cs.wisc.edu Computer Sciences

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### (Quasi-)Newton methods

(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting

### Foundations of Machine Learning On-Line Learning. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu

Foundations of Machine Learning On-Line Learning Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Motivation PAC learning: distribution fixed over time (training and test). IID assumption.

### Solutions of Equations in One Variable. Fixed-Point Iteration II

Solutions of Equations in One Variable Fixed-Point Iteration II Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### Introduction to Online Optimization

Princeton University - Department of Operations Research and Financial Engineering Introduction to Online Optimization Sébastien Bubeck December 14, 2011 1 Contents Chapter 1. Introduction 5 1.1. Statistical

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### Prime Numbers and Irreducible Polynomials

Prime Numbers and Irreducible Polynomials M. Ram Murty The similarity between prime numbers and irreducible polynomials has been a dominant theme in the development of number theory and algebraic geometry.

### No: 10 04. Bilkent University. Monotonic Extension. Farhad Husseinov. Discussion Papers. Department of Economics

No: 10 04 Bilkent University Monotonic Extension Farhad Husseinov Discussion Papers Department of Economics The Discussion Papers of the Department of Economics are intended to make the initial results

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### arxiv:1112.0829v1 [math.pr] 5 Dec 2011

How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly

Load balancing of temporary tasks in the l p norm Yossi Azar a,1, Amir Epstein a,2, Leah Epstein b,3 a School of Computer Science, Tel Aviv University, Tel Aviv, Israel. b School of Computer Science, The

### Projection-free Online Learning

Elad Hazan Technion - Israel Inst. of Tech. Satyen Kale IBM T.J. Watson Research Center ehazan@ie.technion.ac.il sckale@us.ibm.com Abstract The computational bottleneck in applying online learning to massive

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### TD(0) Leads to Better Policies than Approximate Value Iteration

TD(0) Leads to Better Policies than Approximate Value Iteration Benjamin Van Roy Management Science and Engineering and Electrical Engineering Stanford University Stanford, CA 94305 bvr@stanford.edu Abstract

### Trading regret rate for computational efficiency in online learning with limited feedback

Trading regret rate for computational efficiency in online learning with limited feedback Shai Shalev-Shwartz TTI-C Hebrew University On-line Learning with Limited Feedback Workshop, 2009 June 2009 Shai

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### Increasing for all. Convex for all. ( ) Increasing for all (remember that the log function is only defined for ). ( ) Concave for all.

1. Differentiation The first derivative of a function measures by how much changes in reaction to an infinitesimal shift in its argument. The largest the derivative (in absolute value), the faster is evolving.

### The Dirichlet Unit Theorem

Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if

### Several Views of Support Vector Machines

Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min

### Online Learning, Stability, and Stochastic Gradient Descent

Online Learning, Stability, and Stochastic Gradient Descent arxiv:1105.4701v3 [cs.lg] 8 Sep 2011 September 9, 2011 Tomaso Poggio, Stephen Voinea, Lorenzo Rosasco CBCL, McGovern Institute, CSAIL, Brain

### Date: April 12, 2001. Contents

2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........

### A Potential-based Framework for Online Multi-class Learning with Partial Feedback

A Potential-based Framework for Online Multi-class Learning with Partial Feedback Shijun Wang Rong Jin Hamed Valizadegan Radiology and Imaging Sciences Computer Science and Engineering Computer Science

### Online Convex Programming and Generalized Infinitesimal Gradient Ascent

Online Convex Programming and Generalized Infinitesimal Gradient Ascent Martin Zinkevich February 003 CMU-CS-03-110 School of Computer Science Carnegie Mellon University Pittsburgh, PA 1513 Abstract Convex

### GI01/M055 Supervised Learning Proximal Methods

GI01/M055 Supervised Learning Proximal Methods Massimiliano Pontil (based on notes by Luca Baldassarre) (UCL) Proximal Methods 1 / 20 Today s Plan Problem setting Convex analysis concepts Proximal operators

### Online Learning: Theory, Algorithms, and Applications. Thesis submitted for the degree of Doctor of Philosophy by Shai Shalev-Shwartz

Online Learning: Theory, Algorithms, and Applications Thesis submitted for the degree of Doctor of Philosophy by Shai Shalev-Shwartz Submitted to the Senate of the Hebrew University July 2007 This work

### Properties of BMO functions whose reciprocals are also BMO

Properties of BMO functions whose reciprocals are also BMO R. L. Johnson and C. J. Neugebauer The main result says that a non-negative BMO-function w, whose reciprocal is also in BMO, belongs to p> A p,and

### Game Theory: Supermodular Games 1

Game Theory: Supermodular Games 1 Christoph Schottmüller 1 License: CC Attribution ShareAlike 4.0 1 / 22 Outline 1 Introduction 2 Model 3 Revision questions and exercises 2 / 22 Motivation I several solution

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### FACTORING POLYNOMIALS IN THE RING OF FORMAL POWER SERIES OVER Z

FACTORING POLYNOMIALS IN THE RING OF FORMAL POWER SERIES OVER Z DANIEL BIRMAJER, JUAN B GIL, AND MICHAEL WEINER Abstract We consider polynomials with integer coefficients and discuss their factorization

### Interior-Point Methods for Full-Information and Bandit Online Learning

4164 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 7, JULY 2012 Interior-Point Methods for Full-Information and Bandit Online Learning Jacob D. Abernethy, Elad Hazan, and Alexander Rakhlin Abstract

### Economics 1011a: Intermediate Microeconomics

Lecture 12: More Uncertainty Economics 1011a: Intermediate Microeconomics Lecture 12: More on Uncertainty Thursday, October 23, 2008 Last class we introduced choice under uncertainty. Today we will explore

### Convex analysis and profit/cost/support functions

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

### Introduction to Convex Optimization for Machine Learning

Introduction to Convex Optimization for Machine Learning John Duchi University of California, Berkeley Practical Machine Learning, Fall 2009 Duchi (UC Berkeley) Convex Optimization for Machine Learning

### A Sublinear Bipartiteness Tester for Bounded Degree Graphs

A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite

### CONSTANT-SIGN SOLUTIONS FOR A NONLINEAR NEUMANN PROBLEM INVOLVING THE DISCRETE p-laplacian. Pasquale Candito and Giuseppina D Aguí

Opuscula Math. 34 no. 4 2014 683 690 http://dx.doi.org/10.7494/opmath.2014.34.4.683 Opuscula Mathematica CONSTANT-SIGN SOLUTIONS FOR A NONLINEAR NEUMANN PROBLEM INVOLVING THE DISCRETE p-laplacian Pasquale

### {f 1 (U), U F} is an open cover of A. Since A is compact there is a finite subcover of A, {f 1 (U 1 ),...,f 1 (U n )}, {U 1,...

44 CHAPTER 4. CONTINUOUS FUNCTIONS In Calculus we often use arithmetic operations to generate new continuous functions from old ones. In a general metric space we don t have arithmetic, but much of it

### Chapter 7. Sealed-bid Auctions

Chapter 7 Sealed-bid Auctions An auction is a procedure used for selling and buying items by offering them up for bid. Auctions are often used to sell objects that have a variable price (for example oil)

### Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization

Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization Archis Ghate a and Robert L. Smith b a Industrial Engineering, University of Washington, Box 352650, Seattle, Washington,

### 1 Portfolio Selection

COS 5: Theoretical Machine Learning Lecturer: Rob Schapire Lecture # Scribe: Nadia Heninger April 8, 008 Portfolio Selection Last time we discussed our model of the stock market N stocks start on day with

### THE SCHEDULING OF MAINTENANCE SERVICE

THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single

### On Adaboost and Optimal Betting Strategies

On Adaboost and Optimal Betting Strategies Pasquale Malacaria School of Electronic Engineering and Computer Science Queen Mary, University of London Email: pm@dcs.qmul.ac.uk Fabrizio Smeraldi School of

### The Online Set Cover Problem

The Online Set Cover Problem Noga Alon Baruch Awerbuch Yossi Azar Niv Buchbinder Joseph Seffi Naor ABSTRACT Let X = {, 2,..., n} be a ground set of n elements, and let S be a family of subsets of X, S

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### Online Lazy Updates for Portfolio Selection with Transaction Costs

Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence Online Lazy Updates for Portfolio Selection with Transaction Costs Puja Das, Nicholas Johnson, and Arindam Banerjee Department

### Simple and efficient online algorithms for real world applications

Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRA-Lab,

### Reference: Introduction to Partial Differential Equations by G. Folland, 1995, Chap. 3.

5 Potential Theory Reference: Introduction to Partial Differential Equations by G. Folland, 995, Chap. 3. 5. Problems of Interest. In what follows, we consider Ω an open, bounded subset of R n with C 2

### Notes on Symmetric Matrices

CPSC 536N: Randomized Algorithms 2011-12 Term 2 Notes on Symmetric Matrices Prof. Nick Harvey University of British Columbia 1 Symmetric Matrices We review some basic results concerning symmetric matrices.

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Tail inequalities for order statistics of log-concave vectors and applications

Tail inequalities for order statistics of log-concave vectors and applications Rafał Latała Based in part on a joint work with R.Adamczak, A.E.Litvak, A.Pajor and N.Tomczak-Jaegermann Banff, May 2011 Basic

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

Mechanisms for Fair Attribution Eric Balkanski Yaron Singer Abstract We propose a new framework for optimization under fairness constraints. The problems we consider model procurement where the goal is

### A PRIORI ESTIMATES FOR SEMISTABLE SOLUTIONS OF SEMILINEAR ELLIPTIC EQUATIONS. In memory of Rou-Huai Wang

A PRIORI ESTIMATES FOR SEMISTABLE SOLUTIONS OF SEMILINEAR ELLIPTIC EQUATIONS XAVIER CABRÉ, MANEL SANCHÓN, AND JOEL SPRUCK In memory of Rou-Huai Wang 1. Introduction In this note we consider semistable

### Chapter 6. Cuboids. and. vol(conv(p ))

Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both

### Notes on metric spaces

Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### Separation Properties for Locally Convex Cones

Journal of Convex Analysis Volume 9 (2002), No. 1, 301 307 Separation Properties for Locally Convex Cones Walter Roth Department of Mathematics, Universiti Brunei Darussalam, Gadong BE1410, Brunei Darussalam

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### MATH 425, PRACTICE FINAL EXAM SOLUTIONS.

MATH 45, PRACTICE FINAL EXAM SOLUTIONS. Exercise. a Is the operator L defined on smooth functions of x, y by L u := u xx + cosu linear? b Does the answer change if we replace the operator L by the operator

### How to build a probability-free casino

How to build a probability-free casino Adam Chalcraft CCR La Jolla dachalc@ccrwest.org Chris Freiling Cal State San Bernardino cfreilin@csusb.edu Randall Dougherty CCR La Jolla rdough@ccrwest.org Jason

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

### Random graphs with a given degree sequence

Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.

### 1 Fixed Point Iteration and Contraction Mapping Theorem

1 Fixed Point Iteration and Contraction Mapping Theorem Notation: For two sets A,B we write A B iff x A = x B. So A A is true. Some people use the notation instead. 1.1 Introduction Consider a function

### Chapter 4, Arithmetic in F [x] Polynomial arithmetic and the division algorithm.

Chapter 4, Arithmetic in F [x] Polynomial arithmetic and the division algorithm. We begin by defining the ring of polynomials with coefficients in a ring R. After some preliminary results, we specialize

### Subsets of Euclidean domains possessing a unique division algorithm

Subsets of Euclidean domains possessing a unique division algorithm Andrew D. Lewis 2009/03/16 Abstract Subsets of a Euclidean domain are characterised with the following objectives: (1) ensuring uniqueness

### Scheduling Real-time Tasks: Algorithms and Complexity

Scheduling Real-time Tasks: Algorithms and Complexity Sanjoy Baruah The University of North Carolina at Chapel Hill Email: baruah@cs.unc.edu Joël Goossens Université Libre de Bruxelles Email: joel.goossens@ulb.ac.be

### Problem 1 (10 pts) Find the radius of convergence and interval of convergence of the series

1 Problem 1 (10 pts) Find the radius of convergence and interval of convergence of the series a n n=1 n(x + 2) n 5 n 1. n(x + 2)n Solution: Do the ratio test for the absolute convergence. Let a n =. Then,