# AND. Dimitri P. Bertsekas and Paul Tseng

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 SET INTERSECTION THEOREMS AND EXISTENCE OF OPTIMAL SOLUTIONS FOR CONVEX AND NONCONVEX OPTIMIZATION Dimitri P. Bertsekas an Paul Tseng Math. Programming Journal, 2007

2 NESTED SET SEQUENCE INTERSECTIONS Basic Question: Given a neste sequence of nonempty close sets {S k } in R n (S k+1 S k for all k), when is k=0 S k nonempty?

3 NESTED SET SEQUENCE INTERSECTIONS Basic Question: Given a neste sequence of nonempty close sets {S k } in R n (S k+1 S k for all k), when is k=0 S k nonempty? Set intersection theorems are significant in at least four major contexts: Existence of optimal solutions Preservation of closeness by linear transformations Duality gap issue, i.e., equality of optimal values of the primal convex problem an its ual minimize x X, g(x) 0 f(x) { maximize µ 0 q(µ) inf f(x) + µ g(x) } x X min-max = max-min issue, i.e., whether min x max z φ(x, z) = max z min x φ(x, z), where φ is convex in x an concave in z

4 SOME SPECIFIC CONTEXTS I Does a function f : R n (, ] attain a minimum over a set X? This is true iff the intersection of the nonempty sets { x X f(x) γ } is nonempty Level sets of f X

5 SOME SPECIFIC CONTEXTS I Does a function f : R n (, ] attain a minimum over a set X? This is true iff the intersection of the nonempty sets { x X f(x) γ } is nonempty Level sets of f X If C is close, is AC close? N k x S k C y W k y k+1 y k AC Many interesting special cases, e.g., if C 1 an C 2 are close, is C 1 + C 2 close?

6 SOME SPECIFIC CONTEXTS II Preservation of closeness by partial minima: If F (x, u) is close, is p(u) = inf x F (x, u) close? Critical question in the uality gap issue, where F (x, u) = { f(x) if x X, g(x) u, otherwise an p is the primal function. Critical question regaring min-max=maxmin where F (x, u) = { supz Z { φ(x, z) u z } if x X, if x / X. We have min-max=max-min if is close. p(u) = inf F (x, u) x Rn Can be aresse by using the relation Proj ( epi(f ) ) epi(p) cl (Proj ( epi(f ) ))

7 ASYMPTOTIC DIRECTIONS Given a sequence of nonempty neste close sets {S k }, we say that a vector 0 is an asymptotic irection of {S k } if there exists {x k } s. t. x k S k, x k 0, k = 0, 1,... x k, x k x k. S 0 S 1 x 1 S 2 x 0 S 3 x 2 x 3 x 4 x 5 x 6 Asymptotic Sequence 0 Asymptotic Direction

8 ASYMPTOTIC DIRECTIONS Given a sequence of nonempty neste close sets {S k }, we say that a vector 0 is an asymptotic irection of {S k } if there exists {x k } s. t. x k S k, x k 0, k = 0, 1,... x k, x k x k. S 0 S 1 x 1 S 2 x 0 S 3 x 2 x 3 x 4 x 5 x 6 Asymptotic Sequence 0 Asymptotic Direction A sequence {x k } associate with an asymptotic irection as above is calle an asymptotic sequence corresponing to. Generalizes the known notion of asymptotic irection of a set (rather than a neste set sequence).

9 RETRACTIVE ASYMPTOTIC DIRECTIONS An asymptotic sequence {x k } an corresponing asymptotic irection are calle retractive if there exists k 0 such that x k S k, k k. {S k } is calle retractive if all its asymptotic sequences are retractive. x 1 x 2 x 3 x 0 x 4 x 5 Asymptotic Sequence 0 Asymptotic Direction S 2 S 1 S 0 S 1 0 S 2 x 2 x 1 x 0 S 0 0 x 2 x 1 x 0 (a) (b)

10 RETRACTIVE ASYMPTOTIC DIRECTIONS An asymptotic sequence {x k } an corresponing asymptotic irection are calle retractive if there exists k 0 such that x k S k, k k. {S k } is calle retractive if all its asymptotic sequences are retractive. x 1 x 2 x 3 x 0 x 4 x 5 Asymptotic Sequence 0 Asymptotic Direction S 2 S 1 S 0 S 1 0 S 2 x 2 x 1 x 0 S 0 0 x 2 x 1 x 0 (a) Important observation: A retractive asymptotic sequence {x k } (for large k) gets closer to 0 when shifte in the opposite irection. (b)

11 SET INTERSECTION THEOREM Proposition: The intersection of a retractive neste sequence of close sets is nonempty. Key proof ieas: (a) Consier x k a minimum norm vector from S k. (b) The intersection k=0 S k is empty iff {x k } is unboune. (c) An asymptotic sequence {x k } consisting of minimum norm vectors from the S k cannot be retractive, because {x k } eventually gets closer to 0 when shifte opposite to the asymptotic irection. () Hence {x k } is boune. x 1 x 2 x 3 x 0 x 4 x 5 Asymptotic Sequence 0 Asymptotic Direction

12 CALCULUS OF RETRACTIVE SEQUENCES Unions an intersections of retractive set sequences are retractive. Polyheral sets are retractive. Recall the recession cone R C of a convex set C, an its lineality space L C = R C ( R C ). x + a y x y 0 Recession Cone R C Convex Set C For S k :convex, the set of asymptotic irections of {S k } is the set of nonzero k R Sk.

13 CALCULUS OF RETRACTIVE SEQUENCES Unions an intersections of retractive set sequences are retractive. Polyheral sets are retractive. Recall the recession cone R C of a convex set C, an its lineality space L C = R C ( R C ). x + a y x y 0 Recession Cone R C Convex Set C For S k :convex, the set of asymptotic irections of {S k } is the set of nonzero k R Sk. The vector sum of a compact set an a polyheral cone (e.g., a polyheral set) is retractive. The level sets of a continuous concave function {x f(x) γ} are retractive.

14 EXISTENCE OF SOLUTIONS ISSUES Stanar results on existence of minima of convex functions generalize with simple proofs using the set intersection theorem. Use the set intersection theorem, an existence of optimal solution <=> nonemptiness of (nonempty level sets) Example 1: The set of minima of a close convex function f over a close set X is nonempty if there is no asymptotic irection of X that is a irection of recession of f. Example 2: The set of minima of a close quasiconvex function f over a retractive close set X is nonempty if A R L, where A: set of asymptotic irections of X, an γ k f. R = k=0 R S k, L = k=0 L S k, S k = { x f(x) γ k }

15 LINEAR AND QUADRATIC PROGRAMMING Frank-Wolfe Th: Let X be polyheral an f(x) = x Qx + c x where Q is symmetric (not necessarily positive semiefinite). If the minimal value of f over X is finite, there exists a minimum of f of over X. The proof is straightforwar using the set intersection theorem, an existence of optimal solution <=> nonemptiness of (nonempty level sets)

16 LINEAR AND QUADRATIC PROGRAMMING Frank-Wolfe Th: Let X be polyheral an f(x) = x Qx + c x where Q is symmetric (not necessarily positive semiefinite). If the minimal value of f over X is finite, there exists a minimum of f of over X. The proof is straightforwar using the set intersection theorem, an existence of optimal solution <=> nonemptiness of nonempty level sets. Extensions not covere: X can be the vector sum of a compact set an a polyheral cone. f can be of the form f(x) = p(x Qx) + c x where Q is positive semiefinite an p is a polynomial. These extensions nee the subsequent theory. Reason is that level sets of quaratic functions (an polynomial) are not retractive.

17 MULTIPLE SEQUENCE INTERSECTIONS Key question: Given {Sk 1} an {S2 k }, each with nonempty intersection by itself, an with S 1 k S2 k Ø, for all k, when oes the intersection sequence {Sk 1 Sk 2 } have an empty intersection? Sk 1 S2 : Critical Asymptote Examples inicate that the trouble lies with the existence of a critical asymptote. Critical asymptotes roughly are: Common asymptotic irections such that starting at k Sk 2 an looking at the horizon along, we o not meet k Sk 1 (an similarly with the roles of Sk 1 an S2 k reverse).

18 CRITICAL DIRECTIONS We say that an asymptotic irection of {S k }, with k S k Ø is a horizon irection with respect to a set G if for every x G, we have x + α k S k for all α sufficiently large. We say that an asymptotic irection of {S k } is noncritical with respect to a set G if it is either a horizon irection with respect to G or a retractive horizon irection with respect to k S k. Otherwise, is critical with respect to G. Horizon with respect to Rn Retractive

19 CRITICAL DIRECTIONS We say that an asymptotic irection of {S k }, with k S k Ø is a horizon irection with respect to a set G if for every x G, we have x + α k S k for all α sufficiently large. We say that an asymptotic irection of {S k } is noncritical with respect to a set G if it is either a horizon irection with respect to G or a retractive horizon irection with respect to k S k. Otherwise, is critical with respect to G. Horizon with respect to Rn Retractive Example: The asymptotic irections of a level set sequence of a convex quaratic S k = {x x Qx + c x + b γ k }, γ k 0, are noncritical with respect to R n. (Extension: Convex polynomials, biirectionally flat convex fns.) Example: The as. irections of a vector sum S of a compact an a polyheral set are noncritical (are retractive hor. ir. with resp. to S).

20 EXAMPLE OF CRITICAL DIRECTION Sk 1 S2 : Critical Asymptote Two set sequences, all intersections of a finite number of sets are nonempty. shown is the only common asymptotic irection. is noncritical for S 2 with respect to k S 1 k (because it is retractive). is critical for k S 1 k with respect to S2.

21 CRITICAL DIRECTION THEOREM Roughly it says that: For the intersection of a set sequence {Sk 1 S2 k Sr k } to be empty, some common asymptotic irection must be critical for one of the {S j k } with respect to all the others. Critical Direction Theorem: Consier {Sk 1} an {Sk 2 }, each with nonempty intersection by itself. If S 1 k S2 k Ø for all k, an k=0 (S1 k S2 k ) = Ø, there is a common asymptotic irection that is critical for {S 1 k } with respect to k S 2 k (or for {S2 k } with respect to k S 1 k ). Extens to any finite number of sequences {S j k }.

22 CRITICAL DIRECTION THEOREM Roughly it says that: For the intersection of a set sequence {Sk 1 S2 k Sr k } to be empty, some common asymptotic irection must be critical for one of the {S j k } with respect to all the others. Critical Direction Theorem: Consier {Sk 1} an {Sk 2 }, each with nonempty intersection by itself. If S 1 k S2 k Ø for all k, an k=0 (S1 k S2 k ) = Ø, there is a common asymptotic irection that is critical for {S 1 k } with respect to k S 2 k (or for {S2 k } with respect to k S 1 k ). Extens to any finite number of sequences {S j k }. Special Case: The intersection of set sequences efine by convex polynomial functions S j k = {x p j(x) γ j k, j = 1,..., r}, γj k 0, is nonempty, if all the k S j k an S1 k... Sr k are nonempty. (For example p j may be convex quaratic or biirectionally flat.)

23 EXISTENCE OF SOLUTIONS THEOREMS Convex Quaratic/Polynomial Problems: For j = 0, 1,..., r, let f j : R n R be polynomial convex functions. Then the problem minimize f 0 (x) subject to f j (x) 0, j = 1,..., r, has at least one optimal solution if an only if its optimal value is finite.

24 EXISTENCE OF SOLUTIONS THEOREMS Convex Quaratic/Polynomial Problems: For j = 0, 1,..., r, let f j : R n R be polynomial convex functions. Then the problem minimize f 0 (x) subject to f j (x) 0, j = 1,..., r, has at least one optimal solution if an only if its optimal value is finite. Extene Frank-Wolfe Theorem: Let f(x) = x Qx + c x where Q is symmetric, an let X be a close set whose asymptotic irections are retractive horizon irections with respect to X. If the minimal value of f over X is finite, there exists a minimum of f over X.

### A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION

1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Convex analysis and profit/cost/support functions

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### Separation Properties for Locally Convex Cones

Journal of Convex Analysis Volume 9 (2002), No. 1, 301 307 Separation Properties for Locally Convex Cones Walter Roth Department of Mathematics, Universiti Brunei Darussalam, Gadong BE1410, Brunei Darussalam

### MINIMIZATION OF ENTROPY FUNCTIONALS UNDER MOMENT CONSTRAINTS. denote the family of probability density functions g on X satisfying

MINIMIZATION OF ENTROPY FUNCTIONALS UNDER MOMENT CONSTRAINTS I. Csiszár (Budapest) Given a σ-finite measure space (X, X, µ) and a d-tuple ϕ = (ϕ 1,..., ϕ d ) of measurable functions on X, for a = (a 1,...,

### Lecture 5 Principal Minors and the Hessian

Lecture 5 Principal Minors and the Hessian Eivind Eriksen BI Norwegian School of Management Department of Economics October 01, 2010 Eivind Eriksen (BI Dept of Economics) Lecture 5 Principal Minors and

### Chapter 1. Metric Spaces. Metric Spaces. Examples. Normed linear spaces

Chapter 1. Metric Spaces Metric Spaces MA222 David Preiss d.preiss@warwick.ac.uk Warwick University, Spring 2008/2009 Definitions. A metric on a set M is a function d : M M R such that for all x, y, z

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### {f 1 (U), U F} is an open cover of A. Since A is compact there is a finite subcover of A, {f 1 (U 1 ),...,f 1 (U n )}, {U 1,...

44 CHAPTER 4. CONTINUOUS FUNCTIONS In Calculus we often use arithmetic operations to generate new continuous functions from old ones. In a general metric space we don t have arithmetic, but much of it

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### Appendix A. Appendix. A.1 Algebra. Fields and Rings

Appendix A Appendix A.1 Algebra Algebra is the foundation of algebraic geometry; here we collect some of the basic algebra on which we rely. We develop some algebraic background that is needed in the text.

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### 5.1 Derivatives and Graphs

5.1 Derivatives and Graphs What does f say about f? If f (x) > 0 on an interval, then f is INCREASING on that interval. If f (x) < 0 on an interval, then f is DECREASING on that interval. A function has

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### Readings. D Chapter 1. Lecture 2: Constrained Optimization. Cecilia Fieler. Example: Input Demand Functions. Consumer Problem

Economics 245 January 17, 2012 : Example Readings D Chapter 1 : Example The FOCs are max p ( x 1 + x 2 ) w 1 x 1 w 2 x 2. x 1,x 2 0 p 2 x i w i = 0 for i = 1, 2. These are two equations in two unknowns,

### Witt#5e: Generalizing integrality theorems for ghost-witt vectors [not completed, not proofread]

Witt vectors. Part 1 Michiel Hazewinkel Sienotes by Darij Grinberg Witt#5e: Generalizing integrality theorems for ghost-witt vectors [not complete, not proofrea In this note, we will generalize most of

### Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

### 6.254 : Game Theory with Engineering Applications Lecture 5: Existence of a Nash Equilibrium

6.254 : Game Theory with Engineering Applications Lecture 5: Existence of a Nash Equilibrium Asu Ozdaglar MIT February 18, 2010 1 Introduction Outline Pricing-Congestion Game Example Existence of a Mixed

### 3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions

3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.

MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α

### Convex Optimization SVM s and Kernel Machines

Convex Optimization SVM s and Kernel Machines S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola and Stéphane Canu S.V.N.

### Walrasian Demand. u(x) where B(p, w) = {x R n + : p x w}.

Walrasian Demand Econ 2100 Fall 2015 Lecture 5, September 16 Outline 1 Walrasian Demand 2 Properties of Walrasian Demand 3 An Optimization Recipe 4 First and Second Order Conditions Definition Walrasian

### No: 10 04. Bilkent University. Monotonic Extension. Farhad Husseinov. Discussion Papers. Department of Economics

No: 10 04 Bilkent University Monotonic Extension Farhad Husseinov Discussion Papers Department of Economics The Discussion Papers of the Department of Economics are intended to make the initial results

### MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.

MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar

### (a) Write each of p and q as a polynomial in x with coefficients in Z[y, z]. deg(p) = 7 deg(q) = 9

Homework #01, due 1/20/10 = 9.1.2, 9.1.4, 9.1.6, 9.1.8, 9.2.3 Additional problems for study: 9.1.1, 9.1.3, 9.1.5, 9.1.13, 9.2.1, 9.2.2, 9.2.4, 9.2.5, 9.2.6, 9.3.2, 9.3.3 9.1.1 (This problem was not assigned

### Intersecting Families

Intersecting Families Extremal Combinatorics Philipp Zumstein 1 The Erds-Ko-Rado theorem 2 Projective planes Maximal intersecting families 4 Helly-type result A familiy of sets is intersecting if any two

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

### MA4001 Engineering Mathematics 1 Lecture 10 Limits and Continuity

MA4001 Engineering Mathematics 1 Lecture 10 Limits and Dr. Sarah Mitchell Autumn 2014 Infinite limits If f(x) grows arbitrarily large as x a we say that f(x) has an infinite limit. Example: f(x) = 1 x

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Chapter 13: Basic ring theory

Chapter 3: Basic ring theory Matthew Macauley Department of Mathematical Sciences Clemson University http://www.math.clemson.edu/~macaule/ Math 42, Spring 24 M. Macauley (Clemson) Chapter 3: Basic ring

### Notes on metric spaces

Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.

### Metric Spaces Joseph Muscat 2003 (Last revised May 2009)

1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of

### Functions and Equations

Centre for Education in Mathematics and Computing Euclid eworkshop # Functions and Equations c 014 UNIVERSITY OF WATERLOO Euclid eworkshop # TOOLKIT Parabolas The quadratic f(x) = ax + bx + c (with a,b,c

### (x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

### minimal polyonomial Example

Minimal Polynomials Definition Let α be an element in GF(p e ). We call the monic polynomial of smallest degree which has coefficients in GF(p) and α as a root, the minimal polyonomial of α. Example: We

### Date: April 12, 2001. Contents

2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........

### TECHNICAL NOTE. Proof of a Conjecture by Deutsch, Li, and Swetits on Duality of Optimization Problems1

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 102, No. 3, pp. 697-703, SEPTEMBER 1999 TECHNICAL NOTE Proof of a Conjecture by Deutsch, Li, and Swetits on Duality of Optimization Problems1 H. H.

### MOSEK modeling manual

MOSEK modeling manual August 12, 2014 Contents 1 Introduction 1 2 Linear optimization 3 2.1 Introduction....................................... 3 2.1.1 Basic notions.................................. 3

### MA107 Precalculus Algebra Exam 2 Review Solutions

MA107 Precalculus Algebra Exam 2 Review Solutions February 24, 2008 1. The following demand equation models the number of units sold, x, of a product as a function of price, p. x = 4p + 200 a. Please write

### Understanding Basic Calculus

Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other

### A FIRST COURSE IN OPTIMIZATION THEORY

A FIRST COURSE IN OPTIMIZATION THEORY RANGARAJAN K. SUNDARAM New York University CAMBRIDGE UNIVERSITY PRESS Contents Preface Acknowledgements page xiii xvii 1 Mathematical Preliminaries 1 1.1 Notation

### SOLUTIONS. f x = 6x 2 6xy 24x, f y = 3x 2 6y. To find the critical points, we solve

SOLUTIONS Problem. Find the critical points of the function f(x, y = 2x 3 3x 2 y 2x 2 3y 2 and determine their type i.e. local min/local max/saddle point. Are there any global min/max? Partial derivatives

### Math 230.01, Fall 2012: HW 1 Solutions

Math 3., Fall : HW Solutions Problem (p.9 #). Suppose a wor is picke at ranom from this sentence. Fin: a) the chance the wor has at least letters; SOLUTION: All wors are equally likely to be chosen. The

### 2 Complex Functions and the Cauchy-Riemann Equations

2 Complex Functions and the Cauchy-Riemann Equations 2.1 Complex functions In one-variable calculus, we study functions f(x) of a real variable x. Likewise, in complex analysis, we study functions f(z)

### The Ideal Class Group

Chapter 5 The Ideal Class Group We will use Minkowski theory, which belongs to the general area of geometry of numbers, to gain insight into the ideal class group of a number field. We have already mentioned

### The Dirichlet Unit Theorem

Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if

### Math 120 Final Exam Practice Problems, Form: A

Math 120 Final Exam Practice Problems, Form: A Name: While every attempt was made to be complete in the types of problems given below, we make no guarantees about the completeness of the problems. Specifically,

### Solutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR

Solutions Of Some Non-Linear Programming Problems A PROJECT REPORT submitted by BIJAN KUMAR PATEL for the partial fulfilment for the award of the degree of Master of Science in Mathematics under the supervision

### 10. Proximal point method

L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

### On Stability Properties of Economic Solution Concepts

On Stability Properties of Economic Solution Concepts Richard J. Lipton Vangelis Markakis Aranyak Mehta Abstract In this note we investigate the stability of game theoretic and economic solution concepts

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### The Method of Partial Fractions Math 121 Calculus II Spring 2015

Rational functions. as The Method of Partial Fractions Math 11 Calculus II Spring 015 Recall that a rational function is a quotient of two polynomials such f(x) g(x) = 3x5 + x 3 + 16x x 60. The method

### 1 = (a 0 + b 0 α) 2 + + (a m 1 + b m 1 α) 2. for certain elements a 0,..., a m 1, b 0,..., b m 1 of F. Multiplying out, we obtain

Notes on real-closed fields These notes develop the algebraic background needed to understand the model theory of real-closed fields. To understand these notes, a standard graduate course in algebra is

### On the greatest and least prime factors of n! + 1, II

Publ. Math. Debrecen Manuscript May 7, 004 On the greatest and least prime factors of n! + 1, II By C.L. Stewart In memory of Béla Brindza Abstract. Let ε be a positive real number. We prove that 145 1

### THE PRIME NUMBER THEOREM

THE PRIME NUMBER THEOREM NIKOLAOS PATTAKOS. introduction In number theory, this Theorem describes the asymptotic distribution of the prime numbers. The Prime Number Theorem gives a general description

### Slant asymptote. This means a diagonal line y = mx + b which is approached by a graph y = f(x). For example, consider the function: 2x 2 8x.

Math 32 Curve Sketching Stewart 3.5 Man vs machine. In this section, we learn methods of drawing graphs by hand. The computer can do this much better simply by plotting many points, so why bother with

### FIRST YEAR CALCULUS. Chapter 7 CONTINUITY. It is a parabola, and we can draw this parabola without lifting our pencil from the paper.

FIRST YEAR CALCULUS WWLCHENW L c WWWL W L Chen, 1982, 2008. 2006. This chapter originates from material used by the author at Imperial College, University of London, between 1981 and 1990. It It is is

### FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES

FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied

### Notes V General Equilibrium: Positive Theory. 1 Walrasian Equilibrium and Excess Demand

Notes V General Equilibrium: Positive Theory In this lecture we go on considering a general equilibrium model of a private ownership economy. In contrast to the Notes IV, we focus on positive issues such

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### Max-Min Representation of Piecewise Linear Functions

Beiträge zur Algebra und Geometrie Contributions to Algebra and Geometry Volume 43 (2002), No. 1, 297-302. Max-Min Representation of Piecewise Linear Functions Sergei Ovchinnikov Mathematics Department,

### Math 5311 Gateaux differentials and Frechet derivatives

Math 5311 Gateaux differentials and Frechet derivatives Kevin Long January 26, 2009 1 Differentiation in vector spaces Thus far, we ve developed the theory of minimization without reference to derivatives.

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### Chapter 4, Arithmetic in F [x] Polynomial arithmetic and the division algorithm.

Chapter 4, Arithmetic in F [x] Polynomial arithmetic and the division algorithm. We begin by defining the ring of polynomials with coefficients in a ring R. After some preliminary results, we specialize

### The Chebotarev Density Theorem

The Chebotarev Density Theorem Hendrik Lenstra 1. Introduction Consider the polynomial f(x) = X 4 X 2 1 Z[X]. Suppose that we want to show that this polynomial is irreducible. We reduce f modulo a prime

### (January 14, 2009) End k (V ) End k (V/W )

(January 14, 29) [16.1] Let p be the smallest prime dividing the order of a finite group G. Show that a subgroup H of G of index p is necessarily normal. Let G act on cosets gh of H by left multiplication.

### The fundamental group of the Hawaiian earring is not free (International Journal of Algebra and Computation Vol. 2, No. 1 (1992), 33 37) Bart de Smit

The fundamental group of the Hawaiian earring is not free Bart de Smit The fundamental group of the Hawaiian earring is not free (International Journal of Algebra and Computation Vol. 2, No. 1 (1992),

### Polynomial & Rational Functions

4 Polynomial & Rational Functions 45 Rational Functions A function f is a rational function if there exist polynomial functions p and q, with q not the zero function, such that p(x) q(x) for all x for

### Multiplicity. Chapter 6

Chapter 6 Multiplicity The fundamental theorem of algebra says that any polynomial of degree n 0 has exactly n roots in the complex numbers if we count with multiplicity. The zeros of a polynomial are

### The Phase Plane. Phase portraits; type and stability classifications of equilibrium solutions of systems of differential equations

The Phase Plane Phase portraits; type and stability classifications of equilibrium solutions of systems of differential equations Phase Portraits of Linear Systems Consider a systems of linear differential

### 8 Polynomials Worksheet

8 Polynomials Worksheet Concepts: Quadratic Functions The Definition of a Quadratic Function Graphs of Quadratic Functions - Parabolas Vertex Absolute Maximum or Absolute Minimum Transforming the Graph

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### HOMEWORK 4 SOLUTIONS. All questions are from Vector Calculus, by Marsden and Tromba

HOMEWORK SOLUTIONS All questions are from Vector Calculus, by Marsden and Tromba Question :..6 Let w = f(x, y) be a function of two variables, and let x = u + v, y = u v. Show that Solution. By the chain

### Cartesian Products and Relations

Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 14 10/27/2008 MOMENT GENERATING FUNCTIONS Contents 1. Moment generating functions 2. Sum of a ranom number of ranom variables 3. Transforms

### Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product

Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### TRIPLE POSITIVE SOLUTIONS FOR BOUNDARY VALUE PROBLEM OF A NONLINEAR FRACTIONAL DIFFERENTIAL EQUATION. Communicated by Mohammad Asadzadeh

Bulletin of the Iranian Mathematical Society Vol. 33 No. 2 (27), pp -. TRIPLE POSITIVE SOLUTIONS FOR BOUNDARY VALUE PROBLEM OF A NONLINEAR FRACTIONAL DIFFERENTIAL EQUATION R. DEHGHANI AND K. GHANBARI*

### ON A GLOBALIZATION PROPERTY

APPLICATIONES MATHEMATICAE 22,1 (1993), pp. 69 73 S. ROLEWICZ (Warszawa) ON A GLOBALIZATION PROPERTY Abstract. Let (X, τ) be a topological space. Let Φ be a class of realvalued functions defined on X.

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### Metric Spaces. Chapter 1

Chapter 1 Metric Spaces Many of the arguments you have seen in several variable calculus are almost identical to the corresponding arguments in one variable calculus, especially arguments concerning convergence

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### Some Optimization Fundamentals

ISyE 3133B Engineering Optimization Some Optimization Fundamentals Shabbir Ahmed E-mail: sahmed@isye.gatech.edu Homepage: www.isye.gatech.edu/~sahmed Basic Building Blocks min or max s.t. objective as

### MATH 110 Spring 2015 Homework 6 Solutions

MATH 110 Spring 2015 Homework 6 Solutions Section 2.6 2.6.4 Let α denote the standard basis for V = R 3. Let α = {e 1, e 2, e 3 } denote the dual basis of α for V. We would first like to show that β =

### INTRODUCTORY SET THEORY

M.Sc. program in mathematics INTRODUCTORY SET THEORY Katalin Károlyi Department of Applied Analysis, Eötvös Loránd University H-1088 Budapest, Múzeum krt. 6-8. CONTENTS 1. SETS Set, equal sets, subset,

### WARM UP EXERCSE. 2-1 Polynomials and Rational Functions

WARM UP EXERCSE Roots, zeros, and x-intercepts. x 2! 25 x 2 + 25 x 3! 25x polynomial, f (a) = 0! (x - a)g(x) 1 2-1 Polynomials and Rational Functions Students will learn about: Polynomial functions Behavior

### International Journal of Pure and Applied Mathematics Volume 46 No. 1 2008, 37-44

International Journal of Pure and Applied Mathematics Volume 46 No. 1 2008, 37-44 ON THE FIXED POINTS OF A FUNCTION AND THE FIXED POINTS OF ITS COMPOSITE FUNCTIONS Mohammad K. Azarian Department of Mathematics

### PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.

PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include

### Optimisation et simulation numérique.

Optimisation et simulation numérique. Lecture 1 A. d Aspremont. M2 MathSV: Optimisation et simulation numérique. 1/106 Today Convex optimization: introduction Course organization and other gory details...

### This asserts two sets are equal iff they have the same elements, that is, a set is determined by its elements.

3. Axioms of Set theory Before presenting the axioms of set theory, we first make a few basic comments about the relevant first order logic. We will give a somewhat more detailed discussion later, but