Convexity I: Sets and Functions
|
|
- Brittney Jenkins
- 7 years ago
- Views:
Transcription
1 Convexity I: Sets and Functions Lecturer: Ryan Tibshirani Convex Optimization / See supplements for reviews of basic real analysis basic multivariate calculus basic linear algebra
2 Last time: why convexity? Why convexity? Simply put: because we can broadly understand and solve convex optimization problems Nonconvex problems are mostly treated on a case by case basis Reminder: a convex optimization problem is of the form min x D f(x) subject to g i (x) 0, i = 1,... m h j (x) = 0, j = 1,... r where f and g i, i = 1,... m are all convex, and h j, j = 1,... r are affine. Special property: any local minimizer is a global minimizer 2
3 Outline Today: Convex sets Examples Key properties Operations preserving convexity Same, for convex functions 3
4 Convex sets Convex set: C R n such that x, y C = tx + (1 t)y C for all 0 t 1 In words, line segment joining any two elements lies entirely in set Convex combination of x 1,... x k R n : any linear combination θ 1 x θ k x k with θ i 0, i = 1,... k, and k i=1 θ i = 1. Convex hull of a set C, conv(c), is all convex combinations of elements. Always convex 4
5 Examples of convex sets Trivial ones: empty set, point, line Norm ball: {x : x r}, for given norm, radius r Hyperplane: {x : a T x = b}, for given a, b Halfspace: {x : a T x b} Affine space: {x : Ax = b}, for given A, b 5
6 Polyhedron: {x : Ax b}, where inequality is interpreted componentwise. Note: the set {x : Ax b, Cx = d} is also a polyhedron (why?) a1 a2 P a5 a3 a4 Simplex: special case of polyhedra, given by conv{x 0,... x k }, where these points are affinely independent. The canonical example is the probability simplex, conv{e 1,... e n } = {w : w 0, 1 T w = 1} 6
7 Cone: C R n such that Cones x C = tx C for all t 0 Convex cone: cone that is also convex, i.e., x 1, x 2 C = t 1 x 1 + t 2 x 2 C for all t 1, t 2 0 x1 x2 0 Conic combination of x 1,... x k R n : any linear combination θ 1 x θ k x k with θ i 0, i = 1,... k. Conic hull collects all conic combinations 7
8 Examples of convex cones Norm cone: {(x, t) : x t}, for a norm. Under l 2 norm 2, called second-order cone Normal cone: given any set C and point x C, we can define N C (x) = {g : g T x g T y, for all y C} This is always a convex cone, regardless of C Positive semidefinite cone: S n + = {X S n : X 0}, where X 0 means that X is positive semidefinite (and S n is the set of n n symmetric matrices) 8
9 Key properties of convex sets Separating hyperplane theorem: two disjoint convex sets have a separating between hyperplane them a T x b a T x b D C a Formally: if C, D are nonempty convex sets with C D =, then there exists a, b such that C {x : a T x b} D {x : a T x b} 9
10 Supporting hyperplane theorem: a boundary point of a convex set has a supporting hyperplane passing through it Formally: if C is a nonempty convex set, and x 0 bd(c), then there exists a such that C {x : a T x a T x 0 } Both of the above theorems (separating and supporting hyperplane theorems) have partial converses; see Section 2.5 of BV 10
11 11 Operations preserving convexity Intersection: the intersection of convex sets is convex Scaling and translation: if C is convex, then is convex for any a, b ac + b = {ax + b : x C} Affine images and preimages: if f(x) = Ax + b and C is convex then f(c) = {f(x) : x C} is convex, and if D is convex then is convex f 1 (D) = {x : f(x) D}
12 12 Example: linear matrix inequality solution set Given A 1,... A k, B S n, a linear matrix inequality is of the form x 1 A 1 + x 2 A x k A k B for a variable x R k. Let s prove the set C of points x that satisfy the above inequality is convex Approach 1: directly verify that x, y C tx + (1 t)y C. This follows by checking that, for any v, v T ( B k (tx i + (1 t)y i )A i )v 0 i=1 Approach 2: let f : R k S n, f(x) = B k i=1 x ia i. Note that C = f 1 (S n +), affine preimage of convex set
13 13 Example: Fantope Given some integer k 0, the Fantope of order k is { } F k = Z S n : 0 Z I, tr(z) = k where recall the trace operator tr(z) = n i=1 Z ii is the sum of the diagonal entries. Let s prove that F k is convex Approach 1: verify that 0 Z, W I and tr(z) = tr(w ) = I implies the same for tz + (1 t)w Approach 2: recognize the fact that F k = {Z S n : Z 0} {Z S n : Z I} {Z S n : tr(z) = k} intersection of linear inequality and equality constraints (hence like a polyhedron but for matrices)
14 14 More operations preserving convexity Perspective images and preimages: the perspective function is P : R n R ++ R n (where R ++ denotes positive reals), P (x, z) = x/z for z > 0. If C dom(p ) is convex then so is P (C), and if D is convex then so is P 1 (D) Linear-fractional images and preimages: the perspective map composed with an affine function, f(x) = Ax + b c T x + d is called a linear-fractional function, defined on c T x + d > 0. If C dom(f) is convex then so if f(c), and if D is convex then so is f 1 (D)
15 15 Example: conditional probability set Let U, V be random variables over {1,... n} and {1,... m}. Let C R nm be a set of joint distributions for U, V, i.e., each p C defines joint probabilities p ij = P(U = i, V = j) Let D R nm contain corresponding conditional distributions, i.e., each q D defines q ij = P(U = i V = j) Assume C is convex. Let s prove that D is convex. Write { D = q R nm p } ij : q ij = n k=1 p, for some p C = f(c) kj where f is a linear-fractional function, hence D is convex
16 16 Convex functions Convex function: f : R n R such that dom(f) R n convex, and f(tx + (1 t)y) tf(x) + (1 t)f(y) for 0 t 1 and all x, y dom(f) (x, f(x)) (y, f(y)) In words, f lies below the line segment joining f(x), f(y) Concave function: opposite inequality above, so that f concave f convex
17 17 Important modifiers: Strictly convex: f ( tx + (1 t)y ) < tf(x) + (1 t)f(y) for x y and 0 < t < 1. In words, f is convex and has greater curvature than a linear function Strongly convex with parameter m > 0: f m 2 x 2 2 is convex. In words, f is at least as convex as a quadratic function Note: strongly convex strictly convex convex (Analogously for concave functions)
18 18 Examples of convex functions Univariate functions: Exponential function: e ax is convex for any a over R Power function: x a is convex for a 1 or a 0 over R + (nonnegative reals) Power function: x a is concave for 0 a 1 over R + Logarithmic function: log x is concave over R++ Affine function: a T x + b is both convex and concave Quadratic function: 1 2 xt Qx + b T x + c is convex provided that Q 0 (positive semidefinite) Least squares loss: y Ax 2 2 is always convex (since AT A is always positive semidefinite)
19 19 Norm: x is convex for any norm; e.g., l p norms, x p = ( n i=1 x p i ) 1/p for p 1, x = max i=1,...n x i and also operator (spectral) and trace (nuclear) norms, X op = σ 1 (X), X tr = r σ r (X) i=1 where σ 1 (X)... σ r (X) 0 are the singular values of the matrix X
20 20 Indicator function: if C is convex, then its indicator function { 0 x C I C (x) = x / C is convex Support function: for any set C (convex or not), its support function IC(x) = max y C xt y is convex Max function: f(x) = max{x 1,... x n } is convex
21 21 Key properties of convex functions A function is convex if and only if its restriction to any line is convex Epigraph characterization: a function f is convex if and only if its epigraph is a convex set epi(f) = {(x, t) dom(f) R : f(x) t} Convex sublevel sets: if f is convex, then its sublevel sets {x dom(f) : f(x) t} are convex, for all t R. The converse is not true
22 22 First-order characterization: if f is differentiable, then f is convex if and only if dom(f) is convex, and f(y) f(x) + f(x) T (y x) for all x, y dom(f). Therefore for a differentiable convex function f(x) = 0 x minimizes f Second-order characterization: if f is twice differentiable, then f is convex if and only if dom(f) is convex, and 2 f(x) 0 for all x dom(f) Jensen s inequality: if f is convex, and X is a random variable supported on dom(f), then f(e[x]) E[f(X)]
23 23 Operations preserving convexity Nonnegative linear combination: f 1,... f m convex implies a 1 f a m f m convex for any a 1,... a m 0 Pointwise maximization: if f s is convex for any s S, then f(x) = max s S f s (x) is convex. Note that the set S here (number of functions f s ) can be infinite Partial minimization: if g(x, y) is convex in x, y, and C is convex, then f(x) = min y C g(x, y) is convex
24 24 Example: distances to a set Let C be an arbitrary set, and consider the maximum distance to C under an arbitrary norm : f(x) = max y C x y Let s check this is convex: f y (x) = x y is convex in x for any fixed y, so by pointwise maximization rule, f is convex Now let C be convex, and consider the minimum distance to C: f(x) = min y C x y Let s check this is convex: g(x, y) = x y is convex in x, y jointly, and C is assumed convex, so apply partial minimization rule
25 25 More operations preserving convexity Affine composition: f convex implies g(x) = f(ax + b) convex General composition: suppose f = h g, where g : R n R, h : R R, f : R n R. Then: f is convex if h is convex and nondecreasing, g is convex f is convex if h is convex and nonincreasing, g is concave f is concave if h is concave and nondecreasing, g concave f is concave if h is concave and nonincreasing, g convex How to remember these? Think of the chain rule when n = 1: f (x) = h (g(x))g (x) 2 + h (g(x))g (x)
26 26 Vector composition: suppose that f(x) = h ( g(x) ) = h ( g 1 (x),... g k (x) ) where g : R n R k, h : R k R, f : R n R. Then: f is convex if h is convex and nondecreasing in each argument, g is convex f is convex if h is convex and nonincreasing in each argument, g is concave f is concave if h is concave and nondecreasing in each argument, g is concave f is concave if h is concave and nonincreasing in each argument, g is convex
27 27 Example: log-sum-exp function Log-sum-exp function: g(x) = log( k i=1 eat i x+b i ), for fixed a i, b i, i = 1,... k. Often called soft max, as it smoothly approximates max i=1,...k (a T i x + b i) How to show convexity? First, note it suffices to prove convexity of f(x) = log( n i=1 ex i ) (affine composition rule) Now use second-order characterization. Calculate i f(x) = 2 ijf(x) = e x i n l=1 ex l e x i e x i e x j n 1{i = j} l=1 ex l ( n l=1 ex l ) 2 Write 2 f(x) = diag(z) zz T, where z i = e x i /( n l=1 ex l). This matrix is diagonally dominant, hence positive semidefinite
28 28 References and further reading S. Boyd and L. Vandenberghe (2004), Convex optimization, Chapters 2 and 3 J.P. Hiriart-Urruty and C. Lemarechal (1993), Fundamentals of convex analysis, Chapters A and B R. T. Rockafellar (1970), Convex analysis, Chapters 1 10,
3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More information2.3 Convex Constrained Optimization Problems
42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions
More informationOptimisation et simulation numérique.
Optimisation et simulation numérique. Lecture 1 A. d Aspremont. M2 MathSV: Optimisation et simulation numérique. 1/106 Today Convex optimization: introduction Course organization and other gory details...
More informationDuality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725
Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T
More informationWhat is Linear Programming?
Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to
More informationConvex analysis and profit/cost/support functions
CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the
More informationLecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method
Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming
More informationNotes on Symmetric Matrices
CPSC 536N: Randomized Algorithms 2011-12 Term 2 Notes on Symmetric Matrices Prof. Nick Harvey University of British Columbia 1 Symmetric Matrices We review some basic results concerning symmetric matrices.
More informationCHAPTER 9. Integer Programming
CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral
More informationLecture 5 Principal Minors and the Hessian
Lecture 5 Principal Minors and the Hessian Eivind Eriksen BI Norwegian School of Management Department of Economics October 01, 2010 Eivind Eriksen (BI Dept of Economics) Lecture 5 Principal Minors and
More information10. Proximal point method
L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing
More informationLinear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
More informationNonlinear Programming Methods.S2 Quadratic Programming
Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective
More informationA NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION
1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of
More informationDuality of linear conic problems
Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least
More information1 Norms and Vector Spaces
008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)
More informationBindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8
Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e
More information(Basic definitions and properties; Separation theorems; Characterizations) 1.1 Definition, examples, inner description, algebraic properties
Lecture 1 Convex Sets (Basic definitions and properties; Separation theorems; Characterizations) 1.1 Definition, examples, inner description, algebraic properties 1.1.1 A convex set In the school geometry
More informationNotes on Orthogonal and Symmetric Matrices MENU, Winter 2013
Notes on Orthogonal and Symmetric Matrices MENU, Winter 201 These notes summarize the main properties and uses of orthogonal and symmetric matrices. We covered quite a bit of material regarding these topics,
More informationNP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems
NP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems Amir Ali Ahmadi, Alex Olshevsky, Pablo A. Parrilo, and John N. Tsitsiklis Abstract We show that unless P=NP, there exists no
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More informationMATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set.
MATH 304 Linear Algebra Lecture 9: Subspaces of vector spaces (continued). Span. Spanning set. Vector space A vector space is a set V equipped with two operations, addition V V (x,y) x + y V and scalar
More informationSolutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR
Solutions Of Some Non-Linear Programming Problems A PROJECT REPORT submitted by BIJAN KUMAR PATEL for the partial fulfilment for the award of the degree of Master of Science in Mathematics under the supervision
More informationNP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems
NP-hardness of Deciding Convexity of Quartic Polynomials and Related Problems Amir Ali Ahmadi, Alex Olshevsky, Pablo A. Parrilo, and John N. Tsitsiklis Abstract We show that unless P=NP, there exists no
More informationTOPIC 4: DERIVATIVES
TOPIC 4: DERIVATIVES 1. The derivative of a function. Differentiation rules 1.1. The slope of a curve. The slope of a curve at a point P is a measure of the steepness of the curve. If Q is a point on the
More informationWalrasian Demand. u(x) where B(p, w) = {x R n + : p x w}.
Walrasian Demand Econ 2100 Fall 2015 Lecture 5, September 16 Outline 1 Walrasian Demand 2 Properties of Walrasian Demand 3 An Optimization Recipe 4 First and Second Order Conditions Definition Walrasian
More informationLecture 1: Schur s Unitary Triangularization Theorem
Lecture 1: Schur s Unitary Triangularization Theorem This lecture introduces the notion of unitary equivalence and presents Schur s theorem and some of its consequences It roughly corresponds to Sections
More information. P. 4.3 Basic feasible solutions and vertices of polyhedra. x 1. x 2
4. Basic feasible solutions and vertices of polyhedra Due to the fundamental theorem of Linear Programming, to solve any LP it suffices to consider the vertices (finitely many) of the polyhedron P of the
More informationNotes from Week 1: Algorithms for sequential prediction
CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking
More informationLecture 7: Finding Lyapunov Functions 1
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.243j (Fall 2003): DYNAMICS OF NONLINEAR SYSTEMS by A. Megretski Lecture 7: Finding Lyapunov Functions 1
More informationPUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.
PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include
More informationHomework # 3 Solutions
Homework # 3 Solutions February, 200 Solution (2.3.5). Noting that and ( + 3 x) x 8 = + 3 x) by Equation (2.3.) x 8 x 8 = + 3 8 by Equations (2.3.7) and (2.3.0) =3 x 8 6x2 + x 3 ) = 2 + 6x 2 + x 3 x 8
More informationLinear Maps. Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (February 5, 2007)
MAT067 University of California, Davis Winter 2007 Linear Maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (February 5, 2007) As we have discussed in the lecture on What is Linear Algebra? one of
More informationSummer course on Convex Optimization. Fifth Lecture Interior-Point Methods (1) Michel Baes, K.U.Leuven Bharath Rangarajan, U.
Summer course on Convex Optimization Fifth Lecture Interior-Point Methods (1) Michel Baes, K.U.Leuven Bharath Rangarajan, U.Minnesota Interior-Point Methods: the rebirth of an old idea Suppose that f is
More information1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
More informationChapter 6. Cuboids. and. vol(conv(p ))
Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both
More informationPattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
More informationLinear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
More informationIncreasing for all. Convex for all. ( ) Increasing for all (remember that the log function is only defined for ). ( ) Concave for all.
1. Differentiation The first derivative of a function measures by how much changes in reaction to an infinitesimal shift in its argument. The largest the derivative (in absolute value), the faster is evolving.
More informationLinear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.
Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.
More informationConvex Optimization. Lieven Vandenberghe University of California, Los Angeles
Convex Optimization Lieven Vandenberghe University of California, Los Angeles Tutorial lectures, Machine Learning Summer School University of Cambridge, September 3-4, 2009 Sources: Boyd & Vandenberghe,
More informationProximal mapping via network optimization
L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:
More informationCompletely Positive Cone and its Dual
On the Computational Complexity of Membership Problems for the Completely Positive Cone and its Dual Peter J.C. Dickinson Luuk Gijben July 3, 2012 Abstract Copositive programming has become a useful tool
More informationConvex Rationing Solutions (Incomplete Version, Do not distribute)
Convex Rationing Solutions (Incomplete Version, Do not distribute) Ruben Juarez rubenj@hawaii.edu January 2013 Abstract This paper introduces a notion of convexity on the rationing problem and characterizes
More informationALMOST COMMON PRIORS 1. INTRODUCTION
ALMOST COMMON PRIORS ZIV HELLMAN ABSTRACT. What happens when priors are not common? We introduce a measure for how far a type space is from having a common prior, which we term prior distance. If a type
More informationLectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain
Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal
More informationLecture 2: August 29. Linear Programming (part I)
10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.
More informationEigenvalues, Eigenvectors, Matrix Factoring, and Principal Components
Eigenvalues, Eigenvectors, Matrix Factoring, and Principal Components The eigenvalues and eigenvectors of a square matrix play a key role in some important operations in statistics. In particular, they
More informationAdditional Exercises for Convex Optimization
Additional Exercises for Convex Optimization Stephen Boyd Lieven Vandenberghe February 11, 2016 This is a collection of additional exercises, meant to supplement those found in the book Convex Optimization,
More informationIntroduction to Algebraic Geometry. Bézout s Theorem and Inflection Points
Introduction to Algebraic Geometry Bézout s Theorem and Inflection Points 1. The resultant. Let K be a field. Then the polynomial ring K[x] is a unique factorisation domain (UFD). Another example of a
More informationLS.6 Solution Matrices
LS.6 Solution Matrices In the literature, solutions to linear systems often are expressed using square matrices rather than vectors. You need to get used to the terminology. As before, we state the definitions
More informationAdaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
More informationLinear Programming I
Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins
More informationF Matrix Calculus F 1
F Matrix Calculus F 1 Appendix F: MATRIX CALCULUS TABLE OF CONTENTS Page F1 Introduction F 3 F2 The Derivatives of Vector Functions F 3 F21 Derivative of Vector with Respect to Vector F 3 F22 Derivative
More information1 VECTOR SPACES AND SUBSPACES
1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such
More informationLargest Fixed-Aspect, Axis-Aligned Rectangle
Largest Fixed-Aspect, Axis-Aligned Rectangle David Eberly Geometric Tools, LLC http://www.geometrictools.com/ Copyright c 1998-2016. All Rights Reserved. Created: February 21, 2004 Last Modified: February
More information(a) We have x = 3 + 2t, y = 2 t, z = 6 so solving for t we get the symmetric equations. x 3 2. = 2 y, z = 6. t 2 2t + 1 = 0,
Name: Solutions to Practice Final. Consider the line r(t) = 3 + t, t, 6. (a) Find symmetric equations for this line. (b) Find the point where the first line r(t) intersects the surface z = x + y. (a) We
More informationContinuity of the Perron Root
Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North
More informationNonconvex? NP! (No Problem!) Ryan Tibshirani Convex Optimization 10-725/36-725
Nonconvex? NP! (No Problem!) Ryan Tibshirani Convex Optimization 10-725/36-725 1 Beyond the tip? 2 Some takeaway points If possible, formulate task in terms of convex optimization typically easier to solve,
More informationMathematical finance and linear programming (optimization)
Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may
More informationHow To Prove The Dirichlet Unit Theorem
Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if
More informationSF2940: Probability theory Lecture 8: Multivariate Normal Distribution
SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,
More informationIdeal Class Group and Units
Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals
More informationSection 1.1. Introduction to R n
The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to
More informationLecture 13 Linear quadratic Lyapunov theory
EE363 Winter 28-9 Lecture 13 Linear quadratic Lyapunov theory the Lyapunov equation Lyapunov stability conditions the Lyapunov operator and integral evaluating quadratic integrals analysis of ARE discrete-time
More information1 Solving LPs: The Simplex Algorithm of George Dantzig
Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.
More informationMathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
More information1 Homework 1. [p 0 q i+j +... + p i 1 q j+1 ] + [p i q j ] + [p i+1 q j 1 +... + p i+j q 0 ]
1 Homework 1 (1) Prove the ideal (3,x) is a maximal ideal in Z[x]. SOLUTION: Suppose we expand this ideal by including another generator polynomial, P / (3, x). Write P = n + x Q with n an integer not
More informationORIENTATIONS. Contents
ORIENTATIONS Contents 1. Generators for H n R n, R n p 1 1. Generators for H n R n, R n p We ended last time by constructing explicit generators for H n D n, S n 1 by using an explicit n-simplex which
More informationFACTORIZATION OF TROPICAL POLYNOMIALS IN ONE AND SEVERAL VARIABLES. Nathan B. Grigg. Submitted to Brigham Young University in partial fulfillment
FACTORIZATION OF TROPICAL POLYNOMIALS IN ONE AND SEVERAL VARIABLES by Nathan B. Grigg Submitted to Brigham Young University in partial fulfillment of graduation requirements for University Honors Department
More informationAlgebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.
This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra
More informationSection 13.5 Equations of Lines and Planes
Section 13.5 Equations of Lines and Planes Generalizing Linear Equations One of the main aspects of single variable calculus was approximating graphs of functions by lines - specifically, tangent lines.
More informationMetric Spaces. Chapter 7. 7.1. Metrics
Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some
More informationClassification of Cartan matrices
Chapter 7 Classification of Cartan matrices In this chapter we describe a classification of generalised Cartan matrices This classification can be compared as the rough classification of varieties in terms
More information1 Lecture: Integration of rational functions by decomposition
Lecture: Integration of rational functions by decomposition into partial fractions Recognize and integrate basic rational functions, except when the denominator is a power of an irreducible quadratic.
More informationMath 4310 Handout - Quotient Vector Spaces
Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
More informationMaster s Theory Exam Spring 2006
Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem
More informationDiscuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
More informationSolving Systems of Linear Equations
LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how
More informationLecture 4: BK inequality 27th August and 6th September, 2007
CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound
More informationLimits and Continuity
Math 20C Multivariable Calculus Lecture Limits and Continuity Slide Review of Limit. Side limits and squeeze theorem. Continuous functions of 2,3 variables. Review: Limits Slide 2 Definition Given a function
More informationModern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh
Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem
More information9 MATRICES AND TRANSFORMATIONS
9 MATRICES AND TRANSFORMATIONS Chapter 9 Matrices and Transformations Objectives After studying this chapter you should be able to handle matrix (and vector) algebra with confidence, and understand the
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number
More informationLinear Programming: Theory and Applications
Linear Programming: Theory and Applications Catherine Lewis May 11, 2008 1 Contents 1 Introduction to Linear Programming 3 1.1 What is a linear program?...................... 3 1.2 Assumptions.............................
More informationSeveral Views of Support Vector Machines
Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min
More informationMathematical Methods of Engineering Analysis
Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................
More informationInner product. Definition of inner product
Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product
More informationUnderstanding Basic Calculus
Understanding Basic Calculus S.K. Chung Dedicated to all the people who have helped me in my life. i Preface This book is a revised and expanded version of the lecture notes for Basic Calculus and other
More information1 Introduction to Matrices
1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns
More informationMath 120 Final Exam Practice Problems, Form: A
Math 120 Final Exam Practice Problems, Form: A Name: While every attempt was made to be complete in the types of problems given below, we make no guarantees about the completeness of the problems. Specifically,
More informationMATH 425, PRACTICE FINAL EXAM SOLUTIONS.
MATH 45, PRACTICE FINAL EXAM SOLUTIONS. Exercise. a Is the operator L defined on smooth functions of x, y by L u := u xx + cosu linear? b Does the answer change if we replace the operator L by the operator
More information6.207/14.15: Networks Lecture 15: Repeated Games and Cooperation
6.207/14.15: Networks Lecture 15: Repeated Games and Cooperation Daron Acemoglu and Asu Ozdaglar MIT November 2, 2009 1 Introduction Outline The problem of cooperation Finitely-repeated prisoner s dilemma
More informationActually Doing It! 6. Prove that the regular unit cube (say 1cm=unit) of sufficiently high dimension can fit inside it the whole city of New York.
1: 1. Compute a random 4-dimensional polytope P as the convex hull of 10 random points using rand sphere(4,10). Run VISUAL to see a Schlegel diagram. How many 3-dimensional polytopes do you see? How many
More informationOn Minimal Valid Inequalities for Mixed Integer Conic Programs
On Minimal Valid Inequalities for Mixed Integer Conic Programs Fatma Kılınç Karzan June 27, 2013 Abstract We study mixed integer conic sets involving a general regular (closed, convex, full dimensional,
More informationON TORI TRIANGULATIONS ASSOCIATED WITH TWO-DIMENSIONAL CONTINUED FRACTIONS OF CUBIC IRRATIONALITIES.
ON TORI TRIANGULATIONS ASSOCIATED WITH TWO-DIMENSIONAL CONTINUED FRACTIONS OF CUBIC IRRATIONALITIES. O. N. KARPENKOV Introduction. A series of properties for ordinary continued fractions possesses multidimensional
More information19 LINEAR QUADRATIC REGULATOR
19 LINEAR QUADRATIC REGULATOR 19.1 Introduction The simple form of loopshaping in scalar systems does not extend directly to multivariable (MIMO) plants, which are characterized by transfer matrices instead
More informationBX in ( u, v) basis in two ways. On the one hand, AN = u+
1. Let f(x) = 1 x +1. Find f (6) () (the value of the sixth derivative of the function f(x) at zero). Answer: 7. We expand the given function into a Taylor series at the point x = : f(x) = 1 x + x 4 x
More informationDate: April 12, 2001. Contents
2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........
More information