# Introduction and message of the book

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 1 Introduction and message of the book 1.1 Why polynomial optimization? Consider the global optimization problem: P : for some feasible set f := inf x { f(x) : x K } (1.1) K := { x R n : g j (x) 0, j = 1,...,m}, (1.2) where f, g j : R n R are some continuous functions. If one is only interested in finding a local (as opposed to global) minimum then P is a Nonlinear Programming (NLP) problem for which several methods and associated algorithms are already available. But in this book we insist on the fact that P is a global optimization problem, that is, f is the global minimum of f on K. In full generality problem (1.1) is very difficult and there is no general purpose method, even to approximate f. However, and this is one of the messages of this book, if one now restricts oneself to Polynomial Optimization, that is, optimization problems P in (1.1) with the restriction that: f and g j : R n R are all polynomials, j = 1,...,m, then one may approximate f as closely as desired, and sometimes solve P exactly. (In fact one may even consider Semi-Algebraic Optimization, that is, 1

2 2 Introduction and message of the book when f and g j are semi-algebraic functions.) That this is possible is due to the conjunction of two factors. On the one hand, Linear Programming (LP) and Semidefinite Programming (SDP) have become major tools of convex optimization and today s powerful LP and SDP software packages can solve highly nontrivial problems of relatively large size (and even linear programs of extremely large size). On the other hand, remarkable and powerful representation theorems (or positivity certificates) for polynomials that are positive on sets like K in (1.2) were produced in the 1990s by real algebraic geometers and, importantly, the resulting conditions can be checked by solving appropriate semidefinite programs (and linear programs for some representations)! And indeed, in addition to the usual tools from Analysis, Convex Analysis and Linear Algebra already used in optimization, in Polynomial Optimization Algebra may also enter the game. In fact one may find it rather surprising that algebraic aspects of optimization problems defined by polynomials have not been taken into account in a systematic manner earlier. After all, the class of linear/quadratic optimization problems is an important subclass of Polynomial Optimization! But it looks as if we were so familiar with linear and quadratic functions that we forgot that they are polynomials! (It is worth noticing that in the 1960s, Gomory had already introduced some algebraic techniques for attacking (pure) linear integer programs. However, the algebraic techniques described in the present book are different as they come from Real Algebraic Geometry rather than pure algebra.) Even though Polynomial Optimization is a restricted class of optimization problems, it still encompasses a lot of important optimization problems. In particular, it includes the following. Continuous convex and nonconvex optimization problems with linear and/or quadratic costs and constraints, for example inf x { xt A 0 x + b T 0 x : xt A j x + b T j x c j 0, j = 1,...,m}, for some scalars c j, j = 1,...,m, and some real symmetric matrices A j R n n and vectors b j R n, j = 0,...,m. 0/1 optimization problems, modeling a Boolean variable x i {0, 1} via the quadratic polynomial constraint x 2 i x i = 0. For instance, the celebrated MAXCUT problem is the polynomial optimization problem

3 1.2 Message of the book 3 sup { x T Ax : xi 2 x i = 0, i = 1,...,n}, x where the real symmetric matrix A R n n is associated with some given graph with n vertices. Mixed-Integer Linear and NonLinear Programming (MILP and MINLP), for instance: inf x { x T A 0 x + b T 0 x : xt A j x + b T j x c j 0, j= 1,...,m; x [ M,M] n } ; x k Z, k J, for some real symmetric matrices A j R n n, vectors b j R n, j = 0,...,m, and some subset J {1,...,n}. Indeed, it suffices to model the constraint x i [ M,M], i J, with the quadratic inequality constraints M xi 2 0, j J, and the integrality constraints x k Z [ M,M], k J, with the polynomial equality constraints: (x k + M) (x k + M 1) x k (x k 1) (x k M) = 0, k J. 1.2 Message of the book We have already mentioned one message of the book. Polynomial Optimization indeed deserves a special treatment because its algebraic aspects can be taken into account in a systematic manner by invoking powerful results from real algebraic geometry. But there are other important messages Easyness A second message of the book which will become clear in the next chapters, is that the methodology for handling polynomial optimization problems P as defined in (1.1) is rather simple and easy to follow. Firstly, solving a polynomial optimization problem (1.1) is trivially equivalent to solving f = sup { λ : f(x) λ 0, x K }, (1.3) λ which, if f is a polynomial of degree at most d, is in turn equivalent to solving

4 4 Introduction and message of the book f = sup { λ : f λ C d (K) }, (1.4) λ where C d (K) is the convex cone of polynomials of degree at most d which are nonnegative on K. But (1.4) is a finite-dimensional convex optimization problem. Indeed, a polynomial f : R n R of degree d is encoded by its vector f R s(d) of coefficients (e.g. in the usual canonical basis of monomials), where s(d) := ( n+d) n is the dimension of the vector space of polynomials of degree at most d (that is the number of monomials x α = x α 1 1 xα n n, α N n, such that n i=1 α i d). And so C d (K) is a finite-dimensional cone which can be viewed as (or identified with) a convex cone of R s(d). Therefore (with some abuse of notation) (1.4) also reads { ) } f = sup λ λ : f λ ( 10 0 C d (K), (1.5) a convex conic optimization problem in R s(d). Note in passing that the convex formulations (1.3) and (1.5) are proper to the global optimum f and are not valid for a local minimum f>f ˆ! However, (1.5) remains hard to solve because in general there is no simple and tractable characterization of the convex cone C d (K) (even though it is finite dimensional). Then the general methodology that we use follows a simple idea. We first define a (nested) increasing family of convex cones (Cd l(k)) C d(k) such that Cd l (K) Cl+1 d (K) for every l, and each Cd l (K) is the projection of either a polyhedral cone or the intersection of a subspace with the convex cone of positive semidefinite matrices (whose size depends on l). Then we solve the hierarchy of conic optimization problems { ( 10 ) } ρ l = sup λ : f λ Cd l (K), l = 0, 1,... (1.6) λ 0 For each fixed l, the associated conic optimization problem is convex and can be solved efficiently by appropriate methods of convex optimization. For instance, by using some appropriate interior points methods, (1.6) can be solved to arbitrary precision fixed in advance, in time polynomial in its input size. As the Cd l (K) provide a nested sequence of inner approximations of C d (K), ρ l ρ l+1 f for every l. AndtheCd l (K) are chosen so as to ensure the convergence ρ l f as l. So depending on which type of convex approximation is used, (1.6) provides a hierarchy of linear or semidefinite programs (of increasing size) whose respective associated sequences of optimal values both converge to the desired global optimum f.

5 1.2 Message of the book 5 Secondly, the powerful results from Real Algebraic Geometry that we use to justify the convergence ρ l f in the above methodology, are extremely simple to understand and could be presented (without proof) in undergraduate courses of Applied Mathematics, Optimization and/or Operations Research. Of course their proof requires some knowledge of sophisticated material in several branches of mathematics but we will not prove such results, we will only use them! After all, the statement in Fermat s theorem is easy to understand and this theorem may be used with no need to understand its proof. For illustration and to give the flavor of one such important and powerful result, we will repeatedly use the following result which states that a polynomial f which is (strictly) positive on K (as defined in (1.2) and compact) can be written in the form m f(x) = σ 0 (x) + σ j (x)g j (x), x R n, (1.7) j=1 for some polynomials σ j that are Sums of Squares (SOS). By SOS we mean that each σ j, j = 0,...,m, can be written in the form σ j (x) = s j h jk (x) 2, x R n, k=1 for finitely many polynomials h jk, k = 1,...,s j. As one may see, (1.7) provides f with a certificate of its positivity on K. This is because if x K then f(x) 0 follows immediately from (1.7) as σ j (x) 0 (because σ j is SOS) and g j (x) 0 (because x K), for all j. In other words, there is no need to check the positivity of f on K as one may read it directly from (1.7)! Finally, the convex conic optimization problem (1.4) has a dual which is another finite-dimensional convex conic optimization problem. And in fact this classical duality of convex (conic) optimization captures and illustrates the beautiful duality between positive polynomials and moment problems. We will see that the dual of (1.4) is particularly useful for extracting global minimizers of P when the convergence is finite (which, in addition, happens generically!). Depending on which type of positivity certificate is used we call this methodology the moment-lp or moment-sos approach A general methodology The class of polynomial optimization problems contains easy convex problems (e.g. Linear Programming and convex Quadratic Programming) as

6 6 Introduction and message of the book well as NP-hard optimization problems (e.g. the MAXCUT problem already mentioned). Still, the general methodology presented in this book does not distinguish between easy convex problems and nonconvex, discrete and mixedinteger optimization problems! This immediately raises the following issues. How effective can a general purpose approach be for addressing problems which can be so different in nature (e.g. convex, or continuous but possibly with a nonconvex and nonconnected feasible set, or discrete, or mixed-integer, etc.)? Should we not specialize the approach according to the problem on hand, with ad hoc methods for certain categories of problems? Is the general approach reasonably efficient when applied to problems considered easy? Indeed would one trust a general purpose method designed for hard problems, and which would not behave efficiently on easy problems? Indeed, a large class of convex optimization problems are considered easy and can be solved efficiently by several ad hoc methods of convex optimization. Therefore a highly desirable feature of a general purpose approach is the ability somehow to recognize easier convex problems and behave accordingly (even if this may not be as efficient as specific methods tailored to the convex case). A third message of this book is that this is indeed the case for the moment- SOS approach based on semidefinite relaxations, which uses representation results of the form (1.7) based on SOS. This is not the case for the moment-lp approach based on LP-relaxations, which uses other representation results. In our mind this is an important and remarkable feature of the moment- SOS approach. For instance, and as already mentioned, a Boolean variable x i is not treated with any particular attention and is modeled via the quadratic equality constraint x 2 i x i = 0, just one among the many other polynomial equality or inequality constraints in the definition of the feasible set K in (1.2). Running a local minimization algorithm of continuous optimization with such a modeling of a Boolean constraint would not be considered wise (to say the least)! Hence this might justify some doubts concerning the efficiency of the moment-sos approach by lack of specialization. Yet, and remarkably, the resulting semidefinite relaxations

7 1.2 Message of the book 7 still provide the strongest relaxation algorithms for hard combinatorial optimization problems, and recognize easy convex problems as in this latter case convergence is even finite (and sometimes at the first semidefinite relaxation of the hierarchy)! Of course, and especially in view of the present status of semidefinite solvers, the moment-sos approach is still limited to problems of modest size; however, if symmetries or some structured sparsity in the problem data are detected and taken into account, then problems of much larger size can be solved Global optimality conditions Another message of this book is that in the moment-sos approach, generically the convergence ρ l f as l is finite (and genericity will be given a precise meaning)! And so in particular, generically, solving a polynomial optimization problem (1.1) on a compact set K as in (1.2) reduces to solving a single semidefinite program. But of course and as expected, the size of the resulting semidefinite program is not known in advance and can be potentially large. Moreover, the powerful Putinar representation (1.7) of polynomials that are (strictly) positive on K also holds generically for polynomials that are only nonnegative on K. And this translates into global optimality conditions that must be satisfied by global minimizers, provided that each global minimizer satisfies standard well-known constraint qualification, strict complementarity and second-order sufficiency conditions in nonlinear programming. Again, such conditions hold generically. Remarkably, these optimality conditions are the perfect analogues in (nonconvex) polynomial optimization of the Karush Kuhn Tucker (KKT) optimality conditions in convex programming. In particular, and in contrast to the KKT

8 8 Introduction and message of the book conditions in nonconvex programming, the constraints that are important but not active at a global minimizer still play a role in the optimality conditions Extensions The final message is that the above methodology can also be applied in the following situations. To handle semi-algebraic functions, a class of functions much larger than the class of polynomials. For instance one may handle functions like f(x) := min[q 1 (x), q 2 (x)] max[q 3 (x), q 4 (x)]+(q 5 (x) + q 6 (x)) 1/3, where the q i are given polynomials. To handle extensions like parametric and inverse optimization problems. To build up polynomial convex underestimators of a given nonconvex polynomial on a box B R n. To approximate as closely as desired, sets defined with quantifiers, for example the set { x B : f(x, y) 0 for all y such that (x, y) K }, where K R n+p is a set of the form (1.2), and B R n is a simple set. 1.3 Plan of the book The book is divided into three main parts. Part I is introductory and Chapter 2 is dedicated to presenting basic and important results on the representation of polynomials that are positive on a subset K of R n. This problem of real algebraic geometry has a nice dual facet, the so-called K-moment problem in Functional Analysis and Probability. And so results on the algebraic side are complemented with their dual counterparts on the moment side. Interestingly, convex duality in optimization (applied to appropriate convex cones) nicely captures this duality. Chapter 3 describes another characterization of polynomials nonnegative on a closed set K which is of independent interest and is dual to the characterizations of Chapter 2. Now knowledge on K is from moments of a measure supported on K rather than from polynomials that describe the boundary of K. These two dual points of view are exploited in Chapter 4 to provide explicit outer and inner approximations of the cone of polynomials nonnegative on K.

9 1.3 Plan of the book 9 Part II is dedicated to polynomial and semi-algebraic optimization. It describes how to use results of Part I to define hierarchies of convex relaxations whose optimal values provide monotone sequences of lower bounds which converge to the global optimum. Depending on the type of representation (or positivity certificate) used, one obtains a hierarchy of linear programs or semidefinite programs. Their respective merits and drawbacks are analyzed, especially in the light of global optimality conditions. In particular, we describe a global optimality condition which is the exact analogue in nonconvex polynomial optimization of the celebrated KKT optimality conditions in convex optimization. Using the representation results described in Chapter 3, one also obtains a hierarchy of eigenvalue problems which provide a monotone sequence of upper bounds which converges to the global optimum. Notice that most (primal-type) minimization algorithms provide sequences of upper bounds on the global minimum (as they move from a feasible point to another feasible point) but in general their convergence (if it eventually takes place) is guaranteed to a local minimum only. We also (briefly) describe how to use sparsity or symmetry to reduce the computational burden associated with the hierarchy of relaxations. It is worth noticing that the extension from polynomials to semi-algebraic functions (in both the objective function and the description of the feasible set) enlarges significantly the range of potential applications that can be treated. Part III describes some specializations and extensions. Convex polynomial optimization to show that the moment-sos approach somehow recognizes some classes of easy convex problems; in particular the hierarchy of semidefinite relaxations has finite convergence. Some properties of convex polynomials and convex basic semi-algebraic sets are also described and analyzed. Parametric polynomial optimization, that is, optimization problems where the criterion to minimize as well as the constraints that describe the feasible set, may depend on some parameters that belong to a given set. The ultimate and difficult goal is to compute or at least provide some information and/or approximations on the global optimum and the global minimizers, viewed as functions of the parameters. Hence there is a qualitative jump in difficulty as one now searches for functions on some domain (an infinite-dimensional object) rather than a vector x R n (a finite-dimensional object). With this in mind we describe what we call the joint+marginal approach to parametric optimization and show that in this context the moment-sos approach is well suited for providing good approximations (theoretically as closely as

10 10 Introduction and message of the book desired) when the parametric optimization problem is described via polynomials and basic semi-algebraic sets. Inverse polynomial optimization where given a point y K (think of an iterate of some local minimization algorithm) and a polynomial criterion f, one tries to find a polynomial f as close as possible to f and for which y is a global minimizer on K. This problem has interesting potential theoretical and practical applications. For instance, suppose that y K is a current iterate of a local optimization algorithm to minimize f on K, and suppose that solving the inverse problem provides a new criterion f relatively close to f. Should we spend (expensive) additional effort to obtain a better iterate or should we stop (as y solves an optimization problem close to the original one)? In addition, the inverse problem is also a way to measure how illconditioned is the (direct) optimization problem. Convex underestimators. For difficult large scale nonlinear problems and particularly for mixed-integer nonlinear programs (MINLP), the only practical way to approximate the global minimum is to explore an appropriate Branch and Bound search tree in which exploration is guided by lower bounds computed at each node of the tree. The quality of lower bounds is crucial for the overall efficiency of the approach. In general and for obvious reasons, efficient computation of lower bounds is possible only on some appropriate convex relaxation of the problem described at the current node. A standard way to obtain a convex relaxation is to replace the nonconvex objective function with a convex underestimator; similarly, an inequality constraint g(x) 0 is replaced with g(x) 0 for some convex underestimator g of g. Therefore deriving tight convex underestimators is of crucial importance for the quality of the resulting lower bounds. We show that the moment-sos approach is particularly well suited to obtaining tight convex underestimators. Polynomial optimization on sets defined with quantifiers. In this context one is given a set K := { (x, y) : g j (x, y) 0, j = 1,...,m} for some polynomials g j, j = 1,...,m. Given a set B R n (typically a box or an ellipsoid) the goal is to approximate as closely as desired the set K x := { x B : f(x, y) 0 for all y such that (x, y) K }, by a set K l x K x simply defined by K l x ={x : h l(x) 0 } for some polynomial h l. We show how to obtain a sequence of polynomials (h l ) of increasing degree l N, such that the Lebesgue volume vol(k x \ K l x ) tends to zero as l. And so any optimization problem involving the set K x (difficult to handle) can be approximated by substituting K x with K l x for some sufficiently large l.

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### Overview of Math Standards

Algebra 2 Welcome to math curriculum design maps for Manhattan- Ogden USD 383, striving to produce learners who are: Effective Communicators who clearly express ideas and effectively communicate with diverse

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### On Minimal Valid Inequalities for Mixed Integer Conic Programs

On Minimal Valid Inequalities for Mixed Integer Conic Programs Fatma Kılınç Karzan June 27, 2013 Abstract We study mixed integer conic sets involving a general regular (closed, convex, full dimensional,

### Vector Spaces II: Finite Dimensional Linear Algebra 1

John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition

### (Basic definitions and properties; Separation theorems; Characterizations) 1.1 Definition, examples, inner description, algebraic properties

Lecture 1 Convex Sets (Basic definitions and properties; Separation theorems; Characterizations) 1.1 Definition, examples, inner description, algebraic properties 1.1.1 A convex set In the school geometry

### Some Optimization Fundamentals

ISyE 3133B Engineering Optimization Some Optimization Fundamentals Shabbir Ahmed E-mail: sahmed@isye.gatech.edu Homepage: www.isye.gatech.edu/~sahmed Basic Building Blocks min or max s.t. objective as

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### Solving polynomial least squares problems via semidefinite programming relaxations

Solving polynomial least squares problems via semidefinite programming relaxations Sunyoung Kim and Masakazu Kojima August 2007, revised in November, 2007 Abstract. A polynomial optimization problem whose

### A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION

1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of

### Chapter 13: Binary and Mixed-Integer Programming

Chapter 3: Binary and Mixed-Integer Programming The general branch and bound approach described in the previous chapter can be customized for special situations. This chapter addresses two special situations:

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### LEARNING OBJECTIVES FOR THIS CHAPTER

CHAPTER 2 American mathematician Paul Halmos (1916 2006), who in 1942 published the first modern linear algebra book. The title of Halmos s book was the same as the title of this chapter. Finite-Dimensional

### Some representability and duality results for convex mixed-integer programs.

Some representability and duality results for convex mixed-integer programs. Santanu S. Dey Joint work with Diego Morán and Juan Pablo Vielma December 17, 2012. Introduction About Motivation Mixed integer

### Conic optimization: examples and software

Conic optimization: examples and software Etienne de Klerk Tilburg University, The Netherlands Etienne de Klerk (Tilburg University) Conic optimization: examples and software 1 / 16 Outline Conic optimization

### A FIRST COURSE IN OPTIMIZATION THEORY

A FIRST COURSE IN OPTIMIZATION THEORY RANGARAJAN K. SUNDARAM New York University CAMBRIDGE UNIVERSITY PRESS Contents Preface Acknowledgements page xiii xvii 1 Mathematical Preliminaries 1 1.1 Notation

### Lecture 5: Conic Optimization: Overview

EE 227A: Conve Optimization and Applications January 31, 2012 Lecture 5: Conic Optimization: Overview Lecturer: Laurent El Ghaoui Reading assignment: Chapter 4 of BV. Sections 3.1-3.6 of WTB. 5.1 Linear

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### MOSEK modeling manual

MOSEK modeling manual August 12, 2014 Contents 1 Introduction 1 2 Linear optimization 3 2.1 Introduction....................................... 3 2.1.1 Basic notions.................................. 3

### Open and Closed Sets

Open and Closed Sets Definition: A subset S of a metric space (X, d) is open if it contains an open ball about each of its points i.e., if x S : ɛ > 0 : B(x, ɛ) S. (1) Theorem: (O1) and X are open sets.

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities

Algebra 1, Quarter 2, Unit 2.1 Creating, Solving, and Graphing Systems of Linear Equations and Linear Inequalities Overview Number of instructional days: 15 (1 day = 45 60 minutes) Content to be learned

### THE DIMENSION OF A VECTOR SPACE

THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field

### Sec 4.1 Vector Spaces and Subspaces

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA

We Can Early Learning Curriculum PreK Grades 8 12 INSIDE ALGEBRA, GRADES 8 12 CORRELATED TO THE SOUTH CAROLINA COLLEGE AND CAREER-READY FOUNDATIONS IN ALGEBRA April 2016 www.voyagersopris.com Mathematical

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### Lecture 7: Finding Lyapunov Functions 1

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.243j (Fall 2003): DYNAMICS OF NONLINEAR SYSTEMS by A. Megretski Lecture 7: Finding Lyapunov Functions 1

### LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION

LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION Kartik Sivaramakrishnan Department of Mathematics NC State University kksivara@ncsu.edu http://www4.ncsu.edu/ kksivara SIAM/MGSA Brown Bag

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### Appendix A. Appendix. A.1 Algebra. Fields and Rings

Appendix A Appendix A.1 Algebra Algebra is the foundation of algebraic geometry; here we collect some of the basic algebra on which we rely. We develop some algebraic background that is needed in the text.

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Convex Optimization SVM s and Kernel Machines

Convex Optimization SVM s and Kernel Machines S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola and Stéphane Canu S.V.N.

### Integer factorization is in P

Integer factorization is in P Yuly Shipilevsky Toronto, Ontario, Canada E-mail address: yulysh2000@yahoo.ca Abstract A polynomial-time algorithm for integer factorization, wherein integer factorization

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### o-minimality and Uniformity in n 1 Graphs

o-minimality and Uniformity in n 1 Graphs Reid Dale July 10, 2013 Contents 1 Introduction 2 2 Languages and Structures 2 3 Definability and Tame Geometry 4 4 Applications to n 1 Graphs 6 5 Further Directions

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### Basics of Polynomial Theory

3 Basics of Polynomial Theory 3.1 Polynomial Equations In geodesy and geoinformatics, most observations are related to unknowns parameters through equations of algebraic (polynomial) type. In cases where

### MATHEMATICS (CLASSES XI XII)

MATHEMATICS (CLASSES XI XII) General Guidelines (i) All concepts/identities must be illustrated by situational examples. (ii) The language of word problems must be clear, simple and unambiguous. (iii)

### 4.6 Null Space, Column Space, Row Space

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

### Math212a1010 Lebesgue measure.

Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.

### Linear Algebra Done Wrong. Sergei Treil. Department of Mathematics, Brown University

Linear Algebra Done Wrong Sergei Treil Department of Mathematics, Brown University Copyright c Sergei Treil, 2004, 2009, 2011, 2014 Preface The title of the book sounds a bit mysterious. Why should anyone

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### Surface bundles over S 1, the Thurston norm, and the Whitehead link

Surface bundles over S 1, the Thurston norm, and the Whitehead link Michael Landry August 16, 2014 The Thurston norm is a powerful tool for studying the ways a 3-manifold can fiber over the circle. In

### MATHEMATICAL METHODS OF STATISTICS

MATHEMATICAL METHODS OF STATISTICS By HARALD CRAMER TROFESSOK IN THE UNIVERSITY OF STOCKHOLM Princeton PRINCETON UNIVERSITY PRESS 1946 TABLE OF CONTENTS. First Part. MATHEMATICAL INTRODUCTION. CHAPTERS

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### An Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.

An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### Nonlinear Optimization: Algorithms 3: Interior-point methods

Nonlinear Optimization: Algorithms 3: Interior-point methods INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2006 Jean-Philippe Vert,

### Probability and Statistics

CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

### Walrasian Demand. u(x) where B(p, w) = {x R n + : p x w}.

Walrasian Demand Econ 2100 Fall 2015 Lecture 5, September 16 Outline 1 Walrasian Demand 2 Properties of Walrasian Demand 3 An Optimization Recipe 4 First and Second Order Conditions Definition Walrasian

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Semidefinite characterisation of invariant measures for one-dimensional discrete dynamical systems

Semidefinite characterisation of invariant measures for one-dimensional discrete dynamical systems Didier Henrion 1,2,3 April 24, 2012 Abstract Using recent results on measure theory and algebraic geometry,

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### The Dirichlet Unit Theorem

Chapter 6 The Dirichlet Unit Theorem As usual, we will be working in the ring B of algebraic integers of a number field L. Two factorizations of an element of B are regarded as essentially the same if

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### Tensors on a vector space

APPENDIX B Tensors on a vector space In this Appendix, we gather mathematical definitions and results pertaining to tensors. The purpose is mostly to introduce the modern, geometrical view on tensors,

### Section 4.4 Inner Product Spaces

Section 4.4 Inner Product Spaces In our discussion of vector spaces the specific nature of F as a field, other than the fact that it is a field, has played virtually no role. In this section we no longer

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### MA257: INTRODUCTION TO NUMBER THEORY LECTURE NOTES

MA257: INTRODUCTION TO NUMBER THEORY LECTURE NOTES 2016 47 4. Diophantine Equations A Diophantine Equation is simply an equation in one or more variables for which integer (or sometimes rational) solutions

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### An Introduction on SemiDefinite Program

An Introduction on SemiDefinite Program from the viewpoint of computation Hayato Waki Institute of Mathematics for Industry, Kyushu University 2015-10-08 Combinatorial Optimization at Work, Berlin, 2015

### Ideal Class Group and Units

Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

### Several Views of Support Vector Machines

Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min

### Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### PYTHAGOREAN TRIPLES PETE L. CLARK

PYTHAGOREAN TRIPLES PETE L. CLARK 1. Parameterization of Pythagorean Triples 1.1. Introduction to Pythagorean triples. By a Pythagorean triple we mean an ordered triple (x, y, z) Z 3 such that x + y =

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### 5. Linear algebra I: dimension

5. Linear algebra I: dimension 5.1 Some simple results 5.2 Bases and dimension 5.3 Homomorphisms and dimension 1. Some simple results Several observations should be made. Once stated explicitly, the proofs

### Matrix Representations of Linear Transformations and Changes of Coordinates

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that