# LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 LAGRANGIAN RELAXATION TECHNIQUES FOR LARGE SCALE OPTIMIZATION Kartik Sivaramakrishnan Department of Mathematics NC State University kksivara SIAM/MGSA Brown Bag Lunch Seminar NCSU September 27, 2006

2 Thought for the Day In fact, the great watershed in optimization isn t between linearity and nonlinearity, but convexity and nonconvexity. R.T. Rockafellar (SIAM Review 1993)

3 Contents Lagrangian relaxation Properties of Lagrangian dual problem Example 1: Getting good upper bounds Example 2: Exploiting problem structure Solving the Lagrangian dual problem Connections with my current research interests Conclusions and References -1-

4 Lagrangian relaxation: 1 Consider the following primal nonlinear program (P) max f(x) s.t. g(x) 0 h(x) = 0 x X Problem (P) is a nonconvex or mixed integer problem. Want a good upper bound on its optimal value. Problem (P) is easy, but the set X has a decomposable structure. In either case, g(x) 0 and h(x) = 0 are complicating constraints and maximizing f(x) over the set X is easy. Consider the Lagrangian function where u 0. L(x, u, v) = f(x) u T g(x) v T h(x) -2-

5 Lagrangian relaxation: 2 A continuous two person zero sum game between two players 1 and 2. X and Y = {(u, v) : u 0} are the strategy sets for players 1 and 2 L(x, u, v) is the payoff, i.e., the amount that player 1 gets from 2. Player 1 tries to maximize her minimum payoff. Her problem is max x X min (u,v) Y L(x, u, v) which is the original optimization problem (P). Player 2 tries to minimize his maximum payoff. His problem is min (u,v) Y max x X which is the Lagrangian dual problem. L(x, u, v) -3-

6 Properties of Lagrangian dual problem The Lagrangian dual problem (D) can also be written as min u 0,v θ(u, v) where θ(u, v) = max{f(x) u T g(x) v T h(x) : x X}. The objective value of any feasible solution (u, v) to (D) provides an upper bound on the optimal objective f(x ) of (P). The optimal objective θ(u, v ) of (D) provides the best upper bound. θ(u, v ) = f(x ) for convex problems with constraint qualification. For nonconvex problems, θ(u, v ) > f(x ) (duality gap). Note that θ(u, v) is the pointwise supremum of a set of affine functions in u and v. (D) is always convex (regardless of whether (P) is convex or not!) but it is in general a nonsmooth problem. -4-

7 Example 1: Getting good upper bounds Consider the following integer quadratic program We compute the functions max x T Qx s.t. x 2 i = 1, i = 1,..., n. L(x, v) = x T Qx n v i (x 2 i 1) i=1 = x T (Q Diag(v))x + e T v θ(v) = e T v + max x x T (Q Diag(v))x = e T v if (Diag(v) Q) 0. Note, that θ(v) = otherwise. The Lagrangian dual problem is min e T v s.t. (Diag(v) Q) 0.

8 Example 1: Getting good upper bounds This gives the pair of SDPs min v i i s.t. S = Diag(v) Q S 0 max Q X s.t. X ii = 1, i = 1,..., n X 0 which provides the desired upper bound. SDPs can be solved efficiently using interior-point methods. A feasible x in the integer program also gives a feasible X = xx T in the primal SDP, which is a relaxation of the integer program. Goemans & Williamson developed a approximation algorithm for the maxcut problem which solves the above SDP relaxation. The SDP relaxation can be used in a conic branch-cut-price algorithm to solve the original integer program to optimality. -6-

9 Example 2: Exploiting problem structure Consider the problem max s.t. r f i (x i ) i=1 r h i (x i ) = 0 i=1 x i X i where f(x) and g(x) are block separable and X is decomposable, i.e., X = {x = (x 1,..., x r ) : x i X i } The objective function in the Lagrangian dual problem is r θ(v) = max (f i (x i ) v T h i (x i )) x i X i i=1 = r i=1 max (f i (x i ) v T h i (x i )) x i X i -7-

10 Example 2: Exploiting problem structure We have θ(v) = r θ i (v) where i=1 θ i (v) = max x i X i (f i (x i ) v T h i (x i )). Thus, computing θ(v) can be done in parallel. Thus, the Lagrangian dual problem min v θ(v) can be solved more quickly than the original problem. Problems with this structure occur routinely; examples include unit commitment problem, multicommodity flows, two stage stochastic programs with recourse, etc. -8-

11 Solving the Lagrangian dual problem The dual problem is min u 0,v θ(u, v) where θ(u, v) = max{f(x) u T g(x) v T h(x) : x X}. The inner maximization problem (subproblem) can be solved using a general purpose solver (maximizing f(x) over X is easy!). A solution also gives the function value and a subgradient of θ(u, v) at a given point (u, v). The outer problem (master problem) is a nonsmooth problem. Techniques to solve it include: subgradient, cutting plane/column generation, bundle/trust region, and augmented Lagrangian methods (see Kartik s NA seminar talk on October 10th!). A solution to the primal problem (P) can also be recovered from the solution to the dual problem. -9-

12 Connections to my current research Two ongoing research projects Developing a conic interior point decomposition approach for structured block angular SDPs in a parallel computing environment (Example 2 in Lagrangian relaxation). Developing semidefinite programming based branch-cut-price algorithms for solving mixed integer and nonconvex problems to optimality (Example 1 in Lagrangian relaxation). -10-

13 Consider the SDP Preprocessing sparse SDPs into block-angular SDPs: 1 min Q X s.t. X ii = 1, i = 1,..., n, X 0, where Q is the adjacency matrix of the graph C2 = (2,3,4,6) C2 C4 = (5,6,7) C4 C3 C1 = (1,5,7) C1 C3 = (4,6,7) GRAPH CHORDAL EXTENSION OF GRAPH -11-

14 Preprocessing sparse SDPs into block-angular SDPs: 2 Using matrix completion, one can reformulate the earlier SDP as min 4 (Q k X k ) k=1 s.t. X23 1 X13 4 = 0, X34 2 X12 3 = 0, X23 3 X23 4 = 0, Xii k = 1, i = 1,..., C k, k = 1,..., 4, X k 0, k = 1,..., 4, which is in block-angular form. -12-

15 Conclusions Lagrangian relaxation technique is a very general technique (i) to obtain upper bounds (ii) exploit problem structure for difficult and large scale optimization problems. Both facets form important components of my research. Important algorithms in science and engineering solve the Lagrangian dual, examples include support vector machines in data mining, waterfilling algorithm in information theory etc. If you are interested, you must definitely drop by my office (HA 235) for a chat! Also, please attend my NA seminar on the 10th of October :) Background in numerical linear algebra, parallel and distributed computing, mathematical analysis; and linear, nonlinear, and integer programming will be useful. -13-

16 Thank you for your attention!. Questions, Comments, Suggestions? The slides from this talk are available online at kksivara/ publications/siam-mgsa-brownbag.pdf -14-

17 Bibliography 1. C. Lemaréchal, Lagrangian relaxation, In Computational Combinatorial Optimization, M. Jünger and D. Naddef (editors), Springer-Verlag, Heidelberg R.T. Rockafellar, Lagrange multipliers and optimality, SIAM Review, 35(1993), pp C. Lemaréchal and F. Oustry, Semidefinite relaxations and lagrangian duality with application to combinatorial optimization, Rapport de Recherche 3710, INRIA, A. Ruszczyński, Nonlinear Optimization, Princeton University Press, J.B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms, Springer Verlag, New York, G. Nemhauser and L. Wolsey, Integer and Combinatorial Optimization, Wiley Interscience, G. Dantzig and M. Thapa, Linear Programming 2: Theory and Extensions, Springer, New York, S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press boyd/cvxbook/ -15-

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION

1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of

### Big Data - Lecture 1 Optimization reminders

Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### Recovery of primal solutions from dual subgradient methods for mixed binary linear programming; a branch-and-bound approach

MASTER S THESIS Recovery of primal solutions from dual subgradient methods for mixed binary linear programming; a branch-and-bound approach PAULINE ALDENVIK MIRJAM SCHIERSCHER Department of Mathematical

### Introduction and message of the book

1 Introduction and message of the book 1.1 Why polynomial optimization? Consider the global optimization problem: P : for some feasible set f := inf x { f(x) : x K } (1.1) K := { x R n : g j (x) 0, j =

### Absolute Value Programming

Computational Optimization and Aplications,, 1 11 (2006) c 2006 Springer Verlag, Boston. Manufactured in The Netherlands. Absolute Value Programming O. L. MANGASARIAN olvi@cs.wisc.edu Computer Sciences

### An Overview Of Software For Convex Optimization. Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.

An Overview Of Software For Convex Optimization Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu In fact, the great watershed in optimization isn t between linearity

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### Lecture 11: 0-1 Quadratic Program and Lower Bounds

Lecture : - Quadratic Program and Lower Bounds (3 units) Outline Problem formulations Reformulation: Linearization & continuous relaxation Branch & Bound Method framework Simple bounds, LP bound and semidefinite

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### Nonlinear Optimization: Algorithms 3: Interior-point methods

Nonlinear Optimization: Algorithms 3: Interior-point methods INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2006 Jean-Philippe Vert,

### An Introduction on SemiDefinite Program

An Introduction on SemiDefinite Program from the viewpoint of computation Hayato Waki Institute of Mathematics for Industry, Kyushu University 2015-10-08 Combinatorial Optimization at Work, Berlin, 2015

### Some Optimization Fundamentals

ISyE 3133B Engineering Optimization Some Optimization Fundamentals Shabbir Ahmed E-mail: sahmed@isye.gatech.edu Homepage: www.isye.gatech.edu/~sahmed Basic Building Blocks min or max s.t. objective as

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### Minimize subject to. x S R

Chapter 12 Lagrangian Relaxation This chapter is mostly inspired by Chapter 16 of [1]. In the previous chapters, we have succeeded to find efficient algorithms to solve several important problems such

### Support Vector Machines Explained

March 1, 2009 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),

### Summer course on Convex Optimization. Fifth Lecture Interior-Point Methods (1) Michel Baes, K.U.Leuven Bharath Rangarajan, U.

Summer course on Convex Optimization Fifth Lecture Interior-Point Methods (1) Michel Baes, K.U.Leuven Bharath Rangarajan, U.Minnesota Interior-Point Methods: the rebirth of an old idea Suppose that f is

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### Support Vector Machine (SVM)

Support Vector Machine (SVM) CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin

### Introduction to Support Vector Machines. Colin Campbell, Bristol University

Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.

### Introduction to Convex Optimization for Machine Learning

Introduction to Convex Optimization for Machine Learning John Duchi University of California, Berkeley Practical Machine Learning, Fall 2009 Duchi (UC Berkeley) Convex Optimization for Machine Learning

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen

(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### DSL Spectrum Management

DSL Spectrum Management Dr. Jianwei Huang Department of Electrical Engineering Princeton University Guest Lecture of ELE539A March 2007 Jianwei Huang (Princeton) DSL Spectrum Management March 2007 1 /

### Tutorial on Convex Optimization for Engineers Part I

Tutorial on Convex Optimization for Engineers Part I M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de

### Optimization Modeling for Mining Engineers

Optimization Modeling for Mining Engineers Alexandra M. Newman Division of Economics and Business Slide 1 Colorado School of Mines Seminar Outline Linear Programming Integer Linear Programming Slide 2

### An Accelerated First-Order Method for Solving SOS Relaxations of Unconstrained Polynomial Optimization Problems

An Accelerated First-Order Method for Solving SOS Relaxations of Unconstrained Polynomial Optimization Problems Dimitris Bertsimas, Robert M. Freund, and Xu Andy Sun December 2011 Abstract Our interest

### Completely Positive Cone and its Dual

On the Computational Complexity of Membership Problems for the Completely Positive Cone and its Dual Peter J.C. Dickinson Luuk Gijben July 3, 2012 Abstract Copositive programming has become a useful tool

### Using the Simplex Method in Mixed Integer Linear Programming

Integer Using the Simplex Method in Mixed Integer UTFSM Nancy, 17 december 2015 Using the Simplex Method in Mixed Integer Outline Mathematical Programming Integer 1 Mathematical Programming Optimisation

### Optimization Theory for Large Systems

Optimization Theory for Large Systems LEON S. LASDON CASE WESTERN RESERVE UNIVERSITY THE MACMILLAN COMPANY COLLIER-MACMILLAN LIMITED, LONDON Contents 1. Linear and Nonlinear Programming 1 1.1 Unconstrained

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Advances in Convex Optimization: Interior-point Methods, Cone Programming, and Applications

Advances in Convex Optimization: Interior-point Methods, Cone Programming, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA)

### Big Data Optimization at SAS

Big Data Optimization at SAS Imre Pólik et al. SAS Institute Cary, NC, USA Edinburgh, 2013 Outline 1 Optimization at SAS 2 Big Data Optimization at SAS The SAS HPA architecture Support vector machines

### A Simple Introduction to Support Vector Machines

A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear

### A Lagrangian relaxation approach for network inventory control of stochastic revenue management with perishable commodities

Journal of the Operational Research Society (2006), 1--9 2006 Operational Research Society Ltd. All rights reserved. 0160-5682/06 \$30.00 www.palgrave-journals.com/jors A Lagrangian relaxation approach

### ME128 Computer-Aided Mechanical Design Course Notes Introduction to Design Optimization

ME128 Computer-ided Mechanical Design Course Notes Introduction to Design Optimization 2. OPTIMIZTION Design optimization is rooted as a basic problem for design engineers. It is, of course, a rare situation

### Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon

lp.nb 1 Linear Programming, Lagrange Multipliers, and Duality Geoff Gordon lp.nb 2 Overview This is a tutorial about some interesting math and geometry connected with constrained optimization. It is not

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Solving polynomial least squares problems via semidefinite programming relaxations

Solving polynomial least squares problems via semidefinite programming relaxations Sunyoung Kim and Masakazu Kojima August 2007, revised in November, 2007 Abstract. A polynomial optimization problem whose

### Solutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR

Solutions Of Some Non-Linear Programming Problems A PROJECT REPORT submitted by BIJAN KUMAR PATEL for the partial fulfilment for the award of the degree of Master of Science in Mathematics under the supervision

### Optimization Methods in Finance

Optimization Methods in Finance Gerard Cornuejols Reha Tütüncü Carnegie Mellon University, Pittsburgh, PA 15213 USA January 2006 2 Foreword Optimization models play an increasingly important role in financial

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### 10. Proximal point method

L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

### Several Views of Support Vector Machines

Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### MOSEK modeling manual

MOSEK modeling manual August 12, 2014 Contents 1 Introduction 1 2 Linear optimization 3 2.1 Introduction....................................... 3 2.1.1 Basic notions.................................. 3

### Stochastic Optimization Models with Fixed Costs. Outline

Stochastic Optimization Models with Fixed Costs John R. Birge University of Michigan Slide Number 1 Outline Examples Finance Manufacturing Power systems General Formulation Statement Optimality conditions

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### A Distributed Line Search for Network Optimization

01 American Control Conference Fairmont Queen Elizabeth, Montréal, Canada June 7-June 9, 01 A Distributed Line Search for Networ Optimization Michael Zargham, Alejandro Ribeiro, Ali Jadbabaie Abstract

### Convex Programming Tools for Disjunctive Programs

Convex Programming Tools for Disjunctive Programs João Soares, Departamento de Matemática, Universidade de Coimbra, Portugal Abstract A Disjunctive Program (DP) is a mathematical program whose feasible

### Integer factorization is in P

Integer factorization is in P Yuly Shipilevsky Toronto, Ontario, Canada E-mail address: yulysh2000@yahoo.ca Abstract A polynomial-time algorithm for integer factorization, wherein integer factorization

### 1 Polyhedra and Linear Programming

CS 598CSC: Combinatorial Optimization Lecture date: January 21, 2009 Instructor: Chandra Chekuri Scribe: Sungjin Im 1 Polyhedra and Linear Programming In this lecture, we will cover some basic material

### A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems

A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems Sunyoung Kim, Masakazu Kojima and Kim-Chuan Toh October 2013 Abstract. We propose

### Massive Data Classification via Unconstrained Support Vector Machines

Massive Data Classification via Unconstrained Support Vector Machines Olvi L. Mangasarian and Michael E. Thompson Computer Sciences Department University of Wisconsin 1210 West Dayton Street Madison, WI

### Support Vector Machines

Support Vector Machines Charlie Frogner 1 MIT 2011 1 Slides mostly stolen from Ryan Rifkin (Google). Plan Regularization derivation of SVMs. Analyzing the SVM problem: optimization, duality. Geometric

### By W.E. Diewert. July, Linear programming problems are important for a number of reasons:

APPLIED ECONOMICS By W.E. Diewert. July, 3. Chapter : Linear Programming. Introduction The theory of linear programming provides a good introduction to the study of constrained maximization (and minimization)

### Interior Point Methods and Linear Programming

Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either

### Optimisation et simulation numérique.

Optimisation et simulation numérique. Lecture 1 A. d Aspremont. M2 MathSV: Optimisation et simulation numérique. 1/106 Today Convex optimization: introduction Course organization and other gory details...

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### Support Vector Machines with Clustering for Training with Very Large Datasets

Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano

### Equilibrium computation: Part 1

Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium

### Convex Optimization SVM s and Kernel Machines

Convex Optimization SVM s and Kernel Machines S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola and Stéphane Canu S.V.N.

### Interior-Point Algorithms for Quadratic Programming

Interior-Point Algorithms for Quadratic Programming Thomas Reslow Krüth Kongens Lyngby 2008 IMM-M.Sc-2008-19 Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800

### Distributed Machine Learning and Big Data

Distributed Machine Learning and Big Data Sourangshu Bhattacharya Dept. of Computer Science and Engineering, IIT Kharagpur. http://cse.iitkgp.ac.in/~sourangshu/ August 21, 2015 Sourangshu Bhattacharya

### Introduction to Stochastic Optimization in Supply Chain and Logistic Optimization

Introduction to Stochastic Optimization in Supply Chain and Logistic Optimization John R. Birge Northwestern University IMA Tutorial, Stochastic Optimization, September 00 1 Outline Overview Part I - Models

### Optimal Scheduling for Dependent Details Processing Using MS Excel Solver

BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 8, No 2 Sofia 2008 Optimal Scheduling for Dependent Details Processing Using MS Excel Solver Daniela Borissova Institute of

### Electric Company Portfolio Optimization Under Interval Stochastic Dominance Constraints

4th International Symposium on Imprecise Probabilities and Their Applications, Pittsburgh, Pennsylvania, 2005 Electric Company Portfolio Optimization Under Interval Stochastic Dominance Constraints D.

### The Hungarian Method for the Assignment Problem

Chapter 2 The Hungarian Method for the Assignment Problem Harold W. Kuhn Introduction by Harold W. Kuhn This paper has always been one of my favorite children, combining as it does elements of the duality

### Linear Programming: Introduction

Linear Programming: Introduction Frédéric Giroire F. Giroire LP - Introduction 1/28 Course Schedule Session 1: Introduction to optimization. Modelling and Solving simple problems. Modelling combinatorial

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Advanced Lecture on Mathematical Science and Information Science I. Optimization in Finance

Advanced Lecture on Mathematical Science and Information Science I Optimization in Finance Reha H. Tütüncü Visiting Associate Professor Dept. of Mathematical and Computing Sciences Tokyo Institute of Technology

### An interval linear programming contractor

An interval linear programming contractor Introduction Milan Hladík Abstract. We consider linear programming with interval data. One of the most challenging problems in this topic is to determine or tight

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Why is SAS/OR important? For whom is SAS/OR designed?

Fact Sheet What does SAS/OR software do? SAS/OR software provides a powerful array of optimization, simulation and project scheduling techniques to identify the actions that will produce the best results,

### Convex analysis and profit/cost/support functions

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

### Research Article Design of a Distribution Network Using Primal-Dual Decomposition

Mathematical Problems in Engineering Volume 2016, Article ID 7851625, 9 pages http://dx.doi.org/10.1155/2016/7851625 Research Article Design of a Distribution Network Using Primal-Dual Decomposition J.

### Parallel Interior Point Solver for Structured Quadratic Programs: Application to Financial Planning Problems

Parallel Interior Point Solver for Structured uadratic Programs: Application to Financial Planning Problems Jacek Gondzio Andreas Grothey April 15th, 2003 MS-03-001 For other papers in this series see

### Solving Mixed Integer Linear Programs Using Branch and Cut Algorithm

1 Solving Mixed Integer Linear Programs Using Branch and Cut Algorithm by Shon Albert A Project Submitted to the Graduate Faculty of North Carolina State University in Partial Fulfillment of the Requirements

### Linear Programming is the branch of applied mathematics that deals with solving

Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming

### A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Semidefinite characterisation of invariant measures for one-dimensional discrete dynamical systems

Semidefinite characterisation of invariant measures for one-dimensional discrete dynamical systems Didier Henrion 1,2,3 April 24, 2012 Abstract Using recent results on measure theory and algebraic geometry,

### Two-Stage Stochastic Linear Programs

Two-Stage Stochastic Linear Programs Operations Research Anthony Papavasiliou 1 / 27 Two-Stage Stochastic Linear Programs 1 Short Reviews Probability Spaces and Random Variables Convex Analysis 2 Deterministic

### Date: April 12, 2001. Contents

2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........

### Solving Large Scale Optimization Problems via Grid and Cluster Computing

Solving Large Scale Optimization Problems via Grid and Cluster Computing Katsuki Fujisawa, Masakazu Kojima Akiko Takeda and Makoto Yamashita Abstract. Solving large scale optimization problems requires

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### The Simplex Method. yyye

Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #05 1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/

### A New Method for Estimating Maximum Power Transfer and Voltage Stability Margins to Mitigate the Risk of Voltage Collapse

A New Method for Estimating Maximum Power Transfer and Voltage Stability Margins to Mitigate the Risk of Voltage Collapse Bernie Lesieutre Dan Molzahn University of Wisconsin-Madison PSERC Webinar, October

### Introduction: Models, Model Building and Mathematical Optimization The Importance of Modeling Langauges for Solving Real World Problems

Introduction: Models, Model Building and Mathematical Optimization The Importance of Modeling Langauges for Solving Real World Problems Josef Kallrath Structure of the Lecture: the Modeling Process survey

### Stabilizing Acrobot by Using Nonlinear Programming Based on Sliding Mode Controller

5 10 July 2004, Antalya, Turkey Dynamical Systems and Applications, Proceedings, pp. 712 724 Stabilizing Acrobot by Using Nonlinear Programming Based on Sliding Mode Controller Ahmet Yazıcı Electrical

### LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA Hyderabad

LINEAR PROGRAMMING P V Ram B. Sc., ACA, ACMA 98481 85073 Hyderabad Page 1 of 19 Question: Explain LPP. Answer: Linear programming is a mathematical technique for determining the optimal allocation of resources

### Conic optimization: examples and software

Conic optimization: examples and software Etienne de Klerk Tilburg University, The Netherlands Etienne de Klerk (Tilburg University) Conic optimization: examples and software 1 / 16 Outline Conic optimization

### ECON Mathematical Economics - ANSWERS FINAL EXAM. 1. (a)- The consumer's utility maximization problem is written as:

ECON 331 - Mathematical Economics - ANSWERS FINAL EXAM 1. a- The consumer's utility maximization problem is written as: max x,y [xy + y2 + 2x + 2y] s.t. 6x + 10y = m, x 0, y 0 - The associated Lagrangian

### Bindel, Fall 2012 Matrix Computations (CS 6210) Week 8: Friday, Oct 12

Why eigenvalues? Week 8: Friday, Oct 12 I spend a lot of time thinking about eigenvalue problems. In part, this is because I look for problems that can be solved via eigenvalues. But I might have fewer