# M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H

Save this PDF as:

Size: px
Start display at page:

Download "M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H"

## Transcription

1 A N D R E W T U L L O C H M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H T R I N I T Y C O L L E G E T H E U N I V E R S I T Y O F C A M B R I D G E

2

3 Contents 1 Generalities 5 2 Constrained Optimization Lagrangian Multipliers Lagrange Dual Supporting Hyperplanes 8 3 Linear Programming Convexity and Strong Duality Linear Programs Linear Program Duality Complementary Slackness 13 4 Simplex Method Basic Solutions Extreme Points and Optimal Solutions The Simplex Tableau The Simplex Method in Tableau Form 16

4 4 andrew tulloch 5 Advanced Simplex Procedures The Two-Phase Simplex Method Gomory s Cutting Plane Method 17 6 Complexity of Problems and Algorithms Asymptotic Complexity 19 7 The Complexity of Linear Programming A Lower Bound for the Simplex Method The Idea for a New Method 21 8 Graphs and Flows Introduction Minimal Cost Flows 23 9 Transportation and Assignment Problems Non-Cooperative Games Strategic Equilibrium Bibliography 31

5 1 Generalities

6

7 2 Constrained Optimization Minimize f (x) subject to h(x) = b, x X. Objective function f : R n R Vector x R n of decision variables, Functional constraint where h : R n > R m, b R m Regional constraint where X R n. Definition 2.1. The feasible set is X(b) = {x X : h(x) = b}. An inequality of the form g(x) b can be written as g(x) + z = b, where z R m called a slack variable with regional constraint z Lagrangian Multipliers Definition 2.2. Define the Lagrangian of a problem as L(x, λ) = f (x) λ T (h(x) b) (2.1) where λ R m is a vector of Lagrange multipliers Theorem 2.3 (Lagrangian Sufficiency Theorem). Let x X and λ R m such that and h(x) = b. Then x is optimal for P. L(x, λ) = inf x X L(x, λ) (2.2)

8 8 andrew tulloch Proof. min f x X(b) (x ) = min [ f (x) x X(b) λt (h(x ) b)] (2.3) min x X [ f (x ) λ T (h(x ) b)] (2.4) = f (x) λ T (h(x) b) (2.5) = f (x) (2.6) 2.2 Lagrange Dual Definition 2.4. Let φ(b) = inf f (x). (2.7) x X(b) Define the Lagrange dual function g : R m R with g(λ) = in f x X L(x, λ) (2.8) Then, for all λ R m, inf f (x) = inf L(x, λ) inf L(x, λ) = g(λ) (2.9) x X(b) x X(b) x X That is, g(λ) is a lower bound on our optimization function. This motivates the dual problem to maximize g(λ) subject to λ Y, where Y = {λ R m : g(λ) > }. Theorem 2.5 (Duality). From (2.9), we see that the optimal value of the primal is always greater than the optimal value of the dual. This is weak duality. 2.3 Supporting Hyperplanes Fix b R m and consider φ as a function of c R m. Further consider the hyperplane given by α : R m R with α(c) = β λ T (b c) (2.10) Now, try to find φ(b) as follow.

9 mathematics of operations research 9 (i) For each λ, find β λ = max{β : α(c) φ(c), c R m } (2.11) (ii) Choose λ to maximize β λ Definition 2.6. Call α : R m R a supporting hyperplane to φ at b if α(c) = φ(b) λ T (b c) (2.12) and φ(c) φ(b) λ T (b c) (2.13) for all c R m. Theorem 2.7. The following are equivalent (i) There exists a (non-vertical) supporting hyperplane to φ at b, (ii) XXXXXXXXXXXXXXXXXXXXXXXXXX

10

11 3 Linear Programming 3.1 Convexity and Strong Duality Definition 3.1. Let S R n. S is convex if for all δ [0, 1], x, y S implies that δx + (1 δ)y S. f : S R is convex if for all x, y S and δ [0, 1], δ f (x) + (1 δ) f (y) f (δx + (1 δ)y. Visually, the area under the function is a convex set. Definition 3.2. x S is an interior point of S if there exists ɛ > 0 such {y : y x 2 ɛ} S. x S is an extreme point of S if for all y, z S and S (0, 1), x = δy + (1 δ)z implies x = y = z. Theorem 3.3 (Supporting Hyperplane). Suppose that our function φ is convex and b lies in the interior of the set of points where φ is finite. Then there is a (non-vertical) supporting hyperplane to φ at b. Theorem 3.4. Let X(b) = {x X : h(x) b}, φ(b) = inf x X(b) f (x). Then φ is convex if X, f, and h are convex. Proof. Let b 1, b 2 R m such that φ(b 1 ), φ(b 2 ) are defined. Let δ [0, 1] and b = δb 1 + (1 δ)b 2. Consider x 1 X(b 1 ), x 2 X(b 2 ) and let x = δx 1 + (1 δ)x 2.

12 12 andrew tulloch By convexity of Y, x X. By convexity of h, h(x) = h(δx 1 + (1 δ)x 2 ) δh(x 1 ) + (1 δ)h(x 2 ) δb 1 + (1 δ)b 2 = b Thus x X(b), and by convexity of f, φ(b) f (x) = f (δx 1 + (1 δ)x 2 ) δ f (x 1 ) + (1 δ) f (x 2 ) δφ(b 1 ) + (1 δ)φ(b 2 ) 3.2 Linear Programs Definition 3.5. General form of a linear program is min{c T x : Ax b, x 0} (3.1) Standard form of a linear program is min{c T x : Ax = b, x 0} (3.2) 3.3 Linear Program Duality Introduce slack variables to turn problem P into the form min{c T x : Ax z = b, x, z 0} (3.3) We have X = {(x, z) : x, z 0} R m+n. The Lagrangian is L((x, z), λ) = c T x λ T (Ax z b) (3.4) = (c T λ T A)x + λ T z + λ T b (3.5)

13 mathematics of operations research 13 and has a finite minimum if and only if λ Y = {λ : c T λ T A 0, λ 0} (3.6) For a fixed λ Y, the minimum of L is satisfied when (c T λ T A)x = 0 and λ T z = 0, and thus g(λ) = We obtain that the dual problem inf L((x, z), λ) = λ T b (3.7) (x,z) X max{λ T b : A T λ c, λ 0} (3.8) and it can be shown (exercise) that the dual of the dual of a linear program is the original program. 3.4 Complementary Slackness Theorem 3.6. Let x and λ be feasible for P and its dual. Then x and λ are optimal if and only if (c T λ T A)x = 0 (3.9) λ T (Ax b) = 0 (3.10) Proof. If x, λ are optimal, then c T x = λ T b }{{} by strong duality (3.11) = inf x X (ct x λ T (Ax b)) c T x λ T (Ax b) }{{} c T x primal and dual optimality Then the inequalities must be equalities. Thus (3.12) λ T b = c T x λ T (Ax b) = (c T λ T A)x +λ T b (3.13) }{{} =0 and c T x λ T (Ax b) = c T x (3.14) }{{} =0

14 14 andrew tulloch If (c T λ T A)x = 0 and λ T (Ax b) = 0, then c T x = c T λ T (Ax b) = (c T λ T a)x + λ T b = λ T b (3.15) and so by weak duality, x and λ are optimal.

15 4 Simplex Method 4.1 Basic Solutions Maximize c T x subject to Ax = b, x 0, A R m n. Call a solution x R n of Ax = b basic if it has at most m non-zero entries, that is, there exists B {1,..., n} with B = m and x i = 0 if i / B. A basic solution x with x 0 is called a basic feasible solution (bfs). 4.2 Extreme Points and Optimal Solutions We make the following assumptions: (i) The rows of A are linearly independent (ii) Every set of m columns of A are linearly independent. (iii) Every basic solution is non-degenerate - that is, it has exactly m non-zero entries. Theorem 4.1. x is a bfs of Ax = b if and only if it is an extreme point of the set X(b) = {x : Ax = b, x 0}. Theorem 4.2. If the problem has a finite optimum (feasible and bounded), then it has an optimal solution that is a bfs.

16 16 andrew tulloch 4.3 The Simplex Tableau Let A R m n, b R m. Let B be a basis (in the bfs sense), and x R n, such that Ax = b. Then A B x B + A N x N = b (4.1) where A B R m m and A N R m (n m) respectively consist of the columns of A indexed by B and those not indexed by B. Moreover, if x is a basic solution, then there is a basic B such that x N = 0 and A B x B = b, and if x is a basic feasible solution, there is a basis B such that x n = 0, A B x B = b, and x B 0. For every x with Ax = b and every basis B, we have x B = A 1 B (b A Nx N ) (4.2) as we assume that A B has full rank. Thus, f (x) = c T x = c T B x B + c T N x N (4.3) = c T B A 1 B (b A NX N ) + c T N x N (4.4) = CB T A 1 B b + (ct N ct B A 1 B A N)x N (4.5) Assume we can guarantee that A 1 B and x N = 0 is a bfs with b = 0. Then x with x B = A 1 B b f (x ) = C T B A 1 B b (4.6) Assume that we are maximizing c T x. There are two different cases: (i) If C T N CT B A 1 B x is optimal. A N 0, then f (x) c T B A 1 B b for every feasible x, so (ii) If (c T N ct B A 1 B A N) i > 0, then we can increase the objective value by increasing the corresponding row of (x N ) i. 4.4 The Simplex Method in Tableau Form

17 5 Advanced Simplex Procedures 5.1 The Two-Phase Simplex Method Finding an initial bfs is easy if the constraints have the form Ax = b where b 0, as Ax + z = b, (x, z) = (0, b) (5.1) 5.2 Gomory s Cutting Plane Method Used in integer programming (IP). This is a linear program where in addition some of the variables are required to be integral. Assume that for a given integer program we have found an optimal fractional solution x with basis B and let a ij = (A 1 B A j) and a i0 = (A 1 B b) be the entries of the final tableau. If x is not integral, the for some row i, a i0 is not integral. For every feasible solution x, x i = a ij x j x i + a ij x j = a i0. (5.2) j N j N If x is integral, then the left hand side is integral as well, and the inequality must still hold if the right hand side is rounded down. Thus, x i + a ij x j a i0. (5.3) j N Then, we can add this constraint to the problem and solve the augmented program. One can show that this procedure converges in a finite number of steps.

18

19 6 Complexity of Problems and Algorithms 6.1 Asymptotic Complexity We measure complexity as a function of input size. The input of a linear programming problem: c R n, A R m n, b R m is represented in (n + m n + m) k bits if we represent each number using k bits. For two functions f : R R and g : R R write f (n) = O(g(n)) (6.1) if there exists c, n 0 such that for all n n 0, f (n) c g(n) (6.2),... (similarly for Ω, and Θ (Ω + O))

20

21 7 The Complexity of Linear Programming 7.1 A Lower Bound for the Simplex Method Theorem 7.1. There exists a LP of size O(n 2 ), a pivoting rule, and an initial BFS such that the simplex method requires 2 n 1 iterations. Proof. Consider the unit cube in R n, given by constraints 0 x i 1 for i = 1,..., n. Define a spanning path inductively as follows. In dimension 1, go from x 1 = 0 to x 1 = 1. In dimension k, set x k = 0 and follow the path for dimension 1,..., k 1. Then set x = 1, and follow the path for dimension 1,..., k 1 backwards. The objective x n currently increases only once. Instead consider the perturbed unit cube given by the constraints ɛ x 1 1, ɛx i 1 x i 1 ɛx i 1 with ɛ (0, 1 2 ). 7.2 The Idea for a New Method min{c T x : Ax = b, x 0} (7.1) max{b T λ : A T λ c} (7.2) By strong duality, each of these problems has a bounded optimal

22 22 andrew tulloch solution if and only if the following set of constraints is feasible: c T x = b T λ (7.3) Ax = b (7.4) x 0 (7.5) A T λ c (7.6) It is thus enough to decide, for a given A R m n and b R m, whether {x R n : Ax b} =. Definition 7.2. A symmetric matrix D R n n is called positive definite if x T Dx > 0 for every x R n. Alternatively, all eigenvalues of the matrix are strictly positive. Definition 7.3. A set E R n given by E = E(z, D) = {x R n : (x z) T D(x z) 1} (7.7) for a positive definite symmetric D R n n and z R n is called an ellipsoid with center z. Let P = {x R n : Ax b} for some A R m n and b R m. To decide whether P =, we generate a sequence {E t } of ellipsoids E t with centers x t. If x t P, then P is non-empty and the method stops. If x t / P, then one of the constraints is violated - so there exists a row j of A such that a T j x t < b j. Therefore, P is contained in the half-space {x R n : a T j x at j x t, and in particular the intersection of this half-space with E t, which we will call a half-ellipsoid. Theorem 7.4. Let E = E(z, D) be an ellipsoid in R n and a R n = 0. Consider the half-space H = {x R n a T x a T z}, and let z = z + 1 n + 1 D = n2 n 2 1 Da a T Da ( D 2 n + 1 Daa T ) D a T Da (7.8) (7.9) Then D is symmetric and positive definite, and therefore E = E(z, D ) is an ellipsoid. Moreover, E H E and Vol(E ) < e 2(n+1) 1 Vol(E).

23 8 Graphs and Flows 8.1 Introduction Consider a directed graph (network) G = (V, E), Vthe set of vertices, E V V a set of edges. Undirected if E is symmetric. 8.2 Minimal Cost Flows Fill in this stuff from lectures

24

25 9 Transportation and Assignment Problems

26

27 10 Non-Cooperative Games Theorem 10.1 (von Neumann, 1928). Let P R m n. Then max min x X y Y p(x, y) = min max y Y x X p(x, y) (10.1)

28

29 11 Strategic Equilibrium Definition x X is a best response to y Y if p(x, y) = max x X p(x, y). (x, y) X Y is a Nash equilibrium if x is a best response to y and y is a best response to x. Theorem (x, y) X Y is an equilibrium of the matrix game P if and only if min p(x, y ) = max min p(x, y ) (11.1) y Y x X y Y and max p(x, y) = min max p(x, y ). (11.2) x X y Y x X Theorem Let (x, y), (x, y ) X Y be equilibria of the matrix game with payoff matrix P. Then p(x, y) = p(x, y ) and (x, y ) and (x, y) are equilibria as well. Theorem 11.4 (Nash, 1951). Every bimatrix game has an equilibrium.

30

31 12 Bibliography

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Equilibrium computation: Part 1

Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### Can linear programs solve NP-hard problems?

Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf Linear programs Can linear programs solve NP-hard problems? p. 2/9 Can linear programs solve

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### Some representability and duality results for convex mixed-integer programs.

Some representability and duality results for convex mixed-integer programs. Santanu S. Dey Joint work with Diego Morán and Juan Pablo Vielma December 17, 2012. Introduction About Motivation Mixed integer

### Optimization in R n Introduction

Optimization in R n Introduction Rudi Pendavingh Eindhoven Technical University Optimization in R n, lecture Rudi Pendavingh (TUE) Optimization in R n Introduction ORN / 4 Some optimization problems designing

### Convex Programming Tools for Disjunctive Programs

Convex Programming Tools for Disjunctive Programs João Soares, Departamento de Matemática, Universidade de Coimbra, Portugal Abstract A Disjunctive Program (DP) is a mathematical program whose feasible

### IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### Interior Point Methods and Linear Programming

Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### Solutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR

Solutions Of Some Non-Linear Programming Problems A PROJECT REPORT submitted by BIJAN KUMAR PATEL for the partial fulfilment for the award of the degree of Master of Science in Mathematics under the supervision

### Lecture 11: 0-1 Quadratic Program and Lower Bounds

Lecture : - Quadratic Program and Lower Bounds (3 units) Outline Problem formulations Reformulation: Linearization & continuous relaxation Branch & Bound Method framework Simple bounds, LP bound and semidefinite

### 10. Proximal point method

L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### Dantzig-Wolfe bound and Dantzig-Wolfe cookbook

Dantzig-Wolfe bound and Dantzig-Wolfe cookbook thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline LP strength of the Dantzig-Wolfe The exercise from last week... The Dantzig-Wolfe

### Classification of Cartan matrices

Chapter 7 Classification of Cartan matrices In this chapter we describe a classification of generalised Cartan matrices This classification can be compared as the rough classification of varieties in terms

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Two-Stage Stochastic Linear Programs

Two-Stage Stochastic Linear Programs Operations Research Anthony Papavasiliou 1 / 27 Two-Stage Stochastic Linear Programs 1 Short Reviews Probability Spaces and Random Variables Convex Analysis 2 Deterministic

### Advanced Lecture on Mathematical Science and Information Science I. Optimization in Finance

Advanced Lecture on Mathematical Science and Information Science I Optimization in Finance Reha H. Tütüncü Visiting Associate Professor Dept. of Mathematical and Computing Sciences Tokyo Institute of Technology

### Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

### Notes V General Equilibrium: Positive Theory. 1 Walrasian Equilibrium and Excess Demand

Notes V General Equilibrium: Positive Theory In this lecture we go on considering a general equilibrium model of a private ownership economy. In contrast to the Notes IV, we focus on positive issues such

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### NP-Hardness Results Related to PPAD

NP-Hardness Results Related to PPAD Chuangyin Dang Dept. of Manufacturing Engineering & Engineering Management City University of Hong Kong Kowloon, Hong Kong SAR, China E-Mail: mecdang@cityu.edu.hk Yinyu

### Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

### An Expressive Auction Design for Online Display Advertising. AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock

An Expressive Auction Design for Online Display Advertising AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock Li PU & Tong ZHANG Motivation Online advertisement allow advertisers to specify

### Random graphs with a given degree sequence

Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.

### The Max-Distance Network Creation Game on General Host Graphs

The Max-Distance Network Creation Game on General Host Graphs 13 Luglio 2012 Introduction Network Creation Games are games that model the formation of large-scale networks governed by autonomous agents.

### constraint. Let us penalize ourselves for making the constraint too big. We end up with a

Chapter 4 Constrained Optimization 4.1 Equality Constraints (Lagrangians) Suppose we have a problem: Maximize 5, (x 1, 2) 2, 2(x 2, 1) 2 subject to x 1 +4x 2 =3 If we ignore the constraint, we get the

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Bargaining Solutions in a Social Network

Bargaining Solutions in a Social Network Tanmoy Chakraborty and Michael Kearns Department of Computer and Information Science University of Pennsylvania Abstract. We study the concept of bargaining solutions,

### Distributed Machine Learning and Big Data

Distributed Machine Learning and Big Data Sourangshu Bhattacharya Dept. of Computer Science and Engineering, IIT Kharagpur. http://cse.iitkgp.ac.in/~sourangshu/ August 21, 2015 Sourangshu Bhattacharya

### Big Data - Lecture 1 Optimization reminders

Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics

### PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.

PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include

### Math 215 HW #6 Solutions

Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Factorization Theorems

Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

### An interval linear programming contractor

An interval linear programming contractor Introduction Milan Hladík Abstract. We consider linear programming with interval data. One of the most challenging problems in this topic is to determine or tight

### Game Theory: Supermodular Games 1

Game Theory: Supermodular Games 1 Christoph Schottmüller 1 License: CC Attribution ShareAlike 4.0 1 / 22 Outline 1 Introduction 2 Model 3 Revision questions and exercises 2 / 22 Motivation I several solution

### Minimizing costs for transport buyers using integer programming and column generation. Eser Esirgen

MASTER STHESIS Minimizing costs for transport buyers using integer programming and column generation Eser Esirgen DepartmentofMathematicalSciences CHALMERS UNIVERSITY OF TECHNOLOGY UNIVERSITY OF GOTHENBURG

### 4.1 Learning algorithms for neural networks

4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

### Chapter 6: Sensitivity Analysis

Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production

Adaptive Linear Programming Decoding Mohammad H. Taghavi and Paul H. Siegel ECE Department, University of California, San Diego Email: (mtaghavi, psiegel)@ucsd.edu ISIT 2006, Seattle, USA, July 9 14, 2006

### Optimization Methods in Finance

Optimization Methods in Finance Gerard Cornuejols Reha Tütüncü Carnegie Mellon University, Pittsburgh, PA 15213 USA January 2006 2 Foreword Optimization models play an increasingly important role in financial

### Math 312 Homework 1 Solutions

Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

### 9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1

9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 Seffi Naor Computer Science Dept. Technion Haifa, Israel Introduction

### Network Traffic Modelling

University of York Dissertation submitted for the MSc in Mathematics with Modern Applications, Department of Mathematics, University of York, UK. August 009 Network Traffic Modelling Author: David Slade

### Massive Data Classification via Unconstrained Support Vector Machines

Massive Data Classification via Unconstrained Support Vector Machines Olvi L. Mangasarian and Michael E. Thompson Computer Sciences Department University of Wisconsin 1210 West Dayton Street Madison, WI

### Optimization Theory for Large Systems

Optimization Theory for Large Systems LEON S. LASDON CASE WESTERN RESERVE UNIVERSITY THE MACMILLAN COMPANY COLLIER-MACMILLAN LIMITED, LONDON Contents 1. Linear and Nonlinear Programming 1 1.1 Unconstrained

CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen

(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained

### THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

### (Quasi-)Newton methods

(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### The Equivalence of Linear Programs and Zero-Sum Games

The Equivalence of Linear Programs and Zero-Sum Games Ilan Adler, IEOR Dep, UC Berkeley adler@ieor.berkeley.edu Abstract In 1951, Dantzig showed the equivalence of linear programming problems and two-person

### INTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models

Integer Programming INTEGER PROGRAMMING In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is

### Online Learning and Competitive Analysis: a Unified Approach

Online Learning and Competitive Analysis: a Unified Approach Shahar Chen Online Learning and Competitive Analysis: a Unified Approach Research Thesis Submitted in partial fulfillment of the requirements

### Permutation Betting Markets: Singleton Betting with Extra Information

Permutation Betting Markets: Singleton Betting with Extra Information Mohammad Ghodsi Sharif University of Technology ghodsi@sharif.edu Hamid Mahini Sharif University of Technology mahini@ce.sharif.edu

### COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES

COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES Appeared as Chapter 45, Handbook of Game Theory with Economic Applications, Vol. 3 (2002), eds. R. J. Aumann and S. Hart, Elsevier, Amsterdam, pages 1723 1759.

### Manipulability of the Price Mechanism for Data Centers

Manipulability of the Price Mechanism for Data Centers Greg Bodwin 1, Eric Friedman 2,3,4, and Scott Shenker 3,4 1 Department of Computer Science, Tufts University, Medford, Massachusetts 02155 2 School

### A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems Danilo Ardagna 1, Barbara Panicucci 1, Mauro Passacantando 2 1 Politecnico di Milano,, Italy 2 Università di Pisa, Dipartimento

### Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

### DSL Spectrum Management

DSL Spectrum Management Dr. Jianwei Huang Department of Electrical Engineering Princeton University Guest Lecture of ELE539A March 2007 Jianwei Huang (Princeton) DSL Spectrum Management March 2007 1 /

### (a) We have x = 3 + 2t, y = 2 t, z = 6 so solving for t we get the symmetric equations. x 3 2. = 2 y, z = 6. t 2 2t + 1 = 0,

Name: Solutions to Practice Final. Consider the line r(t) = 3 + t, t, 6. (a) Find symmetric equations for this line. (b) Find the point where the first line r(t) intersects the surface z = x + y. (a) We

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### Perron vector Optimization applied to search engines

Perron vector Optimization applied to search engines Olivier Fercoq INRIA Saclay and CMAP Ecole Polytechnique May 18, 2011 Web page ranking The core of search engines Semantic rankings (keywords) Hyperlink

### Sensitivity analysis of utility based prices and risk-tolerance wealth processes

Sensitivity analysis of utility based prices and risk-tolerance wealth processes Dmitry Kramkov, Carnegie Mellon University Based on a paper with Mihai Sirbu from Columbia University Math Finance Seminar,

### SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH

31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### The Open University s repository of research publications and other research outputs

Open Research Online The Open University s repository of research publications and other research outputs The degree-diameter problem for circulant graphs of degree 8 and 9 Journal Article How to cite:

### Convex analysis and profit/cost/support functions

CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

### Recall that two vectors in are perpendicular or orthogonal provided that their dot

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

### Variational approach to restore point-like and curve-like singularities in imaging

Variational approach to restore point-like and curve-like singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure Blanc-Féraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012

### Module1. x 1000. y 800.

Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

### RESULTANT AND DISCRIMINANT OF POLYNOMIALS

RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results

### A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems

A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems Sunyoung Kim, Masakazu Kojima and Kim-Chuan Toh October 2013 Abstract. We propose

Computational Learning Theory Spring Semester, 2009/10 Lecture 11: Sponsored search Lecturer: Yishay Mansour Scribe: Ben Pere, Jonathan Heimann, Alon Levin 11.1 Sponsored Search 11.1.1 Introduction Search

### 1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style

Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with

### ON LIMIT LAWS FOR CENTRAL ORDER STATISTICS UNDER POWER NORMALIZATION. E. I. Pancheva, A. Gacovska-Barandovska

Pliska Stud. Math. Bulgar. 22 (2015), STUDIA MATHEMATICA BULGARICA ON LIMIT LAWS FOR CENTRAL ORDER STATISTICS UNDER POWER NORMALIZATION E. I. Pancheva, A. Gacovska-Barandovska Smirnov (1949) derived four

### Study Guide 2 Solutions MATH 111

Study Guide 2 Solutions MATH 111 Having read through the sample test, I wanted to warn everyone, that I might consider asking questions involving inequalities, the absolute value function (as in the suggested

### Ideal Class Group and Units

Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

### Noncommercial Software for Mixed-Integer Linear Programming

Noncommercial Software for Mixed-Integer Linear Programming J. T. Linderoth T. K. Ralphs December, 2004. Revised: January, 2005. Abstract We present an overview of noncommercial software tools for the

### Creating a More Efficient Course Schedule at WPI Using Linear Optimization

Project Number: ACH1211 Creating a More Efficient Course Schedule at WPI Using Linear Optimization A Major Qualifying Project Report submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial

### A short note on American option prices

A short note on American option Filip Lindskog April 27, 2012 1 The set-up An American call option with strike price K written on some stock gives the holder the right to buy a share of the stock (exercise

### Scheduling a sequence of tasks with general completion costs

Scheduling a sequence of tasks with general completion costs Francis Sourd CNRS-LIP6 4, place Jussieu 75252 Paris Cedex 05, France Francis.Sourd@lip6.fr Abstract Scheduling a sequence of tasks in the acceptation

### Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology

Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem www.cs.huji.ac.il/ yweiss

### Optimal Gateway Selection in Multi-domain Wireless Networks: A Potential Game Perspective

Optimal Gateway Selection in Multi-domain Wireless Networks: A Potential Game Perspective Yang Song, Starsky H.Y. Wong, and Kang-Won Lee Wireless Networking Research Group IBM T. J. Watson Research Center

### Similar matrices and Jordan form

Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

### On the representability of the bi-uniform matroid

On the representability of the bi-uniform matroid Simeon Ball, Carles Padró, Zsuzsa Weiner and Chaoping Xing August 3, 2012 Abstract Every bi-uniform matroid is representable over all sufficiently large

### Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

### Chapter 2: Systems of Linear Equations and Matrices:

At the end of the lesson, you should be able to: Chapter 2: Systems of Linear Equations and Matrices: 2.1: Solutions of Linear Systems by the Echelon Method Define linear systems, unique solution, inconsistent,