M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H"

Transcription

1 A N D R E W T U L L O C H M AT H E M AT I C S O F O P - E R AT I O N S R E S E A R C H T R I N I T Y C O L L E G E T H E U N I V E R S I T Y O F C A M B R I D G E

2

3 Contents 1 Generalities 5 2 Constrained Optimization Lagrangian Multipliers Lagrange Dual Supporting Hyperplanes 8 3 Linear Programming Convexity and Strong Duality Linear Programs Linear Program Duality Complementary Slackness 13 4 Simplex Method Basic Solutions Extreme Points and Optimal Solutions The Simplex Tableau The Simplex Method in Tableau Form 16

4 4 andrew tulloch 5 Advanced Simplex Procedures The Two-Phase Simplex Method Gomory s Cutting Plane Method 17 6 Complexity of Problems and Algorithms Asymptotic Complexity 19 7 The Complexity of Linear Programming A Lower Bound for the Simplex Method The Idea for a New Method 21 8 Graphs and Flows Introduction Minimal Cost Flows 23 9 Transportation and Assignment Problems Non-Cooperative Games Strategic Equilibrium Bibliography 31

5 1 Generalities

6

7 2 Constrained Optimization Minimize f (x) subject to h(x) = b, x X. Objective function f : R n R Vector x R n of decision variables, Functional constraint where h : R n > R m, b R m Regional constraint where X R n. Definition 2.1. The feasible set is X(b) = {x X : h(x) = b}. An inequality of the form g(x) b can be written as g(x) + z = b, where z R m called a slack variable with regional constraint z Lagrangian Multipliers Definition 2.2. Define the Lagrangian of a problem as L(x, λ) = f (x) λ T (h(x) b) (2.1) where λ R m is a vector of Lagrange multipliers Theorem 2.3 (Lagrangian Sufficiency Theorem). Let x X and λ R m such that and h(x) = b. Then x is optimal for P. L(x, λ) = inf x X L(x, λ) (2.2)

8 8 andrew tulloch Proof. min f x X(b) (x ) = min [ f (x) x X(b) λt (h(x ) b)] (2.3) min x X [ f (x ) λ T (h(x ) b)] (2.4) = f (x) λ T (h(x) b) (2.5) = f (x) (2.6) 2.2 Lagrange Dual Definition 2.4. Let φ(b) = inf f (x). (2.7) x X(b) Define the Lagrange dual function g : R m R with g(λ) = in f x X L(x, λ) (2.8) Then, for all λ R m, inf f (x) = inf L(x, λ) inf L(x, λ) = g(λ) (2.9) x X(b) x X(b) x X That is, g(λ) is a lower bound on our optimization function. This motivates the dual problem to maximize g(λ) subject to λ Y, where Y = {λ R m : g(λ) > }. Theorem 2.5 (Duality). From (2.9), we see that the optimal value of the primal is always greater than the optimal value of the dual. This is weak duality. 2.3 Supporting Hyperplanes Fix b R m and consider φ as a function of c R m. Further consider the hyperplane given by α : R m R with α(c) = β λ T (b c) (2.10) Now, try to find φ(b) as follow.

9 mathematics of operations research 9 (i) For each λ, find β λ = max{β : α(c) φ(c), c R m } (2.11) (ii) Choose λ to maximize β λ Definition 2.6. Call α : R m R a supporting hyperplane to φ at b if α(c) = φ(b) λ T (b c) (2.12) and φ(c) φ(b) λ T (b c) (2.13) for all c R m. Theorem 2.7. The following are equivalent (i) There exists a (non-vertical) supporting hyperplane to φ at b, (ii) XXXXXXXXXXXXXXXXXXXXXXXXXX

10

11 3 Linear Programming 3.1 Convexity and Strong Duality Definition 3.1. Let S R n. S is convex if for all δ [0, 1], x, y S implies that δx + (1 δ)y S. f : S R is convex if for all x, y S and δ [0, 1], δ f (x) + (1 δ) f (y) f (δx + (1 δ)y. Visually, the area under the function is a convex set. Definition 3.2. x S is an interior point of S if there exists ɛ > 0 such {y : y x 2 ɛ} S. x S is an extreme point of S if for all y, z S and S (0, 1), x = δy + (1 δ)z implies x = y = z. Theorem 3.3 (Supporting Hyperplane). Suppose that our function φ is convex and b lies in the interior of the set of points where φ is finite. Then there is a (non-vertical) supporting hyperplane to φ at b. Theorem 3.4. Let X(b) = {x X : h(x) b}, φ(b) = inf x X(b) f (x). Then φ is convex if X, f, and h are convex. Proof. Let b 1, b 2 R m such that φ(b 1 ), φ(b 2 ) are defined. Let δ [0, 1] and b = δb 1 + (1 δ)b 2. Consider x 1 X(b 1 ), x 2 X(b 2 ) and let x = δx 1 + (1 δ)x 2.

12 12 andrew tulloch By convexity of Y, x X. By convexity of h, h(x) = h(δx 1 + (1 δ)x 2 ) δh(x 1 ) + (1 δ)h(x 2 ) δb 1 + (1 δ)b 2 = b Thus x X(b), and by convexity of f, φ(b) f (x) = f (δx 1 + (1 δ)x 2 ) δ f (x 1 ) + (1 δ) f (x 2 ) δφ(b 1 ) + (1 δ)φ(b 2 ) 3.2 Linear Programs Definition 3.5. General form of a linear program is min{c T x : Ax b, x 0} (3.1) Standard form of a linear program is min{c T x : Ax = b, x 0} (3.2) 3.3 Linear Program Duality Introduce slack variables to turn problem P into the form min{c T x : Ax z = b, x, z 0} (3.3) We have X = {(x, z) : x, z 0} R m+n. The Lagrangian is L((x, z), λ) = c T x λ T (Ax z b) (3.4) = (c T λ T A)x + λ T z + λ T b (3.5)

13 mathematics of operations research 13 and has a finite minimum if and only if λ Y = {λ : c T λ T A 0, λ 0} (3.6) For a fixed λ Y, the minimum of L is satisfied when (c T λ T A)x = 0 and λ T z = 0, and thus g(λ) = We obtain that the dual problem inf L((x, z), λ) = λ T b (3.7) (x,z) X max{λ T b : A T λ c, λ 0} (3.8) and it can be shown (exercise) that the dual of the dual of a linear program is the original program. 3.4 Complementary Slackness Theorem 3.6. Let x and λ be feasible for P and its dual. Then x and λ are optimal if and only if (c T λ T A)x = 0 (3.9) λ T (Ax b) = 0 (3.10) Proof. If x, λ are optimal, then c T x = λ T b }{{} by strong duality (3.11) = inf x X (ct x λ T (Ax b)) c T x λ T (Ax b) }{{} c T x primal and dual optimality Then the inequalities must be equalities. Thus (3.12) λ T b = c T x λ T (Ax b) = (c T λ T A)x +λ T b (3.13) }{{} =0 and c T x λ T (Ax b) = c T x (3.14) }{{} =0

14 14 andrew tulloch If (c T λ T A)x = 0 and λ T (Ax b) = 0, then c T x = c T λ T (Ax b) = (c T λ T a)x + λ T b = λ T b (3.15) and so by weak duality, x and λ are optimal.

15 4 Simplex Method 4.1 Basic Solutions Maximize c T x subject to Ax = b, x 0, A R m n. Call a solution x R n of Ax = b basic if it has at most m non-zero entries, that is, there exists B {1,..., n} with B = m and x i = 0 if i / B. A basic solution x with x 0 is called a basic feasible solution (bfs). 4.2 Extreme Points and Optimal Solutions We make the following assumptions: (i) The rows of A are linearly independent (ii) Every set of m columns of A are linearly independent. (iii) Every basic solution is non-degenerate - that is, it has exactly m non-zero entries. Theorem 4.1. x is a bfs of Ax = b if and only if it is an extreme point of the set X(b) = {x : Ax = b, x 0}. Theorem 4.2. If the problem has a finite optimum (feasible and bounded), then it has an optimal solution that is a bfs.

16 16 andrew tulloch 4.3 The Simplex Tableau Let A R m n, b R m. Let B be a basis (in the bfs sense), and x R n, such that Ax = b. Then A B x B + A N x N = b (4.1) where A B R m m and A N R m (n m) respectively consist of the columns of A indexed by B and those not indexed by B. Moreover, if x is a basic solution, then there is a basic B such that x N = 0 and A B x B = b, and if x is a basic feasible solution, there is a basis B such that x n = 0, A B x B = b, and x B 0. For every x with Ax = b and every basis B, we have x B = A 1 B (b A Nx N ) (4.2) as we assume that A B has full rank. Thus, f (x) = c T x = c T B x B + c T N x N (4.3) = c T B A 1 B (b A NX N ) + c T N x N (4.4) = CB T A 1 B b + (ct N ct B A 1 B A N)x N (4.5) Assume we can guarantee that A 1 B and x N = 0 is a bfs with b = 0. Then x with x B = A 1 B b f (x ) = C T B A 1 B b (4.6) Assume that we are maximizing c T x. There are two different cases: (i) If C T N CT B A 1 B x is optimal. A N 0, then f (x) c T B A 1 B b for every feasible x, so (ii) If (c T N ct B A 1 B A N) i > 0, then we can increase the objective value by increasing the corresponding row of (x N ) i. 4.4 The Simplex Method in Tableau Form

17 5 Advanced Simplex Procedures 5.1 The Two-Phase Simplex Method Finding an initial bfs is easy if the constraints have the form Ax = b where b 0, as Ax + z = b, (x, z) = (0, b) (5.1) 5.2 Gomory s Cutting Plane Method Used in integer programming (IP). This is a linear program where in addition some of the variables are required to be integral. Assume that for a given integer program we have found an optimal fractional solution x with basis B and let a ij = (A 1 B A j) and a i0 = (A 1 B b) be the entries of the final tableau. If x is not integral, the for some row i, a i0 is not integral. For every feasible solution x, x i = a ij x j x i + a ij x j = a i0. (5.2) j N j N If x is integral, then the left hand side is integral as well, and the inequality must still hold if the right hand side is rounded down. Thus, x i + a ij x j a i0. (5.3) j N Then, we can add this constraint to the problem and solve the augmented program. One can show that this procedure converges in a finite number of steps.

18

19 6 Complexity of Problems and Algorithms 6.1 Asymptotic Complexity We measure complexity as a function of input size. The input of a linear programming problem: c R n, A R m n, b R m is represented in (n + m n + m) k bits if we represent each number using k bits. For two functions f : R R and g : R R write f (n) = O(g(n)) (6.1) if there exists c, n 0 such that for all n n 0, f (n) c g(n) (6.2),... (similarly for Ω, and Θ (Ω + O))

20

21 7 The Complexity of Linear Programming 7.1 A Lower Bound for the Simplex Method Theorem 7.1. There exists a LP of size O(n 2 ), a pivoting rule, and an initial BFS such that the simplex method requires 2 n 1 iterations. Proof. Consider the unit cube in R n, given by constraints 0 x i 1 for i = 1,..., n. Define a spanning path inductively as follows. In dimension 1, go from x 1 = 0 to x 1 = 1. In dimension k, set x k = 0 and follow the path for dimension 1,..., k 1. Then set x = 1, and follow the path for dimension 1,..., k 1 backwards. The objective x n currently increases only once. Instead consider the perturbed unit cube given by the constraints ɛ x 1 1, ɛx i 1 x i 1 ɛx i 1 with ɛ (0, 1 2 ). 7.2 The Idea for a New Method min{c T x : Ax = b, x 0} (7.1) max{b T λ : A T λ c} (7.2) By strong duality, each of these problems has a bounded optimal

22 22 andrew tulloch solution if and only if the following set of constraints is feasible: c T x = b T λ (7.3) Ax = b (7.4) x 0 (7.5) A T λ c (7.6) It is thus enough to decide, for a given A R m n and b R m, whether {x R n : Ax b} =. Definition 7.2. A symmetric matrix D R n n is called positive definite if x T Dx > 0 for every x R n. Alternatively, all eigenvalues of the matrix are strictly positive. Definition 7.3. A set E R n given by E = E(z, D) = {x R n : (x z) T D(x z) 1} (7.7) for a positive definite symmetric D R n n and z R n is called an ellipsoid with center z. Let P = {x R n : Ax b} for some A R m n and b R m. To decide whether P =, we generate a sequence {E t } of ellipsoids E t with centers x t. If x t P, then P is non-empty and the method stops. If x t / P, then one of the constraints is violated - so there exists a row j of A such that a T j x t < b j. Therefore, P is contained in the half-space {x R n : a T j x at j x t, and in particular the intersection of this half-space with E t, which we will call a half-ellipsoid. Theorem 7.4. Let E = E(z, D) be an ellipsoid in R n and a R n = 0. Consider the half-space H = {x R n a T x a T z}, and let z = z + 1 n + 1 D = n2 n 2 1 Da a T Da ( D 2 n + 1 Daa T ) D a T Da (7.8) (7.9) Then D is symmetric and positive definite, and therefore E = E(z, D ) is an ellipsoid. Moreover, E H E and Vol(E ) < e 2(n+1) 1 Vol(E).

23 8 Graphs and Flows 8.1 Introduction Consider a directed graph (network) G = (V, E), Vthe set of vertices, E V V a set of edges. Undirected if E is symmetric. 8.2 Minimal Cost Flows Fill in this stuff from lectures

24

25 9 Transportation and Assignment Problems

26

27 10 Non-Cooperative Games Theorem 10.1 (von Neumann, 1928). Let P R m n. Then max min x X y Y p(x, y) = min max y Y x X p(x, y) (10.1)

28

29 11 Strategic Equilibrium Definition x X is a best response to y Y if p(x, y) = max x X p(x, y). (x, y) X Y is a Nash equilibrium if x is a best response to y and y is a best response to x. Theorem (x, y) X Y is an equilibrium of the matrix game P if and only if min p(x, y ) = max min p(x, y ) (11.1) y Y x X y Y and max p(x, y) = min max p(x, y ). (11.2) x X y Y x X Theorem Let (x, y), (x, y ) X Y be equilibria of the matrix game with payoff matrix P. Then p(x, y) = p(x, y ) and (x, y ) and (x, y) are equilibria as well. Theorem 11.4 (Nash, 1951). Every bimatrix game has an equilibrium.

30

31 12 Bibliography

Proximal mapping via network optimization

Proximal mapping via network optimization L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

More information

3. Linear Programming and Polyhedral Combinatorics

3. Linear Programming and Polyhedral Combinatorics Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

More information

1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where. Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

More information

Equilibrium computation: Part 1

Equilibrium computation: Part 1 Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium

More information

Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725 Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

More information

Can linear programs solve NP-hard problems?

Can linear programs solve NP-hard problems? Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf Linear programs Can linear programs solve NP-hard problems? p. 2/9 Can linear programs solve

More information

Approximation Algorithms

Approximation Algorithms Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

More information

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

More information

Some representability and duality results for convex mixed-integer programs.

Some representability and duality results for convex mixed-integer programs. Some representability and duality results for convex mixed-integer programs. Santanu S. Dey Joint work with Diego Morán and Juan Pablo Vielma December 17, 2012. Introduction About Motivation Mixed integer

More information

Optimization in R n Introduction

Optimization in R n Introduction Optimization in R n Introduction Rudi Pendavingh Eindhoven Technical University Optimization in R n, lecture Rudi Pendavingh (TUE) Optimization in R n Introduction ORN / 4 Some optimization problems designing

More information

Convex Programming Tools for Disjunctive Programs

Convex Programming Tools for Disjunctive Programs Convex Programming Tools for Disjunctive Programs João Soares, Departamento de Matemática, Universidade de Coimbra, Portugal Abstract A Disjunctive Program (DP) is a mathematical program whose feasible

More information

IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2 IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

More information

24. The Branch and Bound Method

24. The Branch and Bound Method 24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

More information

Interior Point Methods and Linear Programming

Interior Point Methods and Linear Programming Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either

More information

OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

More information

Solutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR

Solutions Of Some Non-Linear Programming Problems BIJAN KUMAR PATEL. Master of Science in Mathematics. Prof. ANIL KUMAR Solutions Of Some Non-Linear Programming Problems A PROJECT REPORT submitted by BIJAN KUMAR PATEL for the partial fulfilment for the award of the degree of Master of Science in Mathematics under the supervision

More information

Lecture 11: 0-1 Quadratic Program and Lower Bounds

Lecture 11: 0-1 Quadratic Program and Lower Bounds Lecture : - Quadratic Program and Lower Bounds (3 units) Outline Problem formulations Reformulation: Linearization & continuous relaxation Branch & Bound Method framework Simple bounds, LP bound and semidefinite

More information

10. Proximal point method

10. Proximal point method L. Vandenberghe EE236C Spring 2013-14) 10. Proximal point method proximal point method augmented Lagrangian method Moreau-Yosida smoothing 10-1 Proximal point method a conceptual algorithm for minimizing

More information

3. INNER PRODUCT SPACES

3. INNER PRODUCT SPACES . INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

More information

Dantzig-Wolfe bound and Dantzig-Wolfe cookbook

Dantzig-Wolfe bound and Dantzig-Wolfe cookbook Dantzig-Wolfe bound and Dantzig-Wolfe cookbook thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline LP strength of the Dantzig-Wolfe The exercise from last week... The Dantzig-Wolfe

More information

Classification of Cartan matrices

Classification of Cartan matrices Chapter 7 Classification of Cartan matrices In this chapter we describe a classification of generalised Cartan matrices This classification can be compared as the rough classification of varieties in terms

More information

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm. Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

More information

Two-Stage Stochastic Linear Programs

Two-Stage Stochastic Linear Programs Two-Stage Stochastic Linear Programs Operations Research Anthony Papavasiliou 1 / 27 Two-Stage Stochastic Linear Programs 1 Short Reviews Probability Spaces and Random Variables Convex Analysis 2 Deterministic

More information

Advanced Lecture on Mathematical Science and Information Science I. Optimization in Finance

Advanced Lecture on Mathematical Science and Information Science I. Optimization in Finance Advanced Lecture on Mathematical Science and Information Science I Optimization in Finance Reha H. Tütüncü Visiting Associate Professor Dept. of Mathematical and Computing Sciences Tokyo Institute of Technology

More information

Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

More information

Notes V General Equilibrium: Positive Theory. 1 Walrasian Equilibrium and Excess Demand

Notes V General Equilibrium: Positive Theory. 1 Walrasian Equilibrium and Excess Demand Notes V General Equilibrium: Positive Theory In this lecture we go on considering a general equilibrium model of a private ownership economy. In contrast to the Notes IV, we focus on positive issues such

More information

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm. Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

More information

NP-Hardness Results Related to PPAD

NP-Hardness Results Related to PPAD NP-Hardness Results Related to PPAD Chuangyin Dang Dept. of Manufacturing Engineering & Engineering Management City University of Hong Kong Kowloon, Hong Kong SAR, China E-Mail: mecdang@cityu.edu.hk Yinyu

More information

Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach

Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach Outline Error Bound for Classes of Polynomial Systems and its Applications: A Variational Analysis Approach The University of New South Wales SPOM 2013 Joint work with V. Jeyakumar, B.S. Mordukhovich and

More information

An Expressive Auction Design for Online Display Advertising. AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock

An Expressive Auction Design for Online Display Advertising. AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock An Expressive Auction Design for Online Display Advertising AUTHORS: Sébastien Lahaie, David C. Parkes, David M. Pennock Li PU & Tong ZHANG Motivation Online advertisement allow advertisers to specify

More information

Random graphs with a given degree sequence

Random graphs with a given degree sequence Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.

More information

The Max-Distance Network Creation Game on General Host Graphs

The Max-Distance Network Creation Game on General Host Graphs The Max-Distance Network Creation Game on General Host Graphs 13 Luglio 2012 Introduction Network Creation Games are games that model the formation of large-scale networks governed by autonomous agents.

More information

constraint. Let us penalize ourselves for making the constraint too big. We end up with a

constraint. Let us penalize ourselves for making the constraint too big. We end up with a Chapter 4 Constrained Optimization 4.1 Equality Constraints (Lagrangians) Suppose we have a problem: Maximize 5, (x 1, 2) 2, 2(x 2, 1) 2 subject to x 1 +4x 2 =3 If we ignore the constraint, we get the

More information

Adaptive Online Gradient Descent

Adaptive Online Gradient Descent Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

More information

Bargaining Solutions in a Social Network

Bargaining Solutions in a Social Network Bargaining Solutions in a Social Network Tanmoy Chakraborty and Michael Kearns Department of Computer and Information Science University of Pennsylvania Abstract. We study the concept of bargaining solutions,

More information

Distributed Machine Learning and Big Data

Distributed Machine Learning and Big Data Distributed Machine Learning and Big Data Sourangshu Bhattacharya Dept. of Computer Science and Engineering, IIT Kharagpur. http://cse.iitkgp.ac.in/~sourangshu/ August 21, 2015 Sourangshu Bhattacharya

More information

Big Data - Lecture 1 Optimization reminders

Big Data - Lecture 1 Optimization reminders Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Big Data - Lecture 1 Optimization reminders S. Gadat Toulouse, Octobre 2014 Schedule Introduction Major issues Examples Mathematics

More information

PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.

PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5. PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include

More information

Math 215 HW #6 Solutions

Math 215 HW #6 Solutions Math 5 HW #6 Solutions Problem 34 Show that x y is orthogonal to x + y if and only if x = y Proof First, suppose x y is orthogonal to x + y Then since x, y = y, x In other words, = x y, x + y = (x y) T

More information

Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

More information

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling

Chapter 11. 11.1 Load Balancing. Approximation Algorithms. Load Balancing. Load Balancing on 2 Machines. Load Balancing: Greedy Scheduling Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

More information

Factorization Theorems

Factorization Theorems Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization

More information

An interval linear programming contractor

An interval linear programming contractor An interval linear programming contractor Introduction Milan Hladík Abstract. We consider linear programming with interval data. One of the most challenging problems in this topic is to determine or tight

More information

Game Theory: Supermodular Games 1

Game Theory: Supermodular Games 1 Game Theory: Supermodular Games 1 Christoph Schottmüller 1 License: CC Attribution ShareAlike 4.0 1 / 22 Outline 1 Introduction 2 Model 3 Revision questions and exercises 2 / 22 Motivation I several solution

More information

Minimizing costs for transport buyers using integer programming and column generation. Eser Esirgen

Minimizing costs for transport buyers using integer programming and column generation. Eser Esirgen MASTER STHESIS Minimizing costs for transport buyers using integer programming and column generation Eser Esirgen DepartmentofMathematicalSciences CHALMERS UNIVERSITY OF TECHNOLOGY UNIVERSITY OF GOTHENBURG

More information

4.1 Learning algorithms for neural networks

4.1 Learning algorithms for neural networks 4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

More information

Chapter 6: Sensitivity Analysis

Chapter 6: Sensitivity Analysis Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production

More information

Adaptive Linear Programming Decoding

Adaptive Linear Programming Decoding Adaptive Linear Programming Decoding Mohammad H. Taghavi and Paul H. Siegel ECE Department, University of California, San Diego Email: (mtaghavi, psiegel)@ucsd.edu ISIT 2006, Seattle, USA, July 9 14, 2006

More information

Optimization Methods in Finance

Optimization Methods in Finance Optimization Methods in Finance Gerard Cornuejols Reha Tütüncü Carnegie Mellon University, Pittsburgh, PA 15213 USA January 2006 2 Foreword Optimization models play an increasingly important role in financial

More information

Math 312 Homework 1 Solutions

Math 312 Homework 1 Solutions Math 31 Homework 1 Solutions Last modified: July 15, 01 This homework is due on Thursday, July 1th, 01 at 1:10pm Please turn it in during class, or in my mailbox in the main math office (next to 4W1) Please

More information

9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1

9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 Seffi Naor Computer Science Dept. Technion Haifa, Israel Introduction

More information

Network Traffic Modelling

Network Traffic Modelling University of York Dissertation submitted for the MSc in Mathematics with Modern Applications, Department of Mathematics, University of York, UK. August 009 Network Traffic Modelling Author: David Slade

More information

Massive Data Classification via Unconstrained Support Vector Machines

Massive Data Classification via Unconstrained Support Vector Machines Massive Data Classification via Unconstrained Support Vector Machines Olvi L. Mangasarian and Michael E. Thompson Computer Sciences Department University of Wisconsin 1210 West Dayton Street Madison, WI

More information

Optimization Theory for Large Systems

Optimization Theory for Large Systems Optimization Theory for Large Systems LEON S. LASDON CASE WESTERN RESERVE UNIVERSITY THE MACMILLAN COMPANY COLLIER-MACMILLAN LIMITED, LONDON Contents 1. Linear and Nonlinear Programming 1 1.1 Unconstrained

More information

17.3.1 Follow the Perturbed Leader

17.3.1 Follow the Perturbed Leader CS787: Advanced Algorithms Topic: Online Learning Presenters: David He, Chris Hopman 17.3.1 Follow the Perturbed Leader 17.3.1.1 Prediction Problem Recall the prediction problem that we discussed in class.

More information

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

More information

Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen

Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen (für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained

More information

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

More information

(Quasi-)Newton methods

(Quasi-)Newton methods (Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting

More information

Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

More information

1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES 1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

More information

The Equivalence of Linear Programs and Zero-Sum Games

The Equivalence of Linear Programs and Zero-Sum Games The Equivalence of Linear Programs and Zero-Sum Games Ilan Adler, IEOR Dep, UC Berkeley adler@ieor.berkeley.edu Abstract In 1951, Dantzig showed the equivalence of linear programming problems and two-person

More information

INTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models

INTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models Integer Programming INTEGER PROGRAMMING In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is

More information

Online Learning and Competitive Analysis: a Unified Approach

Online Learning and Competitive Analysis: a Unified Approach Online Learning and Competitive Analysis: a Unified Approach Shahar Chen Online Learning and Competitive Analysis: a Unified Approach Research Thesis Submitted in partial fulfillment of the requirements

More information

Permutation Betting Markets: Singleton Betting with Extra Information

Permutation Betting Markets: Singleton Betting with Extra Information Permutation Betting Markets: Singleton Betting with Extra Information Mohammad Ghodsi Sharif University of Technology ghodsi@sharif.edu Hamid Mahini Sharif University of Technology mahini@ce.sharif.edu

More information

COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES

COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES Appeared as Chapter 45, Handbook of Game Theory with Economic Applications, Vol. 3 (2002), eds. R. J. Aumann and S. Hart, Elsevier, Amsterdam, pages 1723 1759.

More information

Manipulability of the Price Mechanism for Data Centers

Manipulability of the Price Mechanism for Data Centers Manipulability of the Price Mechanism for Data Centers Greg Bodwin 1, Eric Friedman 2,3,4, and Scott Shenker 3,4 1 Department of Computer Science, Tufts University, Medford, Massachusetts 02155 2 School

More information

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems Danilo Ardagna 1, Barbara Panicucci 1, Mauro Passacantando 2 1 Politecnico di Milano,, Italy 2 Università di Pisa, Dipartimento

More information

Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year.

Algebra Unpacked Content For the new Common Core standards that will be effective in all North Carolina schools in the 2012-13 school year. This document is designed to help North Carolina educators teach the Common Core (Standard Course of Study). NCDPI staff are continually updating and improving these tools to better serve teachers. Algebra

More information

DSL Spectrum Management

DSL Spectrum Management DSL Spectrum Management Dr. Jianwei Huang Department of Electrical Engineering Princeton University Guest Lecture of ELE539A March 2007 Jianwei Huang (Princeton) DSL Spectrum Management March 2007 1 /

More information

(a) We have x = 3 + 2t, y = 2 t, z = 6 so solving for t we get the symmetric equations. x 3 2. = 2 y, z = 6. t 2 2t + 1 = 0,

(a) We have x = 3 + 2t, y = 2 t, z = 6 so solving for t we get the symmetric equations. x 3 2. = 2 y, z = 6. t 2 2t + 1 = 0, Name: Solutions to Practice Final. Consider the line r(t) = 3 + t, t, 6. (a) Find symmetric equations for this line. (b) Find the point where the first line r(t) intersects the surface z = x + y. (a) We

More information

Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems. Matrix Factorization Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

More information

Perron vector Optimization applied to search engines

Perron vector Optimization applied to search engines Perron vector Optimization applied to search engines Olivier Fercoq INRIA Saclay and CMAP Ecole Polytechnique May 18, 2011 Web page ranking The core of search engines Semantic rankings (keywords) Hyperlink

More information

Sensitivity analysis of utility based prices and risk-tolerance wealth processes

Sensitivity analysis of utility based prices and risk-tolerance wealth processes Sensitivity analysis of utility based prices and risk-tolerance wealth processes Dmitry Kramkov, Carnegie Mellon University Based on a paper with Mihai Sirbu from Columbia University Math Finance Seminar,

More information

SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH

SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH 31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

The Open University s repository of research publications and other research outputs

The Open University s repository of research publications and other research outputs Open Research Online The Open University s repository of research publications and other research outputs The degree-diameter problem for circulant graphs of degree 8 and 9 Journal Article How to cite:

More information

Convex analysis and profit/cost/support functions

Convex analysis and profit/cost/support functions CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m

More information

Recall that two vectors in are perpendicular or orthogonal provided that their dot

Recall that two vectors in are perpendicular or orthogonal provided that their dot Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

More information

Variational approach to restore point-like and curve-like singularities in imaging

Variational approach to restore point-like and curve-like singularities in imaging Variational approach to restore point-like and curve-like singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure Blanc-Féraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012

More information

Module1. x 1000. y 800.

Module1. x 1000. y 800. Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

More information

RESULTANT AND DISCRIMINANT OF POLYNOMIALS

RESULTANT AND DISCRIMINANT OF POLYNOMIALS RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results

More information

A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems

A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems A Lagrangian-DNN Relaxation: a Fast Method for Computing Tight Lower Bounds for a Class of Quadratic Optimization Problems Sunyoung Kim, Masakazu Kojima and Kim-Chuan Toh October 2013 Abstract. We propose

More information

Lecture 11: Sponsored search

Lecture 11: Sponsored search Computational Learning Theory Spring Semester, 2009/10 Lecture 11: Sponsored search Lecturer: Yishay Mansour Scribe: Ben Pere, Jonathan Heimann, Alon Levin 11.1 Sponsored Search 11.1.1 Introduction Search

More information

1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style

1.5. Factorisation. Introduction. Prerequisites. Learning Outcomes. Learning Style Factorisation 1.5 Introduction In Block 4 we showed the way in which brackets were removed from algebraic expressions. Factorisation, which can be considered as the reverse of this process, is dealt with

More information

ON LIMIT LAWS FOR CENTRAL ORDER STATISTICS UNDER POWER NORMALIZATION. E. I. Pancheva, A. Gacovska-Barandovska

ON LIMIT LAWS FOR CENTRAL ORDER STATISTICS UNDER POWER NORMALIZATION. E. I. Pancheva, A. Gacovska-Barandovska Pliska Stud. Math. Bulgar. 22 (2015), STUDIA MATHEMATICA BULGARICA ON LIMIT LAWS FOR CENTRAL ORDER STATISTICS UNDER POWER NORMALIZATION E. I. Pancheva, A. Gacovska-Barandovska Smirnov (1949) derived four

More information

Study Guide 2 Solutions MATH 111

Study Guide 2 Solutions MATH 111 Study Guide 2 Solutions MATH 111 Having read through the sample test, I wanted to warn everyone, that I might consider asking questions involving inequalities, the absolute value function (as in the suggested

More information

Ideal Class Group and Units

Ideal Class Group and Units Chapter 4 Ideal Class Group and Units We are now interested in understanding two aspects of ring of integers of number fields: how principal they are (that is, what is the proportion of principal ideals

More information

Noncommercial Software for Mixed-Integer Linear Programming

Noncommercial Software for Mixed-Integer Linear Programming Noncommercial Software for Mixed-Integer Linear Programming J. T. Linderoth T. K. Ralphs December, 2004. Revised: January, 2005. Abstract We present an overview of noncommercial software tools for the

More information

Creating a More Efficient Course Schedule at WPI Using Linear Optimization

Creating a More Efficient Course Schedule at WPI Using Linear Optimization Project Number: ACH1211 Creating a More Efficient Course Schedule at WPI Using Linear Optimization A Major Qualifying Project Report submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE in partial

More information

A short note on American option prices

A short note on American option prices A short note on American option Filip Lindskog April 27, 2012 1 The set-up An American call option with strike price K written on some stock gives the holder the right to buy a share of the stock (exercise

More information

Scheduling a sequence of tasks with general completion costs

Scheduling a sequence of tasks with general completion costs Scheduling a sequence of tasks with general completion costs Francis Sourd CNRS-LIP6 4, place Jussieu 75252 Paris Cedex 05, France Francis.Sourd@lip6.fr Abstract Scheduling a sequence of tasks in the acceptation

More information

Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology

Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology Solving NP Hard problems in practice lessons from Computer Vision and Computational Biology Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem www.cs.huji.ac.il/ yweiss

More information

Optimal Gateway Selection in Multi-domain Wireless Networks: A Potential Game Perspective

Optimal Gateway Selection in Multi-domain Wireless Networks: A Potential Game Perspective Optimal Gateway Selection in Multi-domain Wireless Networks: A Potential Game Perspective Yang Song, Starsky H.Y. Wong, and Kang-Won Lee Wireless Networking Research Group IBM T. J. Watson Research Center

More information

Similar matrices and Jordan form

Similar matrices and Jordan form Similar matrices and Jordan form We ve nearly covered the entire heart of linear algebra once we ve finished singular value decompositions we ll have seen all the most central topics. A T A is positive

More information

On the representability of the bi-uniform matroid

On the representability of the bi-uniform matroid On the representability of the bi-uniform matroid Simeon Ball, Carles Padró, Zsuzsa Weiner and Chaoping Xing August 3, 2012 Abstract Every bi-uniform matroid is representable over all sufficiently large

More information

Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

More information

Chapter 2: Systems of Linear Equations and Matrices:

Chapter 2: Systems of Linear Equations and Matrices: At the end of the lesson, you should be able to: Chapter 2: Systems of Linear Equations and Matrices: 2.1: Solutions of Linear Systems by the Echelon Method Define linear systems, unique solution, inconsistent,

More information

QoS optimization for an. on-demand transportation system via a fractional linear objective function

QoS optimization for an. on-demand transportation system via a fractional linear objective function QoS optimization for an Load charge ratio on-demand transportation system via a fractional linear objective function Thierry Garaix, University of Avignon (France) Column Generation 2008 QoS optimization

More information

Interior-Point Algorithms for Quadratic Programming

Interior-Point Algorithms for Quadratic Programming Interior-Point Algorithms for Quadratic Programming Thomas Reslow Krüth Kongens Lyngby 2008 IMM-M.Sc-2008-19 Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800

More information