THE CHINESE UNIVERSITY OF HONG KONG Department of Mathematics MATH5240 Optimization and Modelling (Winter 2011) Homework 2

Transcription

1 THE CHINESE UNIVERSITY OF HONG KONG Department of Mathematics MATH524 Optimization and Modelling (Winter 211) Homework 2 Due Date: 11 th March, 213 Answer all the questions. In this homework set you can use a small pocket calculator. 1. Given an 3 3 symmetric matrix Show that [ x1 x 2 ] x 3 A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 ( = A 1 x 1 + a 12 x 2 + a 13 x 3 a 11 a 11 a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 x 1 x 2 x 3 ) 2 + A ( 2 A 1. x 2 + a ) 2 11a 23 a 12 a 13 x 3 + A 3 A 2 A 2 (x 3) 2, where A i, i = 1, 2, 3 are the determinant of leading principal minors of the matrix A. 2. If S and T are any two sets, the Cartesian product S T of S and T is defined by S T = {(s, t) s S, t T }, as illustrated in Figure 1 for the case when S and T are intervals of the real line. t T S x T S s Figure 1: Graph of Question 2. Show that if S and T are convex sets in R n and R m, respectively, then S T is also convex (in R n+m ).

2 2 3. If f 1, f 2,, f m are functions defined on a convex set in S in R n. Show that if f 1, f 2,, f m are convex and a 1, a 2,, a m, then a 1 f 1 + a 2 f a m f m convex. 4. Examine the convexity/concavity of the following functions: (a) z = x + y e x e x+y (b) z = e x+y + e x y 1 2 y Justify your answers. 5. Let f(x) be a convex function defined on an interval I. Show that if x 1, x 2,, x N I and λ 1, λ 2,, λ N with λ i = 1, then ( N ) f λ i x i λ i f(x i ). (Hint: mathematical induction) Remark: If f(x) is a convex function and X {x i 1,, N} is a random variable with probabilities P (x i ) where P (x i ) = 1, then f (E{x}) E{f(x)} or ( N ) f x i P (x i ) f(x i )P (x i ). 6. Consider the function f(x) = x 1 (x 2 ) 2 [ ] x1 where X = R 2. If possible, determine whether this function is quasiconcave x 2 for x 1 > and x 2 >. If not possible, say why not. 7. Show that the function U is quasiconcave and the function g is increasing. Show that the function f defined by f(x) = g(u(x)) is quasiconcave.

3 3 8. Answer the following questions: (a) Find the linear approximation at (, ) for i. f(x, y) = e xy ii. f(x, y) = ln(1 + x + 2y) (b) Find the quadratic approximation at (, ) for i. f(x, y) = e x+y (xy 1) ii. f(x, y) = ln(1 + x 2 + y 2 ) 9. Use the result of In-Class Exercise 4, Question 8 to show that f(x, y) = 1 x 2 y 2 defined in R 2 is concave. 1. (a) Find the stationary point(s) of each function: i. f(x) = 2x 3 + y 3 3x 2 12x 3y. ii. f(x) = xye ( x2 y 2). (b) Use (a), determine whether the stationary points X for each function f(x) represent maxima, minima, or saddle. 11. For each value of the scalar α, find the set of all stationary points {X f(x) = } of the following function of the two variables x and y f(x, y) = x 2 + y 2 + αxy + x + 2y. Which of these stationary points are global minima? 12. Answer the following questions: (a) Locate/identify all extrema of the following function, if any, where X = (b) Given that [ x1 x 2 f(x) = 5(x 1 3) 2 12(x 2 + 5) 2 + 6x 1 x 2, ] R 2. x 1 1 x 2 5 find the global extrema of above function.

4 4 13. (Bonus) Let f : R n R be a differentiable function. Suppose that a point x is a local minimum of f along every line that passes through x ; that is, the function is minimized at β = for all d R n (a) Show that f(x ) =. g(β) = f(x + βd) (b) Show by example that x need not to be a local minimum of f. Hint: Consider the function of two variables f(x, y) = (y px 2 )(y qx 2 ) where < p < q; that passes through (, ), as show in Figure Figure 2: Graph of f(x, y) = (y px 2 )(y qx 2 ), with p = 1 and q = 4. The origin is a local minimum with respect to every line that passes through it, but is not a local minimum of f. Show that (, ) is a local minimum of f along every line that passes through (, ). Furthermore, if p < m < q, then f(x, mx 2 ) < if x while f(, ) =.

5 5 14. (Bonus) (a) Suppose you have a data set consisting of n observations on three variables y, x 1, x 2 ; the ith observation is denoted (y i, x 1i, x 2i ). You wish to find a linear function of the form y = b 1 x 1 + b 2 x 2 which fits the data as well as possible in the following sense: b 1 and b 2 are chosen so as to minimize the expression Q(b 1, b 2 ) = n (y i b 1 x 1i b 2 x 2i ) 2. Let y R n be the n vector whose ith component is y i, K R n 2 be the n 2 matrix whose ith row is (x 1i, x 2i ). Assume [ that ] the columns of K are linearly b1 independent. Let b be the 2-vector b = R 2. Answer the following questions: i. Show that b 2 Q(b) = (y Kb) T (y Kb). ii. Show that b is a 2 vector such that K T (y Kb ) =. (1) iii. Since the columns of K are linearly independent, the symmetric 2 2 matrix K T K is positive definite and therefore invertible. Deduce that there is only vector b satisfies Eq (1), find an explicit expression for b. iv. Show that Q(b) is minimized when b = b. (b) The Linear Least Squares Regression is the line that minimizes the sum of the square of the errors between the y component of the fitted line and the y component of the data points: E(m, b) = n (y i (mx i + b)) 2. Answer the following questions: i. Verify that and b = are a stationary point of E(m, b). m = n n x iy i n x n i y i n n x2 i ( n x i) 2 n n x2 i y i n x n i x iy i n n x2 i ( n x i) 2

6 6 ii. Show that (x 1 + x x n ) 2 = x x x 2 n + i<j 2x i x j x x x 2 n + (n 1)(x x x 2 n) = n(x x x 2 n). iii. Use (ii) to conclude that (m, b ) is a global minimizer of the function E(m, b).