1 Lecture 0: Convexity and Optimization We say that if f is a once continuously differentiable function on an interval I, and x is a point in the interior of I that x is a critical point of f if f (x) = 0. Critical points of once continuously differentiable functions are important because they are the only points that can be local maxima or minima. In the context of critical points, the second derivative of a function f is important because it helps in determining whether a critical point is a local maximum of minimum. First derivative test Let f be a once continuously differentiable function on an interval I and let x be a critical point. Suppose there is some open interval (a, b) containing x so that for y (a, b) with y < x, we have f (x) > 0 and so that for y (a, b) with y > x, we have f (x) < 0, then f has a local maximum at x. If on the other hand, we have f (y) < 0 when y (a, b) and y < x and f (y) < 0 when y (a, b) and y > x then f has a local minimum at x. Proof of First derivative test We prove the local maximum case. Suppose y (a, b) with y x and f(y) f(x). If y < x, by the mean value theorem, there is some c with y < c < x so that f(x) f(y) = f (c). x y We know, by assumption, that f (c) > 0. Thus f(x) > f(y) which is a contradiction. Suppose instead that x < y. Then there is c with x < c < y so that f(x) f(y) x y = f (c), and by assumption f (c) < 0. Then (since x y is now negative), we still get f(x) > f(y), a contradiction. We can treat the minimum case similarly. Note that under these assumptions, we actually have that x is the unique global maximum (or minimum) on the interval [a, b]. Intuitively, this says we get a maximum if f is increasing as we approach x from the left and decreasing as we leave x to the right. From the First derivative test, we easily obtain: Second derivative test Let f be a once continuously differentiable function. Let x be a critical point for f. If f is twice differentiable at x and f (x) < 0 then f has a local maximum at x. If f is twice differentiable at x and f (x) > 0 then f has a local minimum at x.
2 Proof of the second derivative test It suffices to treat the maximum case as the minimum case proceeds similarly. By the definition of the derivative, we have f (y) = f (x) + f (x)(y x) + o(y x). Since f (x) < 0, there must be a small interval around x, so that to the left of x, we have f positive and to the right, it is negative. We apply the first derivative test. All of this is closely related to the notion of convexity and concavity. Definition A function f(x) is concave if it lies above all its secants. Precisely f is concave if for any a, b, x with x (a, b), we have f(x) b x We say f is strictly concave if under the same conditions f(x) > b x Similarly f is convex if it lies below all its secants. Precisely f is convex if for any a, b, x with x (a, b), we have f(x) b x We say f is strictly convex if under the same conditions f(x) < b x Theorem Let f be twice continuously differentiable. Then f is concave if and only if for every x, we have f (x) 0, and convex if and only if for every x, we have f (x) 0. We will leave the proof of the theorem as an exercise but indicate briefly why this is true locally. If f (x) 0, we have f(y) = f(x) + f (x)(y x) + f (x) (y x) + o((y x) ). We observe that for a, b close to x, if f (x) < 0, we have that (a, f(a)) and (b, f(b)) are below the tangent line to the graph of f at x. Thus the point (x, f(x)) which is on the tangent line is above the secant between (a, f(a)) and (b, f(b)). Concavity has a lot to do with optimization.
3 Example: Resource allocation problems Let f and g be two functions which are continuous on [0, ) and twice continuously differentiable with strictly negative second derivative on (0, ). Let t be a fixed number. Consider W (x) = f(x) + g(t x). [Interpretation: You have t units of a resource and must allocate them between two productive uses. The function of W might represent the value of allocating x units to the first use and t x units to the second use. The concavity represents the fact that each use has diminishing returns.] Under these assumptions, if W (x) has a critical point in (0, t), then the critical point is a maximum. and If, in addition, lim f (x) =, x 0 lim x 0 g (x) =, then we are guaranteed that W (x) has a unique maximum in [0, t]. This is because and lim W (x) =, x 0 lim W (x) =. x t By the intermediate value theorem, there is a zero for W (x) in (0, t). It is unique since W (x) is strictly decreasing. Strictly concave functions on (0, ) whose derivative converge to at 0 are ubiquitous in economics. We give an example. Cobb-Douglas Production function: The Cobb-Douglas production function gives the output of an economy as a function of its inputs (labor and capital). P (K, L) = ck α L α. Here c is a positive constant and α a real number between 0 and. The powers of K and L in the function have been chosen so that P (tk, tl) = tp (K, L). 3
4 That is, if we multiply both the capital and the labor of the economy by t, then we multiply the output by t. Note that if we hold L constant and view P (K, L) as a function of K, then we see this function is defined on (0, ) is strictly concave with derivative going to at 0. An important principle of economics is that we should pay for capital at the rate of the marginal product of capital. We find this rate by taking the derivative in K and getting αck α L. Since we need K units of capital to get the economy to function we pay αck α L α. In this way, we see that α represents the share of the economy that is paid to the holders of capital and α is the share paid to the providers of labor. Another way in which optimization can be applied is to prove inequalities. Arithmetic Geometric mean inequality Let a and b be positive numbers then a b (a + b). This can be proved in a purely algebraic way. Algebraic proof of arithmetic geometric mean inequality a + b a b = (a b ). while Analytic proof of arithmetic geometric mean inequality It suffices to prove the inequality when a + b =. This is because so we just pick t = a+b. Thus what we need to prove is (ta) (tb) = ta b, (ta + tb) = t(a + b), x, when 0 < x <. We let and calculate f (x) = f(x) = x, x x. 4
5 f x x (x) = x. 4( x) 3 4x 3 All terms in the last line are negative so f is strictly concave. The unique critical point is at x =, where equality holds. We have shown that x, since is the maximum. The analytic proof looks a lot messier than the algebraic one, but it is more powerful. For instance, by the same methods, we get that if α, β > 0 and α + β =, then for a, b > 0. a α b β αa + βb, 5
Notation. CHAPTER 4 Linear Programming 1. Graphing Linear Inequalities x apple y means x is less than or equal to y. x y means x is greater than or equal to y. x < y means x is less than y. x > y means
MUST-HAVE MATH TOOLS FOR GRADUATE STUDY IN ECONOMICS William Neilson Department of Economics University of Tennessee Knoxville September 29 28-9 by William Neilson web.utk.edu/~wneilson/mathbook.pdf Acknowledgments
CHPTER 12 Functions You know from calculus that functions play a fundamental role in mathematics. You likely view a function as a kind of formula that describes a relationship between two (or more) quantities.
Mathematics Learning Centre Introduction to Differential Calculus Christopher Thomas c 1997 University of Sydney Acknowledgements Some parts of this booklet appeared in a similar form in the booklet Review
WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT? introduction Many students seem to have trouble with the notion of a mathematical proof. People that come to a course like Math 216, who certainly
1 Base Arithmetic 1.1 Binary Numbers We normally work with numbers in base 10. In this section we consider numbers in base 2, often called binary numbers. In base 10 we use the digits 0, 1, 2, 3, 4, 5,
WHICH SCORING RULE MAXIMIZES CONDORCET EFFICIENCY? DAVIDE P. CERVONE, WILLIAM V. GEHRLEIN, AND WILLIAM S. ZWICKER Abstract. Consider an election in which each of the n voters casts a vote consisting of
Solutions to Homework 6 Mathematics 503 Foundations of Mathematics Spring 2014 3.4: 1. If m is any integer, then m(m + 1) = m 2 + m is the product of m and its successor. That it to say, m 2 + m is the
Chapter 23 Squares Modulo p Revised Version of Chapter 23 We learned long ago how to solve linear congruences ax c (mod m) (see Chapter 8). It s now time to take the plunge and move on to quadratic equations.
Problem 3 If A is divided by B the result is 2/3. If B is divided by C the result is 4/7. What is the result if A is divided by C? Suggested Questions to ask students about Problem 3 The key to this question
7 The Backpropagation Algorithm 7. Learning as gradient descent We saw in the last chapter that multilayered networks are capable of computing a wider range of Boolean functions than networks with a single
A SELF-GUIDE TO O-MINIMALITY CAMERINO TUTORIAL JUNE 2007 Y. PETERZIL, U. OF HAIFA 1. How to read these notes? These notes were written for the tutorial in the Camerino Modnet Summer school. The main source
A Modern Course on Curves and Surfaces Richard S. Palais Contents Lecture 1. Introduction 1 Lecture 2. What is Geometry 4 Lecture 3. Geometry of Inner-Product Spaces 7 Lecture 4. Linear Maps and the Euclidean
How many numbers there are? RADEK HONZIK Radek Honzik: Charles University, Department of Logic, Celetná 20, Praha 1, 116 42, Czech Republic firstname.lastname@example.org Contents 1 What are numbers 2 1.1 Natural
Chapter 2 Switching Algebra and Logic Gates The word algebra in the title of this chapter should alert you that more mathematics is coming. No doubt, some of you are itching to get on with digital design
ONE-DIMENSIONAL RANDOM WALKS 1. SIMPLE RANDOM WALK Definition 1. A random walk on the integers with step distribution F and initial state x is a sequence S n of random variables whose increments are independent,
Doug Ravenel University of Rochester October 15, 2008 s about Euclid s Some s about primes that every mathematician should know (Euclid, 300 BC) There are infinitely numbers. is very elementary, and we
Controllability and Observability of Partial Differential Equations: Some results and open problems Enrique ZUAZUA Departamento de Matemáticas Universidad Autónoma 2849 Madrid. Spain. email@example.com
Chapter 4 Approximating functions by Taylor Polynomials. 4.1 Linear Approximations We have already seen how to approximate a function using its tangent line. This was the key idea in Euler s method. If
Is Piketty s Second Law of Capitalism Fundamental? Per Krusell Institute for International Economic Studies, CEPR, and NBER Anthony A. Smith, Jr. 1 Yale University and NBER April 27, 2015 (first version:
Instructions. Answer each of the questions on your own paper, and be sure to show your work so that partial credit can be adequately assessed. Credit will not be given for answers (even correct ones) without
Truthful Mechanisms for One-Parameter Agents Aaron Archer Éva Tardos y Abstract In this paper, we show how to design truthful (dominant strategy) mechanisms for several combinatorial problems where each
SCILAB IS NOT NAIVE This document has been written by Michaël Baudin from the Scilab Consortium. December 2010 The Scilab Consortium Digiteo. All rights reserved. December 2010 Abstract Most of the time,
Matthias Beck Gerald Marchesi Dennis Pixton Lucas Sabalka Version.5 Matthias Beck A First Course in Complex Analysis Version.5 Gerald Marchesi Department of Mathematics Department of Mathematical Sciences
Orthogonal Bases and the QR Algorithm Orthogonal Bases by Peter J Olver University of Minnesota Throughout, we work in the Euclidean vector space V = R n, the space of column vectors with n real entries
2 Discrete-Time Signals and Systems 2.0 INTRODUCTION The term signal is generally applied to something that conveys information. Signals may, for example, convey information about the state or behavior