Gianni Bosi and Magalì E. Zuanon. Basic Optimization and Financial Mathematics

Transcription

1 Gianni Bosi and Magalì E. Zuanon Basic Optimization and Financial Mathematics

2

3 i Preface It isn t that they can t see the solution. They can t see the problem. G.K. CHESTERTON The Scandal of Father Brown The point of a pin" This booklet contains the notes for a short course in Optimization and Financial Mathematics. It is far from being exhaustive but nevertheless it is entirely self-contained, following a classical course of calculus. The main goal of these notes is to provide students with the essential information about these topics in a primarily theoretical framework. The first part (Chapters 1, 2 and 3) illustrates the basic tools of optimization in several variables, passing through essential linear algebra and quadratic forms. The second part (Chapters 4, 5, 6 and 7) contains Theory of Interest as a mathematical problem topic, which is rather unlike what is done in typical finance courses. In this part we consider the deterministic approach only. In the last Chapter we introduce some arguments of mathematics of life insurance, the understanding of the basic principles of this part being a solid foundation for further study of the theory in a more general setting. The authors apologize in advance for any typos and mistakes. They are grateful to the readers who will be so kind to point out them. Trieste, March 21st, 2012 Gianni BOSI Dipartimento di Scienze Economiche, Aziendali, Matematiche e Statistiche, Università di Trieste, Piazzale Europa 1, Trieste, Italy. giannibo@econ.units.it Magalì E. ZUANON Dipartimento di Metodi Quantitativi, Università degli Studi di Brescia, Contrada Santa Chiara 50, Brescia, Italy. zuanon@eco.unibs.it

4 Contents pag. 1 Some concepts of matrix algebra Basic definitions Determinants The algebraic structure of R n The topological structure of R n Real functions of n real variables General concepts Quadratic forms Differentiability and optimization Convex and concave functions Constrained optimization with equality constraints Exercises The classical financial regimes General concepts Regime of the linear interest Regime of the commercial discount Regime of the compound interest or exponential regime Exercises The regime of the compound interest The final value under the regime of the compound interest Equivalence of interest rates Force of Interest: Continuous compounding Axiomatization of the regime of the compound interest Exercises Annuities and perpetuities Annual payments Non-annual payments Amortization and capital construction by constant instalments 65

5 2 Contents 5.4 Exercises Amortization of a debt General concepts Amortization by constant annual principal shares Amortization by constant instalments Negotiation of a debt: Makeham formula Exercises Actuarial mathematics Probability and Lifetimes Expected Present Values of Insurance Contracts Exercises Solutions to selected exercises 87 Bibliography 97

6 CHAPTER 1 Some concepts of matrix algebra 1.1 Basic definitions In this section we first present the basic concepts concerning matrices. Then we introduce the operations of sum of two matrices, product of a matrix and a real number and finally product of two matrices. Definition (Real Matrix). An m n real matrix (m and n are positive integers) is a collection a a 1n a a 2n A = [a ij ] 1 i m,1 j n =..... a m1... a mn of real numbers a ij (1 i m, j j n) double ordered by row and by column. So, the matrix A = [a ij ] 1 i m,1 j n has m rows and n columns and a ij is precisely the element that belongs both to the i-th row and to the j-th column. In the sequel a real matrix will be simply referred to as a matrix for the sake of simplicity. Example Consider the following matrix ( ) A = Then A is a 2 4 matrix. For example, we have that a 13 = 3 and a 22 = 5.

7 4 Chapter 1. Some concepts of matrix algebra We now present the basic definitions of a square matrix and an identity matrix. Definition (Square Matrix). An n n matrix A = [a ij ] 1 i n,j j n is said to be a square matrix of order n (in this case there are n rows and n columns). In the particular case when n = 1 (there is only one column), we have to do with a column vector a of R m, that is a = a 1 a 2.. a m. Definition (Identity matrix). The identity matrix of order n is the square matrix I n of order n that is defined as follows: I n = so that the elements on the main diagonal of A (i.e. the elements a ii with 1 i n) are all equal to 1 while all the other elements are equal to 0. So, for example, we have I 2 = ( ), I 3 = ,, I 4 = It is possible to define the operations of sum of two matrices and product of a matrix and a real number. Definition (Sum of two matrices). Let two m n matrices A = [a ij ] 1 i m,1 j n and B = [b ij ] 1 i m,1 j n be given. Then the sum C = A+B of the matrices A and B is the m n matrix C = [c ij ] 1 i m,1 j n whose generic element c ij is the sum of the corresponding elements a ij and b ij of A and B, respectively (that is, c ij = a ij + b ij with 1 i m, 1 j n)..

8 1.1. Basic definitions 5 Example Consider the following 2 4 matrices ( ) ( A =, B = ). We have that ( ) A + B = = ( Definition (Product of a matrix and a real number). Let λ be a real number and consider an m n matrix A = [a ij ] 1 i m,1 j n. Then the product λa of λ and A is the m n matrix λa = [λa ij ] 1 i m,1 j n. The following example concerns a linear combination of two matrices. Both the sum and the product by a real number are involved. ). Example Consider the following 2 4 matrices ( ) ( A =, B = ). We want to determine the 2 4 matrix defined as 3A 2B. We have that ( ) ( ) A 2B = 3 2 = ( ) ( ) = + = ( ) = Definition (Product of two matrices). Let A = [a ij ] 1 i m,1 j n be an m n matrix and let B = [b ij ] 1 i n,1 j r be an n r matrix (so A has n columns and B has n rows). Then we define the product row by column of A and B as the m r matrix AB = C = [c ij ] 1 i m,1 j r with m rows and r columns whose generic element c ij is obtained by multiplying the i-th row a i of the matrix A and the j-th column b j of the matrix B, that is n c ij = a ih b hj (i = 1,..., m, j = 1,..., r). h=1 Remark The product of two matrices A and B is not defined whenever the number of columns of A is different from the number of rows of B. We present an example concerning the product of two matrices.

9 6 Chapter 1. Some concepts of matrix algebra Example Consider the following matrices ( ) ( A =, B = ). Notice that BA is not defined since B has 3 columns and A has 2 rows. We have that ( ) ( ) AB = = ( ) ( 2) 1 ( 4) = = ( 1) ( 1) ( 2) 2 ( 4) + ( 1) 6 ( ) = Remark It should be noted that for every square matrix A of order n we have that AI n = I n A = A. Remark The product of two matrices is not commutative. Even if for two matrices A and B both the products AB and BA are defined, it is not true in general that AB = BA. In order to illustrate this fact, consider the following two square matrices of order 2: A = ( ), B = ( ). We have that AB = ( ), BA = ( ). 1.2 Determinants We present the definition and the basic properties of the determinants. We consider determinants as tools in order to establish sufficient conditions for the existence of points of (relative) maximum and minimum. This will be done in the following chapter. In practice, we shall limit ourselves to the consideration of determinants of square matrices of order 2 or 3. Nevertheless, the definition concerns the most general case of a square matrix of arbitrary dimension.

10 1.2. Determinants 7 Definition (Determinant). Let A be a square matrix of order n, A = [a ij ] 1 i n,j j n. Then the determinant of A is the real number det(a) = a a 1n a a 2n a n a nn = ( 1) d(π) a 1π(1) a nπ(n), π where the previous summation is over all permutations (bijective mappings) π : {1,..., n} {1,..., n} and d(π) is the degree of the permutation π (the number of exchanges needed to give the fundamental permutation (1, 2,..., n) starting from the permutation (π(1),..., π(n))). In the sequel we shall consider in particular determinants of square matrices of order 2 and of order 3. Therefore, we need a fast procedure in order to calculate the determinants in these cases. Remark (Determinant of a 2 2 matrix). Given a square matrix of order 2 ( ) a11 a A = 12, a 21 a 22 we have that det(a) = a 11 a 22 a 12 a 21 since the permutation {2, 1} requires one exchange. Remark (Determinant of a 3 3 matrix). Given a square matrix of order 3 we have that A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33, det(a) = a 11 a 22 a 33 + a 12 a 23 a 31 + a 13 a 21 a 32 a 13 a 22 a 31 a 11 a 23 a 32 a 12 a 21 a 33 since the permutations {1, 2, 3}, {2, 3, 1} and {3, 1, 2} require an even number of exchanges while the permutations {3, 2, 1}, {1, 3, 2} and {2, 1, 3} require an odd number of exchanges. There is a fast rule which allows us to immediately arrive at the determinant of a 3 3 matrix. It is called the Sarrus rule".

11 8 Chapter 1. Some concepts of matrix algebra Remark (Sarrus rule). Given a square matrix of order 3 A = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 write the matrix A and the first two columns of A subsequently on the right side, that is a 11 a 12 a 13 a 11 a 12 a 21 a 22 a 23 a 21 a 22 a 31 a 32 a 33 a 31 a 32. Then consider the sum of the products of the elements of the 3 diagonals starting above to the left and ending below to the right (i.e., a 11 a 22 a 33, a 12 a 23 a 31, a 13 a 21 a 32 ) and subtract the sum of the products of the elements of the 3 diagonals starting below to the right and ending above to the left (i.e., a 13 a 22 a 31, a 11 a 23 a 32, a 12 a 21 a 33 ). The resulting number is precisely the determinant of A (i.e., det(a) = a 11 a 22 a 33 +a 12 a 23 a 31 +a 13 a 21 a 32 a 13 a 22 a 31 a 11 a 23 a 32 a 12 a 21 a 33 ). Example (Determinant of some 3 3 matrix). If we consider the square matrix of order 3 A = then we have that det(a) = ( 2) ( 2) + ( 4) 1 3 ( 2) 0 ( 4) 3 ( 2) = = 8. Remark (Determinant of the product λa). It is easily seen that the following property of the determinants holds true :,, (i) for every real number λ and for every square matrix A of order n we have that det (λa) = λ n deta. Definition (Transpose matrix). Let A = [a ij ] 1 i m,1 j n be an m n matrix. Then the transpose of A is the n m matrix A = [a ij ] 1 i n,1 j m whose generic element a ij is defined as follows a ij = a ji 1 i n, 1 j m. It should be noted that the the transpose A of any matrix A is obtained by writing the rows of A as the columns of A or equivalently by writing the columns of A as the rows of A.

12 1.3. The algebraic structure of R n 9 Example Consider the 2 4 matrix ( ) A = Then its transpose A is defined as follows: A = Remark (Determinant of the transpose of a matrix). From the definition of the determinant it is immediate to see that for every square matrix A the determinant of A is equal to the determinant of A (i.e., det(a) = det(a )). Definition (Symmetric matrix). If A = [a ij ] 1 i n,1 j n is a square matrix of order n and we have that A = A, then A is said to be a symmetric matrix. Example Consider the following matrices of order 3: A = 1 0 4, B = We have that A is symmetric while B is not symmetric. 1.3 The algebraic structure of R n In this section we present the basic properties of the space R n with respect to the operations of sum of two elements and product of an element and a real number. As usual, we shall denote by R the set of all real numbers and by R n (n N + ) the n-dimensional Euclidean space consisting of all n-tuples x = (x 1, x 2,..., x n ) of real numbers. We consider on R n the following operations: sum of two elements of R n (+ : R n R n R n ) (x 1, x 2,..., x n ) + (y 1, y 2,..., y n ) = (x 1 + y 1, x 2 + y 2,..., x n + y n );

13 10 Chapter 1. Some concepts of matrix algebra product of a real number and an element of R n ( : R R n R n ) α (x 1, x 2,..., x n ) = (αx 1, αx 2,..., αx n ) The algebraic structure (R n, +, ) (consisting of the underlying space R n together with the operations + and ) is said to be a real vector space. Therefore, we can refer to x R n as a vector of R n. We present the properties according to which (R n, +, ) is a real vector space. Just notice that the definition of (real) vector space is much more general, but goes beyond our purposes. Definition (Vector space). We refer to (R n, +, ) as a real vector space since the following properties are verified: (1) (x + y) + z = x + (y + z) for every x, y, z R n ; (2) There exists a (unique) element 0 = (0,..., 0) such that x+0 = 0+x = x for every x R n ; (3) For every x R n there exists a so called opposite element x = ( x 1,..., x n ) such that x + ( x) = 0; (4) x + y = y + x for every x, y R n ; (5) α(βx) = (αβ)x for every α, β R, for every x R n ; (6) 1x = x for every x R n ; (7) α(x + y) = αx + αy for every α R, for every x, y R n ; (8) (α + β)x = αx + βx for every α, β R, for every x R n. Remark The algebraic structure (R n, +) satisfies the following properties listed above: (1) associative property, (2) existence of a neutral element, (3) existence of an opposite element of any element of R n, (4) commutative property. (R n, +) is said to be a commutative (abelian) group.

14 1.4. The topological structure of R n 11 Definition (Euclidean distance). Given two vectors x = (x 1, x 2,..., x n ) and y = (y 1, y 2,..., y n ) of R n, the Euclidean distance between x and y is the real number defined as follows: d(x, y) = n (x i y i ) 2. i=1 Remark (Properties of the Euclidean distance). The Euclidean distance satisfies the following properties: (0) d(x, y) 0 for every x, y R n ; (1) d(x, y) = 0 if and only if x = y for every x, y R n ; (2) d(x, y) = d(y, x) for every x, y R n (symmetric property); (3) d(x, y) + d(y, z) d(x, z) for every x, y, z R n (triangle inequality). The pair (R n, d) is referred to as a metric space (in the sense that there is a metric (i.e., a distance) d : R n R n R that associates to every pair of vectors of R n a real number in a such a way that the previous properties are verified). Also in this case, the concept of a metric space is much more general, as well as the concept of norm presented in the following definition. Definition (Euclidean norm). The Euclidean norm of any vector x R n is defined to be the real number x = d(x, 0) = n x 2 i. i=1 Remark (Properties of the norm). The (Euclidean) norm satisfies the following properties: (0) x 0 for every x R n ; (1) x = 0 if and only if x = 0 for every x R n ; (2) αx = α x for every α R, for every x R n ; (3) x + y x + y for every x, y R n.

15 12 Chapter 1. Some concepts of matrix algebra 1.4 The topological structure of R n We now present the basic definition of an open ball centered at a point of point of R n. The basic topological concepts will follow. Definition (Open ball centered at a point). Let x 0 be a vector of R n and let r be a positive real number. We denote by B r (x 0 ) the open ball with center at x 0 and radius r, that is defined to be the set of all vectors (points) of R n whose distance from x 0 is smaller than r, that is to say B r (x 0 ) = {x R n : d(x, x 0 ) < r} = x Rn : n (x i x 0 i )2 < r. i= Figure 1.1 Open ball centered at (0, 0) with radius 1.

16 1.4. The topological structure of R n 13 Definition (Accumulation point). A point x 0 R n is a said to be an accumulation point of a set A R n if every open ball with center at x 0 contains infinitely many points of A. Definition (Interior point of a set). A point x 0 R n is said to be an interior point of a set A R n if there exists an open ball centered at x 0 that is contained in A (that is, r R, r > 0 such that B r (x 0 ) A). Definition (Open sets and closed sets). Let A be a subset of R n. Then A is said to be (1) open if every point of A is an interior point of A (that is, for every x 0 A there exists r R, r > 0 such that B r (x 0 ) A); (2) closed if the complement of A, denoted by CA = R n \ A is open (that is, for every x 0 CA there exists r R, r > 0 such that B r (x 0 ) A = ). Example It is easy to check (but here we state without proof) that any open ball B r (x 0 ) is an open set. Further, every closed ball is a closed set. C r (x 0 ) = {x R n : d(x, x 0 ) r} = x Rn : n (x i x 0 i )2 r Remark (properties of the open sets). If we denote by τ the family consisting of all the open subsets of R n, then we have that τ satisfies the following properties: (i) X τ, τ; i=1 (ii) the union of every family of open sets is an open set; (iii) the intersection of any finite family of open sets is an open set. We say that τ is a topology on R n. Definition (Convergent sequence). We say that a sequence {x h } h N + of points of R n converges to a point x 0 R n (that is, x h x 0 ) if for every real number ǫ > 0 there exists a natural number h N + such that d(x h, x 0 ) < ǫ for every h > h (i.e., lim h + d(xh, x 0 ) = 0).

17 14 Chapter 1. Some concepts of matrix algebra Example (Example of a convergent sequence). Consider the sequence of points of R 2 defined as follows: {( 1 h, h )} h N +. We claim that ( 1 h, 1+ 1 h ) (0, 1). Indeed, consider that d(( 1 h, 1+ 1 h ), (0, 1)) = 1 h h = 2 2 h. For any fixed positive real number ǫ we have that 2 2 h < ǫ 2 as soon as h > such that h > 2 ǫ 2 ǫ.. So we only have to consider any natural number h N+ Definition (Bounded set). A subset A of R n is said to be bounded if it is contained in an open ball B r (0) centered at 0 (i.e., if there exists r R, r > 0 such that A B r (0)). Definition (compact set). A subset A of R n is said to be compact if it is closed and bounded.

18 CHAPTER 2 Real functions of n real variables 2.1 General concepts We shall denote by f : (R n )D R a real function of n real variables that is defined on the domain D. So f is a law (o prescription) that associates to every point (vector) x D a well determined real number f(x), that is called the value of f at the point x. From now on (unless a different specification) a vector x R n is thought of as a column vector x 1 x 2 x =... x n Example (Linear functions). A function f : R n R is said to be a linear function if f satisfies the following two conditions; (i) f(x + y) = f(x) + f(y) for all x, y R n ; (ii) f(αx) = αf(x) for all x R n and for all α R. It can be shown that a function f is linear if and only if there exists a (column) vector a R n such that for every x R n n f(x) = a x = (a 1,..., a n ) x = a i x i. i=1

19 16 Chapter 2. Real functions of n real variables Definition (Graph). If f : (R n )D R is a real function of n real variables then the graph of f is the subset of R n+1 that is defined as follows: G f = {(x 1, x 2,..., x n, f(x 1, x 2,..., x n )) : x = (x 1, x 2,..., x n ) D}. It should be noted that in the particular case when n = 2, the graph of a function of two real variables appears as a surface in the three-dimensional space Figure 2.1 Graph of f(x, y) = x 2 + y 2. Definition (Level curve). If f : (R n )D R is a real function of n real variables and c is any real number, then we refer to the level curve (corresponding to c) as the subset D f c of D defined as follows: D f c = {x D : f(x) = c}. Example (f(x, y) = x 2 +y 2 ). Consider the function (positive definite quadratic form) f : R 2 R, f(x, y) = x 2 +y 2. Then for every positive real number c the c-level curve D f c = {x D : x2 + y 2 = c} is a circumference centered at (0, 0) with radius c. We now present, for the sake of completeness, the basic definitions of continuity of a function at a point and finite limit at a point.

20 2.2. Quadratic forms Figure 2.2 Level curves of f(x, y) = x 2 + y 2. Definition Continuity at a point] Let f : (R n )D R be a real function of n real variables and consider a point x 0 D. Then f is said to be continuous at x 0 if f verifies the following condition: for every real number ǫ > 0 there exists a real number δ > 0 such that f(x) f(x 0 ) < ǫ for every x D such that d(x, x 0 ) < δ. Definition (Continuity on a domain). A function f : (R n )D R is said to be continuous (on D) if f is continuous at every point x 0 D. Definition (Finite limit at a point). Let f : (R n )D R be a real function of n real variables and let x 0 be an accumulation point of its domain D. We say that f has finite limit l ( R) as x approaches x 0 ( lim = l) if x x0f(x) for every real number ǫ > 0 there exists a real number δ > 0 such that f(x) l) < ǫ for every x D with 0 < d(x, x 0 ) < δ.

21 18 Chapter 2. Real functions of n real variables Figure 2.3 Level curves of f(x, y) = x 2 + y. 2.2 Quadratic forms In this section we introduce the quadratic forms and study their representations and sign by using the considerations in the previous chapters. Definition (Definition of a quadratic form). A quadratic form q on R n is a function q : R n R of the form n j q(x 1,..., x x ) = a ij x i x j. j=1 i=1

22 2.2. Quadratic forms 19 Definition (Definite and semidefinite quadratic forms). A quadratic form q on R n is said to be (i) positive definite (negative definite) if q(x) > 0 (q(x) < 0) for every x 0 (x R n ); (ii) positive semidefinite (negative semidefinite) if q(x) 0 (q(x) 0) for every x R n ; (iii) indefinite of sign if q(x) > 0 for some x R n and also q(x) < 0 for some x R n. Remark (Associated symmetric matrices). A quadratic form q(x 1,..., x x ) = n j=1 j i=1 a ijx i x j can be represented by means of a symmetric matrix A (of order n) in such a way that q(x) = x Ax for every x R n. To this aim it suffices to consider the following symmetric matrix: A = [a ij ] 1 i n,j j n = 1 a 11 2 a a 1 12 a a 1n 2 a 2n 1 2 a 1n a nn We say that A is the symmetric matrix associated to the quadratic form q. Example (Quadratic form on R 2 ). Consider the quadratic form q on R 2 defined as q(x, y) = x 2 2xy + y 2. If we define x = ( x y then we have that q(x) = x Ax. ), A = ( Example (Quadratic form on R 3 ). Consider the quadratic form on R 3 defined as q(x, y, z) = x 2 + 2y 2 + 3z 2 + 4xy 6xz + 8yz. If we define x = x y z then we have that q(x) = x Ax., A = ) We now introduce the different kinds of definition of a symmetric matrix by using the corresponding notions of definitions of a quadratic form.,.

23 20 Chapter 2. Real functions of n real variables Definition (Positive definite symmetric matrix). The symmetric matrix A of order n is said to be (i) positive definite (negative definite) if the quadratic form q(x) = x Ax is positive definite (negative definite); (ii) positive semidefinite (negative semidefinite) if the quadratic form q(x) = x Ax is positive semidefinite (negative semidefinite);; (iii) indefinite of sign if the quadratic form q(x) = x Ax, q : R n R is indefinite of sign. Example (Matrices definite of sign). (i) The symmetric matrix A = ( ) is indefinite of sign since q(x 1, x 2 ) = x Ax = x 2 1 x2 2 (q(1, 0) = 1 > 0) or negative q(0, 1) = 1 < 0). may be positive (ii) The symmetric matrix A = ( ) is positive definite since q(x 1, x 2 ) = x Ax = x x2 2 all x 0. is greater than zero for (iii) The symmetric matrix A = ( ) is positive semidefinite since q(x 1, x 2 ) = x Ax = x x 1x 2 + x 2 2 nonnegative but it it is equal to zero whenever x 1 = x 2. is always The following concept of principal minor is very useful in order too analyze the definition of a quadratic form.

24 2.2. Quadratic forms 21 Definition (Principal minors). Let A = [a ij ] 1 i n,1 j n be a square matrix of order n. Define, for every 1 h n, A h = [a ij ] 1 i h,1 j h = a a 1h a a 2n a h a hh. The principal minor of order h is defined to be the determinant of A h. We do not prove that following theorem, which contains a characterization of the positive/definition of a symmetric matrix, and therefore of a quadratic form, since we can always refer to the associated symmetric matrix. Theorem (Conditions for positive definitions). Let A be a symmetric matrix of order n. Then we have that: (i) A is positive definite if and only if all the n principal minors are positive; (ii) A is negative definite if and only if all its n principal minors alternate in sign with, in particular, det(a 1 ) < 0, det(a 2 ) > 0, det(a 3 ) < 0,..., and so on; (iii) if at least one principal minor of A is different from 0 but its sign doesn t agree with any of the previous cases, then A is indefinite of sign; (iv) A is positive semidefinite if and only if all its n principal minors are nonnegative and at least one of them is equal to zero; (v) A negative semidefinite if and only if all its n principal minors of odd order are smaller or equal to zero, all its principal minors of even order are greater or equal to zero and at least one of them is equal to zero.

25 22 Chapter 2. Real functions of n real variables Example (Positive definite matrices). (i) The symmetric matrix A = ( is positive definite since det(a 1 ) = 2 > 0 and det(a) = 2 1 = 1 > 0. ) (ii) The symmetric matrix A = is indefinite of sign since det(a 1 ) = 1 > 0 e det(a) = 7 < 0. (iii) The symmetric matrix A = is negative semidefinite since det(a 1 ) = 1 < 0, det(a 2 ) = 0 e det(a) = Differentiability and optimization We introduce the basic concepts related to the differential calculus in n variables. Definition (Partial derivative). Let f : (R n )D R be a real function of n real variables and let x 0 D be an accumulation point of D. If the limit f (x 0 f(x 0 1 ) = lim,..., x0 i 1, x0 i + h, x0 i+1,..., x0 n ) f(x0 ) x i h 0 h exists and it is finite then such a limit is said to be the partial derivative of f with respect to x i at the point x 0.

26 2.3. Differentiability and optimization 23 Definition (Gradient vector). Let f : (R n )D R be a real function of n real variables and let x 0 D be an accumulation point of D. If the partial derivatives f x i (x 0 ) exist for all i {1,..., n}, then we define the gradient vector D 1 f(x 0 ) of f at the point x 0 as follows: D 1 f(x 0 ) = f x 1 (x 0 )... f x n (x 0 ). Theorem (Derivative of compound functions). Let z = f(x 1,..., x n ) be a real function of n real variables such that the partial derivatives f x i (x) exist for every x belonging to the domain of f and for every i {1,..., n}. Consider m real functions of n real variables x i = g i (t 1,..., t m ) (i = 1,..., n) such that the partial derivatives g i t j (t) exist for every t belonging to the common domain of the functions g i. Then, for every j {1,..., m}, we have that z = z x 1 + z x z x n. t j x 1 t j x 2 t j x n t j Example (Derivative of a compound function). Consider the function z = f(x, y) = e x 2y, x = t t 2, y = 2t 2. From Theorem 2.3.3, we have that z = z x + z y = 2t 1 e x 2y = 2t 1 e t2 1 3t 2, dt 1 x t 1 y t 1 z = z dt 2 x x + z t 2 y y t 2 = e x 2y 4e x 2y = 3e t2 1 3t 2. It is clear that, equivalently, we could have considered directly the partial derivatives with respect to t 1 and t 2 of the function f(t 1, t 2 ) = e t2 1 3t 2. Definition (Continuous differentiability). A function f : (R n )D R is said to be continuously differentiable or C 1 on D if the partial derivative f x i (x) exists and it is continuous at every point x D for all i {1,..., n}.

27 24 Chapter 2. Real functions of n real variables Definition (Partial derivatives of higher order). If a function f : (R n )D R is C 1 on D and the partial derivative with respect to x j (j {1,..., n}) of f x i (x) exists for every x D, then we shall write f x j ( ) f (x) = 2 f (x). x i x j x i Further, if 2 f x j x i (x) is continuous at every point x D and for every i, j {1,..., n}, then we shall say that f is C 2 or twice continuously differentiable on D. Example Consider the real function of two real variables defined as Then we have that f x (x, y) = log(x y2 ) + f(x, y) = xlog(x y 2 ). x x y 2, f y (x, y) = 2xy x y 2, 2 f 1 (x, y) = x2 x y 2 y 2 (x y 2 ) 2, 2 f x + y2 (x, y) = 2x y2 (x y 2 ) 2, 2 f x y (x, y) = 2y 3 (x y 2 ) 2 = 2 f (x, y). y x Definition (Hessian matrix). Let f : (R n )D R be a real function of n real variables. If f is C 2 then at every point x 0 D we can define the hessian matrix of f D 2 f(x 0 ) = 2 f x f (x 0 ) 2 f 2 f x n x 1 (x 0 ) x 2 x 1 (x 0 )... 2 f (x 0 ) x x 1 x 2 (x 0 ) f x 1 x n 1 (x 0 ) f x 1 x n (x 0 ).... Hence we have that D 2 f(x 0 ) = [a ij ] 1 i n,1 j n with a ij = 2 f x j x i (x 0 ) (i, j {1,..., n}). 2 f x 2 n (x 0 ).

28 2.3. Differentiability and optimization 25 Remark (symmetric property of the hessian matrix). If f : (R n )D R is C 2 then, according to Schwartz Theorem, we have that for every x D and for every pair (i, j) of indexes the following property is verified: 2 f x j x i (x) = 2 f x i x j (x). Therefore the hessian matrix D 2 f(x) is symmetric for every x D. Definition (points of minimum and maximum). Let f : (R n )D R be a real function of n real variables. Then a point x 0 D is said to be (i) a point of maximum (minimum) if f(x) f(x 0 ) (f(x) f(x 0 )) for every x D; (ii) a point of strict maximum (minimum) if f(x) < f(x 0 ) (f(x) > f(x 0 )) for every x D, x x 0 ; (iii) a point of relative maximum (minimum) if there exists an open ball B r (x 0 ) such that f(x) f(x 0 ) (f(x) f(x 0 )) for every x B r (x 0 ) D; (iv) a point of strict relative maximum (minimum) if there exists an open ball B r (x 0 ) such that f(x) < f(x 0 ) (f(x) > f(x 0 )) for every x B r (x 0 ) D, x x 0. The famous Weierstrass theorem guarantees the existence of a maximum and a minimum of a continuous function on a compact set. We present the statement without the proof. Theorem (Weierstrass Theorem). Let f : (R n )D R be a real function of n real variables. If f is continuous and D is compact, then there exists a point of minimum and a point of maximum for f.. Theorem (Necessary condition for a relative minimum or maximum). If f : (R n )D R is C 1 and x 0 D is a point of relative minimum or maximum that is in addition an interior point of D then it must be f x i (x 0 ) = 0 for i = 1,..., n, or equivalently D 1 f(x 0 ) = 0.

29 26 Chapter 2. Real functions of n real variables Definition (Stationary point). Let f : (R n )D R be a real function of n real variables. We say that x 0 D is a stationary point of f if or equivalently D 1 f(x 0 ) = 0. f x i (x 0 ) = 0 for i = 1,..., n, In order to present sufficient conditions for the existence of relative maxima or minima, we need the Taylor s formula of order 2. Theorem (Taylor s formula of order 2). Let f : (R n )D R be a real function of n real variables which is defined on an open set D. If f is C 2 on D and if x 0 D then there exists a real function of n real variables R 2 (h; x 0 ) such that for every point x 0 + h D such that the segment with extremes x 0 and x 0 + h is contained in D it holds that f(x 0 + h) = f(x 0 ) + [D 1 f(x 0 )] h h D 2 f(x 0 )h + R 2 (h; x 0 ), where the function R 2 (h; x 0 ) is such that R 2 (h; x 0 ) lim h 0 h 2 = 0. We are now ready to prove the following theorem. Theorem (Sufficient condition for relative maximum or minimum). Let f : (R n )D R be a real function of n real variables that is defined on an open set D. If f is C 2 and x 0 D is a stationary point for f then x 0 is (i) a point of strict relative maximum if the hessian matrix D 2 f(x 0 ) is negative definite; (ii) a point of strict relative minimum if the hessian matrix D 2 f(x 0 ) is positive definite. If the hessian matrix D 2 f(x 0 ) is indefinite of sign, then x 0 is neither a point of minimum nor a point of maximum for f. Proof. We shall only prove that statement (ii) holds. The proof of statement (i) is perfectly analogous. Let x 0 D be a stationary point for a real-valued function f which is C 2 on an open set D R n. Assume that the hessian matrix D 2 f(x 0 ) is positive definite. From the Taylor s formula (see Theorem ), since x 0 D is a stationary point of f (hence, D 1 (x 0 ) = 0) we have

30 2.3. Differentiability and optimization 27 that, for every h R n such that the segment with extremes x 0 and x 0 + h is contained in D, f(x 0 + h) = f(x 0 ) h D 2 f(x 0 )h + R 2 (h; x 0 ), and, dividing by h 2, h f(x 0 + h) f(x 0 ) h 2 = 1 2 h D2 f(x 0 h ) h + R 2(h; x 0 ) h 2. The quadratic form q(h) = h D 2 f(x 0 )h is continuous on R n and therefore, from the Weierstrass theorem , attains a minimum value a on the unit circle {v R n : v = 1}, which is a closed and bounded set. On the other hand, since the hessian matrix D 2 f(x 0 ) (and therefore the quadratic form q(h) = h D 2 f(x 0 )h) is positive definite, such a minimum value a is greater than 0. Hence, we have that h 0 < a 2 < 1 2 h D2 f(x 0 h ) h. R 2 (h; x 0 ) Since lim h 0 h 2 = 0, there exists a real number r > 0 such that, if 0 < h < r, then a 4 < R 2(h; x 0 ) h 2 < a 4. Therefore, as soon as 0 < h < r, we have that h f(x 0 + h) f(x 0 ) h 2 = 1 2 h D2 f(x 0 h ) h + R 2(h; x 0 ) h 2 > a 2 a 4 > 0, which implies that x 0 is a point of strict relative minimum for f. Definition (Saddle point). Let f : (R n )D R be a real function of n real variables. Then a point x 0 D is said to be a saddle point of f provided that x 0 is a stationary point and the hessian matrix D 2 f(x 0 ) is indefinite of sign. In this case every open ball centered at x 0 contains points x D such that f(x) < f(x 0 ) and points x D such that f(x) > f(x 0 ). In the following remark we consider the sufficient conditions for the existence of relative maxima or minima in the case of two variables.

31 28 Chapter 2. Real functions of n real variables Figure 2.4 Graph of f(x, y) = x 2 y 2, saddle point x 0 = (0, 0). Remark (Sufficient conditions in the case of two variables). Let f be a real function of two real variables and assume that its domain D is open. From the previous theorem, if f is C 2 and x 0 D is a stationary point for f then x 0 is (i) a point of strict relative maximum if 2 f x 2 (x 0 ) < 0, 2 f x 2 (x 0 ) 2 f y 2 (x 0 ) (ii) a point of strict relative minimum if 2 f x 2 (x 0 ) > 0, 2 f x 2 (x 0 ) 2 f y 2 (x 0 ) [ 2 f x y (x0 )] 2 > 0; [ 2 f x y (x0 )] 2 > 0; (iii) a saddle point for f if 2 f x 2 (x 0 ) 2 f y 2 (x 0 ) [ 2 f x y (x0 )] 2 < 0. Nothing can be said in general if det(d 2 f(x 0 )) = 0. Example (Extremal points in case of 3 variables). Consider the real function of 3 real variables defined as We have that f x (x, y, z) = 4x(x2 +y 2 ) y, f(x, y, z) = (x 2 + y 2 ) 2 + z 2 xy. f y (x, y, z) = 4y(x2 +y 2 ) x, f (x, y, z) = 2z. z

32 2.3. Differentiability and optimization 29 In order to determine the stationary points of f, consider the system 4x(x 2 + y 2 ) y = 0 4y(x 2 + y 2 ) x = 0. z = 0 The previous system is equivalent to the following one: (4x 2 + 4y 2 + 1)(x y) = 0 4y(x 2 + y 2 ) x = 0, z = 0 that in turn is equivalent to the following pair of systems: 4x 2 + 4y = 0 4y(x 2 + y 2 ) x = 0 z = 0, x y = 0 4y(x 2 + y 2 ) x = 0 z = 0. The first one has no solutions in in R 3, while the second one, that can be written as follows: x(8x 2 1) = 0 y = x, z = 0 has solutions (0, 0, 0), ( 2 of the second order are 4, 2 2 f x 2 (x, y, z) = 4(3x2 + y 2 ), 2 f x y (x, y, z) = 2 f (x, y, z) = 8xy 1, y x We have that 2 f x (x, y, z) 2 (x, y, z) Since 2 f y x 4, 0) and ( 2 4, 2 4, 0). The partial derivatives 2 f y 2 (x, y, z) = 4(x2 + 3y 2 ), 2 f y z (x, y, z) = 2 f (x, y, z) = 0. z y 2 f x y (x, y, z) 2 f y 2 (x, y, z) D 2 (f(x, y, z)) = 2 f (x, y, z) = 2, z2 2 f x z (x, y, z) = 2 f (x, y, z) = 0, z x = 16(3x2 + y 2 )(x 2 + 3y 2 ) (8xy 1) 2. 4(3x 2 + y 2 ) 8xy 1 0 8xy 1 4(x 2 + 3y 2 ) we have that det(d 2 f(x, y, z) = 2[16(3x 2 + y 2 )(x 2 + 3y 2 ) (8xy 1) 2 ]). Hence, 2 f x 2 (0, 0, 0) = 0 = det(a 1 ), det(a 2 ) = 1, det(d 2 f(0, 0, 0)) = 2.,

33 30 Chapter 2. Real functions of n real variables The point (0, 0, 0) is a saddle point (therefore it is not an extremal point). If we consider the points ( 2 4, 2 4, 0) and ( 2 4, 2 4, 0), we obtain that D 2 (f(x, y, z)) is positive definite at those points and therefore these are points of relative minimum. 2.4 Convex and concave functions WE present some arguments concerning convexity. Definition (Convex set). A subset D of R n is said to be convex if for all x, y D and for every real number t [0, 1] we have that tx + (1 t)y D. If t [0, 1], then tx + (1 t)y D is said to be a convex linear combination of the points x and y. In this case, tx + (1 t)y is a point of the segment whose extremes are x and y. Definition (Convex and concave functions). A function f : (R n )D R defined on a convex set D R n is said to be (i) convex (concave) if f(tx + (1 t)y) tf(x) + (1 t)f(y) (respectively, f(tx + (1 t)y) tf(x) + (1 t)f(y)) for all x, y D and for every real number 0 < t < 1; (ii) strictly convex (strictly concave) if f(tx + (1 t)y) < tf(x)+(1 t)f(y) (respectively, f(tx + (1 t)y) > tf(x) + (1 t)f(y)) for all x, y D and for every real number 0 < t < 1. The convexity/concavity of a quadratic form is simple to be analyzed by means of the following proposition, which is not proven here. Proposition (Convex quadratic forms). A quadratic form q on R n is convex (concave) if and only if q is positive semidefinite (negative semidefinite). Further, q is strictly convex (concave) if q is positive definite (negative definite).

34 2.4. Convex and concave functions Figure 2.5 The function f(x, y) = sin(x + y) is neither convex nor concave. Example (Strictly convex quadratic form). Consider the following quadratic form on R 2 : q(x, y) = x 2 4xy + 8y 2. Then we have that q(x, y) = x Ax with x = (x, y) and A = ( Since D 2 f(x) A is positive definite, we have that q is strictly convex. We present the proof of the following nice result. Theorem (LocalRelative minimum and convexity). Let f : (R n )D R be a convex (concave) function. Then every point x 0 D that is a point of relative minimum (maximum) is also a point of minimum (maximum) for f on D. Proof. Let f be a convex function on a set D R n and let x 0 be a point of relative minimum for f. Then, from the definition of a point of relative minimum, there exists an open ball B r (x 0 ) such that f(x) f(x 0 ) for every x B r (x 0 ) D. Let x D be a generic point of the domain of f. Consider the convex linear combination z t = tx 0 + (1 t)x (0 < t < 1). Since f is ).

35 32 Chapter 2. Real functions of n real variables convex, we have that f(z t ) = f(tx 0 +(1 t)x) tf(x 0 )+(1 t)f(x) for every real number t such that 0 < t < 1. On the other hand, if t sufficiently near 1 then z t belongs to B r (x 0 ) and therefore f(x 0 ) f(z t ) tf(x 0 )+(1 t)f(x) implies that f(x 0 ) tf(x 0 )+(1 t)f(x). This last inequality can be written as (1 t)f(x 0 ) (1 t)f(x), which is equivalent to f(x 0 ) f(x). Hence, since such a condition is verified for every x D, we have that x 0 is a point of minimum for f. Theorem (Condition for convexity). Let f : (R n )D R be C 2 on an open and convex set D. Then f is convex (concave) if and only if the hessian matrix D 2 f(x) is positive (negative) semidefinite for all x D. If the hessian matrix D 2 f(x) is positive (negative) definite for all x D, then f is strictly convex (concave). Example (Convex function). Consider the function f(x, y) = e x y on R 2. We have that ( ) D 2 e x y e f(x) = x y. e x y e x y Since D 2 f(x) is positive semidefinite, we have that f is convex. Theorem (Convexity and extremal points). Let f : (R n )D R be C 1 on an open and convex set D. If f is convex (concave) and x 0 is a stationary point for f, then x 0 is a point of relative minimum (maximum). If in addition f is strictly convex (concave), then the stationary point x 0 is the unique point of strict minimum (maximum). 2.5 Constrained optimization with equality constraints We now present the some essential material concerning the constrained optimization with equality constraints. Consider m + 1 real functions f, g 1,..., g m : (R n )D R, with m < n. In the [constrained optimization problem with equality constraints we look for the points of minimum and maximum of the function f subject to the constraints g i (x 1,..., x n ) = 0 with i = 1,..., m. Since it is clear that min f(x) = max ( f(x)), we can limit ourselves to the consideration of the following problem:

36 2.5. Constrained optimization with equality constraints 33 max f(x 1,..., x n ) sub g 1 (x 1,..., x n ) = 0. g m (x 1,..., x n ) = 0 (2.5.1) Since we want to reduce ourselves to the consideration of a problem of unconstrained optimazation, let us introduce the Lagrangean function (or simply the Lagrangean) L : D R m R, L(x 1,..., x n, λ 1,..., λ m ) = m f(x 1,..., x n ) λ i g i (x 1,..., x n ). i=1 where the m real variables λ 1,..., λ m are said to be the Lagrange multipliers. Let us now define the set m D 0 = {(x 1,..., x n ) R n : g i (x 1,..., x m ) = 0}, i=1 and let us assume that the functions f, g 1,..., g m are C 1 on D. If a point (x 0, λ 0 ) D 0 R m is a stationary point of the Lagrangean L, then x 0 is a candidate to be a point of maximum for f constrained to g i (x 1,..., x n ) = 0 (i = 1,..., m). A possible justification of such an assertion can be found in the following Theorem, which immediately goes after the definition of a saddle point of the Lagrangean. Let us first notice that, with the following definitions: λ = λ 1. λ m, g(x) = g 1 (x). g m (x) the pervious problem of constrained maximum with equality constraints can be formulated as follows: max f(x), sub g(x) = λ. (2.5.2) Definition (Saddle point of the Lagrangean). If we consider the constrained maximization problem with equality constraints 2.5.2, a point (x 0, λ 0 ) D R m is said to be a saddle point of the Lagrangean L(x, λ) = f(x) λ g(x) if L(x 0, λ 0 ) = max x min L(x, λ). λ

37 34 Chapter 2. Real functions of n real variables Theorem (saddle points and constrained maxima). Consider the constrained maximization problem with equality constraints If (x 0, λ 0 ) is a saddle point of the Lagrangean L(x, λ), then x 0 is a solution of the problem Proof. If (x 0, λ 0 ) is a saddle point of the Lagrangean L(x, λ), then for every λ R m and for every vector x D we have that L(x, λ 0 ) L(x 0, λ 0 ) L(x 0, λ). The last inequality is equivalent to the requirement according to which, for every λ R m, [λ λ 0 ] g(x 0 ) 0, that in turn implies that g(x 0 ) = 0 (that is, x 0 must satisfy the equality constraints). Then the first inequality becomes f(x) λ 0 g(x 0 ) f(x 0 ), which is equivalent to f(x) f(x 0 ) since x satisfies the equality constraints. Therefore x 0 is actually a point of (absolute) maximum as soon we require that the constraints g(x) = 0 are satisfied. Example (extremal points with equality constraints). Consider the following real function of two real variables: f(x, y) = x 2 + y2 2. We want to determine the maximum and the minimum of f conditional to the equality constraints x 2 + y 2 6y = 0. The Lagrangean is L(x, y, λ) = x 2 + y2 2 λ(x2 + y 2 6y). The system of the partial derivatives with respect to the variables x, y e λ equal to zero is x 2 + y 2 6y = 0 2x 2λx = 0 x(1 λ) = 0. y 2λy + 6λ = 0 From the second equation we get x = 0 or else λ = 1. If λ = 1, then we have that x = 0 and y = 6. If λ 1, then we must have x = 0 and y = 0 or y = 6. If y = 0, then we have that λ = 0 and therefore the solutions are x = 0, y = 0 e λ = 0. If y = 6, then it must be λ = 1 and we arrive at a contradiction. Finally, since f(0, 6) = 18 and f(0, 0) = 0, we have that the points (0, 6) e (0, 0) are the points of maximum and respectively minimum given the contraints.

38 2.6. Exercises Exercises Quadratic forms Study the definition of the following quadratic forms on R n : 1. q(x, y, z) = x 2 + y 2 + z 2 2xy + 2yz; 2. q(x, y, z) = x 2 + 2y 2 + z 2 + 2xy 2xz; 3. q(x, y, z) = x 2 + y 2 + z 2 2xz; 4. q(x, y, z) = x 2 + 4y 2 + 2z 2 4xy; 5. q(x, y, z) = x 2 + y 2 + z 2 4xy 4xz; 6. q(x, y, z) = x 2 + 4y 2 + 2z 2 4xy: 7. q(x, y, z) = x 2 + 3y 2 z 2 2xy + 2yz; 8. q(x, y, z) = x 2 + 5y 2 + z 2 2xy 4yz; 9. q(x, y, z) = x 2 + 2y 2 + z 2 2xy + 2xz; 10. q(x, y, z) = x 2 + y 2 + 2z 2 xy xz. Answer the following multiple choice questions concerning a quadratic form q(x) on R n with an associated symmetric matrix A: 11. (a) q(x) is positive definite if there exists a positive minor of A; (b) q(x) is negative definite if det(a 1 ) < 0 and det(a 2 ) < 0; (c) indefinite of sign if det(a 1 ) < 0 and det(a 2 ) < 0; (d) none of the previous assertions is true. 12. (a) q(x) is positive semidefinite if there exists a nonnegative minor of A; (b) q(x) is negative semidefinite if there exists a non positive minor of A; (c) q(x) is indefinite of sign if det(a 1 ) 0 and det(a 2 ) < 0; (d) none of the previous assertions is true.

39 36 Chapter 2. Real functions of n real variables 13. (a) q(x) is negative semidefinite if all the principal minors of A are nonnegative; (b) q(x) is positive definite if det(a 2 ) < 0; (c) indefinite of sign if det(a 1 ) < 0; (d) none of the previous assertions is true. 14. (a) q(x) is positive definite if all the principal minors of A are nonnegative; (b) q(x) is negative definite if all the principal minors of A are nonnegative; (c) indefinite of sign if det(a 2 ) < 0; (d) none of the previous assertions is true. 15. (a) q(x) is positive definite if all the principal minors of A are nonnegative; (b) q(x) is indefinite of sign if det(a 2 ) < 0; (c) q(x) is negative definite if det(a 1 ) < 0; (d) none of the previous assertions is true. 16. (a) q(x) is positive definite if all the principal minors of A are nonnegative; (b) q(x) is negative definite if det(a 1 ) < 0; (c) indefinite of sign if det(a 2 ) < 0; (d) none of the previous assertions is true. Maxima and minima Determine the nature of the stationary points of the following functions: 1. f(x, y) = x 2 x y + 2xy; 2. f(x, y) = x(x y 1); 3. f(x, y) = x 2 + y 2 (x + 1); 4. f(x, y) = x(x + y + 1); 5. f(x, y) = x 2 2xy + y 3 ; 6. f(x, y) = x 2 y(1 2x);

40 2.6. Exercises f(x, y) = x 3 xy + x + y; 8. f(x, y) = 1 x + 1 y + xy. Answer the following multiple choice questions concerning a C 2 real function f : (R n )D R of n real variables and a point x 0 D: 9. (a) a point of strict relative maximum if the gradient at x 0 is D 1 f(x 0 ) = 0; (b) a saddle point of f if the hessian matrix D 2 f(x 0 ) is indefinite of sign; (c) a point of strict relative minimum if the gradient at x 0 is D 1 f(x 0 ) = 0 and the hessian matrix D 2 f(x 0 ) is negative definite; (d) none of the previous assertions is true. 10. (a) if x 0 is an interior point of strict relative maximum then the gradient at x 0 is D 1 f(x 0 ) = 0; (b) (c) (d) if the hessian matrix D 2 f(x 0 ) is positive definite then x 0 is a point of strict relative maximum; if the hessian matrix D 2 f(x 0 ) is positive definite then x 0 is a point of strict relative minimum; none of the previous assertions is true. 11. (a) a point of strict relative minimum if the gradient at x 0 is D 1 f(x 0 ) = 0 and the hessian matrix D 2 f(x 0 ) is positive definite; (b) a saddle point of f if the hessian matrix D 2 f(x 0 ) is indefinite of sign; (c) a point of strict relative maximum if the gradient at x 0 is D 1 f(x 0 ) = 0 and the hessian matrix D 2 f(x 0 ) is positive definite; (d) none of the previous assertions is true. 12. (a) a point of strict relative minimum if the gradient at x 0 is D 1 f(x 0 ) = 0 and the hessian matrix D 2 f(x 0 ) is negative definite; (b) a saddle point of f if the gradient at x 0 is D 1 f(x 0 ) = 0; (c) (d) a saddle point of f if the hessian matrix D 2 f(x 0 ) is negative definite; none of the previous assertions is true. 13. (a) a point of strict relative minimum if the gradient at x 0 is D 1 f(x 0 ) = 0;

41 38 Chapter 2. Real functions of n real variables (b) a saddle point of f if the hessian matrix D 2 f(x 0 ) is positive definite; (c) a point of strict relative maximum if the gradient at x 0 is D 1 f(x 0 ) = 0 and the hessian matrix D 2 f(x 0 ) is positive definite; (d) none of the previous assertions is true.