# THE CONTRACTION MAPPING THEOREM

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 THE CONTRACTION MAPPING THEOREM KEITH CONRAD 1. Introduction Let f : X X be a mapping from a set X to itself. We call a point x X a fixed point of f if f(x) = x. For example, if [a, b] is a closed interval then any continuous function f : [a, b] [a, b] has at least one fixed point. This is a consequence of the intermediate value theorem, as follows. Since f(a) a and f(b) b, we have f(b) b 0 f(a) a. The difference f(x) x is continuous, so by the intermediate value theorem 0 is a value of f(x) x for some x [a, b], and that x is a fixed point of f. Of course, there could be more than one fixed point. We will discuss here the most basic fixed-point theorem in analysis. It is due to Banach and appeared in his Ph.D. thesis (1920, published in 1922). Theorem 1.1. Let (X, d) be a complete metric space and f : X X be a map such that d(f(x), f(x )) cd(x, x ) for some 0 c < 1 and all x and x in X. Then f has a unique fixed point in X. Moreover, for any x 0 X the sequence of iterates x 0, f(x 0 ), f(f(x 0 )),... converges to the fixed point of f. When d(f(x), f(x )) cd(x, x ) for some 0 c < 1 and all x and x in X, f is called a contraction. A contraction shrinks distances by a uniform factor c less than 1 for all pairs of points. Theorem 1.1 is called the contraction mapping theorem or Banach s fixed-point theorem. Example 1.2. A standard procedure to approximate a solution in R to the numerical equation g(x) = 0, where g is differentiable, is to use Newton s method: find an approximate solution x 0 and then compute the recursively defined sequence x n = x n 1 g(x n 1) g (x n 1 ). This recursion amounts to iteration of the function f(x) = x g(x)/g (x) starting from x = x 0. A solution of g(x) = 0 is the same as a solution of f(x) = x: a fixed point of f. To use Newton s method to estimate 3, we take g(x) = x 2 3 and seek a (positive) root of g(x). The Newton recursion is x n = x n 1 x2 n 1 3 2x n 1 = 1 2 ( x n ), x n 1 so f(x) = (1/2)(x + 3/x). The fixed points of f are the square roots of 3. In the following table are iterations of f with three different positive choices of x 0. 1

2 2 KEITH CONRAD n x n x n x n All three sequences of iterates in the table appear to be tending to The last choice of x 0 in the table is quite a bit further away from 3 so the iterations take longer to start resembling 3. To justify the application of the contraction mapping theorem to this setting, we need to find a complete metric space on which f(x) = (1/2)(x+3/x) is a contraction. The set (0, ) doesn t work, since this is not complete. The closed interval X t = [t, ) is complete for any t > 0. For which t is f(x t ) X t and f a contraction on X t? Well, the minimum of f(x) on (0, ) is at x = 3, with f( 3) = 3, so for any t 3 we have x t = f(x) 3 t, hence f(x t ) X t. To find such a t for which f is a contraction on X t, for any positive x and x we have f(x) f(x ) = x x 2 ( 1 3 ) xx. If x t and x t then 1 3/t 2 1 3/xx < 1. Therefore 1 3/xx < 1 as long as 1 3/t 2 > 1, which is equivalent to t 2 > 3/2. Taking 3/2 < t 3 (e.g., t = 3/2 or t = 3), we have f : X t X t and f(x) f(x ) (1/2) x x for x and x in X t. The contraction mapping theorem says the iterates of f starting at any x 0 t will converge to 3. How many iterations are needed to approximate 3 to a desired accuracy will be addressed in Section 2 after we prove the contraction mapping theorem. Although (0, ) is not complete, iterations of f starting from any x > 0 will converge to 3; if we start below 3 then applying f will take us above 3 (because f(x) 3 for all x > 0) and the contraction mapping theorem on [ 3, ) then kicks in to guarantee that further iterations of f will converge to 3. The contraction mapping theorem has many uses in analysis, both pure and applied. After proving the theorem (Section 2) and discussing some generalizations (Section 3), we will give one major application: Picard s theorem (Section 4), which is the basic existence and uniqueness theorem for ordinary differential equations. There are further applications of the contraction mapping theorem to partial differential equations [18], to the Gauss Seidel method of solving systems of linear equations in numerical analysis [17, p. 269], to a proof of the inverse function theorem [8], [15, pp ], and to Google s Page Rank algorithm [9], [12], [19]. A basic introductory account of the ideas of iteration and contraction in analysis with a few applications is in [4], and a comprehensive treatment is in [20]. 2. Proof of the Contraction Mapping Theorem Recalling the notation, f : X X is a contraction with contraction constant c. We want to show f has a unique fixed point, which can be obtained as a limit through iteration of f from any initial value. To show f has at most one fixed point in X, let a and a be fixed points of f. Then d(a, a ) = d(f(a), f(a )) cd(a, a ).

3 THE CONTRACTION MAPPING THEOREM 3 If a a then d(a, a ) > 0 so we can divide by d(a, a ) to get 1 c, which is false. Thus a = a. Next we want to show, for any x 0 X, that the recursively defined iterates x n = f(x n 1 ) for n 1 converge to a fixed point of f. How close is x n to x n+1? For any n 1, d(x n, x n+1 ) = d(f(x n 1 ), f(x n )) cd(x n 1, x n ). Therefore d(x n, x n+1 ) cd(x n 1, x n ) c 2 d(x n 2, x n 1 ) c n d(x 0, x 1 ). Using the expression on the far right as an upper bound on d(x n, x n+1 ) shows the x n s are getting consecutively close at a geometric rate. This implies the x n s are Cauchy: for any m > n, using the triangle inequality several times shows d(x n, x m ) d(x n, x n+1 ) + d(x n+1, x n+2 ) + + d(x m 1, x m ) c n d(x 0, x 1 ) + c n+1 d(x 0, x 1 ) + + c m 1 d(x 0, x 1 ) = (c n + c n c m 1 )d(x 0, x 1 ) (c n + c n+1 + c n+2 + )d(x 0, x 1 ) c n (2.1) = 1 c d(x 0, x 1 ). To prove from this bound that the x n s are Cauchy, choose ε > 0 and then pick N 1 such that (c N /(1 c))d(x 0, x 1 ) < ε. Then for any m > n N, cn d(x n, x m ) 1 c d(x 0, x 1 ) cn 1 c d(x 0, x 1 ) < ε. This proves {x n } is a Cauchy sequence. Since X is complete, the x n s converge in X. Set a = lim n x n in X. To show f(a) = a, we need to know that contractions are continuous. In fact, a contraction is uniformly continuous. This is clear when c = 0 since then f is a constant function. If c > 0 and we are given ε > 0, setting δ = ε/c implies that if d(x, x ) < δ then d(f(x), f(x )) cd(x, x ) < cδ = ε. That proves f is uniformly continuous. Since f is then continuous, from x n a we get f(x n ) f(a). Since f(x n ) = x n+1, f(x n ) a as n. Then f(a) and a are both limits of {x n } n 0. From the uniqueness of limits, a = f(a). This concludes the proof of the contraction mapping theorem. Remark 2.1. When f : X X is a contraction with constant c, any iterate f n is a contraction with constant c n ; the unique fixed point of f will also be the unique fixed point of any f n. Remark 2.2. The contraction mapping theorem admits a converse [3], [16, pp ]. If X is any set (not yet a metric space), c (0, 1), and f : X X is a function such that each iterate f n : X X has a unique fixed point then there is a metric on X making it a complete metric space such that, for this metric, f is a contraction with contraction constant c. For instance, the function f : R R given by f(x) = 2x has 0 as its unique fixed point, and the same applies to its iterates f n (x) = 2 n x. Therefore there is a metric on R with respect to which it is complete and the function x 2x is a contraction, so in particular 2 n 0 in this metric: how strange! There are examples where f has a unique fixed point but an iterate of f does not, such as f : R R by f(x) = 1 x. Here f 2 (x) = f(f(x)) = x for all x in R. Corollary 2.3. Let f : X X be a contraction on a complete metric space and Y X be a closed subset such that f(y ) Y. Then the unique fixed point of f is in Y.

4 4 KEITH CONRAD Proof. Since Y is a closed subset of a complete metric space, it is complete. Then we can apply the contraction mapping theorem to f : Y Y, so f has a fixed point in Y. Since f has only one fixed point in X, it must lie in Y. The proof of the contraction mapping theorem yields useful information about the rate of convergence towards the fixed point, as follows. Corollary 2.4. Let f be a contraction mapping on a complete metric space X, with contraction constant c and fixed point a. For any x 0 X, with f-iterates {x n }, we have the estimates (2.2) d(x n, a) cn 1 c d(x 0, f(x 0 )), (2.3) d(x n, a) cd(x n 1, a), and (2.4) d(x n, a) c 1 c d(x n 1, x n ). Proof. From (2.1), for m > n we have d(x n, x m ) cn 1 c d(x 0, x 1 ) = cn 1 c d(x 0, f(x 0 )). The right side is independent of m. Let m to get d(x n, a) (c n /(1 c))d(x 0, f(x 0 )). To derive (2.3), from a being a fixed point we get (2.5) d(x n, a) = d(f(x n 1 ), f(a)) cd(x n 1, a). Applying the triangle inequality to d(x n 1, a) on the right side of (2.5) using the three points x n 1, x n, and a, and isolating d(x n, a) yields That s (2.4), so we re done. d(x n, a) c(d(x n 1, x n ) + d(x n, a)), d(x n, a) c 1 c d(x n 1, x n ). The three inequalities in Corollary 2.4 serve different purposes. The inequality (2.2) tells us, in terms of the distance between x 0 and f(x 0 ) = x 1, how many times we need to iterate f starting from x 0 to be certain that we are within a specified distance from the fixed point. This is an upper bound on how long we need to compute. It is called an a priori estimate. Inequality (2.3) shows that once we find a term by iteration within some desired distance of the fixed point, all further iterates will be within that distance. However, (2.3) is not so useful as an error estimate since both sides of (2.3) involve the unknown fixed point. The inequality (2.4) tells us, after each computation, how much closer we are to the fixed point in terms of the previous two iterations. This kind of estimate, called an a posteriori estimate, is very important because if two successive iterations are nearly equal, (2.4) guarantees that we are very close to the fixed point. For example, if c = 2/3 and d(x n 1, x n ) < 1/10 10 then (2.4) tells us d(x n, a) < 2/10 10 < 1/10 9.

5 THE CONTRACTION MAPPING THEOREM 5 Example 2.5. To compute 3 to 4 digits after the decimal point, we use the recursion from Example 1.2: 3 is a fixed point of f(x) = (1/2)(x + 3/x). In that example we saw a contraction constant for f on [3/2, ) is c = 1/2. Taking x 0 = 3/2, so x 0 f(x 0 ) = 1/4, (2.2) becomes x n n+1. The right side is less than 1/10 4 for n = 13, which means x 13 lies within 1/10 4 of 3. However, this iteration gives us that much accuracy a lot sooner: x and x , so x 3 x By (2.4) with c = 1/2, x 4 3 x 4 x 3, so x 4 gives us 3 accurately to 4 (and in fact a bit more) places after the decimal point. The iteration we are using here to approximate 3 comes from Newton s method, which has its own error bounds separate from the bounds for general contraction mappings. When Newton s method converges, it does so doubly-exponentially (something like c 2n and not just c n ), so essentially doubling the number of correct digits after each iteration while a typical contraction mapping produces at most one new correct digit after each iteration once the approximations get close enough to the fixed point. Example 2.6. Graphs of y = cos x and y = x intersect once (see the figure below), which means the cosine function has a unique fixed point in R. From the graph, this fixed point lies in [0, 1]. We will show this point can be approximated through iteration. y = cos x y = x The function cos x is not a contraction mapping on R: there is no c (0, 1) such that cos x cos y c x y for all x and y. Indeed, if there were such a c then (cos x cos y)/(x y) c whenever x y, and letting y x implies sin x c for all x, which is false. However, cos x is a contraction mapping on [0, 1]. First let s check cos x maps [0, 1] to [0, 1]. Since cosine is decreasing on [0, 1] and cos 1.54, cos([0, 1]) [0, 1]. To find c < 1 such that cos x cos y c x y for all x and y in [0, 1], we can t get by with elementary algebra as we did for the function (1/2)(x+3/x) in Example 1.2. We will use the mean-value theorem: for any differentiable function f, f(x) f(y) = f (t)(x y) for some t between x and y, so bounding (cos t) = sin t over [0, 1] will give us a contraction constant. Since sin t is increasing on [0, 1], for all t in [0, 1] we have sin t = sin t sin Therefore cos x cos y.8415 x y, for all x and y in [0, 1]. Since [0, 1] is complete, the contraction mapping theorem tells us that there is a unique solution of cos x = x in [0, 1] and it can be found by iterating the function cos x starting from any initial point in [0, 1]. Here are some iterations of cos x starting from x 0 = 0:

6 6 KEITH CONRAD n x n The estimates (2.2) and (2.4) with x 0 = 0 become (2.6) x n a.8415n , x n a x n 1 x n. We can use these bounds to know with certainty a value of n for which x n a <.001. The first bound falls below.001 for the first time when n = 51, while the second one falls below.001 much earlier, at n = 22. Taking n = 22, the second upper bound on x 22 a in (2.6) is a little less than.0009, so a lies between x and x The iterative process achieves the same accuracy in fewer steps if we begin the iteration closer to the fixed point. Let s start at x 0 =.7 instead of at x 0 = 0. Here are some iterations: n x n Now (2.2) and (2.4) with x 0 =.7 say x n a.8415n cos(.7), x n a x n 1 x n. The first upper bound on x n a falls below.001 for n = 35, and the second upper bound falls below.001 when n = 16. Using the second bound, a lies between x and x Example 2.7. We can use the contraction mapping theorem to compute inverses using the computationally less expensive operations of addition and multiplication. The linear polynomial f(x) = mx + b has fixed point b/(1 m), and this function is a contraction mapping on R when m < 1. (In fact x n = m n x 0 + (1 + m + m m n 1 )b = m n x 0 + mn 1 m 1 b, which implies that the three inequalities in Corollary 2.3 are in this case all equalities, so that corollary is sharp: the convergence of iterates under a contraction mapping can t be any faster in general than what Corollary 2.3 says.) For s 0, to obtain 1/s as a fixed point of f(x) we seek b and m such that b/(1 m) = 1/s and m < 1. Choosing b to satisfy 1 bs < 1, so b is an initial approximation to 1/s (their product is

7 THE CONTRACTION MAPPING THEOREM 7 not too far from 1), the function f(x) = (1 bs)x + b is a contraction on R with fixed point 1/s. For example, if s 1 < 1 then we can use f(x) = (1 s)x+1. For this contraction and x 0 = 0, we have x n = 1 + (1 s) + + (1 s) n, which means the convergence of geometric series can be viewed as a special case of the contraction mapping theorem. (Logically this is circular, since we used geometric series in the proof of the contraction mapping theorem; see (2.1).) We can also make iterations converge to 1/s using Newton s method on g(x) = 1/x s, which amount to viewing 1/s as the fixed point of f(x) = x g(x)/g (x) = 2x sx 2. From the mean-value theorem (or the triangle inequality), check that f is a contraction on [1/s ε, 1/s + ε] when ε < 1/(2s). So if x 0 1/s < 1/(2s) then iterates of f starting at x 0 will converge to 1/s by the contraction mapping theorem. This method of approximating 1/s converges much faster than the iterates of a linear polynomial: check by induction that the nth iterate of f(x) is 1/s (sx 1) 2n /s, so if x 0 1/s < 1/(2s) then the nth iterate of f at x 0 differs from 1/s by less than (1/2) 2n (1/s), which decays to 0 exponentially rather than geometrically. 3. Generalizations In some situations a function is not a contraction but an iterate of it is. This turns out to suffice to get the conclusion of the contraction mapping theorem for the original function. Theorem 3.1. If X is a complete metric space and f : X X is a mapping such that some iterate f N : X X is a contraction, then f has a unique fixed point. Moreover, the fixed point of f can be obtained by iteration of f starting from any x 0 X. Proof. By the contraction mapping theorem, f N has a unique fixed point. Call it a, so f N (a) = a. To show a is the only possible fixed point of f, observe that a fixed point of f is a fixed point of f N, and thus must be a. To show a really is a fixed point of f, we note that f(a) = f(f N (a)) = f N (f(a)), so f(a) is a fixed point of f N. Therefore f(a) and a are both fixed points of f N. Since f N has a unique fixed point, f(a) = a. We now show that for any x 0 X the points f n (x 0 ) converge to a as n. Consider the iterates f n (x 0 ) as n runs through a congruence class modulo N. That is, pick 0 r N 1 and look at the points f Nk+r (x 0 ) as k. Since f Nk+r (x 0 ) = f Nk (f r (x 0 )) = (f N ) k (f r (x 0 )), these points can be viewed (for each r) as iterates of f N starting at the point y 0 = f r (x 0 ). Since f N is a contraction, these iterates of f N (from any initial point, such as y 0 ) must tend to a by the contraction mapping theorem. This limit is independent of the value of r in the range {1,..., N 1}, so all N sequences {f Nk+r (x 0 )} k 1 tend to a as k. This shows (3.1) f n (x 0 ) a as n. We will use Theorem 3.1 in Section 4 to solve differential equations. Example 3.2. The graphs of y = e x and y = x intersect once, so e a = a for a unique real number a. See the graph below.

8 8 KEITH CONRAD y = e x y = x a The function f(x) = e x is not a contraction on R (for instance, f( 2) f(0) 6.38 > 2 0 ), but its second iterate g(x) = f 2 (x) = e e x is a contraction on R: by the mean-value theorem g(x) g(y) = g (t)(x y) for some t between x and y, where g (t) = e e t e t = e (t+e t) e 1 (since t + e t 1 for all real t). Hence f 2 has contraction constant 1/e.367 < 1. By Theorem 3.1 the solution to e a = a can be approximated by iteration of f starting with any real number. Iterating f enough times with x 0 = 0 suggests a.567. To prove this approximation is correct, one can generalize the error estimates in Corollary 2.4 to apply to a function having an iterate as a contraction. Alternatively, check f([ε, 1]) [ε, 1] when 0 ε 1/e, and when 0 < ε 1/e the function f is a contraction on [ε, 1]. Therefore Corollary 2.4 can be applied directly to f by starting iterates at any x 0 (0, 1]. Remark 3.3. When f N is a contraction, f N is continuous, but that does not imply f is continuous. For example, let f : [0, 1] [0, 1] by f(x) = 0 for 0 x 1/2 and f(x) = 1/2 for 1/2 < x 1. Then f(f(x)) = 0 for all x, so f 2 is a contraction but f is discontinuous. It was important in the proof of the contraction mapping theorem that the contraction constant c be strictly less than 1. That gave us control over the rate of convergence of f n (x 0 ) to the fixed point since c n 0 as n. If instead of f being a contraction we suppose d(f(x), f(x )) < d(x, x ) whenever x x in X then we lose that control and indeed a fixed point need not exist. Example 3.4. Let I be a closed interval in R and f : I I be differentiable with f (t) < 1 for all t. Then the mean-value theorem implies f(x) f(x ) < x x for x x in I. The following three functions all fit this condition, where I = [1, ) in the first case and I = R in the second and third cases: f(x) = x + 1 x, f(x) = x 2 + 1, f(x) = log(1 + e x ). In each case, f(x) > x, so none of these functions has a fixed point in I. (One could consider to be a fixed point, and f n (x) as n. See [5] for a general theorem in this direction.) Despite such examples, there is a fixed-point theorem when d(f(x), f(x )) < d(x, x ) for all x x provided the space is compact, which is not the case in the previous example. This theorem, which is next, is due to Edelstein [7].

9 THE CONTRACTION MAPPING THEOREM 9 Theorem 3.5. Let X be a compact metric space. If f : X X satisfies d(f(x), f(x )) < d(x, x ) when x x in X, then f has a unique fixed point in X and the fixed point can be found as the limit of f n (x 0 ) as n for any x 0 X. Proof. To show f has at most one fixed point in X, suppose f has two fixed points a a. Then d(a, a ) = d(f(a), f(a )) < d(a, a ). This is impossible, so a = a. To prove f actually has a fixed point, we will look at the function X [0, ) given by x d(x, f(x)). This measures the distance between each point and its f-value. A fixed point of f is where this function takes the value 0. Since X is compact, the function d(x, f(x)) takes on its minimum value: there is an a X such that d(a, f(a)) d(x, f(x)) for all x X. We ll show by contradiction that a is a fixed point for f. If f(a) a then the hypothesis about f in the theorem (taking x = a and x = f(a)) says d(f(a), f(f(a))) < d(a, f(a)), which contradicts the minimality of d(a, f(a)) among all numbers d(x, f(x)). So f(a) = a. Finally, we show for any x 0 X that the sequence x n = f n (x 0 ) converges to a as n. This can t be done as in the proof of the contraction mapping theorem since we don t have the contraction constant to help us out. Instead we will exploit compactness. If for some k 0 we have x k = a then x k+1 = f(x k ) = f(a) = a, and more generally x n = a for all n k, so x n a since the terms of the sequence equal a for all large n. Now we may assume instead that x n a for all n. Then 0 < d(x n+1, a) = d(f(x n ), f(a)) < d(x n, a), so the sequence of numbers d(x n, a) is decreasing and positive. Thus it has a limit l = lim n d(x n, a) 0. We will show l = 0 (so d(x n, a) 0, which means x n a in X). By compactness of X, the sequence {x n } has a convergent subsequence x ni, say x ni y X. The function f is continuous, so f(x ni ) f(y), which says x ni +1 f(y) as i. Since d(x n, a) l as n, d(x ni, a) l and d(x ni +1, a) l as i. By continuity of the metric, d(x ni, a) d(y, a) and d(x ni +1, a) = d(f(x ni ), a) d(f(y), a). Having already shown these limits are l, (3.2) d(y, a) = l = d(f(y), a) = d(f(y), f(a)). If y a then d(f(y), f(a)) < d(y, a), but this contradicts (3.2). So y = a, which means l = d(y, a) = 0. That shows d(x n, a) 0 as n. The proof of Edelstein s theorem does not yield any error estimate on the rate of convergence to the fixed point, since we proved convergence to the fixed point without having to make estimates along the way. Edelstein did not assume for his main theorem in [7] that X is compact, as in Theorem 3.5. He assumed instead that there is some x 0 X whose sequence of iterates f n (x 0 ) has a convergent subsequence and he proved this limit is the unique fixed point of f and that the whole sequence f n (x 0 ) tends to that fixed point. It is natural to wonder if the compactness of X might force f in Theorem 3.5 to be a contraction after all, so the usual contraction mapping theorem would apply. For instance, the ratios d(f(x), f(x ))/d(x, x ) for x x are always less than 1, so they should be less than or equal to some definite constant c < 1 from compactness (a continuous real-valued function on a compact set achieves its supremum as a value). But this reasoning is bogus, because d(f(x), f(x ))/d(x, x ) is defined not on the compact set X X but rather on X X {(x, x) : x X} where the diagonal is removed, and this is not compact (take a

10 10 KEITH CONRAD look at the special case X = [0, 1], for instance). There is no way to show f in Theorem 3.5 has to be a contraction since there are examples where it isn t. Example 3.6. Let f : [0, 1] [0, 1] by f(x) = 1 1+x, so f(x) f(y) = x y / (1+x)(1+y). When x y we have f(x) f(y) / x y = 1/(1 + x)(1 + y) < 1, so f(x) f(y) < x y and f(x) f(y) / x y gets arbitrarily close to 1 when x and y are sufficiently close to 0. Therefore f is not a contraction on [0, 1] with respect to the usual metric. Theorem 3.5 says f has a unique fixed point in [0, 1] and f n (x 0 ) tends to this point as n for any choice of x 0. Of course, it is easy to find the fixed point: x = 1/(1 + x) in [0, 1] at x = ( 1 + 5)/ This example does not actually require Theorem 3.5. One way around it is to check that if 1/2 x 1 then 1/2 f(x) 2/3, so f : [1/2, 1] [1/2, 1] with max x [1/2,1] f (x) = 4/9. We could apply the contraction mapping theorem to f on [1/2, 1] to find the fixed point. A second method is to check that f 2 (x) = (1 + x)/(2 + x) is a contraction on [0, 1] (the derivative of (1 + x)/(2 + x) on [0, 1] has absolute value at most 1/4), so we could apply the contraction mapping theorem to f 2. Example 3.7. Following [2], let f : [0, 1] [0, 1] by f(x) = x/(1 + x). Since f(x) f(y) / x y = 1/(1 + x)(1 + y) for x y, the ratio f(x) f(y) / x y for x y can be made arbitrarily close to 1 by taking x and y sufficiently close to 0. What makes this example different from the previous one is that, since 0 is now the fixed point, f does not restrict to a contraction on any neighborhood of its fixed point. Moreover, since f n (x) = x/(1 + nx), for x y the ratio f n (x) f n (y) / x y = 1/(1+nx)(1+ny) is arbitrarily close to 1 when x and y are sufficiently close to 0, so no iterate f n is a contraction on any neighborhood of the fixed point in [0, 1]. A similar example on [0, 1] with 1 as the fixed point uses g(x) = 1 f(1 x) = 1/(2 x). 4. Differential Equations As a significant application of the contraction mapping theorem, we will solve the initial value problem (4.1) dy dt = f(t, y), y(t 0) = y 0 for nice functions f. A solution to (4.1) is a differentiable function y(t) defined on a neighborhood of t 0 such that dy dt = f(t, y(t)), y(t 0) = y 0. Before stating the basic existence and uniqueness theorem for such initial value problems, we look at some examples to appreciate the scope of the theorem we will prove. Example 4.1. The initial value problem dy dt = y2, y(0) = 1 has a unique solution passing through the point (0,1) that is found using separation of variables and integration: y(t) = 1 1 t.

11 THE CONTRACTION MAPPING THEOREM 11 Note that although the right side of the differential equation makes sense everywhere, the solution blows up in finite time (at t = 1). See the blue curve in the figure below. Other curves in the figure are other solutions to dy/dt = y 2 with y(0) > 0 (having the form y = 1/(c t)) Example 4.2. The initial value problem dy dt = y2 t, y(0) = 1 has a solution, but the solution can t be expressed in terms of elementary functions or their integrals (theorem of Liouville [14, p. 70 ff.]). Like the previous example, the solution blows up in finite time even though the right side makes sense everywhere. Take a look at the slope field digram below to be convinced visually of the finite blow-up time for the solution curve passing through (0, 1); for a proof of this see Appendix A.

12 12 KEITH CONRAD Example 4.3. The initial value problem dy dt = y2/3, y(0) = 0, has two solutions: y(t) = 0 and y(t) = t 3 /27. These examples show that to solve (4.1) in any kind of generality, we will have to be satisfied with a solution that exists only locally (i.e., not everywhere that f makes sense), and some constraint is needed on f to have a unique solution near t 0. The condition we will introduce on f(t, y) to guarantee a local unique solution is a Lipschitz condition in the second variable. That is, we will assume there is a constant K > 0 such that f(t, y 1 ) f(t, y 2 ) K y 1 y 2 for all y 1 and y 2. The right side doesn t involve t, so this is a Lipschitz condition in the second variable that is uniform in the first variable. Most functions are not globally Lipschitz. For instance, the single-variable function h(x) = x 2 doesn t satisfy h(a) h(b) K a b for some K > 0 and all a and b. But any function with continuous first derivative is locally Lipschitz: Lemma 4.4. Let U R 2 be open and f : U R be a C 1 -function. Then it is locally Lipschitz in its second variable unifomly in the first: for any (t 0, y 0 ) U, there is a constant K > 0 such that f(t, y 1 ) f(t, y 2 ) K y 1 y 2 for all t near t 0 and all y 1 and y 2 near y 0. Proof. The open set U contains a rectangle around (t 0, y 0 ), say I J. Here I and J are closed intervals (of positive length) with t 0 I, y 0 J, and I J U. Applying the mean-value theorem to f in its second variable, for any t I and y 1 and y 2 in J f(t, y 1 ) f(t, y 2 ) = f y (t, y 3)(y 1 y 2 ) for some y 3 between y 1 and y 2, depending possibly on t. Then f(t, y 1 ) f(t, y 2 ) = f y (t, y 3) y 1 y 2 sup f y (p) y 1 y 2. p I J Let K = sup p I J ( f/ y)(p), which is finite since a continuous function on a compact set is bounded. Example 4.5. The function f(t, y) = sin(ty) is C 1 on all of R 2 and is Lipschitz on any vertical strip [ R, R] R: if t R then for any y 1 and y 2 in R, for some y 3 between y 1 and y 2. Therefore sin(ty 1 ) sin(ty 2 ) = t cos(ty 3 )(ty 1 ty 2 ) sin(ty 1 ) sin(ty 2 ) t 2 y 1 y 2 R 2 y 1 y 2. Example 4.6. The function f(t, y) = y 2/3 is not C 1 at (0, 0) and in fact is not even Lipschitz in any neighborhood of (0, 0): if y 2/3 1 y 2/3 2 K y 1 y 2 for some K > 0 and all y 1, y 2 near 0 then taking y 2 = 0 shows y 2/3 K y for all y near 0, but for y 0 that s the same as y 1/3 K, and y 1/3 is unbounded as y 0. Our next lemma controls the growth of a hypothetical solution to (4.1) near t 0.

13 THE CONTRACTION MAPPING THEOREM 13 Lemma 4.7. Let U R 2 be open and f : U R be continuous. For any (t 0, y 0 ) U there are r, R > 0 such that the rectangle [t 0 r, t 0 + r] [y 0 R, y 0 + R] is in U and, for any δ (0, r], a solution y(t) to (4.1) that is defined on [t 0 δ, t 0 +δ] must satisfy y(t) y 0 R for t t 0 δ. y 0 + R y 0 y 0 R U t 0 r t 0 t 0 + r Proof. Suppose y(t) satisfies (4.1) near t 0. For t near t 0, (t, y(t)) U because U is open and y(t) is continuous, so f(t, y(t)) makes sense. For t near t 0, t t 0 f(s, y(s)) ds has t-derivative f(t, y(t)), which is also the t-derivative of y(t), so by (4.1) y(t) and t t 0 f(s, y(s)) ds differ by a constant near t 0. At t = t 0 the difference is y(t 0 ) 0 = y 0, so (4.2) y(t) = y 0 + for t near t 0. Pick a rectangle t t 0 f(s, y(s)) ds [t 0 r, t 0 + r] [y 0 R, y 0 + R] in U that is centered at (t 0, y 0 ) and let B > 0 be an upper bound of f on the rectangle. For t near t 0, (4.2) implies (4.3) y(t) y 0 B t t 0. Notice B, r, and R are determined wholly by f and not by any solutions to (4.1). Shrink r so that Br R, i.e., replace r with min(r, R/B). (We could alternatively try to increase R to achieve this inequality, but increasing the size of the rectangle would usually change B and there is also the constraint that the rectangle should be inside of U.) Pick δ (0, r] and suppose there is a solution y(t) to (4.1) for t t 0 δ. In particular, (t, y(t)) U for t t 0 δ. From (4.3), y(t) y 0 Br R, so (t, y(t)) is in the rectangle for t t 0 r. Here is the key existence and uniqueness theorem for ODEs. Theorem 4.8 (Picard). Let R = [a, b] [y 0 R, y 0 + R] R 2 be a rectangle and f : R R be a continuous function on the rectangle that is Lipschitz in its second variable uniformly in its first variable. Let B > 0 be an upper bound on f over this rectangle. For t 0 [a, b], let r = min( t 0 a, t 0 b ). The initial value problem (4.1) has a unique solution on the interval [t 0 r 0, t 0 + r 0 ], where r 0 = min(r, R/B). In particular, if U R 2 is open and f : U R is C 1 then (4.1) has a unique local solution passing through any point in its domain.

14 14 KEITH CONRAD y 0 + R y 0 y 0 R a t 0 b Proof. Set M = {h C[a, b] : h(t) y 0 R for t [a, b]}. The constant function h(t) = y 0 is in M, so M. For any h M, the graph {(t, h(t)) : t [a, b]} lies in R. Giving C[a, b] the sup-norm sup, M is a closed subspace of C[a, b]. Since C[a, b] is complete in the sup-norm, M is also complete in the sup-norm. Considering the formula (4.2), let F : M M by (4.4) (F h)(t) := y 0 + t t 0 f(s, h(s)) ds. The integration makes sense, since (s, h(s)) R for all s [a, b], and moreover (F h)(t) y 0 B t t 0 Br R, so F (M) M. We will show the contraction mapping theorem can be applied to find a fixed point of F in M. A fixed point of F is a solution to (4.1) by the fundamental theorem of calculus. The solutions of (4.1) and (4.2) are the same. The advantage of (4.2) over (4.1) is that continuous functions have continuous antiderivatives but they might not have a derivative; integration is a smoother operation on functions than differentiation. To illustrate different techniques, we will show how the contraction mapping theorem can be applied to find a fixed point of F in two ways: (1) an iterate F n is a contraction on M for n 0, so Theorem 3.1 applies, (2) F is a contraction on M using a norm other than the sup-norm, but in which M is still complete. Proof of (1): For h 1, h 2 M and t [a, b], (F h 1 )(t) (F h 2 )(t) = so (4.5) (F h 1 )(t) (F h 2 )(t) t t 0 (f(s, h 1 (s)) f(s, h 2 (s))) ds, f(s, h 1 (s)) f(s, h 2 (s)) ds, where denotes [t 0,t] if t 0 t and [t,t 0 ] if t < t 0. We are assuming f is Lipschitz in its second variable uniformly in the first, say f(t, y 1 ) f(t, y 2 ) K y 1 y 2 for (t, y 1 ) and

15 (t, y 2 ) in R. Then (4.6) (4.7) THE CONTRACTION MAPPING THEOREM 15 (F h 1 )(t) (F h 2 )(t) K h 1 (s) h 2 (s) ds K h 1 h 2 sup t t 0 K h 1 h 2 sup (b a). Thus F h 1 F h 2 sup K(b a) h 1 h 2 sup. If K(b a) < 1 then F is a contraction on M. (This already gives us local existence and uniqueness of a solution by shrinking [a, b] small enough around t 0 so that b a < 1/K. But we want to show there is a solution on our original [a, b], so we proceed in a different way.) To cover the case K(b a) 1, we look at how iterates of F shrink distances. Since we get (F 2 h 1 )(t) (F 2 h 2 )(t) (F 2 h 1 )(t) (F 2 h 2 )(t) = t t 0 (f(s, (F h 1 )(s)) f(s, (F h 2 )(s))) ds, K (F h 1 )(s) (F h 2 )(s) ds K K h 1 h 2 sup s t 0 ds by (4.7) = K 2 h 1 h 2 sup s t 0 ds. by the Lipschitz condition Since s t 0 ds = t t 0 2 /2 (check this separately for t t 0 and t < t 0 ), we have (F 2 h 1 )(t) (F 2 h 2 )(t) K 2 h 1 h 2 sup t t = (K t t 0 ) 2 h 1 h 2 sup. 2 By induction, using the formula s t 0 n ds = t t 0 n+1 /(n + 1) for n 0, (F n h 1 )(t) (F n h 2 )(t) (K t t 0 ) n h 1 h 2 sup n! for all n 0 and t [a, b]. Since t t 0 b a, F n h 1 F n h 2 sup (K(b a))n h 1 h 2 sup. n! When n 0, (K(b a)) n /n! < 1, so F n is a contraction on M in the sup-norm. Thus F has a unique fixed point in M by Theorem 3.1. Proof of (2): Define a new norm on C[a, b] by h := sup e K t t0 h(t), t [a,b] where K is the Lipschitz constant for f on R. To check this is a norm on C[a, b], for any t [a, b] we have h 1 (t) + h 2 (t) h 1 (t) + h 2 (t), so e K t t 0 h 1 (t) + h 2 (t) e K t t 0 h 1 (t) + e K t t 0 h 2 (t) h 1 + h 2.

16 16 KEITH CONRAD Taking the sup of the left side over all t [a, b] shows satisfies the triangle inequality. The other conditions to be a vector space norm are easily checked. How does compare to the sup-norm on C[a, b]? For t [a, b], so for any h C[a, b] Taking the sup over t [a, b], e K(b a) e K t t 0 1, e K(b a) h(t) e K t t 0 h(t) h(t). (4.8) e K(b a) h sup h h sup. Thus and the sup-norm bound each other from above and below up to scaling factors, so these two norms have the same open sets and the same convergent sequences in C[a, b]. In particular, C[a, b] and its subset M are both complete relative to since they are complete with respect to the sup-norm. Returning to (4.6), multiply both sides by e K t t0 : e K t t0 (F h 1 )(t) (F h 2 )(t) e K t t 0 K h 1 (s) h 2 (s) ds = e K t t 0 e K t t 0 = h 1 h 2 e K t t 0 Ke K s t 0 e K s t 0 h 1 (s) h 2 (s) ds Ke K s t 0 h 1 h 2 ds Ke K s t 0 ds. To compute the integral in this last expression, we take cases depending on whether t 0 t or t 0 > t. If t 0 t then t t 0 = t t 0 and s t 0 = s t 0 for t 0 s t, so t Ke K s t0 ds = Ke K(s t0) ds = e K(t t0) 1. t 0 If t 0 > t then t t 0 = t 0 t and s t 0 = t 0 s for t s t 0, so t0 Ke K s t0 ds = Ke K(t0 s) ds = e K(t0 t) 1. In either case the value is e K t t0 1, so e K t 0 t Ke K s t0 ds = e K t t0 (e K t t0 1) = 1 e K t0 t. Therefore t e K t t 0 (F h 1 )(t) (F h 2 )(t) h 1 h 2 (1 e K t t 0 ) (1 e K(b a) ) h 1 h 2. Taking the supremum of the left side over all t [a, b], F h 1 F h 2 (1 e K(b a) ) h 1 h 2. Since K(b a) > 0, 1 e K(b a) < 1, so F : M M is a contraction with respect to. Since M is complete with respect to, there is a unique fixed point of F in M by the contraction mapping theorem.

17 THE CONTRACTION MAPPING THEOREM 17 The proof of Picard s theorem says that a solution to (4.1) near t 0 can be found as a limit of the sequence {y n (t)} where y 0 (t) = y 0 is a constant function and t y n+1 (t) = y 0 + f(s, y n (s)) ds. t 0 These functions y n (t), defined recursively by integration, are called Picard iterates. Picard s proof of Theorem 4.8 appeared in 1890 [13] (30 years before Banach stated the general contraction mapping theorem), and the basic idea was already used in a special case by Liouville in Remark 4.9. In the proof of Picard s theorem we introduced an integral operator F in (4.4) and gave two ways of showing it has a fixed point. The same ideas can be used to prove in two ways that for any continuous function f C[a, b] the (Volterra) integral equation y(t) = f(t) + t a k(t, s)y(s) ds has a unique solution y C[a, b] for any given continuous f and k. See [11, pp ]. Corollary If f : [a, b] R R is Lipschitz in its second variable uniformly in its first variable, then the differential equation (4.1) has a unique solution on [a, b] given any initial condition at a point t 0 [a, b]. Proof. In the beginning of the proof of Picard s theorem, the business with R and shrinking the domain on which the differential equation is studied near t 0 was carried out to put ourselves in the position of having a domain where f is Lipschitz in its second variable uniformly in the first variable and such that we define M to contain only continuous functions whose graph remains in this domain for f. Therefore if we assume from the outset that f has the relevant Lipschitz condition and its second variable runs over R, we can drop the preliminary part of the proof of Theorem 4.8 from consideration and just take M = C[a, b]. Define F : M M as in (4.4). Example The function f(t, y) = sin(ty) is Lipschitz in y uniformly in t on any set of the form [ r, r] R, so the differential equation dy/dt = sin(ty) with y(0) = 1 has a unique solution on [ r, r] for all r. Thus the solution y is defined on all of R. Remark If we drop the local Lipschitz condition on f then we lose the uniqueness of the solution to (4.1), as we saw in Example 4.3, but we don t lose existence. It is a theorem of Peano that as long as f(t, y) is continuous, (4.1) will have a solution curve passing through any point (t 0, y 0 ). In one method of proof [6, pp ], the basic idea is to express f as the uniform limit of polynomial functions P n (t, y) using the Stone-Weierstrass theorem. Polynomials are C 1, so by Picard s theorem (essentially the contraction mapping theorem), the initial value problems y (t) = P n (t, y), y(t 0 ) = y 0, are each uniquely locally solvable near t 0. Denote the solutions as y n. It can be shown, using a compactness theorem in spaces of continuous functions, that some subsequence of the y n s converges to a solution to the original initial value problem y = f(t, y). For a second proof of Peano s theorem, using a different fixed-point theorem, see [20, Prop. 2.14]. Picard s theorem is applicable to more general first-order differential equations than (4.1), such as ( ) dy 2 (4.9) t 2 + y 2 + = 1, y(0) = 1 dt 2.

18 18 KEITH CONRAD Setting t = 0 yields 1/4 + y (0) 2 = 1, so y (0) = ± 3/2. We have to make a choice of y (0) before we can pin down a particular solution, since the differential equation does not determine y (0). Taking y (0) = 3/2, we want to solve dy dt = 1 t 2 y 2, y(0) = 1 2, while taking y (0) = 3/2 leads to the equation dy dt = 1 t 2 y 2, y(0) = 1 2. Each of these has a unique solution near t = 0 using Picard s theorem. More generally, a first-order ordinary differential equation looks like g(t, y, y ) = 0 with some initial conditions y(t 0 ) = y 0 and y (t 0 ) = z 0 (so g(t 0, y 0, z 0 ) = 0). Assuming g(t, y, z) is a C 1 -function of 3 variables, as long as ( g/ z)(t 0, y 0, z 0 ) 0 the implicit function theorem tells us there is an open set U (t 0, y 0 ) in R 2 and a C 1 -function f : U R such that z 0 = f(t 0, y 0 ) and for all (t, y) near (t 0, y 0 ), g(t, y, z) = 0 if and only if z = f(t, y). Therefore the differential equation g(t, y, y ) = 0 locally near (t 0, y 0, z 0 ) looks like y = f(t, y), which is exactly the kind of differential equation we solved locally and uniquely by Picard s theorem. So Picard s theorem implies local existence and uniqueness of fairly general first-order ODEs. Picard s theorem generalizes [1, Chap. 4] to a first-order ordinary differential equation with a vector-valued function: dy = f(t, y(t)) dt where f and y have values in R n and f is C 1. Essentially the only change needed to extend the proof of Picard s theorem from the 1-dimensional case to higher dimensions is the use of an integral (rather than differential) form of the mean-value theorem to show a C 1 -function is locally Lipschitz. Appendix A. A Differential Inequality In Example 4.2 it was stated that the initial value problem y (t) = y(t) 2 t, y(0) = 1 has a solution that blows up in finite time. To estimate the blow-up time, let Y (t) = 1/y(t) and see where Y (t) = 0. From Y (t) = ty (t) 2 1 and Y (0) = 1, a computer algebra package has Y (t) = 0 at t Theorem A.1. The solution to y (t) = y(t) 2 t satisfying y(0) = 1 is undefined somewhere before t = This is weaker than what numerics suggest (i.e., the blow-up time is around 1.125), but proving something sharper requires a more careful analysis than we wish to develop. An expert in non-linear ODEs told me that in practice nobody tries to prove very sharp approximations for blow-up times (mere existence of a blow-up time usually suffices). Proof. We know y(t) is defined for small t > 0. Assume y(t) is defined for 0 t < c. We will show for a suitable c that y(t) c/(c t) for 0 t < c, so y(t) as t c +. Therefore y(t) has to be undefined for some t c.

19 THE CONTRACTION MAPPING THEOREM 19 Set z(t) = c/(c t), with c still to be determined, so d dt (y z) = y2 t dz dt = y 2 c t (c t) ( 2 = y 2 z c = (y z)(y + z) + ) z 2 t (c 1)c (c t) 2 t By calculus, (c 1)c/(c t) 2 t 0 for 0 t < c as long as c 1 (4/27)c 2, which happens for c between the two roots of x 1 = (4/27)x 2. The roots are approximately and So taking c = 1.221, we have (y(t) z(t)) (y(t) z(t))(y(t) + z(t)) for 0 t < c. Using an integrating factor, this differential inequality is the same as d dt (e t 0 (y(s)+z(s)) ds (y(t) z(t)) ) 0. Since e t 0 (y(s)+z(s)) ds (y(t) z(t)) t=0 = 0, e t 0 (y(s)+z(s)) ds (y(t) z(t)) 0 for t 0, so y(t) z(t) 0 because the exponential factor is positive. Thus y(t) z(t) = c/(c t). References [1] V. I. Arnol d, Ordinary Differential Equations, Springer-Verlag, New York, [2] D. G. Bennett and B. Fisher, On a Fixed Point Theorem for Compact Metric Spaces, Math. Magazine 47 (1974), [3] C. Bessaga, On the Converse of the Banach Fixed-point Principle, Colloq. Math. 7 (1959), [4] V. Bryant, Metric Spaces: Iteration and Application, Cambridge Univ. Press, Cambridge, [5] A. F. Beardon, Contractions on the real line, Amer. Math. Monthly 113 (2006), [6] R. C. Buck (ed.), Studies in Modern Analysis, Vol. 1, Prentice-Hall, Englewood Cliffs, NJ, [7] M. Edelstein, On Fixed and Periodic Points Under Contractive Mappings, J. London Math. Soc. 37 (1962), [8] M. Freeman, The Inverse as a Fixed Point in Function Space, Amer. Math. Monthly 83 (1976), [9] D. J. Higham and A. Taylor, The Sleekest Link Algorithm, aas 96106/rep pdf. [10] J. E. Hutchinson, Fractals and Self-Similarity, Indiana Univ. Math. J. 30 (1981), [11] A. N. Kolmogorov and S. Fomin, Introductory Real Analysis, Dover, New York, [12] A. Langville and C. Meyer, Google s PageRank and Beyond: The Science of Search Engine Rankings, Princeton Univ. Press, Princeton, [13] E. Picard, Mémoire sur la théorie des équations aux dérivées partielles et la méthode des approximations successives, J. Math. Pures et Appl. 6 (1890), [14] J. F. Ritt, Integration in Finite Terms, Columbia Univ. Press, New York, [15] W. Rudin, Principles of Mathematical Analysis, 3rd ed., McGraw Hill, New York, [16] E. Schechter, Handbook of Analysis and its Applications, Academic Press, San Diego, [17] C. Wagner, A Generic Approach to Iterative Methods, Math. Mag. 55 (1982), [18] W. Walter, An Elementary Proof of the Cauchy Kowalevsky Theorem, Amer. Math. Monthly 92 (1985), [19] B. White, How Google Ranks Web Pages, PageRank.pdf. [20] E. Zeidler, Nonlinear Functional Analysis and its Applications I: Fixed-Point Theorems, Springer- Verlag New York, 1986.

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### THE BANACH CONTRACTION PRINCIPLE. Contents

THE BANACH CONTRACTION PRINCIPLE ALEX PONIECKI Abstract. This paper will study contractions of metric spaces. To do this, we will mainly use tools from topology. We will give some examples of contractions,

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### Course 221: Analysis Academic year , First Semester

Course 221: Analysis Academic year 2007-08, First Semester David R. Wilkins Copyright c David R. Wilkins 1989 2007 Contents 1 Basic Theorems of Real Analysis 1 1.1 The Least Upper Bound Principle................

### {f 1 (U), U F} is an open cover of A. Since A is compact there is a finite subcover of A, {f 1 (U 1 ),...,f 1 (U n )}, {U 1,...

44 CHAPTER 4. CONTINUOUS FUNCTIONS In Calculus we often use arithmetic operations to generate new continuous functions from old ones. In a general metric space we don t have arithmetic, but much of it

### Fixed Point Theorems

Fixed Point Theorems Definition: Let X be a set and let T : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation

### Notes on metric spaces

Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.

### k=1 k2, and therefore f(m + 1) = f(m) + (m + 1) 2 =

Math 104: Introduction to Analysis SOLUTIONS Alexander Givental HOMEWORK 1 1.1. Prove that 1 2 +2 2 + +n 2 = 1 n(n+1)(2n+1) for all n N. 6 Put f(n) = n(n + 1)(2n + 1)/6. Then f(1) = 1, i.e the theorem

### The Mean Value Theorem

The Mean Value Theorem THEOREM (The Extreme Value Theorem): If f is continuous on a closed interval [a, b], then f attains an absolute maximum value f(c) and an absolute minimum value f(d) at some numbers

### Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics

Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights

### TOPIC 3: CONTINUITY OF FUNCTIONS

TOPIC 3: CONTINUITY OF FUNCTIONS. Absolute value We work in the field of real numbers, R. For the study of the properties of functions we need the concept of absolute value of a number. Definition.. Let

### Metric Spaces. Chapter 1

Chapter 1 Metric Spaces Many of the arguments you have seen in several variable calculus are almost identical to the corresponding arguments in one variable calculus, especially arguments concerning convergence

### Practice with Proofs

Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Lectures 5-6: Taylor Series

Math 1d Instructor: Padraic Bartlett Lectures 5-: Taylor Series Weeks 5- Caltech 213 1 Taylor Polynomials and Series As we saw in week 4, power series are remarkably nice objects to work with. In particular,

### Taylor Polynomials and Taylor Series Math 126

Taylor Polynomials and Taylor Series Math 26 In many problems in science and engineering we have a function f(x) which is too complicated to answer the questions we d like to ask. In this chapter, we will

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### Sequences and Convergence in Metric Spaces

Sequences and Convergence in Metric Spaces Definition: A sequence in a set X (a sequence of elements of X) is a function s : N X. We usually denote s(n) by s n, called the n-th term of s, and write {s

### Metric Spaces Joseph Muscat 2003 (Last revised May 2009)

1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of

### 2 Complex Functions and the Cauchy-Riemann Equations

2 Complex Functions and the Cauchy-Riemann Equations 2.1 Complex functions In one-variable calculus, we study functions f(x) of a real variable x. Likewise, in complex analysis, we study functions f(z)

### x a x 2 (1 + x 2 ) n.

Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number

### Compactness in metric spaces

MATHEMATICS 3103 (Functional Analysis) YEAR 2012 2013, TERM 2 HANDOUT #2: COMPACTNESS OF METRIC SPACES Compactness in metric spaces The closed intervals [a, b] of the real line, and more generally the

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

### Numerical methods for finding the roots of a function

Numerical methods for finding the roots of a function The roots of a function f (x) are defined as the values for which the value of the function becomes equal to zero. So, finding the roots of f (x) means

### 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is:

CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### Course 421: Algebraic Topology Section 1: Topological Spaces

Course 421: Algebraic Topology Section 1: Topological Spaces David R. Wilkins Copyright c David R. Wilkins 1988 2008 Contents 1 Topological Spaces 1 1.1 Continuity and Topological Spaces...............

### Lecture notes on Ordinary Differential Equations. Plan of lectures

1 Lecture notes on Ordinary Differential Equations Annual Foundation School, IIT Kanpur, Dec.3-28, 2007. by Dept. of Mathematics, IIT Bombay, Mumbai-76. e-mail: sivaji.ganesh@gmail.com References Plan

### MATH 425, PRACTICE FINAL EXAM SOLUTIONS.

MATH 45, PRACTICE FINAL EXAM SOLUTIONS. Exercise. a Is the operator L defined on smooth functions of x, y by L u := u xx + cosu linear? b Does the answer change if we replace the operator L by the operator

### THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS

THE FUNDAMENTAL THEOREM OF ALGEBRA VIA PROPER MAPS KEITH CONRAD 1. Introduction The Fundamental Theorem of Algebra says every nonconstant polynomial with complex coefficients can be factored into linear

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### Differentiability and some of its consequences Definition: A function f : (a, b) R is differentiable at a point x 0 (a, b) if

Differentiability and some of its consequences Definition: A function f : (a, b) R is differentiable at a point x 0 (a, b) if f(x)) f(x 0 ) x x 0 exists. If the it exists for all x 0 (a, b) then f is said

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### Inner Product Spaces

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

### Limits and Continuity

Math 20C Multivariable Calculus Lecture Limits and Continuity Slide Review of Limit. Side limits and squeeze theorem. Continuous functions of 2,3 variables. Review: Limits Slide 2 Definition Given a function

### since by using a computer we are limited to the use of elementary arithmetic operations

> 4. Interpolation and Approximation Most functions cannot be evaluated exactly: x, e x, ln x, trigonometric functions since by using a computer we are limited to the use of elementary arithmetic operations

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### Learning Objectives for Math 165

Learning Objectives for Math 165 Chapter 2 Limits Section 2.1: Average Rate of Change. State the definition of average rate of change Describe what the rate of change does and does not tell us in a given

### ORDERS OF ELEMENTS IN A GROUP

ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### 6. Metric spaces. In this section we review the basic facts about metric spaces. d : X X [0, )

6. Metric spaces In this section we review the basic facts about metric spaces. Definitions. A metric on a non-empty set X is a map with the following properties: d : X X [0, ) (i) If x, y X are points

### n k=1 k=0 1/k! = e. Example 6.4. The series 1/k 2 converges in R. Indeed, if s n = n then k=1 1/k, then s 2n s n = 1 n + 1 +...

6 Series We call a normed space (X, ) a Banach space provided that every Cauchy sequence (x n ) in X converges. For example, R with the norm = is an example of Banach space. Now let (x n ) be a sequence

### 1 Fixed Point Iteration and Contraction Mapping Theorem

1 Fixed Point Iteration and Contraction Mapping Theorem Notation: For two sets A,B we write A B iff x A = x B. So A A is true. Some people use the notation instead. 1.1 Introduction Consider a function

### Chapter 6 SEQUENCES. sn 1+1,s 1 =1 is again an abbreviation for the sequence { 1, 1 2, 1 } n N ε,d (s n,s) <ε.

Chapter 6 SEQUENCES 6.1 Sequences A sequence is a function whose domain is the set of natural numbers N. If s is a sequence, we usually denote its value at n by s n instead of s (n). We may refer to the

### Lecture 3: Fourier Series: pointwise and uniform convergence.

Lecture 3: Fourier Series: pointwise and uniform convergence. 1. Introduction. At the end of the second lecture we saw that we had for each function f L ([, π]) a Fourier series f a + (a k cos kx + b k

### SOLUTIONS TO EXERCISES FOR. MATHEMATICS 205A Part 3. Spaces with special properties

SOLUTIONS TO EXERCISES FOR MATHEMATICS 205A Part 3 Fall 2008 III. Spaces with special properties III.1 : Compact spaces I Problems from Munkres, 26, pp. 170 172 3. Show that a finite union of compact subspaces

### Definition of Vertical Asymptote The line x = a is called a vertical asymptote of f (x) if at least one of the following is true: f (x) =

Vertical Asymptotes Definition of Vertical Asymptote The line x = a is called a vertical asymptote of f (x) if at least one of the following is true: lim f (x) = x a lim f (x) = lim x a lim f (x) = x a

### Chapter 2 Limits Functions and Sequences sequence sequence Example

Chapter Limits In the net few chapters we shall investigate several concepts from calculus, all of which are based on the notion of a limit. In the normal sequence of mathematics courses that students

### MTH4100 Calculus I. Lecture notes for Week 8. Thomas Calculus, Sections 4.1 to 4.4. Rainer Klages

MTH4100 Calculus I Lecture notes for Week 8 Thomas Calculus, Sections 4.1 to 4.4 Rainer Klages School of Mathematical Sciences Queen Mary University of London Autumn 2009 Theorem 1 (First Derivative Theorem

### LINE INTEGRALS OF VECTOR FUNCTIONS: GREEN S THEOREM. Contents. 2. Green s Theorem 3

LINE INTEGRALS OF VETOR FUNTIONS: GREEN S THEOREM ontents 1. A differential criterion for conservative vector fields 1 2. Green s Theorem 3 1. A differential criterion for conservative vector fields We

### MATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION. Chapter 4: Fourier Series and L 2 ([ π, π], µ) ( 1 π

MATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION Chapter 4: Fourier Series and L ([, π], µ) Square Integrable Functions Definition. Let f : [, π] R be measurable. We say that f

### Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011

Basic Concepts of Point Set Topology Notes for OU course Math 4853 Spring 2011 A. Miller 1. Introduction. The definitions of metric space and topological space were developed in the early 1900 s, largely

### ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING. 0. Introduction

ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING Abstract. In [1] there was proved a theorem concerning the continuity of the composition mapping, and there was announced a theorem on sequential continuity

### No: 10 04. Bilkent University. Monotonic Extension. Farhad Husseinov. Discussion Papers. Department of Economics

No: 10 04 Bilkent University Monotonic Extension Farhad Husseinov Discussion Papers Department of Economics The Discussion Papers of the Department of Economics are intended to make the initial results

### A Note on Di erential Calculus in R n by James Hebda August 2010.

A Note on Di erential Calculus in n by James Hebda August 2010 I. Partial Derivatives o Functions Let : U! be a real valued unction deined in an open neighborhood U o the point a =(a 1,...,a n ) in the

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Solutions to Homework 10

Solutions to Homework 1 Section 7., exercise # 1 (b,d): (b) Compute the value of R f dv, where f(x, y) = y/x and R = [1, 3] [, 4]. Solution: Since f is continuous over R, f is integrable over R. Let x

### Representation of functions as power series

Representation of functions as power series Dr. Philippe B. Laval Kennesaw State University November 9, 008 Abstract This document is a summary of the theory and techniques used to represent functions

### Analysis via Uniform Error Bounds Hermann Karcher, Bonn. Version May 2001

Analysis via Uniform Error Bounds Hermann Karcher, Bonn. Version May 2001 The standard analysis route proceeds as follows: 1.) Convergent sequences and series, completeness of the reals, 2.) Continuity,

### Problem Set. Problem Set #2. Math 5322, Fall December 3, 2001 ANSWERS

Problem Set Problem Set #2 Math 5322, Fall 2001 December 3, 2001 ANSWERS i Problem 1. [Problem 18, page 32] Let A P(X) be an algebra, A σ the collection of countable unions of sets in A, and A σδ the collection

### Properties of BMO functions whose reciprocals are also BMO

Properties of BMO functions whose reciprocals are also BMO R. L. Johnson and C. J. Neugebauer The main result says that a non-negative BMO-function w, whose reciprocal is also in BMO, belongs to p> A p,and

### CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.

CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,

### CONTINUED FRACTIONS AND PELL S EQUATION. Contents 1. Continued Fractions 1 2. Solution to Pell s Equation 9 References 12

CONTINUED FRACTIONS AND PELL S EQUATION SEUNG HYUN YANG Abstract. In this REU paper, I will use some important characteristics of continued fractions to give the complete set of solutions to Pell s equation.

### (Refer Slide Time: 00:00:56 min)

Numerical Methods and Computation Prof. S.R.K. Iyengar Department of Mathematics Indian Institute of Technology, Delhi Lecture No # 3 Solution of Nonlinear Algebraic Equations (Continued) (Refer Slide

### Change of Continuous Random Variable

Change of Continuous Random Variable All you are responsible for from this lecture is how to implement the Engineer s Way (see page 4) to compute how the probability density function changes when we make

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### MATH 2300 review problems for Exam 3 ANSWERS

MATH 300 review problems for Exam 3 ANSWERS. Check whether the following series converge or diverge. In each case, justify your answer by either computing the sum or by by showing which convergence test

### INTRODUCTION TO THE CONVERGENCE OF SEQUENCES

INTRODUCTION TO THE CONVERGENCE OF SEQUENCES BECKY LYTLE Abstract. In this paper, we discuss the basic ideas involved in sequences and convergence. We start by defining sequences and follow by explaining

### Functional analysis and its applications

Department of Mathematics, London School of Economics Functional analysis and its applications Amol Sasane ii Introduction Functional analysis plays an important role in the applied sciences as well as

### UNIT - I LESSON - 1 The Solution of Numerical Algebraic and Transcendental Equations

UNIT - I LESSON - 1 The Solution of Numerical Algebraic and Transcendental Equations Contents: 1.0 Aims and Objectives 1.1 Introduction 1.2 Bisection Method 1.2.1 Definition 1.2.2 Computation of real root

### 2.1 Functions. 2.1 J.A.Beachy 1. from A Study Guide for Beginner s by J.A.Beachy, a supplement to Abstract Algebra by Beachy / Blair

2.1 J.A.Beachy 1 2.1 Functions from A Study Guide for Beginner s by J.A.Beachy, a supplement to Abstract Algebra by Beachy / Blair 21. The Vertical Line Test from calculus says that a curve in the xy-plane

### Prof. Girardi, Math 703, Fall 2012 Homework Solutions: Homework 13. in X is Cauchy.

Homework 13 Let (X, d) and (Y, ρ) be metric spaces. Consider a function f : X Y. 13.1. Prove or give a counterexample. f preserves convergent sequences if {x n } n=1 is a convergent sequence in X then

### CHAPTER 3. Sequences. 1. Basic Properties

CHAPTER 3 Sequences We begin our study of analysis with sequences. There are several reasons for starting here. First, sequences are the simplest way to introduce limits, the central idea of calculus.

### FACTORING POLYNOMIALS IN THE RING OF FORMAL POWER SERIES OVER Z

FACTORING POLYNOMIALS IN THE RING OF FORMAL POWER SERIES OVER Z DANIEL BIRMAJER, JUAN B GIL, AND MICHAEL WEINER Abstract We consider polynomials with integer coefficients and discuss their factorization

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### BOUNDED, ASYMPTOTICALLY STABLE, AND L 1 SOLUTIONS OF CAPUTO FRACTIONAL DIFFERENTIAL EQUATIONS. Muhammad N. Islam

Opuscula Math. 35, no. 2 (215), 181 19 http://dx.doi.org/1.7494/opmath.215.35.2.181 Opuscula Mathematica BOUNDED, ASYMPTOTICALLY STABLE, AND L 1 SOLUTIONS OF CAPUTO FRACTIONAL DIFFERENTIAL EQUATIONS Muhammad

### Fourier Series Chapter 3 of Coleman

Fourier Series Chapter 3 of Coleman Dr. Doreen De eon Math 18, Spring 14 1 Introduction Section 3.1 of Coleman The Fourier series takes its name from Joseph Fourier (1768-183), who made important contributions

### Notes on weak convergence (MAT Spring 2006)

Notes on weak convergence (MAT4380 - Spring 2006) Kenneth H. Karlsen (CMA) February 2, 2006 1 Weak convergence In what follows, let denote an open, bounded, smooth subset of R N with N 2. We assume 1 p

### Microeconomic Theory: Basic Math Concepts

Microeconomic Theory: Basic Math Concepts Matt Van Essen University of Alabama Van Essen (U of A) Basic Math Concepts 1 / 66 Basic Math Concepts In this lecture we will review some basic mathematical concepts

### Math 2602 Finite and Linear Math Fall 14. Homework 9: Core solutions

Math 2602 Finite and Linear Math Fall 14 Homework 9: Core solutions Section 8.2 on page 264 problems 13b, 27a-27b. Section 8.3 on page 275 problems 1b, 8, 10a-10b, 14. Section 8.4 on page 279 problems

### Sets and functions. {x R : x > 0}.

Sets and functions 1 Sets The language of sets and functions pervades mathematics, and most of the important operations in mathematics turn out to be functions or to be expressible in terms of functions.

### N E W S A N D L E T T E R S

N E W S A N D L E T T E R S 73rd Annual William Lowell Putnam Mathematical Competition Editor s Note: Additional solutions will be printed in the Monthly later in the year. PROBLEMS A1. Let d 1, d,...,

### Lecture 3: Solving Equations Using Fixed Point Iterations

cs41: introduction to numerical analysis 09/14/10 Lecture 3: Solving Equations Using Fixed Point Iterations Instructor: Professor Amos Ron Scribes: Yunpeng Li, Mark Cowlishaw, Nathanael Fillmore Our problem,

### 3.7 Non-autonomous linear systems of ODE. General theory

3.7 Non-autonomous linear systems of ODE. General theory Now I will study the ODE in the form ẋ = A(t)x + g(t), x(t) R k, A, g C(I), (3.1) where now the matrix A is time dependent and continuous on some

### correct-choice plot f(x) and draw an approximate tangent line at x = a and use geometry to estimate its slope comment The choices were:

Topic 1 2.1 mode MultipleSelection text How can we approximate the slope of the tangent line to f(x) at a point x = a? This is a Multiple selection question, so you need to check all of the answers that

### Module MA1S11 (Calculus) Michaelmas Term 2016 Section 3: Functions

Module MA1S11 (Calculus) Michaelmas Term 2016 Section 3: Functions D. R. Wilkins Copyright c David R. Wilkins 2016 Contents 3 Functions 43 3.1 Functions between Sets...................... 43 3.2 Injective

### Mathematics Pre-Test Sample Questions A. { 11, 7} B. { 7,0,7} C. { 7, 7} D. { 11, 11}

Mathematics Pre-Test Sample Questions 1. Which of the following sets is closed under division? I. {½, 1,, 4} II. {-1, 1} III. {-1, 0, 1} A. I only B. II only C. III only D. I and II. Which of the following

### DIFFERENTIABILITY OF COMPLEX FUNCTIONS. Contents

DIFFERENTIABILITY OF COMPLEX FUNCTIONS Contents 1. Limit definition of a derivative 1 2. Holomorphic functions, the Cauchy-Riemann equations 3 3. Differentiability of real functions 5 4. A sufficient condition

### MATH301 Real Analysis Tutorial Note #3

MATH301 Real Analysis Tutorial Note #3 More Differentiation in Vector-valued function: Last time, we learn how to check the differentiability of a given vector-valued function. Recall a function F: is

### Infinite series, improper integrals, and Taylor series

Chapter Infinite series, improper integrals, and Taylor series. Introduction This chapter has several important and challenging goals. The first of these is to understand how concepts that were discussed

### Problem 1 (10 pts) Find the radius of convergence and interval of convergence of the series

1 Problem 1 (10 pts) Find the radius of convergence and interval of convergence of the series a n n=1 n(x + 2) n 5 n 1. n(x + 2)n Solution: Do the ratio test for the absolute convergence. Let a n =. Then,

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### Linear and quadratic Taylor polynomials for functions of several variables.

ams/econ 11b supplementary notes ucsc Linear quadratic Taylor polynomials for functions of several variables. c 010, Yonatan Katznelson Finding the extreme (minimum or maximum) values of a function, is

### 4.3 Lagrange Approximation

206 CHAP. 4 INTERPOLATION AND POLYNOMIAL APPROXIMATION Lagrange Polynomial Approximation 4.3 Lagrange Approximation Interpolation means to estimate a missing function value by taking a weighted average

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### Max-Min Representation of Piecewise Linear Functions

Beiträge zur Algebra und Geometrie Contributions to Algebra and Geometry Volume 43 (2002), No. 1, 297-302. Max-Min Representation of Piecewise Linear Functions Sergei Ovchinnikov Mathematics Department,