# 1 Fixed Point Iteration and Contraction Mapping Theorem

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 1 Fixed Point Iteration and Contraction Mapping Theorem Notation: For two sets A,B we write A B iff x A = x B. So A A is true. Some people use the notation instead. 1.1 Introduction Consider a function y = g(x) where x,y R n : y 1. y n g 1 (x 1,...,x n ) =. g n (x 1,...,x n ) We assume that g(x) is defined for x D where D is a subset of R n. The goal is to find a solution x of the fixed point equation g(x) = x. A method to find x is the fixed point iteration: Pick an initial guess x () D and define for k =,1,2,... x (k+1) := g(x (k) ) Note that this may not converge. But if the sequence x (k) converges, and the function g is continuous, the limit x must be a solution of the fixed point equation. 1.2 Contraction Mapping Theorem The following theorem is called Contraction Mapping Theorem or Banach Fixed Point Theorem. Theorem 1. Consider a set D R n and a function g: D R n. Assume 1. D is closed (i.e., it contains all limit points of sequences in D) 2. x D = g(x) D 3. The mapping g is a contraction on D: There exists q < 1 such that x,y D: g(x) g(y) q x y (1) Then 1. there exists a unique x D with g(x ) = x 2. for any x () D the fixed point iterates given by x (k+1) := g(x (k) ) converge to x as k 3. x (k) satisfies the a-priori error estimate x (k) x qk x (1) x () (2) and the a-posteriori error estimate x (k) x q x (k) x (k 1) (3)

2 Proof. Pick x () D and define x (k) for k = 1,2,... by x (k) := g(x (k 1) ). We have from the contraction property (1) x (k+1) x (k) = g(x (k) ) g(x (k 1) ) q x (k) x (k 1) (4) and hence x (k+1) x (k) q k x (1) x () (5) Let d := x (1) x (). We have from the triangle inequality and (5) x (k) x (k+l) x (k) x (k+1) + + x (k+l 1) x (k+l) q k d + + q k+l 1 d = q k d (1 + q + + q l 1 ) x (k) x (k+l) q k d 1 using the sum of the geometric series l 1 j= q j j= q j = 1/(). Note that (6) shows that the sequence x (k) is a Cauchy sequence. Therefore it must converge to a limit x R n (since the space R n is complete). As D is closed, we must have x D. We need to show that x = g(x ): We have x (k+1) = g(x (k) ), hence lim k x(k+1) = lim g(x (k) ) k The limit of the left hand side is x. Note that because of (1) the function g must be continuous. Therefore lim k g(x(k) ) = g( lim x (k) ) = g(x ). k Next we need to show that the fixed point x is unique. Assume that we have fixed points x = g(x ) and y = g(y ). Then we obtain using the contraction property (1) x y = g(x ) g(y ) q x y implying () x y and therefore x y =, i.e., x = y. The a-priori estimate (2) follows from (6) by letting l tend to infinity. For the a-posteriori estimate use (2) with k = 1 for x () := x (k), x (1) = x (k+1). (6) 1.3 Proving the Contraction Property The contraction property is related to the Jacobian g (x) which is an n n matrix for each point x D. If the matrix norm satisfies g (x) q < 1 then the mapping g must be a contraction: Theorem 2. Assume the set D R n is convex and the function g: D R n has continuous partial derivatives g j in D. If k for q < 1 the matrix norm of the Jacobian satisfies x D: g (x) q (7) the mapping g is a contraction in D and satisfies (1). Proof. Let x,y D. Then the points on the straight line from x to y are given by x +t(y x) for t [,1]. As D is convex all these points are contained in D. Let G(t) := g ( x +t(y x) ), then by the chain rule we have G (t) = g ( x +t(y x) ) (y x) and 1 1 g(y) g(x) = G(1) G() = G (t)dt = g ( x +t(y x) ) (y x)dt As an integral of a continuous function is a limit of Riemann sums the triangle inequality implies b a F(t)dt b a F(t) dt: g(y) g(x) 1 g ( x +t(y x) ) (y x)dt 1 g ( x +t(y x) ) y x dt q y x }{{} q This is usually the easiest method to prove that a given mapping g is a contraction, see the examples in sections 1.5, 1.6.

3 1.4 A-priori and a-posteriori error estimates The error estimates (2), (3) are useful for figuring out how many iterations we need. For this we need to know the contraction constant q (typically we get this from (7)). A-priori estimate: For an initial guess x () we can find x (1). Without computing anything else we then have the error bound x (k) x qk x (1) x () for all future iterates x (k), before ( a-priori ) we actually compute them. We can e.g. 1 q use this to find a value k such that x (k) x is below a given tolerance. A-posteriori estimate: After we have actually computed x (k) ( a-posteriori ) we would like to know where the true solution x is located. Let δ k := q x (k) x (k 1), D k := {x x x (k) δ k } The a-posteriori estimate states that x is contained in the set D k. Note: the radius δ k of D k decreases at least by a factor of q with each iteration: δ k+1 qδ k the sets D k are nested: D 1 D 2 D 3 To show D k+1 D k assume x D k+1. Then x x (k) x x (k+1) + x (k+1) x (k) ( ) q x }{{} + 1 (k+1) x (k) (4) 1 q x (k) x (k 1) = δk (8) δ k+1 If we use the -norm: x (k) x δ k means that for each component x j we have a bracket x j [x (k) j δ k,x (k) j + δ k ], i.e., the set D k is a square/cube/hypercube with side length 2δ k centered in x (k). 1.5 Example We want to solve the nonlinear system where we have g(x) = 1 [ 1 x2 sin(x 1 + x 2 ) 2 + x 1 + cos(x 1 x 2 ) x 1 = 1 [1 x 2 sin(x 1 + x 2 )] x 2 = 1 [2 + x 1 + cos(x 1 x 2 )] ]. First we want to show that g is a contraction using Theorem 2. Therefore we first have to find the Jacobian g (x): g (x) = 1 cos(x1 + x 2 ) 1 cos(x 1 + x 2 ) 1 sin(x 1 x 2 ) sin(x 1 x 2 ) Let A := g (x). Let us use the -norm. We need to find an upper bound for A = max{ a 11 + a 12, a 21 + a 22 }. We obtain for any x 1,x 2 R a 11 = 1 cos(x 1 + x 2 ) 1, a 12 = 1 1 cos(x 1 + x 2 ) 1 (1 + 1) a 21 = 1 1 sin(x 1 x 2 ) 1 (1 + 1), a 22 1 sin(x 1 x 2 ) 1 Therefore for any x R 2 we have g (x) 3 = q < 1. By Theorem 2 we therefore obtain that g is a contraction for all of R 2. We now want to use Theorem 1. We need to pick a set D such that the three assumptions of the theorem are satisfied. We consider two choices:

4 First choice D = R 2 : We can use the set D = R 2. This set is closed. For any x R 2 we certainly have that g(x) R 2. We have also shown that g is a contraction for all of R 2. Therefore we obtain from Theorem 1 that the nonlinear system g(x) = x has exactly one solution x in all of R 2. Second choice D = [ 1,1] [ 1,1]: We can use for D the square with 1 x 1 1 and 1 x 2 1. This is a closed set (the boundary of the square is included). We now have to check that for x D we have that y = g(x) D: We have using 1 sinα 1, 1 cosα 1 2 = 1 (1 1 1) y 1 = 1 [1 x 2 sin(x 1 + x 2 )] 1 3 ( ) = = 1 (2 1 1) y 2 = 1 [2 + x 1 + cos(x 1 x 2 )] 1 4 ( ) = therefore y D and the second assumption of the theorem is satisfied. We already showed that g is a contraction for all of R 2, so the third assumption definitely holds for x,y D. We can now apply Theorem 1 and obtain that the nonlinear system has exactly one solution x which is located in the square D = [ 1,1] [ 1,1]. Numerical Computation: containing x : We start with the initial guess x () = (,). After each iteration we find δ k and the square D k k x (k) δ k D k 1 (.1,.3) [.2857,.2286] [.1714,.4286] 2 (.36,.38) 3. 2 [.151,.66] [.2785,.3376] 3 (.3594,.2993) [.3221,.3967] [.2956,.33] 4 (.3717,.31) [.3664,.377] [.2996,.37] 5 (.3689,.33) [.3677,.371] [.31,.34] Note: (i) δ k decreases at least by a factor of q =.3 with each iteration. (ii) The sets D k are nested: D 1 D 2 D Using the Fixed Point Theorem without the Assumption g(d) D The tricky part in using the contraction mapping theorem is to find a set D for which both the 2nd and 3rd assumption of the fixed point theorem hold: x D = g(x) D g is a contraction on D Typically we can prove that g (x) q < 1 for x in some convex region D. We suspect that there is a solution x of the fixed point equation in D. But it may not be true that g(x) D for all x D. In this case we may be able to prove a result by computing a few iterates x (k) : Start with k = and an initial guess x () D. Then repeat let k := k + 1 and compute x (k) := g(x (k 1) ) compute δ k := q x (k) x (k 1), let D k := {x x x (k) δ k } until either D k D or x (k) / D. If the iterates exit from the set D we cannot conclude anything. But as long as the points x (k) stay inside D we have δ k+1 qδ k and D k+1 D k. So we expect that for some k the condition D k D will be satisfied (if x (k) converges to a limit in the interior of D the loop must terminate with D k D; but in general it is possible that the loop never terminates). If the loop does terminate with D k D for k = K we have the following result:

5 Theorem 3. Let D R n and assume that the function g: D R n satisfies for q < 1 x,y D : g(x) g(y) q x y Let x () D and define for k =,1,2,... x (k+1) := g(x (k) ), δ k := q x (k) x (k 1), D k := {x x x (k) δ k } If for some K we have x (K 1) D and D K D there holds the equation g(x) = x has a unique solution x in D this solution satisfies x D k for all k K Proof. Let x D K. We want to show that g(x) D K : As D K D the contraction property gives using the definition of D k and δ k g(x) x (K) q x x (K 1) q x x (K) + q x (K) x (K 1) qδ K + ()δ K = δ K As D K is closed and D K D the set D := D K satisfies all three assumptions of the fixed point theorem Theorem 1. Hence there is a unique solution x D. The a-posteriori estimate (3) states that x D k for all iterates x (k) with k K. Assume that there is another fixed point y D with g(y ) = y. Then As q < 1 we must have y x =. Summary: y x = g(y ) g(x ) q y x Find a convex set D for which you suspect x D and where you can show g (x) q < 1 Pick x () D and perform the fixed point iteration: for each iteration: find x (k) and D k if x (k) / D: stop (we can t conclude anything) if D k D: success: there is a unique solution x D, and there holds x D k for this and all following iterations Example: Let g(x) := 1 x1 x 1 x x 2 + x 1 x Then the Jacobian is g (x) = 1 1 x2 x 1 3 x x 1 x 2 Let us try to use D = [,a] [.a] with a 1 and the -norm. We then obtain for x D that g (x) 1 3 max{1 + a,a a 2 } For a = 1 we get g (x) 4 3 which is too large. So we try a =.6 which gives g (x) =: q < 1. Therefore g is a contraction on D = [,.6] [,.6]. Note that g( ) = / D, so D does not satisfy all three assumptions of Theorem 1. For x () = we obtain x (1) = (.33333,.33333) D, x (2) = (.4741,.45679) D, x (3) = (.47,.51393) D, x (4) = (.39929,.5449) D, x (5) = (.39449,.55238) D, D 1 = [.4229, ] [.4229, ] D D 2 = [.12829,.68653] [.17767,.73591] D D 3 = [.27791,.53629] [.38474,.64313] D D 4 = [.33926,.45933] [.4845,.652] D D 5 = [.36761,.42138] [.52549,.57926] D Therefore we can conclude from Theorem 3 that there exists a unique solution x D = [,.6] [,.6]. This solution x is located in the smaller square D 5. For k = 5,6,7,... we obtain x D k where D k is a square with side length 2δ k. As δ k q k 5 δ 5 ( ) 2.8 k we can obtain arbitrarily small squares containing the solution if we choose k sufficiently large.

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Analysis MA131. University of Warwick. Term

Analysis MA131 University of Warwick Term 1 01 13 September 8, 01 Contents 1 Inequalities 5 1.1 What are Inequalities?........................ 5 1. Using Graphs............................. 6 1.3 Case

### 1 Norms and Vector Spaces

008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)

### k=1 k2, and therefore f(m + 1) = f(m) + (m + 1) 2 =

Math 104: Introduction to Analysis SOLUTIONS Alexander Givental HOMEWORK 1 1.1. Prove that 1 2 +2 2 + +n 2 = 1 n(n+1)(2n+1) for all n N. 6 Put f(n) = n(n + 1)(2n + 1)/6. Then f(1) = 1, i.e the theorem

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### THE BANACH CONTRACTION PRINCIPLE. Contents

THE BANACH CONTRACTION PRINCIPLE ALEX PONIECKI Abstract. This paper will study contractions of metric spaces. To do this, we will mainly use tools from topology. We will give some examples of contractions,

### Lecture 3: Solving Equations Using Fixed Point Iterations

cs41: introduction to numerical analysis 09/14/10 Lecture 3: Solving Equations Using Fixed Point Iterations Instructor: Professor Amos Ron Scribes: Yunpeng Li, Mark Cowlishaw, Nathanael Fillmore Our problem,

### v 1. v n R n we have for each 1 j n that v j v n max 1 j n v j. i=1

1. Limits and Continuity It is often the case that a non-linear function of n-variables x = (x 1,..., x n ) is not really defined on all of R n. For instance f(x 1, x 2 ) = x 1x 2 is not defined when x

### Section 4.7. More on Area

Difference Equations to Differential Equations Section 4.7 More on Area In Section 4. we motivated the definition of the definite integral with the idea of finding the area of a region in the plane. However,

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### Markov Chains, part I

Markov Chains, part I December 8, 2010 1 Introduction A Markov Chain is a sequence of random variables X 0, X 1,, where each X i S, such that P(X i+1 = s i+1 X i = s i, X i 1 = s i 1,, X 0 = s 0 ) = P(X

### n k=1 k=0 1/k! = e. Example 6.4. The series 1/k 2 converges in R. Indeed, if s n = n then k=1 1/k, then s 2n s n = 1 n + 1 +...

6 Series We call a normed space (X, ) a Banach space provided that every Cauchy sequence (x n ) in X converges. For example, R with the norm = is an example of Banach space. Now let (x n ) be a sequence

### 0 ( x) 2 = ( x)( x) = (( 1)x)(( 1)x) = ((( 1)x))( 1))x = ((( 1)(x( 1)))x = ((( 1)( 1))x)x = (1x)x = xx = x 2.

SOLUTION SET FOR THE HOMEWORK PROBLEMS Page 5. Problem 8. Prove that if x and y are real numbers, then xy x + y. Proof. First we prove that if x is a real number, then x 0. The product of two positive

### 0 <β 1 let u(x) u(y) kuk u := sup u(x) and [u] β := sup

456 BRUCE K. DRIVER 24. Hölder Spaces Notation 24.1. Let Ω be an open subset of R d,bc(ω) and BC( Ω) be the bounded continuous functions on Ω and Ω respectively. By identifying f BC( Ω) with f Ω BC(Ω),

### A sequence of real numbers or a sequence in R is a mapping f : N R.

A sequence of real numbers or a sequence in R is a mapping f : N R. A sequence of real numbers or a sequence in R is a mapping f : N R. Notation: We write x n for f(n), n N and so the notation for a sequence

### Second Project for Math 377, fall, 2003

Second Project for Math 377, fall, 003 You get to pick your own project. Several possible topics are described below or you can come up with your own topic (subject to my approval). At most two people

### MATHEMATICAL BACKGROUND

Chapter 1 MATHEMATICAL BACKGROUND This chapter discusses the mathematics that is necessary for the development of the theory of linear programming. We are particularly interested in the solutions of a

### Analysis via Uniform Error Bounds Hermann Karcher, Bonn. Version May 2001

Analysis via Uniform Error Bounds Hermann Karcher, Bonn. Version May 2001 The standard analysis route proceeds as follows: 1.) Convergent sequences and series, completeness of the reals, 2.) Continuity,

### 3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field

3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field 77 3.8 Finding Antiderivatives; Divergence and Curl of a Vector Field Overview: The antiderivative in one variable calculus is an important

### Nonlinear Algebraic Equations. Lectures INF2320 p. 1/88

Nonlinear Algebraic Equations Lectures INF2320 p. 1/88 Lectures INF2320 p. 2/88 Nonlinear algebraic equations When solving the system u (t) = g(u), u(0) = u 0, (1) with an implicit Euler scheme we have

### Limits and Continuity

Math 20C Multivariable Calculus Lecture Limits and Continuity Slide Review of Limit. Side limits and squeeze theorem. Continuous functions of 2,3 variables. Review: Limits Slide 2 Definition Given a function

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### Fixed Point Theorems

Fixed Point Theorems Definition: Let X be a set and let T : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation

### Taylor Polynomials and Taylor Series Math 126

Taylor Polynomials and Taylor Series Math 26 In many problems in science and engineering we have a function f(x) which is too complicated to answer the questions we d like to ask. In this chapter, we will

### Math 5311 Gateaux differentials and Frechet derivatives

Math 5311 Gateaux differentials and Frechet derivatives Kevin Long January 26, 2009 1 Differentiation in vector spaces Thus far, we ve developed the theory of minimization without reference to derivatives.

### Theorem 2. If x Q and y R \ Q, then. (a) x + y R \ Q, and. (b) xy Q.

Math 305 Fall 011 The Density of Q in R The following two theorems tell us what happens when we add and multiply by rational numbers. For the first one, we see that if we add or multiply two rational numbers

### be a nested sequence of closed nonempty connected subsets of a compact metric space X. Prove that

Problem 1A. Let... X 2 X 1 be a nested sequence of closed nonempty connected subsets of a compact metric space X. Prove that i=1 X i is nonempty and connected. Since X i is closed in X, it is compact.

### Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning

Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning slides: http://www.gatsby.ucl.ac.uk/~szabo/ teaching.html Gatsby Unit March 22, 2016 Overview 1 Concepts from functional

### Solutions of Equations in One Variable. Fixed-Point Iteration II

Solutions of Equations in One Variable Fixed-Point Iteration II Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### 4.3 Limit of a Sequence: Theorems

4.3. LIMIT OF A SEQUENCE: THEOREMS 5 4.3 Limit of a Sequence: Theorems These theorems fall in two categories. The first category deals with ways to combine sequences. Like numbers, sequences can be added,

### Course 221: Analysis Academic year , First Semester

Course 221: Analysis Academic year 2007-08, First Semester David R. Wilkins Copyright c David R. Wilkins 1989 2007 Contents 1 Basic Theorems of Real Analysis 1 1.1 The Least Upper Bound Principle................

### 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is:

CONVERGENCE OF FOURIER SERIES 1. Periodic Fourier series. The Fourier expansion of a 2π-periodic function f is: with coefficients given by: a n = 1 π f(x) a 0 2 + a n cos(nx) + b n sin(nx), n 1 f(x) cos(nx)dx

### LINE INTEGRALS OF VECTOR FUNCTIONS: GREEN S THEOREM. Contents. 2. Green s Theorem 3

LINE INTEGRALS OF VETOR FUNTIONS: GREEN S THEOREM ontents 1. A differential criterion for conservative vector fields 1 2. Green s Theorem 3 1. A differential criterion for conservative vector fields We

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Metric Spaces Joseph Muscat 2003 (Last revised May 2009)

1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of

### Math 2602 Finite and Linear Math Fall 14. Homework 9: Core solutions

Math 2602 Finite and Linear Math Fall 14 Homework 9: Core solutions Section 8.2 on page 264 problems 13b, 27a-27b. Section 8.3 on page 275 problems 1b, 8, 10a-10b, 14. Section 8.4 on page 279 problems

### CHAPTER IV NORMED LINEAR SPACES AND BANACH SPACES

CHAPTER IV NORMED LINEAR SPACES AND BANACH SPACES DEFINITION A Banach space is a real normed linear space that is a complete metric space in the metric defined by its norm. A complex Banach space is a

### MATH 425, PRACTICE FINAL EXAM SOLUTIONS.

MATH 45, PRACTICE FINAL EXAM SOLUTIONS. Exercise. a Is the operator L defined on smooth functions of x, y by L u := u xx + cosu linear? b Does the answer change if we replace the operator L by the operator

### 1. Please write your name in the blank above, and sign & date below. 2. Please use the space provided to write your solution.

Name : Instructor: Marius Ionescu Instructions: 1. Please write your name in the blank above, and sign & date below. 2. Please use the space provided to write your solution. 3. If you need extra pages

### The Power Method for Eigenvalues and Eigenvectors

Numerical Analysis Massoud Malek The Power Method for Eigenvalues and Eigenvectors The spectrum of a square matrix A, denoted by σ(a) is the set of all eigenvalues of A. The spectral radius of A, denoted

### Iterative Techniques in Matrix Algebra. Jacobi & Gauss-Seidel Iterative Techniques II

Iterative Techniques in Matrix Algebra Jacobi & Gauss-Seidel Iterative Techniques II Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin

### TOPIC 3: CONTINUITY OF FUNCTIONS

TOPIC 3: CONTINUITY OF FUNCTIONS. Absolute value We work in the field of real numbers, R. For the study of the properties of functions we need the concept of absolute value of a number. Definition.. Let

### The Real Numbers. Here we show one way to explicitly construct the real numbers R. First we need a definition.

The Real Numbers Here we show one way to explicitly construct the real numbers R. First we need a definition. Definitions/Notation: A sequence of rational numbers is a funtion f : N Q. Rather than write

### Solving nonlinear equations in one variable

Chapter Solving nonlinear equations in one variable Contents.1 Bisection method............................. 7. Fixed point iteration........................... 8.3 Newton-Raphson method........................

### 3.1. Sequences and Their Limits Definition (3.1.1). A sequence of real numbers (or a sequence in R) is a function from N into R.

CHAPTER 3 Sequences and Series 3.. Sequences and Their Limits Definition (3..). A sequence of real numbers (or a sequence in R) is a function from N into R. Notation. () The values of X : N! R are denoted

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### 21st Annual Iowa Collegiate Mathematics Competition ISU, Saturday, February 21, 2015 Problems by Razvan Gelca

21st Annual Iowa Collegiate Mathematics Competition ISU, Saturday, February 21, 215 Problems by Razvan Gelca SOLUTIONS 1. Asked about his age, a boy replied: I have a sister, and four years ago when she

### x a x 2 (1 + x 2 ) n.

Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number

### Numerical methods for finding the roots of a function

Numerical methods for finding the roots of a function The roots of a function f (x) are defined as the values for which the value of the function becomes equal to zero. So, finding the roots of f (x) means

### Notes on weak convergence (MAT Spring 2006)

Notes on weak convergence (MAT4380 - Spring 2006) Kenneth H. Karlsen (CMA) February 2, 2006 1 Weak convergence In what follows, let denote an open, bounded, smooth subset of R N with N 2. We assume 1 p

### Math 230a HW3 Extra Credit

Math 230a HW3 Extra Credit Erik Lewis November 15, 2006 1 Extra Credit 16. Regard Q, the set of all rational numbers, as a metric space, with d(p, q) = p q. Let E be the set of all p Q such that 2 < p

### Section 3 Sequences and Limits

Section 3 Sequences and Limits Definition A sequence of real numbers is an infinite ordered list a, a 2, a 3, a 4,... where, for each n N, a n is a real number. We call a n the n-th term of the sequence.

### CHAPTER 2. Inequalities

CHAPTER 2 Inequalities In this section we add the axioms describe the behavior of inequalities (the order axioms) to the list of axioms begun in Chapter 1. A thorough mastery of this section is essential

### Homework # 3 Solutions

Homework # 3 Solutions February, 200 Solution (2.3.5). Noting that and ( + 3 x) x 8 = + 3 x) by Equation (2.3.) x 8 x 8 = + 3 8 by Equations (2.3.7) and (2.3.0) =3 x 8 6x2 + x 3 ) = 2 + 6x 2 + x 3 x 8

### Practice Problems Solutions

Practice Problems Solutions The problems below have been carefully selected to illustrate common situations and the techniques and tricks to deal with these. Try to master them all; it is well worth it!

### Notes: Chapter 2 Section 2.2: Proof by Induction

Notes: Chapter 2 Section 2.2: Proof by Induction Basic Induction. To prove: n, a W, n a, S n. (1) Prove the base case - S a. (2) Let k a and prove that S k S k+1 Example 1. n N, n i = n(n+1) 2. Example

### Math 317 HW #5 Solutions

Math 317 HW #5 Solutions 1. Exercise 2.4.2. (a) Prove that the sequence defined by x 1 = 3 and converges. x n+1 = 1 4 x n Proof. I intend to use the Monotone Convergence Theorem, so my goal is to show

### COS 511: Foundations of Machine Learning. Rob Schapire Lecture #14 Scribe: Qian Xi March 30, 2006

COS 511: Foundations of Machine Learning Rob Schapire Lecture #14 Scribe: Qian Xi March 30, 006 In the previous lecture, we introduced a new learning model, the Online Learning Model. After seeing a concrete

### Chapter 2 Limits Functions and Sequences sequence sequence Example

Chapter Limits In the net few chapters we shall investigate several concepts from calculus, all of which are based on the notion of a limit. In the normal sequence of mathematics courses that students

### P -adic numbers And Bruhat-Tits Tree

P -adic numbers And Bruhat-Tits Tree In the first part of these notes we give a brief introduction to p-adic numbers. In the second part we discuss some properties of the Bruhat-Tits tree. It is mostly

### Convergence and stability results for a class of asymptotically quasi-nonexpansive mappings in the intermediate sense

Functional Analysis, Approximation and Computation 7 1) 2015), 57 66 Published by Faculty of Sciences and Mathematics, University of Niš, Serbia Available at: http://www.pmf.ni.ac.rs/faac Convergence and

### Power Series. We already know how to express one function as a series. Take a look at following equation: 1 1 r = r n. n=0

Math -0 Calc Power, Taylor, and Maclaurin Series Survival Guide One of the harder concepts that we have to become comfortable with during this semester is that of sequences and series We are working with

### Lecture 5 Principal Minors and the Hessian

Lecture 5 Principal Minors and the Hessian Eivind Eriksen BI Norwegian School of Management Department of Economics October 01, 2010 Eivind Eriksen (BI Dept of Economics) Lecture 5 Principal Minors and

### Section 3 Sequences and Limits, Continued.

Section 3 Sequences and Limits, Continued. Lemma 3.6 Let {a n } n N be a convergent sequence for which a n 0 for all n N and it α 0. Then there exists N N such that for all n N. α a n 3 α In particular

### Scalar Valued Functions of Several Variables; the Gradient Vector

Scalar Valued Functions of Several Variables; the Gradient Vector Scalar Valued Functions vector) valued function of n variables: Let us consider a scalar (i.e., numerical, rather than y = φ(x) = φ(x 1,

### Math 140a - HW 1 Solutions

Math 140a - HW 1 Solutions Problem 1 (WR Ch 1 #1). If r is rational (r 0) and x is irrational, prove that r + x and rx are irrational. Solution. Given that r is rational, we can write r = a b for some

### Riesz-Fredhölm Theory

Riesz-Fredhölm Theory T. Muthukumar tmk@iitk.ac.in Contents 1 Introduction 1 2 Integral Operators 1 3 Compact Operators 7 4 Fredhölm Alternative 14 Appendices 18 A Ascoli-Arzelá Result 18 B Normed Spaces

### CHAPTER 3. Sequences. 1. Basic Properties

CHAPTER 3 Sequences We begin our study of analysis with sequences. There are several reasons for starting here. First, sequences are the simplest way to introduce limits, the central idea of calculus.

### Triangle. A triangle is a geometrical figure. Tri means three. So Triangle is a geometrical figure having 3 angles.

Triangle A triangle is a geometrical figure. Tri means three. So Triangle is a geometrical figure having 3 angles. A triangle is consisting of three line segments linked end to end. As the figure linked

### Selected problems and solutions

Selected problems and solutions 1. Prove that (0, 1) = R. Solution. Define f : (0, 1) R by 1 x 0.5 + 2, x < 0.5 f(x) = 1 x 0.5 2, x > 0.5 0, x = 0.5 Note that the image of (0, 0.5) under f is (, 0), the

### Lecture 1 Newton s method.

Lecture 1 Newton s method. 1 Square roots 2 Newton s method. The guts of the method. A vector version. Implementation. The existence theorem. Basins of attraction. The Babylonian algorithm for finding

### Chapter 5: Application: Fourier Series

321 28 9 Chapter 5: Application: Fourier Series For lack of time, this chapter is only an outline of some applications of Functional Analysis and some proofs are not complete. 5.1 Definition. If f L 1

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### MATH SOLUTIONS TO PRACTICE FINAL EXAM. (x 2)(x + 2) (x 2)(x 3) = x + 2. x 2 x 2 5x + 6 = = 4.

MATH 55 SOLUTIONS TO PRACTICE FINAL EXAM x 2 4.Compute x 2 x 2 5x + 6. When x 2, So x 2 4 x 2 5x + 6 = (x 2)(x + 2) (x 2)(x 3) = x + 2 x 3. x 2 4 x 2 x 2 5x + 6 = 2 + 2 2 3 = 4. x 2 9 2. Compute x + sin

### Lectures 5-6: Taylor Series

Math 1d Instructor: Padraic Bartlett Lectures 5-: Taylor Series Weeks 5- Caltech 213 1 Taylor Polynomials and Series As we saw in week 4, power series are remarkably nice objects to work with. In particular,

### Chapter 2 Sequences and Series

Chapter 2 Sequences and Series 2. Sequences A sequence is a function from the positive integers (possibly including 0) to the reals. A typical example is a n = /n defined for all integers n. The notation

### Prof. Girardi, Math 703, Fall 2012 Homework Solutions: Homework 13. in X is Cauchy.

Homework 13 Let (X, d) and (Y, ρ) be metric spaces. Consider a function f : X Y. 13.1. Prove or give a counterexample. f preserves convergent sequences if {x n } n=1 is a convergent sequence in X then

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### ANALYTICAL MATHEMATICS FOR APPLICATIONS 2016 LECTURE NOTES Series

ANALYTICAL MATHEMATICS FOR APPLICATIONS 206 LECTURE NOTES 8 ISSUED 24 APRIL 206 A series is a formal sum. Series a + a 2 + a 3 + + + where { } is a sequence of real numbers. Here formal means that we don

### Mathematical Induction. Mary Barnes Sue Gordon

Mathematics Learning Centre Mathematical Induction Mary Barnes Sue Gordon c 1987 University of Sydney Contents 1 Mathematical Induction 1 1.1 Why do we need proof by induction?.... 1 1. What is proof by

### Metric Spaces. Chapter 1

Chapter 1 Metric Spaces Many of the arguments you have seen in several variable calculus are almost identical to the corresponding arguments in one variable calculus, especially arguments concerning convergence

### Critical points of once continuously differentiable functions are important because they are the only points that can be local maxima or minima.

Lecture 0: Convexity and Optimization We say that if f is a once continuously differentiable function on an interval I, and x is a point in the interior of I that x is a critical point of f if f (x) =

### 1 Error in Euler s Method

1 Error in Euler s Method Experience with Euler s 1 method raises some interesting questions about numerical approximations for the solutions of differential equations. 1. What determines the amount of

### Sequences and Convergence in Metric Spaces

Sequences and Convergence in Metric Spaces Definition: A sequence in a set X (a sequence of elements of X) is a function s : N X. We usually denote s(n) by s n, called the n-th term of s, and write {s

### CHAPTER 3. Fourier Series

`A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 4 A COLLECTION OF HANDOUTS ON PARTIAL DIFFERENTIAL

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### 11. Nonlinear equations with one variable

EE103 (Fall 2011-12) 11. Nonlinear equations with one variable definition and examples bisection method Newton s method secant method 11-1 Definition and examples x is a zero (or root) of a function f

### Taylor and Maclaurin Series

Taylor and Maclaurin Series In the preceding section we were able to find power series representations for a certain restricted class of functions. Here we investigate more general problems: Which functions

### INTRODUCTION TO THE CONVERGENCE OF SEQUENCES

INTRODUCTION TO THE CONVERGENCE OF SEQUENCES BECKY LYTLE Abstract. In this paper, we discuss the basic ideas involved in sequences and convergence. We start by defining sequences and follow by explaining

19 LINEAR QUADRATIC REGULATOR 19.1 Introduction The simple form of loopshaping in scalar systems does not extend directly to multivariable (MIMO) plants, which are characterized by transfer matrices instead

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Problem Set. Problem Set #2. Math 5322, Fall December 3, 2001 ANSWERS

Problem Set Problem Set #2 Math 5322, Fall 2001 December 3, 2001 ANSWERS i Problem 1. [Problem 18, page 32] Let A P(X) be an algebra, A σ the collection of countable unions of sets in A, and A σδ the collection

### Math 317 HW #7 Solutions

Math 17 HW #7 Solutions 1. Exercise..5. Decide which of the following sets are compact. For those that are not compact, show how Definition..1 breaks down. In other words, give an example of a sequence

### f (x) = x 2 (x) = (1 + x 2 ) 2 (x) = (1 + x 2 ) 2 + 8x 2 (1 + x 2 ) 3 8x (1 + x 2 ) x (1 + x 2 ) 3 48x3 f(0) = tan 1 (0) = 0 f (0) = 2

Math 5 Exam # Practice Problem Solutions. Find the Maclaurin series for tan (x (feel free just to write out the first few terms. Answer: Let f(x = tan (x. Then the first few derivatives of f are: f (x

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### Cork Institute of Technology. CIT Mathematics Examination, Paper 2 Sample Paper A

Cork Institute of Technology CIT Mathematics Examination, 2015 Paper 2 Sample Paper A Answer ALL FIVE questions. Each question is worth 20 marks. Total marks available: 100 marks. The standard Formulae