# 5. Convergence of sequences of random variables

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 5. Convergence of sequences of random variables Throughout this chapter we assume that {X, X 2,...} is a sequence of r.v. and X is a r.v., and all of them are defined on the same probability space (Ω, F, P ). Stochastic convergence formalizes the idea that a sequence of r.v. sometimes is expected to settle into a pattern. The pattern may for instance be that: there is a convergence of (ω) in the classical sense to a fixed value X(ω), for each and every event ω; the probability that the distance between from a particular r.v. X exceeds any prescribed positive value decreases and converges to zero; the series formed by calculating the expected value of the (absolute or quadratic) distance between and X converges to zero; the distribution of particular r.v. X. may grow increasingly similar to the distribution of a Just as in analysis, we can distinguish among several types of convergence (Rohatgi, 976, p. 240). Thus, in this chapter we investigate modes of convergence of sequences of r.v.: almost sure convergence ( ); convergence in probability ( P ); convergence in quadratic mean or in L 2 ( q.m. ); convergence in L or in mean ( ); L See of random variables. 26

2 convergence in distribution ( ). d Two laws of large numbers and central limit theorems are also stated. It is important for the reader to be familiarized with all these modes of convergence, the way they can be related and with the applications of such results and understand their considerable significance in probability, statistics and stochastic processes. 5. Modes of convergence The first four modes of convergence (, where =, P, q.m., L ) pertain to the sequence of r.v. and to X as functions of Ω, while the fifth ( ) d is related to the convergence of d.f. (Karr, 993, p. 35). 5.. Convergence of r.v. as functions on Ω Motivation 5. Almost sure convergence (Karr, 993, p. 35) Almost sure convergence or convergence with probability one is the probabilistic version of pointwise convergence known from elementary real analysis. Definition 5.2 Almost sure convergence (Karr, 993, p. 35; Rohatgi, 976, p. 249) The sequence of r.v. {X, X 2,...} is said to converge almost surely to a r.v. X if ({ }) P w : lim (ω) = X(ω) =. (5.) n + In this case we write (or with probability ). Exercise 5.3 Almost sure convergence Let {X, X 2,...} be a sequence of r.v. such that Bernoulli( ), n IN. n Prove that X n 0, by deriving P ({ = 0, for every m n n 0 }) and observing that this probability does not converge to as n 0 + for all values of m (Rohatgi, 976, p. 252, Example 9). 27

3 Motivation 5.4 Convergence in probability (Karr, 993, p. 35; of random variables) Convergence in probability essentially means that the probability that X exceeds any prescribed, strictly positive value converges to zero. The basic idea behind this type of convergence is that the probability of an unusual outcome becomes smaller and smaller as the sequence progresses. Definition 5.5 Convergence in probability (Karr, 993, p. 36; Rohatgi, 976, p. 243) The sequence of r.v. {X, X 2,...} is said to converge in probability to a r.v. X denoted by X P n if lim P ({ X > ɛ}) = 0, (5.2) n + for every ɛ > 0. Remarks 5.6 Convergence in probability (Rohatgi, 976, p. 243; of random variables) The definition of convergence in probability says nothing about the convergence of r.v. to r.v. X in the sense in which it is understood in real analysis. Thus, X P n does not imply that, given ɛ > 0, we can find an N such that X < ɛ, for n N. Definition 5.5 speaks only of the convergence of the sequence of probabilities P ( X > ɛ) to zero. Formally, Definition 5.5 means that ɛ, δ > 0, N δ : P ({ X > ɛ}) < δ, n N δ. (5.3) The concept of convergence in probability is used very often in statistics. For example, an estimator is called consistent if it converges in probability to the parameter being estimated. Convergence in probability is also the type of convergence established by the weak law of large numbers. 28

4 Exercise 5.7 Convergence in probability Let {X, X 2,...} be a sequence of r.v. such that Bernoulli( ), n IN. n (a) Prove that X P n 0, by obtaining P ({ > ɛ}), for 0 < ɛ < and ɛ (Rohatgi, 976, pp , Example 5). (b) Verify that E(X k n) E(X k ), where k IN and X d = 0. Exercise 5.8 Convergence in probability does not imply convergence of kth. moments Let {X, X 2,...} be a sequence of r.v. such that, x = 0 n P ({ = x}) = n, x = n 0, otherwise. = d n Bernoulli( ), n IN, i.e. n (5.4) Prove that X P n 0, however E(Xn) k E(X k ), where k IN and the r.v. X is degenerate at 0 (Rohatgi, 976, p. 247, Remark 3). Motivation 5.9 Convergence in quadratic mean and in L We have just seen that convergence in probability does not imply the convergence of moments, namely of orders 2 or. Definition 5.0 Convergence in quadratic mean or in L 2 (Karr, 993, p. 36) Let X, X, X 2,... belong to L 2. Then the sequence of r.v. {X, X 2,...} is said to converge to X in quadratic mean (or in L 2 q.m. L ) denoted by (or X 2 n ) if lim E [ ( X) 2] = 0. (5.5) n + Exercise 5. Convergence in quadratic mean Let {X, X 2,...} be a sequence of r.v. such that Bernoulli ( n). q.m. Prove that, where the r.v. X is degenerate at 0 (Rohatgi, 976, p. 247, Example 6). 29

5 Exercise 5.2 Convergence in quadratic mean (bis) Let {X, X 2,...} be a sequence of r.v. with P ({ ) = ± n} = q.m. Prove that, where the r.v. X is degenerate at 0 (Rohatgi, 976, p. 252, Example ). 2. Exercise 5.3 Convergence in quadratic mean implies convergence of 2nd. moments (Karr, 993, p. 58, Exercise 5.6(a)) Prove that q.m. E(X 2 n) E(X 2 ) (Rohatgi, 976, p. 248, proof of Theorem 8). Exercise 5.4 Convergence in quadratic mean of partial sums (Karr, 993, p. 59, Exercise 5.) Let X, X 2,... be pairwise uncorrelated r.v. with mean zero and partial sums S n = n i= X i. Prove that if there is a constant c such that V (X i ) c, for every i, then S n n α q.m. 0 for all α > 2. Definition 5.5 Convergence in mean or in L (Karr, 993, p. 36) Let X, X, X 2,... belong to L. Then the sequence of r.v. {X, X 2,...} is said to converge to X in mean (or in L L ) denoted by X n if lim E ( X ) = 0. (5.6) n + Exercise 5.6 Convergence in mean implies convergence of st. moments (Karr, 993, p. 58, Exercise 5.6(b)) Prove that L E( ) E(X). 220

6 5..2 Convergence in distribution Motivation 5.7 Convergence in distribution ( of random variables) Convergence in distribution is very frequently used in practice, most often it arises from the application of the central limit theorem. Definition 5.8 Convergence in distribution (Karr, 993, p. 36; Rohatgi, 976, pp. 240 ) The sequence of r.v. {X, X 2,...} converges to X in distribution denoted by if lim F (x) = F X (x), (5.7) n + for all x at which F X is continuous. d Remarks 5.9 Convergence in distribution ( of random variables; Karr, 993, p. 36; Rohatgi, 976, p. 242) With this mode of convergence, we increasingly expect to see the next r.v. in a sequence of r.v. becoming better and better modeled by a given d.f., as seen in Exercise It must be noted that it is quite possible for a given sequence of d.f. to converge to a function that is not a d.f., as shown in Exercise 5.2. The requirement that only the continuity points of F X essential, as we shall see in exercises 5.22 and should be considered is The convergence in distribution does not imply the convergence of corresponding p.(d.)f., as shown in Exercise Exercise 5.20 Convergence in distribution Let X, X 2,..., be i.i.d. r.v. with common p.d.f. f(x) = {, θ 0 < x < θ 0, otherwise, (5.8) where 0 < θ < +, and X (n) = max,...,n X i. Prove that X d (n) θ (Rohatgi, 976, p. 24, Example 2). 22

7 Exercise 5.2 A sequence of d.f. converging to a non d.f. Consider the sequence of d.f. { 0, x < n F Xn (x) =, x n, (5.9) where F Xn (x) is the d.f. of the r.v. degenerate at x = n. Verify that F Xn (x) converges to a function (that is identically equal to 0!!!) which is not a d.f. (Rohatgi, 976, p. 24, Example ). Exercise 5.22 The requirement that only the continuity points of F X should be considered is essential Let Uniform (, + ) 2 n 2 n and X be a r.v. degenerate at. 2 (a) Prove that d (Karr, 993, p. 42). ( (b) Verify that F Xn 2) = for each n, and these values do not converge to F ( ) 2 X 2 =. Is there any contradiction with the convergence in distribution previously proved? (Karr, 993, p. 42.) Exercise 5.23 The requirement that only the continuity points of F X should be considered is essential (bis) Let Uniform ( 0, n) and X a r.v. degenerate at 0. Prove that X d n, even though F Xn (0) = 0, for all n, and F X (0) =, that is, the convergence of d.f. fails at the point x = 0 where F X is discontinuous ( of random variables). Exercise 5.24 Convergence in distribution does not imply convergence of corresponding p.(d.)f. Let {X, X 2,...} be a sequence of r.v. with p.f. {, x = 2 + n P ({ = x}) = 0, otherwise. (a) Prove that d, where X a r.v. degenerate at 2. (5.0) (b) Verify that none of the p.f. P ({ = x}) assigns any probability to the point x = 2, for all n, and that P ({ = x}) 0 for all x (Rohatgi, 976, p. 242, Example 4). 222

8 The following table condenses the definitions of convergence of sequences of r.v. Mode of convergence Assumption Defining condition (almost sure) P ({w : (ω) (ω)}) = P (in probability) P ({ Xn X > ɛ}) 0, for all ɛ > 0 q.m (in quadratic mean) X, X, X 2,... L 2 E [ ( X) 2] 0 L (in L ) X, X, X 2,... L E ( X ) 0 d (in distribution) FXn (x) F X (x), at continuity points x of F X Exercise 5.25 Modes of convergence and uniqueness of limit (Karr, 993, p. 58, Exercise 5.) Prove that for all five forms of convergence the limit is unique. In particular: (a) if and Y, where =, P, q.m., L, then X Y ; (b) if d and d Y, then X d Y ; Exercise 5.26 Modes of convergence and the vector space structure of the family of r.v. (Karr, 993, p. 58, Exercise 5.2) Prove that, for =, P, q.m., L, X n X 0, (5.) i.e. the four function-based forms of convergence are compatible with the vector space structure of the family of r.v. 223

9 5..3 Alternative criteria The definition of almost sure convergence and its verification are far from trivial. More tractable criteria have to be stated... Proposition 5.27 Relating almost sure convergence and convergence in probability (Karr, 993, p. 37; Rohatgi, 976, p. 249) iff ({ }) ɛ > 0, lim P sup X k X > ɛ = 0, (5.2) n + k n i.e. Y n = sup X k X P 0. (5.3) k n Remarks 5.28 Relating almost sure convergence and convergence in probability (Karr, 993, p. 37; Rohatgi, 976, p. 250, Remark 6) Proposition 5.27 states an equivalent form of almost sure convergence that illuminates its relationship to convergence in probability. 0 means that, ɛ, η > 0, n 0 IN : P Indeed, we can write, equivalently, that lim P n + ({ }) sup X k > ɛ < η. (5.4) k n 0 ( ) { X k > ɛ} = 0, (5.5) k n for ɛ > 0 arbitrary. Exercise 5.29 Relating almost sure convergence and convergence in probability Prove Proposition 5.27 (Karr, 993, p. 37; Rohatgi, 976, p. 250). 224

10 Exercise 5.30 Relating almost sure convergence and convergence in probability (bis) Let {X, X 2,...} be a sequence of r.v. with P ({ ) = ± n} = Prove that, where the r.v. X is degenerate at 0, by using (5.5) (Rohatgi, 976, p. 252). 2. Theorem 5.3 Cauchy criterion (Rohatgi, 976, p. 270) ({ }) lim P sup +m ɛ =, ɛ > 0. (5.6) n + m Exercise 5.32 Cauchy criterion Prove Theorem 5.3 (Rohatgi, 976, pp ). Definition 5.33 Complete convergence (Karr, 993, p. 38) The sequence of r.v. {X, X 2,...} is said to converge completely to X if + n= P ({ X > ɛ} ) < +, (5.7) for every ɛ > 0. The next results relate complete convergence, which is stronger than almost sure convergence, and sometimes more convenient to establish (Karr, 993, p. 37). Proposition 5.34 Relating almost sure convergence and complete convergence (Karr, 993, p. 38) + n= P ({ X > ɛ} ) < +, ɛ > 0. (5.8) Remark 5.35 Relating almost sure convergence and complete convergence (Karr, 993, p. 38) P X iff the probabilities P ({ X > ɛ} ) converge to zero, while if (but not only if) the convergence of probabilities P ({ X > ɛ} ) is fast enough that their sum, + n= P ({ X > ɛ} ), is finite. 225

11 Exercise 5.36 Relating almost sure convergence and complete convergence Prove Proposition 5.34, by using the (st.) Borel-Cantelli lemma (Karr, 993, p. 38). Theorem 5.37 Almost sure convergence of a sequence of independent r.v. (Rohatgi, 976, p. 265) Let {X, X 2,...} be a sequence of independent r.v. Then 0 + P ({ > ɛ} ) < +, ɛ > 0. (5.9) n= Exercise 5.38 Almost sure convergence of a sequence of independent r.v. Prove Theorem 5.37 (Rohatgi, 976, pp ). The definition of convergence in distribution is cumbersome because of the proviso regarding continuity points of the limit d.f. F X. An alternative criterion follows. Theorem 5.39 Alternative criterion for convergence in distribution (Karr, 993, p. 38) Let C be the set of bounded, continuous functions f : IR IR. Then X d n E[f( )] E[f(X)], f C. (5.20) Remark 5.40 Alternative criterion for convergence in distribution (Karr, 993, p. 38) Theorem 5.39 provides a criterion for convergence in distribution which is superior to the definition of convergence in distribution in that one need not to deal with continuity points of the limit d.f. Exercise 5.4 Alternative criterion for convergence in distribution Prove Theorem 5.39 (Karr, 993, pp ). Since in the proof of Theorem 5.39 the continuous functions used to approximate indicator functions can be taken to be arbitrarily smooth we can add a sufficient condition that guarantees convergence in distribution. 226

12 Corollary 5.42 Sufficient condition for convergence in distribution (Karr, 993, p. 39) Let: Then k be a fixed non-negative integer; C (k) be the space of bounded, k times uniformly continuously differentiable functions f : IR IR. E[f( )] E[f(X)], f C (k) d. (5.2) The next table summarizes the alternative criteria and sufficient conditions for almost sure convergence and convergence in distribution of sequences of r.v. Alternative criterion or sufficient condition ɛ > 0, lim n + P ({ sup k n X k X > ɛ }) = 0 Y n = sup k n X k X P 0 lim n + P ({sup m +m ɛ}) =, ɛ > 0 + n= P ({ X > ɛ} ) < +, ɛ > 0 E[f( )] E[f(X)], f C d E[f( )] E[f(X)], f C (k) for a fixed k IN 0 d Mode of convergence 227

13 5.2 Relationships among the modes of convergence Given the plethora of modes of convergence, it is natural to inquire how they can be always related or hold true in the presence of additional assumptions (Karr, 993, pp. 40 and 42) Implications always valid Proposition 5.43 Almost sure convergence implies convergence in probability (Karr, 993, p. 40; Rohatgi, 976, p. 250) P. (5.22) Exercise 5.44 Almost sure convergence implies convergence in probability Prove Proposition 5.43 (Karr, 993, p. 40; Rohatgi, 976, p. 25). Proposition 5.45 Convergence in quadratic mean implies convergence in L (Karr, 993, p. 40) q.m. L. (5.23) Exercise 5.46 Convergence in quadratic mean implies convergence in L Prove Proposition 5.45, by applying Cauchy-Schwarz s inequality (Karr, 993, p. 40). Proposition 5.47 Convergence in L (Karr, 993, p. 4) implies convergence in probability L P. (5.24) Exercise 5.48 Convergence in L implies convergence in probability Prove Proposition 5.47, by using Chebyshev s inequality (Karr, 993, p. 4). Proposition 5.49 Convergence in probability implies convergence in distribution (Karr, 993, p. 4) X P n X d n. (5.25) 228

14 Exercise 5.50 Convergence in probability implies convergence in distribution Prove Proposition 5.49, (Karr, 993, p. 4). Figure 5. shows that convergence in distribution is the weakest form of convergence, since it is implied by all other types of convergence studied so far. q.m. L P Xn d Figure 5.: Implications always valid between modes of convergence Counterexamples Counterexamples to all implications among the modes of convergence (and more!) are condensed in Figure 5.2 and presented by means of several exercises. q.m. L P Xn d Figure 5.2: Counterexamples to implications among the modes of convergence. Before proceeding with exercises, recall exercises 5.3 and 5.7 which pertain to the sequence of r.v. {X, X 2,...}, where Bernoulli( ), n IN. In the first exercise n we proved that X n 0, whereas in the second one we concluded that X P n 0. Thus, combining these results we can state that X P n

15 Exercise 5.5 Almost sure convergence does not imply convergence in quadratic mean Let {X, X 2,...} be a sequence of r.v. such that, x = 0 n P ({ = x}) = n, x = n 0, otherwise. (5.26) Prove that 0, and, hence, X P n 0 and X d L n 0, but 0 and X q.m. n 0 (Karr, 993, p. 4, Counterexample a)). Exercise 5.52 Almost sure convergence does not imply convergence in quadratic mean (bis) Let {X, X 2,...} be a sequence of r.v. such that, x = 0 n r P ({ = x}) =, x = n n r 0, otherwise, (5.27) where r 2. Prove that 0, but q.m. 0 (Rohatgi, 976, p. 252, Example 0). Exercise 5.53 Convergence in quadratic mean does not imply almost sure convergence Let Bernoulli ( n). q.m. Prove that 0, but X n 0 (Rohatgi, 976, p. 252, Example 9). Exercise 5.54 Convergence in L does not imply convergence in quadratic mean Let {X, X 2,...} be a sequence of r.v. such that, x = 0 n P ({ = x}) = n, x = n 0, otherwise. (5.28) Prove that 0 and L 0, however Counterexample b)). q.m. 0 (Karr, 993, p. 4, 230

16 Exercise 5.55 Convergence in probability does not imply almost sure convergence For each positive integer n there exists integers m and k (uniquely determined) such that n = 2 k + m, m = 0,,..., 2 k, k = 0,, 2,... (5.29) Thus, for n =, k = m = 0; for n = 5, k = 2, m=; and so on. Define r.v., for n =, 2,..., on Ω = [0, ] by { 2 k m, w < m+ 2 (ω) = k 2 k 0, otherwise. (5.30) Let the probability distribution of be given by P ({I}) =length of the interval I Ω. Thus,, x = 0 2 k P ({ = x}) =, x = 2 2 k k 0, otherwise. (5.3) Prove that P 0, but 0 (Rohatgi, 976, pp. 25 2, Example 8). Exercise 5.56 Convergence in distribution does not imply convergence in probability Let {X, X 2,...} be a sequence of r.v. such that 0, x < 0 F Xn (x) = 2, 0 x < n, x, (5.32) i.e. Bernoulli ( + 2 n). Prove that X d n, where X Bernoulli ( P 2), but Xn (Karr, 993, p. 42, Counterexample d)). Exercise 5.57 Convergence in distribution does not imply convergence in probability (bis) Let X, X, X 2,... be i.i.d. r.v. and let the joint p.f. of (X, ) be P ({X = 0, = }) = P ({X =, = 0}) =. 2 Prove that X d n, but X P n (Rohatgi, 976, p. 247, Remark 2). 23

17 5.2.3 Implications of restricted validity Proposition 5.58 Convergence in distribution to a constant implies convergence in probability (Karr, 993, p. 40; Rohatgi, 976, p. 246) Let {X, X 2,...} be a sequence of r.v. and c IR. Then X d n c X P n c. (5.33) Remark 5.59 Convergence in distribution to a constant is equivalent to convergence in probability (Rohatgi, 976, p. 246) If we add to the previous result the fact that P c d c, we can conclude that X P n c X d n c. (5.34) Exercise 5.60 Convergence in distribution to a constant implies convergence in probability Prove Proposition 5.58 (Karr, 993, p. 40). Definition 5.6 Uniform integrability (Karr, 993, p. 42) A sequence of r.v. {X, X 2,...} is uniformly integrable if L for each n IN and if lim sup E( ; { > a}) = 0. (5.35) a + n Recall that the expected value of a r.v. X over an event A is given by E(X; A) = E(X A ). Proposition 5.62 Alternative criterion for uniform integrability (Karr, 993, p. 43) A sequence of r.v. {X, X 2,...} is uniformly integrable iff sup n E( ) < + and {X, X 2,...} is uniformly absolutely continuous: for each ɛ > 0 there is δ > 0 such that sup n E( ; A) < ɛ whenever P (A) > δ. Proposition 5.63 Combining convergence in probability and uniform integrability is equivalent to convergence in L (Karr, 993, p. 44) Let X, X, X 2,... L. Then X P n and {X, X 2,...} is uniformly integrable L X n. (5.36) 232

18 Exercise 5.64 Combining convergence in probability and uniform integrability is equivalent to convergence in L Prove Proposition 5.63 (Karr, 993, p. 44). Exercise 5.65 Combining convergence in probability of the sequence of r.v. and convergence of sequence of the means implies convergence in L (Karr, 993, p. 60, Exercise 5.6) Let X, X, X 2,... be positive r.v. Prove that if P and E( ) E(X), then L. Exercise 5.66 Increasing character and convergence in probability combined imply almost sure convergence (Karr, 993, p. 60, Exercise 5.5) Prove that if X X 2... and X P n, then. Exercise 5.67 Strictly decreasing and positive character and convergence in probability combined imply almost sure convergence (Rohatgi, 976, p. 252, Theorem 3) Let {X, X 2,...} be a strictly decreasing sequence of positive r.v. Prove that if X P n 0 then 0. References Karr, A.F. (993). Probability. Springer-Verlag. Resnick, S.I. (999). A Probability Path. Birkhäuser. (QA RES.49925) Rohatgi, V.K. (976). An Introduction to Probability Theory and Mathematical Statistics. John Wiley & Sons. (QA /4.ROH.34909) 233

### x a x 2 (1 + x 2 ) n.

Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number

### THE CENTRAL LIMIT THEOREM TORONTO

THE CENTRAL LIMIT THEOREM DANIEL RÜDT UNIVERSITY OF TORONTO MARCH, 2010 Contents 1 Introduction 1 2 Mathematical Background 3 3 The Central Limit Theorem 4 4 Examples 4 4.1 Roulette......................................

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### Notes on metric spaces

Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.

### n k=1 k=0 1/k! = e. Example 6.4. The series 1/k 2 converges in R. Indeed, if s n = n then k=1 1/k, then s 2n s n = 1 n + 1 +...

6 Series We call a normed space (X, ) a Banach space provided that every Cauchy sequence (x n ) in X converges. For example, R with the norm = is an example of Banach space. Now let (x n ) be a sequence

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Separation Properties for Locally Convex Cones

Journal of Convex Analysis Volume 9 (2002), No. 1, 301 307 Separation Properties for Locally Convex Cones Walter Roth Department of Mathematics, Universiti Brunei Darussalam, Gadong BE1410, Brunei Darussalam

### Some Notes on Taylor Polynomials and Taylor Series

Some Notes on Taylor Polynomials and Taylor Series Mark MacLean October 3, 27 UBC s courses MATH /8 and MATH introduce students to the ideas of Taylor polynomials and Taylor series in a fairly limited

### HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

### Gambling Systems and Multiplication-Invariant Measures

Gambling Systems and Multiplication-Invariant Measures by Jeffrey S. Rosenthal* and Peter O. Schwartz** (May 28, 997.. Introduction. This short paper describes a surprising connection between two previously

### Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics

Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### I. Pointwise convergence

MATH 40 - NOTES Sequences of functions Pointwise and Uniform Convergence Fall 2005 Previously, we have studied sequences of real numbers. Now we discuss the topic of sequences of real valued functions.

### Wald s Identity. by Jeffery Hein. Dartmouth College, Math 100

Wald s Identity by Jeffery Hein Dartmouth College, Math 100 1. Introduction Given random variables X 1, X 2, X 3,... with common finite mean and a stopping rule τ which may depend upon the given sequence,

### FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL

FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint

### Mathematics for Econometrics, Fourth Edition

Mathematics for Econometrics, Fourth Edition Phoebus J. Dhrymes 1 July 2012 1 c Phoebus J. Dhrymes, 2012. Preliminary material; not to be cited or disseminated without the author s permission. 2 Contents

### Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1

Further Study on Strong Lagrangian Duality Property for Invex Programs via Penalty Functions 1 J. Zhang Institute of Applied Mathematics, Chongqing University of Posts and Telecommunications, Chongqing

### Ri and. i=1. S i N. and. R R i

The subset R of R n is a closed rectangle if there are n non-empty closed intervals {[a 1, b 1 ], [a 2, b 2 ],..., [a n, b n ]} so that R = [a 1, b 1 ] [a 2, b 2 ] [a n, b n ]. The subset R of R n is an

### Introduction to Monte-Carlo Methods

Introduction to Monte-Carlo Methods Bernard Lapeyre Halmstad January 2007 Monte-Carlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities

### Math212a1010 Lebesgue measure.

Math212a1010 Lebesgue measure. October 19, 2010 Today s lecture will be devoted to Lebesgue measure, a creation of Henri Lebesgue, in his thesis, one of the most famous theses in the history of mathematics.

### {f 1 (U), U F} is an open cover of A. Since A is compact there is a finite subcover of A, {f 1 (U 1 ),...,f 1 (U n )}, {U 1,...

44 CHAPTER 4. CONTINUOUS FUNCTIONS In Calculus we often use arithmetic operations to generate new continuous functions from old ones. In a general metric space we don t have arithmetic, but much of it

### Probability and Statistics

CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

### 2 Complex Functions and the Cauchy-Riemann Equations

2 Complex Functions and the Cauchy-Riemann Equations 2.1 Complex functions In one-variable calculus, we study functions f(x) of a real variable x. Likewise, in complex analysis, we study functions f(z)

### BANACH AND HILBERT SPACE REVIEW

BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but

### Independence of Four Projective Criteria for the Weak Invariance Principle

Alea 5, 21 27 (2009) Independence of Four Projective Criteria for the Weak Invariance Principle Olivier Durieu Laboratoire de Mathematiques Raphaël Salem, UMR 6085 CNRS-Université de Rouen URL: http://www.univ-rouen.fr/lmrs/persopage/durieu/

### Option Pricing for a General Stock Model in Discrete Time

University of Wisconsin Milwaukee UWM Digital Commons Theses and Dissertations August 2014 Option Pricing for a General Stock Model in Discrete Time Cindy Lynn Nichols University of Wisconsin-Milwaukee

### 1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let

Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as

### Tiers, Preference Similarity, and the Limits on Stable Partners

Tiers, Preference Similarity, and the Limits on Stable Partners KANDORI, Michihiro, KOJIMA, Fuhito, and YASUDA, Yosuke February 7, 2010 Preliminary and incomplete. Do not circulate. Abstract We consider

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### An example of a computable

An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### 4 Lyapunov Stability Theory

4 Lyapunov Stability Theory In this section we review the tools of Lyapunov stability theory. These tools will be used in the next section to analyze the stability properties of a robot controller. We

### A Note on the Ruin Probability in the Delayed Renewal Risk Model

Southeast Asian Bulletin of Mathematics 2004 28: 1 5 Southeast Asian Bulletin of Mathematics c SEAMS. 2004 A Note on the Ruin Probability in the Delayed Renewal Risk Model Chun Su Department of Statistics

### About the inverse football pool problem for 9 games 1

Seventh International Workshop on Optimal Codes and Related Topics September 6-1, 013, Albena, Bulgaria pp. 15-133 About the inverse football pool problem for 9 games 1 Emil Kolev Tsonka Baicheva Institute

### P (A) = lim P (A) = N(A)/N,

1.1 Probability, Relative Frequency and Classical Definition. Probability is the study of random or non-deterministic experiments. Suppose an experiment can be repeated any number of times, so that we

### Section 3 Sequences and Limits, Continued.

Section 3 Sequences and Limits, Continued. Lemma 3.6 Let {a n } n N be a convergent sequence for which a n 0 for all n N and it α 0. Then there exists N N such that for all n N. α a n 3 α In particular

### Series Convergence Tests Math 122 Calculus III D Joyce, Fall 2012

Some series converge, some diverge. Series Convergence Tests Math 22 Calculus III D Joyce, Fall 202 Geometric series. We ve already looked at these. We know when a geometric series converges and what it

### ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING. 0. Introduction

ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING Abstract. In [1] there was proved a theorem concerning the continuity of the composition mapping, and there was announced a theorem on sequential continuity

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

### CONVERGENCE OF SEQUENCES OF ITERATES OF RANDOM-VALUED VECTOR FUNCTIONS

C O L L O Q U I U M M A T H E M A T I C U M VOL. 97 2003 NO. 1 CONVERGENCE OF SEQUENCES OF ITERATES OF RANDOM-VALUED VECTOR FUNCTIONS BY RAFAŁ KAPICA (Katowice) Abstract. Given a probability space (Ω,

### Open and Closed Sets

Open and Closed Sets Definition: A subset S of a metric space (X, d) is open if it contains an open ball about each of its points i.e., if x S : ɛ > 0 : B(x, ɛ) S. (1) Theorem: (O1) and X are open sets.

### MOP 2007 Black Group Integer Polynomials Yufei Zhao. Integer Polynomials. June 29, 2007 Yufei Zhao yufeiz@mit.edu

Integer Polynomials June 9, 007 Yufei Zhao yufeiz@mit.edu We will use Z[x] to denote the ring of polynomials with integer coefficients. We begin by summarizing some of the common approaches used in dealing

### BOUNDED, ASYMPTOTICALLY STABLE, AND L 1 SOLUTIONS OF CAPUTO FRACTIONAL DIFFERENTIAL EQUATIONS. Muhammad N. Islam

Opuscula Math. 35, no. 2 (215), 181 19 http://dx.doi.org/1.7494/opmath.215.35.2.181 Opuscula Mathematica BOUNDED, ASYMPTOTICALLY STABLE, AND L 1 SOLUTIONS OF CAPUTO FRACTIONAL DIFFERENTIAL EQUATIONS Muhammad

### LEARNING OBJECTIVES FOR THIS CHAPTER

CHAPTER 2 American mathematician Paul Halmos (1916 2006), who in 1942 published the first modern linear algebra book. The title of Halmos s book was the same as the title of this chapter. Finite-Dimensional

### Direct Algorithms for Checking Coherence and Making Inferences from Conditional Probability Assessments

Direct Algorithms for Checking Coherence and Making Inferences from Conditional Probability Assessments Peter Walley, Renato Pelessoni a and Paolo Vicig a a Dipartimento di Matematica Applicata B. de Finetti,

### MA651 Topology. Lecture 6. Separation Axioms.

MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples

### Every Positive Integer is the Sum of Four Squares! (and other exciting problems)

Every Positive Integer is the Sum of Four Squares! (and other exciting problems) Sophex University of Texas at Austin October 18th, 00 Matilde N. Lalín 1. Lagrange s Theorem Theorem 1 Every positive integer

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Chapter 1 - σ-algebras

Page 1 of 17 PROBABILITY 3 - Lecture Notes Chapter 1 - σ-algebras Let Ω be a set of outcomes. We denote by P(Ω) its power set, i.e. the collection of all the subsets of Ω. If the cardinality Ω of Ω is

### THE BANACH CONTRACTION PRINCIPLE. Contents

THE BANACH CONTRACTION PRINCIPLE ALEX PONIECKI Abstract. This paper will study contractions of metric spaces. To do this, we will mainly use tools from topology. We will give some examples of contractions,

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Statistiek (WISB361)

Statistiek (WISB361) Final exam June 29, 2015 Schrijf uw naam op elk in te leveren vel. Schrijf ook uw studentnummer op blad 1. The maximum number of points is 100. Points distribution: 23 20 20 20 17

### The Delta Method and Applications

Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and

### Uniform Multicommodity Flow through the Complete Graph with Random Edge-capacities

Combinatorics, Probability and Computing (2005) 00, 000 000. c 2005 Cambridge University Press DOI: 10.1017/S0000000000000000 Printed in the United Kingdom Uniform Multicommodity Flow through the Complete

### Probability and statistics; Rehearsal for pattern recognition

Probability and statistics; Rehearsal for pattern recognition Václav Hlaváč Czech Technical University in Prague Faculty of Electrical Engineering, Department of Cybernetics Center for Machine Perception

### Mathematical Methods of Engineering Analysis

Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................

### POWER SETS AND RELATIONS

POWER SETS AND RELATIONS L. MARIZZA A. BAILEY 1. The Power Set Now that we have defined sets as best we can, we can consider a sets of sets. If we were to assume nothing, except the existence of the empty

### Properties of moments of random variables

Properties of moments of rom variables Jean-Marie Dufour Université de Montréal First version: May 1995 Revised: January 23 This version: January 14, 23 Compiled: January 14, 23, 1:5pm This work was supported

### CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.

CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,

### No: 10 04. Bilkent University. Monotonic Extension. Farhad Husseinov. Discussion Papers. Department of Economics

No: 10 04 Bilkent University Monotonic Extension Farhad Husseinov Discussion Papers Department of Economics The Discussion Papers of the Department of Economics are intended to make the initial results

### CONTINUED FRACTIONS AND PELL S EQUATION. Contents 1. Continued Fractions 1 2. Solution to Pell s Equation 9 References 12

CONTINUED FRACTIONS AND PELL S EQUATION SEUNG HYUN YANG Abstract. In this REU paper, I will use some important characteristics of continued fractions to give the complete set of solutions to Pell s equation.

### Stochastic Processes and Advanced Mathematical Finance. Laws of Large Numbers

Steven R. Dunbar Department of Mathematics 203 Avery Hall University of Nebraska-Lincoln Lincoln, NE 68588-0130 http://www.math.unl.edu Voice: 402-472-3731 Fax: 402-472-8466 Stochastic Processes and Advanced

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### FIRST YEAR CALCULUS. Chapter 7 CONTINUITY. It is a parabola, and we can draw this parabola without lifting our pencil from the paper.

FIRST YEAR CALCULUS WWLCHENW L c WWWL W L Chen, 1982, 2008. 2006. This chapter originates from material used by the author at Imperial College, University of London, between 1981 and 1990. It It is is

### 88 CHAPTER 2. VECTOR FUNCTIONS. . First, we need to compute T (s). a By definition, r (s) T (s) = 1 a sin s a. sin s a, cos s a

88 CHAPTER. VECTOR FUNCTIONS.4 Curvature.4.1 Definitions and Examples The notion of curvature measures how sharply a curve bends. We would expect the curvature to be 0 for a straight line, to be very small

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### 0 <β 1 let u(x) u(y) kuk u := sup u(x) and [u] β := sup

456 BRUCE K. DRIVER 24. Hölder Spaces Notation 24.1. Let Ω be an open subset of R d,bc(ω) and BC( Ω) be the bounded continuous functions on Ω and Ω respectively. By identifying f BC( Ω) with f Ω BC(Ω),

### Differentiating under an integral sign

CALIFORNIA INSTITUTE OF TECHNOLOGY Ma 2b KC Border Introduction to Probability and Statistics February 213 Differentiating under an integral sign In the derivation of Maximum Likelihood Estimators, or

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Probability Generating Functions

page 39 Chapter 3 Probability Generating Functions 3 Preamble: Generating Functions Generating functions are widely used in mathematics, and play an important role in probability theory Consider a sequence

### Math 5311 Gateaux differentials and Frechet derivatives

Math 5311 Gateaux differentials and Frechet derivatives Kevin Long January 26, 2009 1 Differentiation in vector spaces Thus far, we ve developed the theory of minimization without reference to derivatives.

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### Vectors, Gradient, Divergence and Curl.

Vectors, Gradient, Divergence and Curl. 1 Introduction A vector is determined by its length and direction. They are usually denoted with letters with arrows on the top a or in bold letter a. We will use

### Stochastic Inventory Control

Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the

### Spectral Measure of Large Random Toeplitz Matrices

Spectral Measure of Large Random Toeplitz Matrices Yongwhan Lim June 5, 2012 Definition (Toepliz Matrix) The symmetric Toeplitz matrix is defined to be [X i j ] where 1 i, j n; that is, X 0 X 1 X 2 X n

### Taylor Polynomials and Taylor Series Math 126

Taylor Polynomials and Taylor Series Math 26 In many problems in science and engineering we have a function f(x) which is too complicated to answer the questions we d like to ask. In this chapter, we will

### 160 CHAPTER 4. VECTOR SPACES

160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

### Exercises with solutions (1)

Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal

### Metric Spaces Joseph Muscat 2003 (Last revised May 2009)

1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of

### Note on some explicit formulae for twin prime counting function

Notes on Number Theory and Discrete Mathematics Vol. 9, 03, No., 43 48 Note on some explicit formulae for twin prime counting function Mladen Vassilev-Missana 5 V. Hugo Str., 4 Sofia, Bulgaria e-mail:

### Math 55: Discrete Mathematics

Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What

### AN EXPANSION FORMULA FOR FRACTIONAL DERIVATIVES AND ITS APPLICATION. Abstract

AN EXPANSION FORMULA FOR FRACTIONAL DERIVATIVES AND ITS APPLICATION T. M. Atanackovic 1 and B. Stankovic 2 Dedicated to Professor Ivan H. Dimovski on the occasion of his 7th birthday Abstract An expansion

### The Lebesgue Measure and Integral

The Lebesgue Measure and Integral Mike Klaas April 12, 2003 Introduction The Riemann integral, defined as the limit of upper and lower sums, is the first example of an integral, and still holds the honour

### TOPIC 3: CONTINUITY OF FUNCTIONS

TOPIC 3: CONTINUITY OF FUNCTIONS. Absolute value We work in the field of real numbers, R. For the study of the properties of functions we need the concept of absolute value of a number. Definition.. Let

### arxiv: v1 [math.ho] 7 Apr 2016

ON EXISTENCE OF A TRIANGLE WITH PRESCRIBED BISECTOR LENGTHS S. F. OSINKIN arxiv:1604.03794v1 [math.ho] 7 Apr 2016 Abstract. We suggest a geometric visualization of the process of constructing a triangle

### ZERO-DIVISOR GRAPHS OF POLYNOMIALS AND POWER SERIES OVER COMMUTATIVE RINGS

ZERO-DIVISOR GRAPHS OF POLYNOMIALS AND POWER SERIES OVER COMMUTATIVE RINGS M. AXTELL, J. COYKENDALL, AND J. STICKLES Abstract. We recall several results of zero divisor graphs of commutative rings. We

### 2. Methods of Proof Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try.

2. METHODS OF PROOF 69 2. Methods of Proof 2.1. Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try. Trivial Proof: If we know q is true then

### TOPIC 4: DERIVATIVES

TOPIC 4: DERIVATIVES 1. The derivative of a function. Differentiation rules 1.1. The slope of a curve. The slope of a curve at a point P is a measure of the steepness of the curve. If Q is a point on the

### 1 Prior Probability and Posterior Probability

Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which

### Practice with Proofs

Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab

Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?

### Section 6-2 Mathematical Induction

6- Mathematical Induction 457 In calculus, it can be shown that e x k0 x k k! x x x3!! 3!... xn n! where the larger n is, the better the approximation. Problems 6 and 6 refer to this series. Note that

### F. ABTAHI and M. ZARRIN. (Communicated by J. Goldstein)

Journal of Algerian Mathematical Society Vol. 1, pp. 1 6 1 CONCERNING THE l p -CONJECTURE FOR DISCRETE SEMIGROUPS F. ABTAHI and M. ZARRIN (Communicated by J. Goldstein) Abstract. For 2 < p