Properties of MLE: consistency, asymptotic normality. Fisher information.

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Properties of MLE: consistency, asymptotic normality. Fisher information."

Transcription

1 Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout this course. Law of Large Numbers (LLN): If the distributio of the i.i.d. sample X,..., X is such that X has fiite expectatio, i.e. EX <, the the sample average X X X = EX coverges to its expectatio i probability, which meas that for ay arbitrarily small α >, P( X EX > θ) as. Note. Wheever we will use the LLN below we will simply say that the average coverges to its expectatio ad will ot metio i what sese. More mathematically iclied studets are welcome to carry out these steps more rigorously, especially whe we use LLN i combiatio with the Cetral Limit Theorem. Cetral Limit Theorem (CLT): If the distributio of the i.i.d. sample X,..., X is such that X has fiite expectatio ad variace, i.e. EX < ad π = Var(X) <, the (X EX ) d N(, π ) coverges i distributio to ormal distributio with zero mea ad variace π, which meas that for ay iterval [a, b], b x P (X EX ) [a, b] π e dx. 6 a

2 I other words, the radom variable (X EX ) will behave like a radom variable from ormal distributio whe gets large. Exercise. Illustrate CLT by geeratig Beroulli radom varibles B(p) (or oe Biomial r.v. B(, p)) ad the computig (X EX ). Repeat this may times ad use dfittool to see that this radom quatity will be well approximated by ormal distributio. We will prove that MLE satisfies (usually) the followig two properties called cosistecy ad asymptotic ormality.. Cosistecy. We say that a estimate ϕˆ is cosistet if ϕˆ ϕ i probability as, where ϕ is the true ukow parameter of the distributio of the sample.. Asymptotic Normality. We say that ϕˆ is asymptotically ormal if (ϕˆ ϕ ) d N(, π ) where π is called the asymptotic variace of the estimate ϕˆ. Asymptotic ormality says that the estimator ot oly coverges to the ukow parameter, but it coverges fast eough, at a rate /. Cosistecy of MLE. To make our discussio as simple as possible, let us assume that a likelihood fuctio is smooth ad behaves i a ice way like show i figure 3., i.e. its maximum is achieved at a uique poit ϕ. ˆ (...4 ( PSfrag replacemets ϕˆ ϕ Figure 3.: Maximum Likelihood Estimator (MLE) Suppose that the data X,..., X is geerated from a distributio with ukow parameter ϕ ad ϕˆ is a MLE. Why ϕˆ coverges to the ukow parameter ϕ? This is ot immediately obvious ad i this sectio we will give a sketch of why this happes. 7

3 First of all, MLE ϕˆ is the maximizer of L ( = i= log X i which is a log-likelihood fuctio ormalized by (of course, this does ot affect maximizatio). Notice that fuctio L ( depeds o data. Let us cosider a fuctio l( = log ad defie L( = E l(, where E deotes the expectatio with respect to the true ukow parameter ϕ of the sample X,..., X. If we deal with cotiuous distributios the L( = (log x )x ϕ )dx. By law of large umbers, for ay ϕ, L ( E l( = L(. Note that L( does ot deped o the sample, it oly depeds o ϕ. We will eed the followig Lemma. We have that for ay ϕ, L( L(ϕ ). Moreover, the iequality is strict, L( < L(ϕ ), uless which meas that P = P. Proof. Let us cosider the differece P ( = ϕ )) =. L( L(ϕ ) = E (log log ϕ )) = E log f ( ϕ ). Sice log t t, we ca write E log E = x ϕ )dx f ( ϕ) ϕ) x x ϕ) = x dx x ϕ )dx = =. Both itegrals are equal to because we are itegratig the probability desity fuctios. This proves that L( L(ϕ ). The secod statemet of Lemma is also clear. 8

4 We will use this Lemma to sketch the cosistecy of the MLE. Theorem: Uder some regularity coditios o the family of distributios, MLE ϕˆ is cosistet, i.e. ϕˆ ϕ as. The statemet of this Theorem is ot very precise but but rather tha provig a rigorous mathematical statemet our goal here is to illustrate the mai idea. Mathematically iclied studets are welcome to come up with some precise statemet. L ( L( PSfrag replacemets ˆϕ ϕ ϕ Figure 3.: Illustratio to Theorem. Proof. We have the followig facts:. ϕˆ is the maximizer of L ( (by defiitio).. ϕ is the maximizer of L( (by Lemma). 3. ϕ we have L ( L( by LLN. This situatio is illustrated i figure 3.. Therefore, sice two fuctios L ad L are gettig closer, the poits of maximum should also get closer which exactly meas that ϕˆ ϕ. Asymptotic ormality of MLE. Fisher iformatio. We wat to show the asymptotic ormality of MLE, i.e. to show that (ϕˆ ϕ ) d N(, πmle ) for some πmle ad compute πmle. This asymptotic variace i some sese measures the quality of MLE. First, we eed to itroduce the otio called Fisher Iformatio. Let us recall that above we defied the fuctio l( = log. To simplify the otatios we will deote by l (, l (, etc. the derivatives of l( with respect to ϕ. Defiitio. (Fisher iformatio.) Fisher iformatio of a radom variable X with distributio P from the family {P : ϕ } is defied by I(ϕ ) = E (l (ϕ )) E log. ϕ = 9

5 Remark. Let us give a very iformal iterpretatio of Fisher iformatio. The derivative l (ϕ ) = (log ϕ )) = f ( ϕ) ϕ ) ca be iterpreted as a measure of how quickly the distributio desity or p.f. will chage whe we slightly chage the parameter ϕ ear ϕ. Whe we square this ad take expectatio, i.e. average over X, we get a averaged versio of this measure. So if Fisher iformatio is large, this meas that the distributio will chage quickly whe we move the parameter, so the distributio with parameter ϕ is quite differet ad ca be well distiguished from the distributios with parameters ot so close to ϕ. This meas that we should be able to estimate ϕ well based o the data. O the other had, if Fisher iformatio is small, this meas that the distributio is very similar to distributios with parameter ot so close to ϕ ad, thus, more difficult to distiguish, so our estimatio will be worse. We will see precisely this behavior i Theorem below. Next lemma gives aother ofte coveiet way to compute Fisher iformatio. Lemma. We have, E l (ϕ ) E ϕ log ϕ ) = I(ϕ ). ad Proof. First of all, we have Also, sice p.d.f. itegrates to, l ( = (log ) = f ( (log ) = f ( (f ( ). f ( x dx =, if we take derivatives of this equatio with respect to ϕ (ad iterchage derivative ad itegral, which ca usually be doe) we will get, ϕ x dx = ad ϕ x dx = f (x dx =. To fiish the proof we write the followig computatio E l ( ϕ ) = E log ϕ ) = (log x ϕ )) x ϕ )dx ϕ f f ( x ϕ (x ϕ) = x ϕ )dx x ϕ)) x ϕ) = f (x ϕ )dx E (l (ϕ )) = I(ϕ = I(ϕ ).

6 We are ow ready to prove the mai result of this sectio. Theorem. (Asymptotic ormality of MLE.) We have, (ϕˆ ϕ ) N,. I(ϕ ) As we ca see, the asymptotic variace/dispersio of the estimate aroud true parameter will be smaller whe Fisher iformatio is larger. Proof. Sice MLE ϕˆ is maximizer of L ( = i= log X i, we have Let us use the Mea Value Theorem a) b) a b L (ϕˆ) =. = f (c) or a) = b) + f (c)(a b) for c [a, b] with = L (, a = ϕˆ ad b = ϕ. The we ca write, for some ϕ ˆ [ˆ ϕ, ϕ ]. From here we get that = L (ϕˆ) = L (ϕ ) + L (ϕˆ)(ϕˆ ϕ ) ˆ ϕ ϕ = L (ϕ) L (ϕ ) ad (ϕˆ ϕ ) =. (3..) L ( ˆ ( ˆ ϕ ) L ϕ ) Sice by Lemma i the previous sectio we kow that ϕ is the maximizer of L(, we have L (ϕ ) = E l ( ϕ ) =. (3..) Therefore, the umerator i (3..) L (ϕ ) = = l (X i ϕ ) i= l (X i ϕ ) E l (X ϕ ) N, Var (l (X ϕ )) i= (3..3) coverges i distributio by Cetral Limit Theorem. Next, let us cosider the deomiator i (3..). First of all, we have that for all ϕ, L ( = l (X i E l (X by LLN. (3..4) Also, sice ϕˆ [ˆ ϕ, ϕ ϕ ϕ, we have ˆ ] ad by cosistecy result of previous sectio, ˆ ϕ ϕ. Usig this together with (..3) we get L (ϕˆ) E l (X ϕ ) = I(ϕ ) by Lemma above.

7 Combiig this with (3..3) we get L (ϕ ) d N, Var (l (X ϕ )). L (ϕˆ) (I(ϕ )) Fially, the variace, Var (l (X ϕ )) = E (l (ϕ )) (E l (x ϕ )) = I(ϕ ) where i the last equality we used the defiitio of Fisher iformatio ad (3..). Let us compute Fisher iformatio for some particular distributios. Example. The family of Beroulli distributios B(p) has p.f. ad takig the logarithm x p) = p x ( p) x log x p) = x log p + ( x) log( p). The secod derivative with respect to parameter p is x x log x p) =, log x p) = x x. p p p p p ( p) The the Fisher iformatio ca be computed as I(p) = E log X p) = EX + EX = p + p =. p p ( p) p ( p) p( p) The MLE of p is ˆp = X ad the asymptotic ormality result states that (ˆp p ) N(, p ( p )) which, of course, also follows directly from the CLT. Example. The family of expoetial distributios E() has p.d.f. e x, x x ) =, x < ad, therefore, This does ot deped o X ad we get log x ) = log x log x ) =. I() = E log ) =. Therefore, the MLE ˆ = /X is asymptotically ormal ad (ˆ ) N(, ).

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

Confidence Intervals for the Mean of Non-normal Data Class 23, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom

Confidence Intervals for the Mean of Non-normal Data Class 23, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom Cofidece Itervals for the Mea of No-ormal Data Class 23, 8.05, Sprig 204 Jeremy Orloff ad Joatha Bloom Learig Goals. Be able to derive the formula for coservative ormal cofidece itervals for the proportio

More information

Sampling Distribution And Central Limit Theorem

Sampling Distribution And Central Limit Theorem () Samplig Distributio & Cetral Limit Samplig Distributio Ad Cetral Limit Samplig distributio of the sample mea If we sample a umber of samples (say k samples where k is very large umber) each of size,

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

ORDERS OF GROWTH KEITH CONRAD

ORDERS OF GROWTH KEITH CONRAD ORDERS OF GROWTH KEITH CONRAD Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really wat to uderstad their behavior It also helps you better grasp topics i calculus

More information

Chapter 14 Nonparametric Statistics

Chapter 14 Nonparametric Statistics Chapter 14 Noparametric Statistics A.K.A. distributio-free statistics! Does ot deped o the populatio fittig ay particular type of distributio (e.g, ormal). Sice these methods make fewer assumptios, they

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

1 Hypothesis testing for a single mean

1 Hypothesis testing for a single mean BST 140.65 Hypothesis Testig Review otes 1 Hypothesis testig for a sigle mea 1. The ull, or status quo, hypothesis is labeled H 0, the alterative H a or H 1 or H.... A type I error occurs whe we falsely

More information

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means) CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

Methods of Evaluating Estimators

Methods of Evaluating Estimators Math 541: Statistical Theory II Istructor: Sogfeg Zheg Methods of Evaluatig Estimators Let X 1, X 2,, X be i.i.d. radom variables, i.e., a radom sample from f(x θ), where θ is ukow. A estimator of θ is

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

The Euler Totient, the Möbius and the Divisor Functions

The Euler Totient, the Möbius and the Divisor Functions The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship

More information

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

The geometric series and the ratio test

The geometric series and the ratio test The geometric series ad the ratio test Today we are goig to develop aother test for covergece based o the iterplay betwee the it compariso test we developed last time ad the geometric series. A ote about

More information

Economics 140A Confidence Intervals and Hypothesis Testing

Economics 140A Confidence Intervals and Hypothesis Testing Ecoomics 140A Cofidece Itervals ad Hypothesis Testig Obtaiig a estimate of a parameter is ot the al purpose of statistical iferece because it is highly ulikely that the populatio value of a parameter is

More information

if A S, then X \ A S, and if (A n ) n is a sequence of sets in S, then n A n S,

if A S, then X \ A S, and if (A n ) n is a sequence of sets in S, then n A n S, Lecture 5: Borel Sets Topologically, the Borel sets i a topological space are the σ-algebra geerated by the ope sets. Oe ca build up the Borel sets from the ope sets by iteratig the operatios of complemetatio

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

Module 4: Mathematical Induction

Module 4: Mathematical Induction Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

Riemann Sums y = f (x)

Riemann Sums y = f (x) Riema Sums Recall that we have previously discussed the area problem I its simplest form we ca state it this way: The Area Problem Let f be a cotiuous, o-egative fuctio o the closed iterval [a, b] Fid

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

Math C067 Sampling Distributions

Math C067 Sampling Distributions Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Lesson 12. Sequences and Series

Lesson 12. Sequences and Series Retur to List of Lessos Lesso. Sequeces ad Series A ifiite sequece { a, a, a,... a,...} ca be thought of as a list of umbers writte i defiite order ad certai patter. It is usually deoted by { a } =, or

More information

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio

More information

Measurable Functions

Measurable Functions Measurable Fuctios Dug Le 1 1 Defiitio It is ecessary to determie the class of fuctios that will be cosidered for the Lebesgue itegratio. We wat to guaratee that the sets which arise whe workig with these

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

TAYLOR SERIES, POWER SERIES

TAYLOR SERIES, POWER SERIES TAYLOR SERIES, POWER SERIES The followig represets a (icomplete) collectio of thigs that we covered o the subject of Taylor series ad power series. Warig. Be prepared to prove ay of these thigs durig the

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or p-series (the Compariso Test), but of

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

3. Continuous Random Variables

3. Continuous Random Variables Statistics ad probability: 3-1 3. Cotiuous Radom Variables A cotiuous radom variable is a radom variable which ca take values measured o a cotiuous scale e.g. weights, stregths, times or legths. For ay

More information

Section 9.2 Series and Convergence

Section 9.2 Series and Convergence Sectio 9. Series ad Covergece Goals of Chapter 9 Approximate Pi Prove ifiite series are aother importat applicatio of limits, derivatives, approximatio, slope, ad cocavity of fuctios. Fid challegig atiderivatives

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Using Excel to Construct Confidence Intervals

Using Excel to Construct Confidence Intervals OPIM 303 Statistics Ja Stallaert Usig Excel to Costruct Cofidece Itervals This hadout explais how to costruct cofidece itervals i Excel for the followig cases: 1. Cofidece Itervals for the mea of a populatio

More information

4.3. The Integral and Comparison Tests

4.3. The Integral and Comparison Tests 4.3. THE INTEGRAL AND COMPARISON TESTS 9 4.3. The Itegral ad Compariso Tests 4.3.. The Itegral Test. Suppose f is a cotiuous, positive, decreasig fuctio o [, ), ad let a = f(). The the covergece or divergece

More information

2.7 Sequences, Sequences of Sets

2.7 Sequences, Sequences of Sets 2.7. SEQUENCES, SEQUENCES OF SETS 67 2.7 Sequeces, Sequeces of Sets 2.7.1 Sequeces Defiitio 190 (sequece Let S be some set. 1. A sequece i S is a fuctio f : K S where K = { N : 0 for some 0 N}. 2. For

More information

Simulation and Monte Carlo integration

Simulation and Monte Carlo integration Chapter 3 Simulatio ad Mote Carlo itegratio I this chapter we itroduce the cocept of geeratig observatios from a specified distributio or sample, which is ofte called Mote Carlo geeratio. The ame of Mote

More information

Statistical inference: example 1. Inferential Statistics

Statistical inference: example 1. Inferential Statistics Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

More information

Joint Probability Distributions and Random Samples

Joint Probability Distributions and Random Samples STAT5 Sprig 204 Lecture Notes Chapter 5 February, 204 Joit Probability Distributios ad Radom Samples 5. Joitly Distributed Radom Variables Chapter Overview Joitly distributed rv Joit mass fuctio, margial

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

Lecture 7: Borel Sets and Lebesgue Measure

Lecture 7: Borel Sets and Lebesgue Measure EE50: Probability Foudatios for Electrical Egieers July-November 205 Lecture 7: Borel Sets ad Lebesgue Measure Lecturer: Dr. Krisha Jagaatha Scribes: Ravi Kolla, Aseem Sharma, Vishakh Hegde I this lecture,

More information

AQA STATISTICS 1 REVISION NOTES

AQA STATISTICS 1 REVISION NOTES AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if

More information

Definition. Definition. 7-2 Estimating a Population Proportion. Definition. Definition

Definition. Definition. 7-2 Estimating a Population Proportion. Definition. Definition 7- stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio

More information

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1) BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

More information

1 n. n > dt. t < n 1 + n=1

1 n. n > dt. t < n 1 + n=1 Math 05 otes C. Pomerace The harmoic sum The harmoic sum is the sum of recirocals of the ositive itegers. We kow from calculus that it diverges, this is usually doe by the itegral test. There s a more

More information

Infinite Sequences and Series

Infinite Sequences and Series CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

Confidence Intervals for One Mean with Tolerance Probability

Confidence Intervals for One Mean with Tolerance Probability Chapter 421 Cofidece Itervals for Oe Mea with Tolerace Probability Itroductio This procedure calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) with

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

Lecture 5: Span, linear independence, bases, and dimension

Lecture 5: Span, linear independence, bases, and dimension Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;

More information

Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series

Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series 8 Fourier Series Our aim is to show that uder reasoable assumptios a give -periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series

More information

Divergence of p 1/p. Adrian Dudek. adrian.dudek[at]anu.edu.au

Divergence of p 1/p. Adrian Dudek. adrian.dudek[at]anu.edu.au Divergece of / Adria Dudek adria.dudek[at]au.edu.au Whe I was i high school, my maths teacher cheekily told me that it s ossible to add u ifiitely may umbers ad get a fiite umber. She the illustrated this

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

Discrete Random Variables and Probability Distributions. Random Variables. Chapter 3 3.1

Discrete Random Variables and Probability Distributions. Random Variables. Chapter 3 3.1 UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig

More information

USING STATISTICAL FUNCTIONS ON A SCIENTIFIC CALCULATOR

USING STATISTICAL FUNCTIONS ON A SCIENTIFIC CALCULATOR USING STATISTICAL FUNCTIONS ON A SCIENTIFIC CALCULATOR Objective:. Improve calculator skills eeded i a multiple choice statistical eamiatio where the eam allows the studet to use a scietific calculator..

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Theorems About Power Series

Theorems About Power Series Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by

More information

1. MATHEMATICAL INDUCTION

1. MATHEMATICAL INDUCTION 1. MATHEMATICAL INDUCTION EXAMPLE 1: Prove that for ay iteger 1. Proof: 1 + 2 + 3 +... + ( + 1 2 (1.1 STEP 1: For 1 (1.1 is true, sice 1 1(1 + 1. 2 STEP 2: Suppose (1.1 is true for some k 1, that is 1

More information

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about

More information

Lecture 4: Cheeger s Inequality

Lecture 4: Cheeger s Inequality Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

More information

Hypothesis Tests Applied to Means

Hypothesis Tests Applied to Means The Samplig Distributio of the Mea Hypothesis Tests Applied to Meas Recall that the samplig distributio of the mea is the distributio of sample meas that would be obtaied from a particular populatio (with

More information

Unbiased Estimation. Topic 14. 14.1 Introduction

Unbiased Estimation. Topic 14. 14.1 Introduction Topic 4 Ubiased Estimatio 4. Itroductio I creatig a parameter estimator, a fudametal questio is whether or ot the estimator differs from the parameter i a systematic maer. Let s examie this by lookig a

More information

Measures of Central Tendency

Measures of Central Tendency Measures of Cetral Tedecy A studet s grade will be determied by exam grades ( each exam couts twice ad there are three exams, HW average (couts oce, fial exam ( couts three times. Fid the average if the

More information

STATISTICAL METHODS FOR BUSINESS

STATISTICAL METHODS FOR BUSINESS STATISTICAL METHODS FOR BUSINESS UNIT 7: INFERENTIAL TOOLS. DISTRIBUTIONS ASSOCIATED WITH SAMPLING 7.1.- Distributios associated with the samplig process. 7.2.- Iferetial processes ad relevat distributios.

More information

THE LEAST SQUARES REGRESSION LINE and R 2

THE LEAST SQUARES REGRESSION LINE and R 2 THE LEAST SQUARES REGRESSION LINE ad R M358K I. Recall from p. 36 that the least squares regressio lie of y o x is the lie that makes the sum of the squares of the vertical distaces of the data poits from

More information

Advanced Probability Theory

Advanced Probability Theory Advaced Probability Theory Math5411 HKUST Kai Che (Istructor) Chapter 1. Law of Large Numbers 1.1. σ-algebra, measure, probability space ad radom variables. This sectio lays the ecessary rigorous foudatio

More information

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals Overview Estimatig the Value of a Parameter Usig Cofidece Itervals We apply the results about the sample mea the problem of estimatio Estimatio is the process of usig sample data estimate the value of

More information

8.1 Arithmetic Sequences

8.1 Arithmetic Sequences MCR3U Uit 8: Sequeces & Series Page 1 of 1 8.1 Arithmetic Sequeces Defiitio: A sequece is a comma separated list of ordered terms that follow a patter. Examples: 1, 2, 3, 4, 5 : a sequece of the first

More information

Estimating the Mean and Variance of a Normal Distribution

Estimating the Mean and Variance of a Normal Distribution Estimatig the Mea ad Variace of a Normal Distributio Learig Objectives After completig this module, the studet will be able to eplai the value of repeatig eperimets eplai the role of the law of large umbers

More information

Lecture 10: Hypothesis testing and confidence intervals

Lecture 10: Hypothesis testing and confidence intervals Eco 514: Probability ad Statistics Lecture 10: Hypothesis testig ad cofidece itervals Types of reasoig Deductive reasoig: Start with statemets that are assumed to be true ad use rules of logic to esure

More information

Lecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k.

Lecture 3. denote the orthogonal complement of S k. Then. 1 x S k. n. 2 x T Ax = ( ) λ x. with x = 1, we have. i = λ k x 2 = λ k. 18.409 A Algorithmist s Toolkit September 17, 009 Lecture 3 Lecturer: Joatha Keler Scribe: Adre Wibisoo 1 Outlie Today s lecture covers three mai parts: Courat-Fischer formula ad Rayleigh quotiets The

More information

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

More information

NPTEL STRUCTURAL RELIABILITY

NPTEL STRUCTURAL RELIABILITY NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics

More information

Lesson 17 Pearson s Correlation Coefficient

Lesson 17 Pearson s Correlation Coefficient Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

More information

Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork

Solutions to Selected Problems In: Pattern Classification by Duda, Hart, Stork Solutios to Selected Problems I: Patter Classificatio by Duda, Hart, Stork Joh L. Weatherwax February 4, 008 Problem Solutios Chapter Bayesia Decisio Theory Problem radomized rules Part a: Let Rx be the

More information

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here). BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

More information

Distributions of Order Statistics

Distributions of Order Statistics Chapter 2 Distributios of Order Statistics We give some importat formulae for distributios of order statistics. For example, where F k: (x)=p{x k, x} = I F(x) (k, k + 1), I x (a,b)= 1 x t a 1 (1 t) b 1

More information

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr.

Learning outcomes. Algorithms and Data Structures. Time Complexity Analysis. Time Complexity Analysis How fast is the algorithm? Prof. Dr. Algorithms ad Data Structures Algorithm efficiecy Learig outcomes Able to carry out simple asymptotic aalysisof algorithms Prof. Dr. Qi Xi 2 Time Complexity Aalysis How fast is the algorithm? Code the

More information

Stat 104 Lecture 16. Statistics 104 Lecture 16 (IPS 6.1) Confidence intervals - the general concept

Stat 104 Lecture 16. Statistics 104 Lecture 16 (IPS 6.1) Confidence intervals - the general concept Statistics 104 Lecture 16 (IPS 6.1) Outlie for today Cofidece itervals Cofidece itervals for a mea, µ (kow σ) Cofidece itervals for a proportio, p Margi of error ad sample size Review of mai topics for

More information

LECTURE 13: Cross-validation

LECTURE 13: Cross-validation LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

More information

Section 7-3 Estimating a Population. Requirements

Section 7-3 Estimating a Population. Requirements Sectio 7-3 Estimatig a Populatio Mea: σ Kow Key Cocept This sectio presets methods for usig sample data to fid a poit estimate ad cofidece iterval estimate of a populatio mea. A key requiremet i this sectio

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information