Maximum Likelihood Estimators.

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Maximum Likelihood Estimators."

Transcription

1 Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio of beta distributio later, at this poit we oly eed to kow that this isi a cotiuous distributio o the iterval [0, ]. This ca be doe by typig X=betard(5,2,00,). Let us fit differet distributios by usig a distributio fittig tool dfittool. We try to fit ormal distributio ad beta distributio to this sample ad the results are displayed i figure 2.. Desity samples ~ Beta(5,2) Normal fit Beta fit Cumulative probability samples ~ Beta(5,2) Normal fit Beta fit Figure 2.: Fittig a radom sample of size 00 from Beta(5, 2). (a) Histogram of the data ad p.d.f.s of fitted ormal (solid lie) ad beta (dashed lie) distributios; (b) Empirical c.d.f. ad c.d.f.s of fitted ormal ad beta distributios. Besides the graphs, the distributio fittig tool outputs the followig iformatio: Distributio: Normal Log likelihood:

2 Domai: -If < y < If Mea: Variace: Parameter Estimate Std. Err. mu sigma Estimated covariace of parameter estimates: mu sigma mu e-020 sigma e e-005 Distributio: Beta Log likelihood: Domai: 0 < y < Mea: Variace: Parameter Estimate Std. Err. a b Estimated covariace of parameter estimates: a b a b The value Log likelihood idicates that the tool uses the maximum likelihood estimators to fit the distributio, which will be the topic of the ext few lectures. Notice the Parameter estimates - give the data dfittool estimates the ukow parameters of the distributio ad the graphs the p.d.f. or c.d.f. correspodig to these parameters. Sice the data was geerated from beta distributio, it is ot surprisig that beta distributio fit seems better tha ormal distributio fit, which is particularly clear from figure 2. (b), that compares how estimated c.d.f. fits the empirical c.d.f. Empirical c.d.f. is defied as F (x) = I(X i x) i= where I(X x) is the idicator that X i is x. I other words, F (x) is the proportio of observatios below level x. Oe ca ask several questios about this example:. How to estimate the ukow parameters of a distributio give the data from this distributio? 8

3 2. How good are these estimates, are they close to the actual true parameters? 3. Does the data come from a particular type of distributio, for example, ormal or beta distributio? I the ext few lectures we will study the first two questios ad we will assume that we kow what type of distributio the sample comes from, so we oly do ot kow the parameters of the distributio. I the cotext of the above example, we would be told that the data comes from beta distributio, but the parameters (5, 2) would be ukow. Of course, i geeral we might ot kow what kid of distributio the data comes from - we will study this type of questios later whe we look at the so called goodess-of-fit hypotheses tests. I particular, we will see graphs like 2. (b) agai whe we study the Kolmogorov-Smirov goodess-of-fit test. Example. We cosider a dataset of various body measuremets from [] (dataset ca be dowloaded from joural s website), icludig weight, height, waist girth, abdome girth, etc. First, we use Matlab fittig tool to fit weight ad waist girth of me ad wome (separately) with logormal distributio, see figure 2.2 (a) ad (b). Wikipedia article about ormal distributio gives a referece to a 932 book Problems of Relative Growth by Julia Huxley for the explaatio why the sizes of full-grow aimals are approximately log-ormal. Oe short explaatio is cosistecy betwee liear ad volume dimesios - if liear dimesios are logormal ad volume dimesios are proportioal to cube of liear dimesios the they also are logormal. Assumptio that sizes are ormal would violate this cosistecy, sice the cube of ormal is ot ormal. We observe, hovewer, that the fit of wome s waist with logormal is ot very accurate. Later i the class we will lear several statistical tests to decide if the data comes from a certai distributio or a family of distributios, but here is a preview of what s to come. Chi-squared goodess-of-fit test rejects the hypothesis that the distributio of logarithms of wome s waists is ormal: [h,p,stats]=chi2gof(log_wome_waist) h =, p = e-004 stats = chi2stat: df: 5 edges: [x9 double] O: [ ] E: [x8 double] ad so does Lilliefor s test (adjusted Kolmogorov-Smirov test): [h,p,stats]=lillietest(log_wome_waist) h =, p = 0, stats = The same tests accept the hypotheses that other variables have logormal distributio. Author s i [] suggest that we ca fit wome s waist with Gamma distributio. Sice Gamma 9

4 Cumulative probability wome s weight logormal fit wome me s weight logormal fit me Cumulative probability wome s waist logormal fit wome me s waist logormal fit me Cumulative probability wome s waist (shifted) Gamma fit ormal fit Figure 2.2: Fittig weight (upper left) ad waist girth (upper right) with logormal distributio. Lower left: fittig wome s waist with shifted Gamma ad ormal distributios. does ot have a traslatio (shift) parameter, whe we fit Gamma distributio we ca either add to it a shift parameter or istead shift all data to start at zero. I figure 2.2 (c) we fit Gamma ad, for the sake of illustratio, ormal distributio, to wome s waist sample. As we ca see, Gamma fits the data better tha logormal ad much better tha ormal. To fid the parameters of fitted Gamma distributio we use Matlab gamfit fuctio: param=gamfit(wome_waist_shift) param = Chi-squared goodess-of-fit test for a specific (fitted) Gamma distributio: [h,p,stats]=chi2gof(wome_waist_shift, 0

5 h = 0, p = , stats = chi2stat: , df: 7 accepts the hypothesis that the sample has Gamma distributio (2.87, 4.496). This test is ot accurate i some sese, which will be explaied later. Oe ca also check that Gamma distributio fits well other variables - me s waist girth, weight of me ad weight of wome. Let us cosider a family of distributios P idexed by a parameter (which could be a vector of parameters) ϕ that belogs to a set. For example, we could cosider a family of ormal distributios N(, α 2 ) i which case the parameter would be ϕ = (, α 2 ) - the mea ad variace of the distributio. Let f(x ϕ) be either a probability fuctio (i case of discrete distributio) or a probability desity fuctio (cotiuous case) of the distributio P. Suppose we are give a i.i.d. sample X,..., X with ukow distributio P from this family, i.e. parameter ϕ is ukow. A likelihood fuctio is defied by (ϕ) = f(x ϕ)... f(x ϕ). We thik of the sample X,..., X as give umbers ad we thik of as a fuctio of the parameter ϕ oly. The likelihood fuctio has a clear iterpretatio. For example, if our distributios are discrete the the probability fuctio f(x ϕ) = P (X = x) is the probability to observe a poit x ad the likelihood fuctio (ϕ) = f(x ϕ)... f(x ϕ) = P (X )... P (X ) = P (X,..., X ) is the probability to observe the sample X,..., X whe the parameters of the distributio are equal to ϕ. I the cotiuous case the likelihood fuctio (ϕ) is the probability desity fuctio of the vector (X,..., X ). Defiitio: (Maximum Likelihood Estimators.) Suppose that there exists a parameter ϕˆ that maximizes the likelihood fuctio (ϕ) o the set of possible parameters, i.e. (ϕˆ) = max (ϕ). The ϕˆ is called the Maximum Likelihood Estimator (MLE). Whe fidig the MLE it sometimes easier to maximize the log-likelihood fuctio sice (ϕ) maximize log (ϕ) maximize maximizig is equivalet to maximizig log. Log-likelihood fuctio ca be writte as log (ϕ) = log f(x i ϕ). i= Let us give several examples of computig the MLE.

6 Example. Beroulli distributio B(p). X = {0, }, P(X = ) = p, P(X = 0) = p, p [0, ]. Probability fuctio i this case is give by p, x = f(x p) = p, x = 0 = p x ( p) x. Likelihood fuctio is (p) = f(x p)f(x 2 p)... f(x p) # of s ( p) # of 0 s X +...+X = p = p ( p) (X +...+X ) ad the log-likelihood fuctio is log (p) = (X X ) log p + ( (X X )) log( p). To maximize this over p [0, ] let us fid the critical poit (log (p)) = 0, (X X ) ( (X X )) = 0. p p Solvig this for p gives, X X p = = X ad, therefore, the proportio of successes pˆ = X i the sample is the MLEstimator of the ukow true probability of success, which is a very atural ad ituitive estimator. For example, by law of large umbers, we kow that X EX = p i probability (we will recall this defiitio i the ext lecture), which meas that our estimate will approximate the ukow parameter p well whe we get more ad more data. Remark. I each example, oce we compute the estimate of parameters, we ca try to prove directly, usig the explicit form of the estimate, that it approximates well the ukow parameters, as we did i Example. However, i the ext lecture we will describe i a geeral settig that MLE has good properties. Example 2. Normal distributio N(, α 2 ). The p.d.f. of ormal distributio is ad, therefore, likelihood fuctio is (X ) f(x (, α 2 )) = 2α e. (, α 2 ) = e (X i ) α i= 2

7 ad log-likelihood fuctio is log (, α 2 ) = (X i ) 2 log log α 2 2α 2 i= d (X i ) 2 = 2 (X i ) = 0 log log α (X i X ) (X i X ) 2 = 0 α α 3 αˆ2 = (X X ) 2 i. The ormal distributio fit i figure 2. correspods to these parameters (ˆ, αˆ2). Exercise. Geerate a ormal sample i Matlab ad fit it with a ormal distributio usig dfittool. The plot a p.d.f. or c.d.f. correspodig to MLE above ad compare this with dfittool. Let us give oe more example of MLE. Uiform distributio U[0, ϕ] o the iterval [0, ϕ]. This distributio has p.d.f., 0 x ϕ, f(x ϕ) = 0, otherwise. = log log α (X i ) α 2 We wat to maximize the log-likelihood with respect to < < ad α 2 > 0. First, obviously, for ay α we eed to miimize (X i ) 2 over. The critical poit coditio is d i= i= ad solvig this for we get that ˆ = X. We ca plug this estimate i the log-likelihood ad it remais to maximize over α. The critical poit coditio reads, 2α 2 i= ad solvig this for α we obtai that the MLE of α 2 is i= i= The likelihood fuctio (ϕ) = f(x i ϕ) = i= = ϕ I(X,..., X [0, ϕ]) ϕ I(max(X,..., X ) ϕ). 3

8 Here the idicator fuctio I(A) equals to if evet A happes ad 0 otherwise. What the idicator above meas is that the likelihood will be equal to 0 if at least oe of the factors is 0 ad this will happe if at least oe observatio X i will fall outside of the allowed iterval [0, ϕ]. Aother way to say it is that the maximum amog observatios will exceed ϕ, i.e. ad (ϕ) = 0 if ϕ < max(x,..., X ), (ϕ) = if ϕ max(x,..., X ). ϕ Therefore, lookig at the figure 2.3 we see that ϕˆ = max(x,..., X ) is the MLE (ϕ) PSfrag replacemets max(x,..., X ) ϕ Figure 2.3: MLE for the uiform distributio. Sometimes it is ot so easy to fid the maximum of the likelihood fuctio as i the examples above ad oe might have to do it umerically. Also, MLE does ot always exist. Here is a example: let us cosider uiform distributio U[0, ϕ) ad defie the desity by, 0 x < ϕ, f(x ϕ) = 0, otherwise. The differece is that we excluded the poit ϕ by settig f(ϕ ϕ) = 0. The the likelihood fuctio is ϕ i= (ϕ) = f(x i ϕ) = I(max(X,..., X ) < ϕ) 4

9 ad the maximum at the poit ϕˆ = max(x,..., X ) is ot achieved. Of course, this is a artificial example that shows that sometimes oe eeds to be careful. Refereces: [] Grete Heiz, Louis J. Peterso, Roger W. Johso, Carter J. Kerk, (2003) Explorig Relatioships i Body Dimesios. Joural of Statistics Educatio, Volume, Number 2. 5

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

Sampling Distribution And Central Limit Theorem

Sampling Distribution And Central Limit Theorem () Samplig Distributio & Cetral Limit Samplig Distributio Ad Cetral Limit Samplig distributio of the sample mea If we sample a umber of samples (say k samples where k is very large umber) each of size,

More information

Chapter 14 Nonparametric Statistics

Chapter 14 Nonparametric Statistics Chapter 14 Noparametric Statistics A.K.A. distributio-free statistics! Does ot deped o the populatio fittig ay particular type of distributio (e.g, ormal). Sice these methods make fewer assumptios, they

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

Lesson 17 Pearson s Correlation Coefficient

Lesson 17 Pearson s Correlation Coefficient Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

More information

1 Hypothesis testing for a single mean

1 Hypothesis testing for a single mean BST 140.65 Hypothesis Testig Review otes 1 Hypothesis testig for a sigle mea 1. The ull, or status quo, hypothesis is labeled H 0, the alterative H a or H 1 or H.... A type I error occurs whe we falsely

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Joint Probability Distributions and Random Samples

Joint Probability Distributions and Random Samples STAT5 Sprig 204 Lecture Notes Chapter 5 February, 204 Joit Probability Distributios ad Radom Samples 5. Joitly Distributed Radom Variables Chapter Overview Joitly distributed rv Joit mass fuctio, margial

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

One-sample test of proportions

One-sample test of proportions Oe-sample test of proportios The Settig: Idividuals i some populatio ca be classified ito oe of two categories. You wat to make iferece about the proportio i each category, so you draw a sample. Examples:

More information

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means) CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

Notes on Hypothesis Testing

Notes on Hypothesis Testing Probability & Statistics Grishpa Notes o Hypothesis Testig A radom sample X = X 1,..., X is observed, with joit pmf/pdf f θ x 1,..., x. The values x = x 1,..., x of X lie i some sample space X. The parameter

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

1 Correlation and Regression Analysis

1 Correlation and Regression Analysis 1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Definition. Definition. 7-2 Estimating a Population Proportion. Definition. Definition

Definition. Definition. 7-2 Estimating a Population Proportion. Definition. Definition 7- stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio

More information

NPTEL STRUCTURAL RELIABILITY

NPTEL STRUCTURAL RELIABILITY NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

Stat 104 Lecture 16. Statistics 104 Lecture 16 (IPS 6.1) Confidence intervals - the general concept

Stat 104 Lecture 16. Statistics 104 Lecture 16 (IPS 6.1) Confidence intervals - the general concept Statistics 104 Lecture 16 (IPS 6.1) Outlie for today Cofidece itervals Cofidece itervals for a mea, µ (kow σ) Cofidece itervals for a proportio, p Margi of error ad sample size Review of mai topics for

More information

Lecture 4: Cheeger s Inequality

Lecture 4: Cheeger s Inequality Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

More information

Confidence Intervals. CI for a population mean (σ is known and n > 30 or the variable is normally distributed in the.

Confidence Intervals. CI for a population mean (σ is known and n > 30 or the variable is normally distributed in the. Cofidece Itervals A cofidece iterval is a iterval whose purpose is to estimate a parameter (a umber that could, i theory, be calculated from the populatio, if measuremets were available for the whole populatio).

More information

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about

More information

2.7 Sequences, Sequences of Sets

2.7 Sequences, Sequences of Sets 2.7. SEQUENCES, SEQUENCES OF SETS 67 2.7 Sequeces, Sequeces of Sets 2.7.1 Sequeces Defiitio 190 (sequece Let S be some set. 1. A sequece i S is a fuctio f : K S where K = { N : 0 for some 0 N}. 2. For

More information

x : X bar Mean (i.e. Average) of a sample

x : X bar Mean (i.e. Average) of a sample A quick referece for symbols ad formulas covered i COGS14: MEAN OF SAMPLE: x = x i x : X bar Mea (i.e. Average) of a sample x i : X sub i This stads for each idividual value you have i your sample. For

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

Distributions of Order Statistics

Distributions of Order Statistics Chapter 2 Distributios of Order Statistics We give some importat formulae for distributios of order statistics. For example, where F k: (x)=p{x k, x} = I F(x) (k, k + 1), I x (a,b)= 1 x t a 1 (1 t) b 1

More information

Math C067 Sampling Distributions

Math C067 Sampling Distributions Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

More information

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

More information

Measures of Central Tendency

Measures of Central Tendency Measures of Cetral Tedecy A studet s grade will be determied by exam grades ( each exam couts twice ad there are three exams, HW average (couts oce, fial exam ( couts three times. Fid the average if the

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

, a Wishart distribution with n -1 degrees of freedom and scale matrix.

, a Wishart distribution with n -1 degrees of freedom and scale matrix. UMEÅ UNIVERSITET Matematisk-statistiska istitutioe Multivariat dataaalys D MSTD79 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multivariat dataaalys D, 5 poäg.. Assume that

More information

AQA STATISTICS 1 REVISION NOTES

AQA STATISTICS 1 REVISION NOTES AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if

More information

Lesson 15 ANOVA (analysis of variance)

Lesson 15 ANOVA (analysis of variance) Outlie Variability -betwee group variability -withi group variability -total variability -F-ratio Computatio -sums of squares (betwee/withi/total -degrees of freedom (betwee/withi/total -mea square (betwee/withi

More information

3. Greatest Common Divisor - Least Common Multiple

3. Greatest Common Divisor - Least Common Multiple 3 Greatest Commo Divisor - Least Commo Multiple Defiitio 31: The greatest commo divisor of two atural umbers a ad b is the largest atural umber c which divides both a ad b We deote the greatest commo gcd

More information

Using Excel to Construct Confidence Intervals

Using Excel to Construct Confidence Intervals OPIM 303 Statistics Ja Stallaert Usig Excel to Costruct Cofidece Itervals This hadout explais how to costruct cofidece itervals i Excel for the followig cases: 1. Cofidece Itervals for the mea of a populatio

More information

Parametric (theoretical) probability distributions. (Wilks, Ch. 4) Discrete distributions: (e.g., yes/no; above normal, normal, below normal)

Parametric (theoretical) probability distributions. (Wilks, Ch. 4) Discrete distributions: (e.g., yes/no; above normal, normal, below normal) 6 Parametric (theoretical) probability distributios. (Wilks, Ch. 4) Note: parametric: assume a theoretical distributio (e.g., Gauss) No-parametric: o assumptio made about the distributio Advatages of assumig

More information

Confidence Intervals for the Population Mean

Confidence Intervals for the Population Mean Cofidece Itervals Math 283 Cofidece Itervals for the Populatio Mea Recall that from the empirical rule that the iterval of the mea plus/mius 2 times the stadard deviatio will cotai about 95% of the observatios.

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

Descriptive Statistics

Descriptive Statistics Descriptive Statistics We leared to describe data sets graphically. We ca also describe a data set umerically. Measures of Locatio Defiitio The sample mea is the arithmetic average of values. We deote

More information

3. Continuous Random Variables

3. Continuous Random Variables Statistics ad probability: 3-1 3. Cotiuous Radom Variables A cotiuous radom variable is a radom variable which ca take values measured o a cotiuous scale e.g. weights, stregths, times or legths. For ay

More information

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number. GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all

More information

Chapter 10. Hypothesis Tests Regarding a Parameter. 10.1 The Language of Hypothesis Testing

Chapter 10. Hypothesis Tests Regarding a Parameter. 10.1 The Language of Hypothesis Testing Chapter 10 Hypothesis Tests Regardig a Parameter A secod type of statistical iferece is hypothesis testig. Here, rather tha use either a poit (or iterval) estimate from a simple radom sample to approximate

More information

Statistics Lecture 14. Introduction to Inference. Administrative Notes. Hypothesis Tests. Last Class: Confidence Intervals

Statistics Lecture 14. Introduction to Inference. Administrative Notes. Hypothesis Tests. Last Class: Confidence Intervals Statistics 111 - Lecture 14 Itroductio to Iferece Hypothesis Tests Admiistrative Notes Sprig Break! No lectures o Tuesday, March 8 th ad Thursday March 10 th Exteded Sprig Break! There is o Stat 111 recitatio

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

Lecture 5: Span, linear independence, bases, and dimension

Lecture 5: Span, linear independence, bases, and dimension Lecture 5: Spa, liear idepedece, bases, ad dimesio Travis Schedler Thurs, Sep 23, 2010 (versio: 9/21 9:55 PM) 1 Motivatio Motivatio To uderstad what it meas that R has dimesio oe, R 2 dimesio 2, etc.;

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

Lecture 10: Hypothesis testing and confidence intervals

Lecture 10: Hypothesis testing and confidence intervals Eco 514: Probability ad Statistics Lecture 10: Hypothesis testig ad cofidece itervals Types of reasoig Deductive reasoig: Start with statemets that are assumed to be true ad use rules of logic to esure

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

CHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Means and Proportions

CHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Means and Proportions CHAPTER 8: CONFIDENCE INTERVAL ESTIMATES for Meas ad Proportios Itroductio: We wat to kow the value of a parameter for a populatio. We do t kow the value of this parameter for the etire populatio because

More information

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals

Overview. Learning Objectives. Point Estimate. Estimation. Estimating the Value of a Parameter Using Confidence Intervals Overview Estimatig the Value of a Parameter Usig Cofidece Itervals We apply the results about the sample mea the problem of estimatio Estimatio is the process of usig sample data estimate the value of

More information

Simple Linear Regression

Simple Linear Regression Simple Liear Regressio We have bee itroduced to the otio that a categorical variable could deped o differet levels of aother variable whe we discussed cotigecy tables. We ll exted this idea to the case

More information

Inference on Proportion. Chapter 8 Tests of Statistical Hypotheses. Sampling Distribution of Sample Proportion. Confidence Interval

Inference on Proportion. Chapter 8 Tests of Statistical Hypotheses. Sampling Distribution of Sample Proportion. Confidence Interval Chapter 8 Tests of Statistical Hypotheses 8. Tests about Proportios HT - Iferece o Proportio Parameter: Populatio Proportio p (or π) (Percetage of people has o health isurace) x Statistic: Sample Proportio

More information

Economics 140A Confidence Intervals and Hypothesis Testing

Economics 140A Confidence Intervals and Hypothesis Testing Ecoomics 140A Cofidece Itervals ad Hypothesis Testig Obtaiig a estimate of a parameter is ot the al purpose of statistical iferece because it is highly ulikely that the populatio value of a parameter is

More information

Gregory Carey, 1998 Linear Transformations & Composites - 1. Linear Transformations and Linear Composites

Gregory Carey, 1998 Linear Transformations & Composites - 1. Linear Transformations and Linear Composites Gregory Carey, 1998 Liear Trasformatios & Composites - 1 Liear Trasformatios ad Liear Composites I Liear Trasformatios of Variables Meas ad Stadard Deviatios of Liear Trasformatios A liear trasformatio

More information

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1) BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

More information

Confidence Intervals for One Mean with Tolerance Probability

Confidence Intervals for One Mean with Tolerance Probability Chapter 421 Cofidece Itervals for Oe Mea with Tolerace Probability Itroductio This procedure calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) with

More information

Module 4: Mathematical Induction

Module 4: Mathematical Induction Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Statistical Methods. Chapter 1: Overview and Descriptive Statistics

Statistical Methods. Chapter 1: Overview and Descriptive Statistics Geeral Itroductio Statistical Methods Chapter 1: Overview ad Descriptive Statistics Statistics studies data, populatio, ad samples. Descriptive Statistics vs Iferetial Statistics. Descriptive Statistics

More information

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here). BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

Confidence Intervals and Sample Size

Confidence Intervals and Sample Size 8/7/015 C H A P T E R S E V E N Cofidece Itervals ad Copyright 015 The McGraw-Hill Compaies, Ic. Permissio required for reproductio or display. 1 Cofidece Itervals ad Outlie 7-1 Cofidece Itervals for the

More information

Robust and Resistant Regression

Robust and Resistant Regression Chapter 13 Robust ad Resistat Regressio Whe the errors are ormal, least squares regressio is clearly best but whe the errors are oormal, other methods may be cosidered. A particular cocer is log-tailed

More information

Chapter 5: Inner Product Spaces

Chapter 5: Inner Product Spaces Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

More information

Chapter 7: Confidence Interval and Sample Size

Chapter 7: Confidence Interval and Sample Size Chapter 7: Cofidece Iterval ad Sample Size Learig Objectives Upo successful completio of Chapter 7, you will be able to: Fid the cofidece iterval for the mea, proportio, ad variace. Determie the miimum

More information

Confidence Intervals

Confidence Intervals Cofidece Itervals Cofidece Itervals are a extesio of the cocept of Margi of Error which we met earlier i this course. Remember we saw: The sample proportio will differ from the populatio proportio by more

More information

Research Method (I) --Knowledge on Sampling (Simple Random Sampling)

Research Method (I) --Knowledge on Sampling (Simple Random Sampling) Research Method (I) --Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact

More information

1 The Gaussian channel

1 The Gaussian channel ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.

More information

MEI Structured Mathematics. Module Summary Sheets. Statistics 2 (Version B: reference to new book)

MEI Structured Mathematics. Module Summary Sheets. Statistics 2 (Version B: reference to new book) MEI Mathematics i Educatio ad Idustry MEI Structured Mathematics Module Summary Sheets Statistics (Versio B: referece to ew book) Topic : The Poisso Distributio Topic : The Normal Distributio Topic 3:

More information

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed. This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

More information

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or p-series (the Compariso Test), but of

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

15.075 Exam 3. Instructor: Cynthia Rudin TA: Dimitrios Bisias. November 22, 2011

15.075 Exam 3. Instructor: Cynthia Rudin TA: Dimitrios Bisias. November 22, 2011 15.075 Exam 3 Istructor: Cythia Rudi TA: Dimitrios Bisias November 22, 2011 Gradig is based o demostratio of coceptual uderstadig, so you eed to show all of your work. Problem 1 A compay makes high-defiitio

More information

The Euler Totient, the Möbius and the Divisor Functions

The Euler Totient, the Möbius and the Divisor Functions The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship

More information

Chapter 10 Student Lecture Notes 10-1

Chapter 10 Student Lecture Notes 10-1 Chapter 0 tudet Lecture Notes 0- Basic Busiess tatistics (9 th Editio) Chapter 0 Two-ample Tests with Numerical Data 004 Pretice-Hall, Ic. Chap 0- Chapter Topics Comparig Two Idepedet amples Z test for

More information

A gentle introduction to Expectation Maximization

A gentle introduction to Expectation Maximization A getle itroductio to Expectatio Maximizatio Mark Johso Brow Uiversity November 2009 1 / 15 Outlie What is Expectatio Maximizatio? Mixture models ad clusterig EM for setece topic modelig 2 / 15 Why Expectatio

More information

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio

More information

Descriptive statistics deals with the description or simple analysis of population or sample data.

Descriptive statistics deals with the description or simple analysis of population or sample data. Descriptive statistics Some basic cocepts A populatio is a fiite or ifiite collectio of idividuals or objects. Ofte it is impossible or impractical to get data o all the members of the populatio ad a small

More information

Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is

Trigonometric Form of a Complex Number. The Complex Plane. axis. ( 2, 1) or 2 i FIGURE 6.44. The absolute value of the complex number z a bi is 0_0605.qxd /5/05 0:45 AM Page 470 470 Chapter 6 Additioal Topics i Trigoometry 6.5 Trigoometric Form of a Complex Number What you should lear Plot complex umbers i the complex plae ad fid absolute values

More information

Institute for the Advancement of University Learning & Department of Statistics

Institute for the Advancement of University Learning & Department of Statistics Istitute for the Advacemet of Uiversity Learig & Departmet of Statistics Descriptive Statistics for Research (Hilary Term, 00) Lecture 5: Cofidece Itervals (I.) Itroductio Cofidece itervals (or regios)

More information

Alternatives To Pearson s and Spearman s Correlation Coefficients

Alternatives To Pearson s and Spearman s Correlation Coefficients Alteratives To Pearso s ad Spearma s Correlatio Coefficiets Floreti Smaradache Chair of Math & Scieces Departmet Uiversity of New Mexico Gallup, NM 8730, USA Abstract. This article presets several alteratives

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find 1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information