Simulation and Monte Carlo integration

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Simulation and Monte Carlo integration"

Transcription

1 Chapter 3 Simulatio ad Mote Carlo itegratio I this chapter we itroduce the cocept of geeratig observatios from a specified distributio or sample, which is ofte called Mote Carlo geeratio. The ame of Mote Carlo was applied to a class of mathematical methods first by scietists workig o the developmet of uclear weapos i Los Alamos i the 1940s. For history of Mote Carlo see Kalos ad Whitelock (1986), Mote Carlo Methods, Vol. I: Basics, Wiley. 3.1 The law of large umbers There are two types of laws of large umbers, a strog oe ad a weak oe. They may be thought of as the theoretical backgroud for Mote Carlo methods. Below is a strog law of large umbers. Theorem 1. If X 1, X 2,..., X,... are idepedet ad idetically distributed radom variables with EX k µ, k 1, 2,..., the ( k1 P lim X ) k µ 1. Proof. See page 85 of Billigsley, P. (1995), Probability ad Measure, 3rd editio, Wiley. The ext oe is a weak law of large umbers. Theorem 2. If X 1, X 2,..., X,... are idepedet ad idetically distributed radom variables with EX k µ ad E(X k µ) 2 σ 2, k 1, 2,..., the, for ay positive costat ε, ( lim P k1 X ) k µ ε 0. Proof. It is ot so hard, ad is left as a homework. 41

2 42 CHAPTER 3. SIMULATION AND MONTE CARLO INTEGRATION 3.2 The hit-or-miss method Cosider a evet A whose probability is p. Defie a sequece of idepedet ad idetically distributed Beroulli radom variables X 1, X 2,..., X by 1, if evet A occurs, X k k 1, 2,... 0, otherwise, You may imagie {X 1 as hit ad {X 0 as miss. The momet estimator ad maximum likelihood estimator of p based o X 1, X 2,..., X are the same, with k1 ˆp X k. This estimator is ubiased, that is, E ˆp p. More importatly, it follows from the strog law of large umbers that ( k1 P lim X ) k p 1. The earliest documeted use of radom samplig to fid the solutio to a itegral seems to be that of Comte de Buffo, which is ofte called Buffo s eedle problem. Here we estimate π by the hit-or miss method. Example 1 (Estimatio of π). Suppose that a disk with radius 1 meter is put iside a square whose legth is 2 meters. We toss a little eedle to hit the square. The questio is: what is the probability that the eedle lies withi the disk? The aswer is π 4. To perform the Mote Carlo method, let U 1 ad U 2 be idepedet radom variables uiformly distributed o the iterval [ 1, 1], ad they may be treated as two sides of a square. Defie a radom variable that represets the evet of hittig the disk, { 1, if U 2 X 1 + U2 2 1, 0, otherwise. The EX P (U U 2 2 1) π 4. For a sequece of such i.i.d. variables, X 1,..., X, by the strog law of large umbers we obtai with probability 1. lim k1 X k π 4,

3 3.3. MONTE CARLO INTEGRATION 43 pimcfuctio(){ # is the umber of simulatios u1ruif(, -1, 1) # geerate a uiform variable U_1 u2ruif(, -1, 1) # U_2 xrep(0, ) # format of X x[u1^2 + u2^2 <1]1 # X variable pimc4*mea(x) # pi estimator pimc > pimc(10) [1] 4 > pimc(100) [1] 3.28 > pimc(1000) [1] 3.06 > pimc(10000) [1] > pimc(100000) [1] > pimc( ) [1] Mote Carlo itegratio Suppose that g(x), x [0, 1], is a real ad cotiuous fuctio. The questio is: how to estimate the itegral g(x)dx? There may be may approximatios for this itegral. Here we use the Mote Carlo method. 1 0 Example 2 (Estimatio of π). Cosider the fuctio It is easy to show that 1 0 g(u)du π. g(u) 4 1 u 2, u [0, 1]. To use the Mote Carlo method, suppose that U 1, U 2,..., U are idepedet radom variables uiformly distributed o the iterval [0, 1]. The X 1 g(u 1 ), X 2 g(u 2 ),..., X g(u ) are i.i.d radom variables, with mea EX k Eg(U k ) 1 0 g(u)du π. pimcfuctio(){ # is the umber of simulatios uruif() # geerate a sequece of uiform U radom umbers g4*sqrt(1-u^2) # g(u) pimcmea(g) # pi estimator pimc

4 44 CHAPTER 3. SIMULATION AND MONTE CARLO INTEGRATION 3.4 Geeratig radom umbers This sectio describes some approaches to geerate radom variables or vectors with specified distributios. Whe talkig about geerate a radom object, we mea a algorithm whose output is a object of the desired type. To geerate a radom variable (or vector) havig a target distributio F (x) o R d, we typically start from a sequece of i.i.d. uiform radom umbers. Thus, our task is: give a i.i.d. sequece U 1, U 2,... that follow a uiform distributio o the iterval [0, 1], fid m d ad a determiistic fuctio g : [0, 1] m R d such that the distributio fuctio of g(u 1,..., U m ) is F (x) The iverse method Theorem. Let X be a radom variable with distributio fuctio F (x). (1) For every t [0, 1], P (F (X) t) t. I particular, if F (x) is cotiuous o the real lie, the U F (X) is uiformly distributed o [0, 1]. (2) Defie the iverse of F as F 1 (y) if{x, F (x) y, 0 < y < 1. If U is a uiform radom variable o [0, 1], the F 1 (U) has distributio fuctio F (x). Example 1. A expoetial desity fuctio is of the form { λ exp( λx), x > 0, f(x) 0, x 0, where λ is a positive costat. The cumulative distributio fuctio { 1 exp( λx), x > 0, F (x) 0, x 0, is strictly icreasig ad cotiuous o [0, ), with the iverse F 1 (x) 1 l(1 x), 0 < x < 1. λ Thus, we could use the followig procedure to geerate expoetial radom umbers. rexp fuctio(, lambda){ u ruif() # geerate uiform radom umbers g -1/lambda*log(1-u) # expoetial radom umbers g

5 3.4. GENERATING RANDOM NUMBERS 45 Oe way to see whether a data set comes from a particular distributio is the q-q plot, which is based o sample order statistics (or sample quatiles) ad theoretical quatiles. For a set of observatios x 1, x 2,..., x, we put them i order, x 1: x 2: x :. ( ) k If the target distributio fuctio is F (x), the the quatile for +1 is F 1 k +1, k 1,...,. A q-q plot is a scatterplot of poits ( ( )) k x k:, F 1, k 1,...,. + 1 These poits would be close to a straight lie if the data set actually comes from F ( ). Below is a example for a expoetial case. x rexp(50, 1) # geerate 50 expoetial radom umbers sq sort(x) # sample quatiles tq fuctio(u){-log(1-u) # expoetial quatile fuctio with lambda1 plot(sq, tq(1:50/51), xlab"sample quatiles", ylab"theoretical quatiles") The ext example is for geeratig ormal observatios, but ot usig the iverse method. Example 2 (Geeratig ormal observatios). To simulate ormal radom variables, Box ad Muller (1958) suggested the followig procedure. Let U 1 ad U 2 be idepedet radom variables uiformly distributed o the iterval (0, 1). Defie X 1 ad X 2 by X 1 2 l U 1 cos(2πu 2 ), X 2 2 l U 1 si(2πu 2 ). It ca be show that X 1 ad X 2 are idepedet stadard ormal radom variables. rormalfuctio(){ u1ruif() u2ruif() x (-2*log(u1))^(1/2)*cos(2*pi*u2) x qqorm(rormal(100)) qqlie(rormal(100)) Rejectio method We wat to geerate radom umbers from a target distributio with the desity f(x).

6 46 CHAPTER 3. SIMULATION AND MONTE CARLO INTEGRATION Suppose that f(x) ca be rewritte as f(x) h(x)g(x), x R, h(u)h(u)du where h(x) is a oegative fuctio, ad g(x) is aother desity fuctio that has a simple form. The rejectio procedure is: Step 1. Geerate Y g(x). Step 2. Geerate U uiform(0, 1). Step 3. Accept Y is U h(y ) 1, otherwise go back to Step 1. There are two discrete evets ivolved i the above procedure: a success (Y is accepted), ad o success (Y is rejected). To cofirm the procedure makes sese, we eed evaluate the coditioal probability P (Y x success). Note that P (Y xad success) P (Y x, U h(y ) 1) (joit desity of Y ad U) dydu y x y x x y x,u h(y) 1 y x,u h(y) 1 ( du u h(y) 1 h(y)g(y)dy g(y) 1dydu ) g(y)dy h(y)g(y)dy, x R, from which we obtai P (success) lim P (Y x ad success) h(y)g(y)dy. x Thus, P (Y x success) P (Y x ad success) P (success) x h(y)g(y)dy, x R. h(y)g(y)dy This meas the above procedure works theoretically.

7 3.4. GENERATING RANDOM NUMBERS 47 Example 1. Let f(x) 4 π 1 1+x 2, 0 < x < 1, 0, otherwise. We will use the rejectio method to geerate radom umbers from this distributio. Choose g(x) to be a uiform desity, 1, 0 < x < 1, g(x) 0, otherwise, ad h(x) f(x). The, the algorithm is re fuctio(){ hfuctio(x){4/pi*(1+x^2)^(-1) y ruif() u ruif() re y[u < h(y) & h(y) <1] #if(u>h(y)) repeat re > re(100) [1] [8] [15] [22] [29] [36] [43] Example 2. Cosider the followig algorithm that was proposed by Marsaglia ad Bray 1 : (1) Geerate U ad V idepedet uiform ( 1, 1) radom variables. (2) Set W U 2 + V 2. (3) If W > 1, go to Step (1). (4) Set Z ( 2 l W )/W, ad let X 1 UZ ad X 2 V Z. Show that the radom variables X 1 ad X 2 are idepedet ad ormally distributed with mea zero ad variace 1. Use this algorithm to geerate 100 ormal radom umbers, ad obtai a ormal Q-Q plot. 1 Marsaglia ad Bray (1964), A coveiet method for geeratig ormal variables, SIAM Review, vol. 6,

8 48 CHAPTER 3. SIMULATION AND MONTE CARLO INTEGRATION 3.5 Importace samplig Suppose that we have a -dimesioal itegral I g(x)f(x)dx R that eeds to evaluate, where f(x) is a desity fuctio. The Mote Carlo procedure is to take a radom sample X 1,..., X from this distributio, ad the to form the mea g 1 g(x k ). Whe R g 2 (x)f(x)dx <, the strog large umber law asserts that with probability 1. Sice g estimates I, we ca write 1 k1 g(x k ) I, k1 g I + error. This error is a radom variable, with mea 0 ad variace var(g ) 1 { g 2 (x)f(x)dx I 2. R What we wat is to reduce the error (variace). But, f(x) is ot ecessarily the best desity fuctio to use i the Mote Carlo simulatio eve though it appears i the itegrad. A differet desity fuctio, f(x), ca be itroduced ito the itegral as follows: g(x)f(x) I f(x)dx, R f(x) where f(x) 0, f(x)dx R 1, ad g(x)f(x) < except for perhaps a (coutable) set of poits. How to f(x) choose a appropriate f is a trick i importace samplig.

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

1 Introduction to reducing variance in Monte Carlo simulations

1 Introduction to reducing variance in Monte Carlo simulations Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

Estimating the Mean and Variance of a Normal Distribution

Estimating the Mean and Variance of a Normal Distribution Estimatig the Mea ad Variace of a Normal Distributio Learig Objectives After completig this module, the studet will be able to eplai the value of repeatig eperimets eplai the role of the law of large umbers

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

Notes on Hypothesis Testing

Notes on Hypothesis Testing Probability & Statistics Grishpa Notes o Hypothesis Testig A radom sample X = X 1,..., X is observed, with joit pmf/pdf f θ x 1,..., x. The values x = x 1,..., x of X lie i some sample space X. The parameter

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

Joint Probability Distributions and Random Samples

Joint Probability Distributions and Random Samples STAT5 Sprig 204 Lecture Notes Chapter 5 February, 204 Joit Probability Distributios ad Radom Samples 5. Joitly Distributed Radom Variables Chapter Overview Joitly distributed rv Joit mass fuctio, margial

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Chapter 4 Multivariate distributions

Chapter 4 Multivariate distributions Chapter 4 Multivariate distributios k Multivariate Distributios All the results derived for the bivariate case ca be geeralized to RV. The joit CDF of,,, k will have the form: P(x, x,, x k ) whe the RVs

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means)

CHAPTER 7: Central Limit Theorem: CLT for Averages (Means) CHAPTER 7: Cetral Limit Theorem: CLT for Averages (Meas) X = the umber obtaied whe rollig oe six sided die oce. If we roll a six sided die oce, the mea of the probability distributio is X P(X = x) Simulatio:

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

Inverse Gaussian Distribution

Inverse Gaussian Distribution 5 Kauhisa Matsuda All rights reserved. Iverse Gaussia Distributio Abstract Kauhisa Matsuda Departmet of Ecoomics The Graduate Ceter The City Uiversity of New York 65 Fifth Aveue New York NY 6-49 Email:

More information

Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series

Our aim is to show that under reasonable assumptions a given 2π-periodic function f can be represented as convergent series 8 Fourier Series Our aim is to show that uder reasoable assumptios a give -periodic fuctio f ca be represeted as coverget series f(x) = a + (a cos x + b si x). (8.) By defiitio, the covergece of the series

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

More information

The Euler Totient, the Möbius and the Divisor Functions

The Euler Totient, the Möbius and the Divisor Functions The Euler Totiet, the Möbius ad the Divisor Fuctios Rosica Dieva July 29, 2005 Mout Holyoke College South Hadley, MA 01075 1 Ackowledgemets This work was supported by the Mout Holyoke College fellowship

More information

Distributions of Order Statistics

Distributions of Order Statistics Chapter 2 Distributios of Order Statistics We give some importat formulae for distributios of order statistics. For example, where F k: (x)=p{x k, x} = I F(x) (k, k + 1), I x (a,b)= 1 x t a 1 (1 t) b 1

More information

1 The Gaussian channel

1 The Gaussian channel ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

Section 7: Free electron model

Section 7: Free electron model Physics 97 Sectio 7: ree electro model A free electro model is the simplest way to represet the electroic structure of metals. Although the free electro model is a great oversimplificatio of the reality,

More information

1 Review of Probability

1 Review of Probability Copyright c 27 by Karl Sigma 1 Review of Probability Radom variables are deoted by X, Y, Z, etc. The cumulative distributio fuctio (c.d.f.) of a radom variable X is deoted by F (x) = P (X x), < x

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

More information

Discrete Random Variables and Probability Distributions. Random Variables. Chapter 3 3.1

Discrete Random Variables and Probability Distributions. Random Variables. Chapter 3 3.1 UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig

More information

AQA STATISTICS 1 REVISION NOTES

AQA STATISTICS 1 REVISION NOTES AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1) BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

More information

Sampling Distribution And Central Limit Theorem

Sampling Distribution And Central Limit Theorem () Samplig Distributio & Cetral Limit Samplig Distributio Ad Cetral Limit Samplig distributio of the sample mea If we sample a umber of samples (say k samples where k is very large umber) each of size,

More information

Measures of Central Tendency

Measures of Central Tendency Measures of Cetral Tedecy A studet s grade will be determied by exam grades ( each exam couts twice ad there are three exams, HW average (couts oce, fial exam ( couts three times. Fid the average if the

More information

Math 113 HW #11 Solutions

Math 113 HW #11 Solutions Math 3 HW # Solutios 5. 4. (a) Estimate the area uder the graph of f(x) = x from x = to x = 4 usig four approximatig rectagles ad right edpoits. Sketch the graph ad the rectagles. Is your estimate a uderestimate

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

A gentle introduction to Expectation Maximization

A gentle introduction to Expectation Maximization A getle itroductio to Expectatio Maximizatio Mark Johso Brow Uiversity November 2009 1 / 15 Outlie What is Expectatio Maximizatio? Mixture models ad clusterig EM for setece topic modelig 2 / 15 Why Expectatio

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

15.075 Exam 3. Instructor: Cynthia Rudin TA: Dimitrios Bisias. November 22, 2011

15.075 Exam 3. Instructor: Cynthia Rudin TA: Dimitrios Bisias. November 22, 2011 15.075 Exam 3 Istructor: Cythia Rudi TA: Dimitrios Bisias November 22, 2011 Gradig is based o demostratio of coceptual uderstadig, so you eed to show all of your work. Problem 1 A compay makes high-defiitio

More information

Infinite Sequences and Series

Infinite Sequences and Series CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 11 04/01/2008. Sven Zenker

Parameter estimation for nonlinear models: Numerical approaches to solving the inverse problem. Lecture 11 04/01/2008. Sven Zenker Parameter estimatio for oliear models: Numerical approaches to solvig the iverse problem Lecture 11 04/01/2008 Sve Zeker Review: Trasformatio of radom variables Cosider probability distributio of a radom

More information

An Efficient Polynomial Approximation of the Normal Distribution Function & Its Inverse Function

An Efficient Polynomial Approximation of the Normal Distribution Function & Its Inverse Function A Efficiet Polyomial Approximatio of the Normal Distributio Fuctio & Its Iverse Fuctio Wisto A. Richards, 1 Robi Atoie, * 1 Asho Sahai, ad 3 M. Raghuadh Acharya 1 Departmet of Mathematics & Computer Sciece;

More information

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. The Weak Law of Large Numbers

Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. The Weak Law of Large Numbers Steve R. Dubar Departmet o Mathematics 203 Avery Hall Uiversity o Nebrasa-Licol Licol, NE 68588-0130 http://www.math.ul.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics i Probability Theory ad Stochastic

More information

Unbiased Estimation. Topic 14. 14.1 Introduction

Unbiased Estimation. Topic 14. 14.1 Introduction Topic 4 Ubiased Estimatio 4. Itroductio I creatig a parameter estimator, a fudametal questio is whether or ot the estimator differs from the parameter i a systematic maer. Let s examie this by lookig a

More information

Lesson 17 Pearson s Correlation Coefficient

Lesson 17 Pearson s Correlation Coefficient Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

More information

Lecture 10: Hypothesis testing and confidence intervals

Lecture 10: Hypothesis testing and confidence intervals Eco 514: Probability ad Statistics Lecture 10: Hypothesis testig ad cofidece itervals Types of reasoig Deductive reasoig: Start with statemets that are assumed to be true ad use rules of logic to esure

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

Math C067 Sampling Distributions

Math C067 Sampling Distributions Math C067 Samplig Distributios Sample Mea ad Sample Proportio Richard Beigel Some time betwee April 16, 2007 ad April 16, 2007 Examples of Samplig A pollster may try to estimate the proportio of voters

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

Theorems About Power Series

Theorems About Power Series Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

NOTES ON PROBABILITY Greg Lawler Last Updated: March 21, 2016

NOTES ON PROBABILITY Greg Lawler Last Updated: March 21, 2016 NOTES ON PROBBILITY Greg Lawler Last Updated: March 21, 2016 Overview This is a itroductio to the mathematical foudatios of probability theory. It is iteded as a supplemet or follow-up to a graduate course

More information

LECTURE 13: Cross-validation

LECTURE 13: Cross-validation LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

More information

Inference on Proportion. Chapter 8 Tests of Statistical Hypotheses. Sampling Distribution of Sample Proportion. Confidence Interval

Inference on Proportion. Chapter 8 Tests of Statistical Hypotheses. Sampling Distribution of Sample Proportion. Confidence Interval Chapter 8 Tests of Statistical Hypotheses 8. Tests about Proportios HT - Iferece o Proportio Parameter: Populatio Proportio p (or π) (Percetage of people has o health isurace) x Statistic: Sample Proportio

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number. GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

Hypergeometric Distributions

Hypergeometric Distributions 7.4 Hypergeometric Distributios Whe choosig the startig lie-up for a game, a coach obviously has to choose a differet player for each positio. Similarly, whe a uio elects delegates for a covetio or you

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles

The following example will help us understand The Sampling Distribution of the Mean. C1 C2 C3 C4 C5 50 miles 84 miles 38 miles 120 miles 48 miles The followig eample will help us uderstad The Samplig Distributio of the Mea Review: The populatio is the etire collectio of all idividuals or objects of iterest The sample is the portio of the populatio

More information

THE HEIGHT OF q-binary SEARCH TREES

THE HEIGHT OF q-binary SEARCH TREES THE HEIGHT OF q-binary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average

More information

Monte Carlo Methods and Importance Sampling

Monte Carlo Methods and Importance Sampling Lecture Notes for Stat 578C c Eric C. Aderso Statistical Geetics 2 October 999 (subbi for E.A Thompso) Mote Carlo Methods ad Importace Samplig History ad defiitio: The term Mote Carlo was apparetly first

More information

Chapter 5 Discrete Probability Distributions

Chapter 5 Discrete Probability Distributions Slides Prepared by JOHN S. LOUCKS St. Edward s Uiversity Slide Chapter 5 Discrete Probability Distributios Radom Variables Discrete Probability Distributios Epected Value ad Variace Poisso Distributio

More information

STATISTICAL METHODS FOR BUSINESS

STATISTICAL METHODS FOR BUSINESS STATISTICAL METHODS FOR BUSINESS UNIT 7: INFERENTIAL TOOLS. DISTRIBUTIONS ASSOCIATED WITH SAMPLING 7.1.- Distributios associated with the samplig process. 7.2.- Iferetial processes ad relevat distributios.

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

Hand-book on STATISTICAL DISTRIBUTIONS for experimentalists

Hand-book on STATISTICAL DISTRIBUTIONS for experimentalists Iteral Report SUF PFY/96 Stockholm, December 996 st revisio, 3 October 998 last modificatio September 7 Had-book o STATISTICAL DISTRIBUTIONS for experimetalists by Christia Walck Particle Physics Group

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

Central Limit Theorem and Its Applications to Baseball

Central Limit Theorem and Its Applications to Baseball Cetral Limit Theorem ad Its Applicatios to Baseball by Nicole Aderso A project submitted to the Departmet of Mathematical Scieces i coformity with the requiremets for Math 4301 (Hoours Semiar) Lakehead

More information

Chapter 5: Inner Product Spaces

Chapter 5: Inner Product Spaces Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

More information

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here). BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

More information

LIMIT MODEL FOR THE RELIABILITY OF A REGULAR AND HOMOGENEOUS SERIES- PARALLEL SYSTEM

LIMIT MODEL FOR THE RELIABILITY OF A REGULAR AND HOMOGENEOUS SERIES- PARALLEL SYSTEM REVSTAT Statistical Joural Volume 7, Number 3, November 2009, 227 243 LIMIT MODEL FOR THE RELIABILITY OF A REGULAR AND HOMOGENEOUS SERIES- PARALLEL SYSTEM Authors: Paula Reis CEAUL ad EST, Politecic Istitut

More information

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find 1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.

More information

THE ABRACADABRA PROBLEM

THE ABRACADABRA PROBLEM THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

More information

A GENERAL FRAMEWORK OF IMPORTANCE SAMPLING FOR VALUE-AT-RISK AND CONDITIONAL VALUE-AT-RISK. Lihua Sun L. Jeff Hong

A GENERAL FRAMEWORK OF IMPORTANCE SAMPLING FOR VALUE-AT-RISK AND CONDITIONAL VALUE-AT-RISK. Lihua Sun L. Jeff Hong Proceedigs of the 2009 Witer Simulatio Coferece M. D. Rossetti, R. R. Hill, B. Johasso, A. Duki, ad R. G. Igalls, eds. A GENERAL FRAMEWORK OF IMPORTANCE SAMPLING FOR VALUE-AT-RISK AND CONDITIONAL VALUE-AT-RISK

More information

Exploratory Data Analysis

Exploratory Data Analysis 1 Exploratory Data Aalysis Exploratory data aalysis is ofte the rst step i a statistical aalysis, for it helps uderstadig the mai features of the particular sample that a aalyst is usig. Itelliget descriptios

More information

TAYLOR SERIES, POWER SERIES

TAYLOR SERIES, POWER SERIES TAYLOR SERIES, POWER SERIES The followig represets a (icomplete) collectio of thigs that we covered o the subject of Taylor series ad power series. Warig. Be prepared to prove ay of these thigs durig the

More information

Multi-server Optimal Bandwidth Monitoring for QoS based Multimedia Delivery Anup Basu, Irene Cheng and Yinzhe Yu

Multi-server Optimal Bandwidth Monitoring for QoS based Multimedia Delivery Anup Basu, Irene Cheng and Yinzhe Yu Multi-server Optimal Badwidth Moitorig for QoS based Multimedia Delivery Aup Basu, Iree Cheg ad Yizhe Yu Departmet of Computig Sciece U. of Alberta Architecture Applicatio Layer Request receptio -coectio

More information

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean 1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.

More information

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown

Z-TEST / Z-STATISTIC: used to test hypotheses about. µ when the population standard deviation is unknown Z-TEST / Z-STATISTIC: used to test hypotheses about µ whe the populatio stadard deviatio is kow ad populatio distributio is ormal or sample size is large T-TEST / T-STATISTIC: used to test hypotheses about

More information

Confidence Interval Estimation of the Shape. Parameter of Pareto Distribution. Using Extreme Order Statistics

Confidence Interval Estimation of the Shape. Parameter of Pareto Distribution. Using Extreme Order Statistics Applied Mathematical Scieces, Vol 6, 0, o 93, 467-4640 Cofidece Iterval Estimatio of the Shape Parameter of Pareto Distributio Usig Extreme Order Statistics Aissa Omar, Kamarulzama Ibrahim ad Ahmad Mahir

More information

1 Hypothesis testing for a single mean

1 Hypothesis testing for a single mean BST 140.65 Hypothesis Testig Review otes 1 Hypothesis testig for a sigle mea 1. The ull, or status quo, hypothesis is labeled H 0, the alterative H a or H 1 or H.... A type I error occurs whe we falsely

More information

COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S 2 CONTROL CHART FOR THE CHANGES IN A PROCESS

COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S 2 CONTROL CHART FOR THE CHANGES IN A PROCESS COMPARISON OF THE EFFICIENCY OF S-CONTROL CHART AND EWMA-S CONTROL CHART FOR THE CHANGES IN A PROCESS Supraee Lisawadi Departmet of Mathematics ad Statistics, Faculty of Sciece ad Techoology, Thammasat

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

arxiv:1506.03481v1 [stat.me] 10 Jun 2015

arxiv:1506.03481v1 [stat.me] 10 Jun 2015 BEHAVIOUR OF ABC FOR BIG DATA By Wetao Li ad Paul Fearhead Lacaster Uiversity arxiv:1506.03481v1 [stat.me] 10 Ju 2015 May statistical applicatios ivolve models that it is difficult to evaluate the likelihood,

More information

x : X bar Mean (i.e. Average) of a sample

x : X bar Mean (i.e. Average) of a sample A quick referece for symbols ad formulas covered i COGS14: MEAN OF SAMPLE: x = x i x : X bar Mea (i.e. Average) of a sample x i : X sub i This stads for each idividual value you have i your sample. For

More information

A Mathematical Perspective on Gambling

A Mathematical Perspective on Gambling A Mathematical Perspective o Gamblig Molly Maxwell Abstract. This paper presets some basic topics i probability ad statistics, icludig sample spaces, probabilistic evets, expectatios, the biomial ad ormal

More information

Chapter One BASIC MATHEMATICAL TOOLS

Chapter One BASIC MATHEMATICAL TOOLS Chapter Oe BAIC MATHEMATICAL TOOL As the reader will see, the study of the time value of moey ivolves substatial use of variables ad umbers that are raised to a power. The power to which a variable is

More information