SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation"

Transcription

1 SYSM 6304: Risk and Decision Analysis Lecture 3 Monte Carlo Simulation M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas September 19, 2015

2 Outline Motivating Example 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

3 Outline Motivating Example 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

4 Motivation Motivating Example Until now we have seen how to fit distributions to data. The objective of Monte Carlo simulation is to generate data from distributions. Even if we have exact formulas for the distribution functions of individual random variables, it is not always possible (or easy) to generate distribution functions of their sum, or product, etc.

5 Motivation (Cont d) Motivating Example In applications such as supply chain management or project management, we often have available the distribution functions of the constituent parts of a large and complex system. Monte Carlo simulation allows us to generate samples for each constituent random variable, combine those to generate samples for the overall random variable, and then combine the samples of the overall random variable. These samples can then be used to estimate various quantities about the overall random variable, such as mean, variance, tail values, etc.

6 Toy Manufacturing Example X 1 X 3 Start Finish X 2 X 4 Parts simultaneously start at stations 1 and 2, then move to 3 and 4 respectively. When both stations 3 and 4 finish, the process is complete. Y = max{x 1 + X 3, X 2 + X 4 }.

7 Toy Manufacturing Example No. 2 X 1 X 3 Start Finish X 2 X 4 Parts simultaneously start at stations 1 and 2, then move to 3 and 4 respectively. When both stations 3 and 4 finish, the process is complete. Y 2 = max{max{x 1, X 2 } + X 3, X 2 + X 4 }.

8 General Approach Motivating Example From historical records we can generate cumulative distribution functions (cdfs) of the individual random variables X 1 through X 4. Even if we had formulas for the cdfs of the four random variabes X 1 through X 4, it would be extremely difficult to find a formula for the cdf of Y.

9 General Approach (Cont d) So instead we can generate lots of random samples of each of the four random variables X 1 through X 4, use these to compute lots of random samples of Y. We can use these samples to estimate various quantities, e.g., the mean and variance of Y. We can try to fit some distribution to these randomly generated samples, to get an approximate cdf of Y. We can fit an empirical distribution to the data, and estimate how close it is to the true distribution. Usually the middle bullet is not attempted.

10 Pertinent Questions Motivating Example Given cdfs of individual random variables X 1 through X 4, how do we generate samples of the random variables X 1 through X 4 with the specified distribution? How can we generate an empirical distribution of the random variabl Y? How well does this empirical distribution approximate the true but unknown distribution function?

11 Outline Motivating Example 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

12 Outline Motivating Example 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

13 Percentile Approach to Sampling Φ X x The grid points are uniformly spaced on the vertical axis, though not on the horizontal axis.

14 Generating Samples Using Uniform Distribution Suppose a cdf Φ X is specified. How can we generate samples of X with this distribution? Suppose Z is uniformly distributed on [0, 1], and let Φ X ( ) denote the distribution function of X. Then the r.v. Φ 1 X (Z) has the same distribution as X. To generate samples x 1,..., x n of X according to the distribution Φ X ( ), first generate samples z 1,..., z n with the uniform distribution, and then define x i = Φ 1 X (z i), i = 1,..., n.

15 Generating Samples Using Uniform Distribution (Cont d) The matlab command rand(n,m) generates an n m matrix of random (actually pseudo-random) numbers that are uniformly distributed. In particular, rand (n) generates m uniformly distributed random numbers. By substituting these numbers into Φ 1 X, we can generate the desired samples of X. Note that matlab provides inverse cdfs for many widely used distributions, such as Gaussian (normal), Poisson, etc. In addition, the function stblinv.m can be used to invert a given stable distribution, while triinv can be used to invert a triangular distribution.

16 Generating Samples Using Uniform Distribution (Cont d) If there are k independent random variables X 1,..., X k, we can generate a k n array of uniformly distributed (pseudo-)random numbers Z by using the command Z = rand(k,n). Denote the entries of the k n matrix Z as z 11,..., z kn. Then we can generate n independent samples for each of the k random variables via x ij = Φ 1 X i (z ij ), i = 1,..., k, j = 1,..., n.

17 Outline Motivating Example 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

18 Monte Carlo Simulation Suppose Y = f(x 1,..., X k ) where X 1,..., X k are independent random variables. How can we generate samples of Y? Generate n independent samples each of the k random variables; call them x 11,..., x kn. Compute the samples y i = f(x 1i,..., x ki ), i = 1,..., n. Construct the empirical distribution function ˆΦ Y (u) = n I {u yi }. i=1

19 Toy Manufacturing Example Revisited Recall the example where Y = max{x 1 + X 3, X 2 + X 4 }. By substituting the samples into this formula, we can generate n independent samples of Y. So the question now arises: What do we do with these samples?

20 Empirical Distribution Function Suppose Y is the random variable of interest, and we have n independent samples of Y, call them y 1,..., y n. For each value of y, define the empirical distribution function ˆΦ Y (y) = 1 n n i=1 I yi y, where I denotes the indicator function it equals one if the statement in the subscript is true, and zero if it is false. So specifically ˆΦ Y (y) is just the fraction of the n samples that are smaller than or equal to y.

21 Empirical Distribution Function (Cont d) To construct the empirical distribution function, first sort all the samples y 1,..., y n in increasing order; call the result (y) 1,..., (y) n. Then construct a staircase function that jumps by 1/n at each sample (y) i. That is the empirical distribution function.

22 Depiction: Empirical Distribution ˆΦ Y (u) (y) 1 (y) 2 (y) 3 (y) 4 (y) 5 (y) 6 u

23 Theory Behind Monte Carlo Simulation Theory allows us to say just how many samples we need to draw, to get a desired level of accuracy of the estimate, with a given level of confidence. With confidence 1 δ it can be said that the true but unknown probability distribution function Φ Y (u) satisfies max u ˆΦ Y (u) Φ Y (u) θ, where θ(n, δ) = ( 1 2n log 2 ) 1/2. δ

24 Theory Behind Monte Carlo Simulation (Cont d) Turning around this inequality, if we want to approximate Φ Y (u) to accuracy θ with confidence 1 δ, then the minimum number of samples needed is n 1 2θ 2 log 2 δ. With this many samples, true but unknown probability distribution function Φ Y (u) lies within a band of width ɛ around the empirical probability distribution ˆΦ Y (u). For example, to approximate Φ Y ( ) to accuracy with confidence 95%, we require n 2951 samples. If we wish to be 99% sure, then we require 4, 239 samples.

25 Depiction: Error Bands for Empirical Distribution Thin green and vermilion staircase functions show upper and lower bounds for true but unknown distribution Φ Y ( ) These can be used to bound percentile values of Y with specified confidence and accuracy. ˆΦ Y (u) y i1 y i2 y i3 y i4 y i5 y i6 u

26 Estimating Value at Risk One of the most common uses of Monte Carlo simulation is estimating the Value at Risk (VaR). Suppose wish to determine a value V such that Pr{Y > V } α, where α is a pre-specified level. Usual values of α are 0.01 or If α = 0.5, then V is the called the 95% Value at Risk, whereas if α = 0.01 then V is called the 99% Value at Risk.

27 Estimating Value at Risk (Cont d) We can express the VaR in terms of the complementary cdf: V = Φ 1 1 (1 α) = Φ (α). Y The difficulty however is that we don t know the true cdf Φ or the true ccdf Φ. This is where we can use the empirical distribution. Y

28 Estimating Value at Risk (Cont d) Suppose α = 0.05, so that we wish to estimate the 95% VaR. Choose θ = α/2 = Then choose the desired confidence level δ, and the corresponding number of samples n according to n 1 2θ 2 log 2 δ. With this many samples, we know that the empirical distribution function ˆΦ Y is within θ of the true but unknown distribution function.

29 Estimating Value at Risk (Cont d) Now compute ˆV according to ˆΦ Y ( ˆV ) 1 α/2 = 1 θ. Then, with confidence 1 δ, we can say that Φ Y ( ˆV ) 1 α. Therefore ˆV is an estimated VaR, at a confidence level of δ.

30 Estimating Value at Risk (Cont d) Often the VaR is estimated of some function of Y. For example, suppose Y is the time needed to complete some manufacturing job. The manufacturer receives a bonus for early completion and pays a penalty for late completion. We wish to estimate the VaR of the bonus/penalty. This situation can be modeled by defining the bonus B as a function of Y, with negative bonus corresponding to a penalty.

31 Estimating Value at Risk (Cont d) Once a level α is specified, we wish to estimate the value V B such that Pr{B(Y ) V B } = 1 α. Again, using the empirical distribution of Y, we can construct a corresponding empirical distribution of the bonus B, and use that to estimate the VaR of the bonus.

32 Estimating Value at Risk (Cont d) But often there is a simpler way to do this. If the bonus is a monotonic function of the time to completion (which is a reasonable assumption), then we simply compute (or estimate) the VaR of Y, and substitute that into the formula for the bonus.

33 Estimating Percentiles The VaR calculation applies to the far end of the distribution. The same philosophy can also be applied to estimating other percentiles, such the median for example. Suppose we wish to estimate the median value of Y. We have the empirical estimate ˆΦ Y, and we have chosen the number of samples n such that, with confidence 1 δ, we can assert that ˆΦ Y (u) Φ Y (u) θ u. Now the median corresponds to Φ 1 T (0.5). So we can compute ˆΦ 1 1 Y (0.5 θ) and ˆΦ Y (0.5 + θ). These numbers give a range for the median. To estimate other percentiles, just replace 0.5 by the desired number.

34 Hoeffding s Inequality If the random variable Y is bounded, then a very useful estimate known as Hoeffding s inequality becomes applicable. Note that if popular models such as Gaussian or log-normal distributions are used to model various quantities, then in principle the random variables are not bounded, and Hoeffding s inequality does not apply. But if triangular distributions (for example) are used, then Hoeffding s inequality does apply.

35 Hoeffding s Inequality (Cont d) Suppose Y is a random variable assuming values in a finite interval [a, b]. Suppose y 1,..., y n are independent samples of Y, and define ˆµ Y = 1 n n i=1 y i to be the empirical mean of Y. Let µ Y denote the true but unknown mean of Y. Hoeffding s inequality states that Pr{ ˆµ Y µ Y > ɛ} 2 exp( 2nɛ 2 /(b a) 2 ).

36 Hoeffding s Inequality (Cont d) Therefore, to estimate the quantity µ Y to within a specified accuracy ɛ with confidence 1 δ, we require ( ) (b a)2 2 n 2ɛ 2 log δ samples. We can also compute the accuracy ɛ in terms of the number of samples n and the confidence δ. [ ( )] b a 2 1/2 ɛ = 2n log. δ

37 Outline Motivating Example Example with Bounded Distributions Example with Unbounded Distributions 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

38 Outline Motivating Example Example with Bounded Distributions Example with Unbounded Distributions 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

39 Example with Bounded Distributions Example with Unbounded Distributions Specification of Individual Random Variables Suppose X 1 has a triangular distribution with minimum a 1 = 1, mode b 1 = 2, and maximum c 1 = 6. X 2 has a triangular distribution with minimum a 2 = 1, mode b 2 = 3, and maximum c 2 = 5. X 3 has a triangular distribution with minimum a 3 = 3, mode b 3 = 5, and maximum c 3 = 7. X 4 has a triangular distribution with minimum a 4 = 2, mode b 4 = 5, and maximum c 4 = 8.

40 Example with Bounded Distributions Example with Unbounded Distributions Determination of the Number of Samples Let us choose θ = 0.01, δ = This leads to n = 1 2θ 2 log 2 δ = Let us round this up to 26,500 samples. Repeating earlier steps leads to the empirical distribution shown in the next slide.

41 Example with Bounded Distributions Example with Unbounded Distributions Empirical Distribution of Processing Time 1 Empirical Distribution Function Phi hat(y) Values of Y

42 Estimating Median Processing Time Example with Bounded Distributions Example with Unbounded Distributions Following earlier steps, we find that [8.7431, ] to be the 99% confidence interval for the median processing time.

43 Estimating the Mean Processing Time Example with Bounded Distributions Example with Unbounded Distributions Because Y is now bounded, lying between 3 and 13, we can apply Hoeffding s inequality. Because we have 26,500 samples, we can compute the achievable accuracy at a confidence level of 1 δ using the formula [ ( )] b a 2 1/2 ɛ = 2n log. δ turns out to be ˆµ Y = Therefore we can assert with confidence 1 δ that the true mean µ(y ) lies in the interval [ˆµ Y ɛ, ˆµ Y + ɛ].

44 Example with Bounded Distributions Example with Unbounded Distributions Estimating the Mean Processing Time (Cont d) In the present case, choosing δ = 0.01 leads to ɛ = The empirical mean, that is, the average of the 26,500 samples of Y, In the present case, we can state with 99% confidence that the true mean of Y lies in the interval [8.7730, ]. This estimate does not differ too much from the estimate for the median, which is [8.7431, ]. This is because the empirical distribution of Y is not very skewed.

45 Outline Motivating Example Example with Bounded Distributions Example with Unbounded Distributions 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

46 Toy Manufacturing Example: Reprise Example with Bounded Distributions Example with Unbounded Distributions X 1 X 3 Start Finish X 2 X 4 Parts simultaneously start at stations 1 and 2, then move to 3 and 4 respectively. When both stations 3 and 4 finish, the process is complete. Y = max{x 1 + X 3, X 2 + X 4 }.

47 Example with Bounded Distributions Example with Unbounded Distributions Specification of Individual Random Variables Suppose X 1 is log-normally distributed with mean µ 1 = 1 and standard deviation (of log X 1 ) of s 1 = 0.5. X 2 has a triangular distribution with minimum a 2 = 1, mode b 2 = 2, and maximum c 2 = 6. X 3 is log-normally distributed with mean µ 1 = 2.25 and standard deviation (of log X 3 ) of s 1 = X 4 has a triangular distribution with minimum a 4 = 5, mode b 4 = 9, and maximum c 4 = 16.

48 Generation of Samples Example with Bounded Distributions Example with Unbounded Distributions Suppose we wish to approximate the distribution function of the total processing time Y to an accuracy of 0.025, with a confidence of Therefore θ = 0.025, δ = 0.01, which means that we require n = 1 2θ 2 log 2 δ = 4288 samples. We can round this up to n = 4300.

49 Generation of Samples (Cont d) Example with Bounded Distributions Example with Unbounded Distributions By using the appropriate matlab commands, we can generate n samples for each of the four random variables. By substituting into the expression Y = max{x 1 + X 3, X 2 + X 4 }. we can generate n independent samples of Y. This leads to the empirical distribution function shown in the next slide.

50 Example with Bounded Distributions Example with Unbounded Distributions Empirical Distribution of Processing Time 1 Empirical Distribution Function via Monte Carlo Simulation Empirical Distribution Function of Y Values of Y1

51 Example with Bounded Distributions Example with Unbounded Distributions Estimating Value at Risk of Processing Time We have chosen θ = So we can estimate the 1 θ Value at Risk (97.5% VaR) of Y using the empirical distribution. Note that 1 2θ = 095. So, with confidence 1 δ = 0.99, we can say that the 95% VaR of the empirical distribution of Y is no larger than the 97.5% VaR of the true but unknown distribution of Y. This value turns out to be Therefore we are 99% sure that the 97.5% VaR of Y is not larger than this number.

52 Estimating the Median Example with Bounded Distributions Example with Unbounded Distributions We would like to estimate the median value of Y, which is Φ 1 Y (0.5). By finding the range of values [ˆΦ 1 1 Y (0.5 θ), ˆΦ Y (0.5 + θ)], we can get an estimate for the mediam value of Y, with confidence 1 δ. This interval turns out to be [ , ]. So we are 99% sure that the median value of Y lies in this interval. Because the log-normal distribution is unbounded, we cannot apply Hoeffding s inequality to this problem.

53 Outline Motivating Example Definition and Characterization of Independence Covariance and Correlation Coefficient 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

54 Outline Motivating Example Definition and Characterization of Independence Covariance and Correlation Coefficient 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

55 Independence of Real Random Variables Definition and Characterization of Independence Covariance and Correlation Coefficient There are two equivalent ways of defining independence in this case. X, Y are independent if or equivalently Φ X,Y (a, b) = Φ X (a) Φ Y (b) x, y, φ X,Y (x, y) = φ X (x) φ Y (y) x, y.

56 Sums of Independent Random Variables Definition and Characterization of Independence Covariance and Correlation Coefficient Suppose X, Y are independent r.v.s with densities φ X and φ Y respectively. Then the r.v. Z = X + Y has the density φ Z (z) = = z z φ X (u)φ Y (z u)du φ X (z v)φ Y (v)dv. In other words, the density of X + Y is the convolution of the densities of X and Y. If X and Y are not indepedent, then this statement is false in general.

57 Outline Motivating Example Definition and Characterization of Independence Covariance and Correlation Coefficient 1 Motivating Example 2 3 Example with Bounded Distributions Example with Unbounded Distributions 4 Definition and Characterization of Independence Covariance and Correlation Coefficient

58 Covariance Motivating Example Definition and Characterization of Independence Covariance and Correlation Coefficient Suppose X, Y are real-valued r.v.s. Define the expected values E[X], E[Y ], variances V (X), V (Y ), and standard deviations σ(x) = (V (X)) 1/2 and σ(y ) = (V (Y )) 1/2. The quantity C(X, Y ) = E[(X E(X))(Y E(Y ))] = E[XY ] E[X]E[Y ] is called the covariance of X and Y.

59 Correlation Coefficient Definition and Characterization of Independence Covariance and Correlation Coefficient ρ(x, Y ) = C(X, Y ) E[XY ] E[X]E[Y ] = σ(x)σ(y ) σ(x)σ(y ) is called the correlation coefficient between X and Y. ρ(x, Y ) is always in the interval [ 1, 1]. If ρ(x, Y ) > 0 we say that X, Y are positively correlated; if ρ(x, Y ) < 0 we say that X, Y are negatively correlated; and if ρ(x, Y ) = 0 we say that X, Y are uncorrelated. Note that the correlation coefficient is invariant under linear transformations. So if a, b, c, d are real numbers, then ρ(x, Y ) = ρ(ax + b, cy + d).

60 Correlation Coefficient 2 Definition and Characterization of Independence Covariance and Correlation Coefficient Common Misinterpretation: If X, Y are uncorrelated, then they are independent. The correlation coefficient ρ(x, Y ) tells only whether E[XY ] is more or less than the product E[X]E[Y ]. Fact: If X, Y are independent, then E(XY ) = E(X) E(Y ). Therefore ρ(x, Y ) = 0 if X, Y are independent. But the converse is not true at all!

61 More Than Two Random Variables Definition and Characterization of Independence Covariance and Correlation Coefficient We will discuss the case where there are, not just two, but d 2 real-valued random variables X 1,..., X d. In this case we can define an d d covariance matrix C defined by c ij = E[X i X j ] E[X i ]E[X j ], i, j = 1,..., d. Then C is a symmetric and positive semidefinite matrix, that is, all of its eigenvalues are real and nonnegative.

62 Multivariate Gaussian Distribution Definition and Characterization of Independence Covariance and Correlation Coefficient Suppose d 2 is some integer, µ R d, and Σ is a d d symmetric and positive definite matrix. Then the d-dimensional joint density function φ X (x) = 1 (2π) d/2 det(σ) 1/2 exp( (1/2)(x µ)t Σ 1 (x µ)) defines the d-dimensional Gaussian distribution with mean µ and covariance matrix Σ. It is easy to check that it is a generalization of the one-dimensional Gaussian density function φ(x) = 1 2πσ exp( (x µ) 2 /2σ 2 ).

63 Multivariate Gaussian Distribution 2 Definition and Characterization of Independence Covariance and Correlation Coefficient The d-dimensional Gaussian distribution defines a collection of d random variables with the properties that and covariance matrix Σ. E(X) = µ, that is, E(X i ) = µ i, Important Property: It is easy to see that the d random variables are pairwise uncorrelated if and only if the matrix Σ is diagonal. However, for Gaussian distributions only, it can be shown that if Σ is diagonal, then the d random variables are also pairwise independent.

64 Simulating Correlated Gaussian Variables Definition and Characterization of Independence Covariance and Correlation Coefficient The Matlab command norminv can be used with scalars as well as matrices. Thus if x is an n-dimensional vector consisting of samples generated using the uniform distribution, then y = norminv(x, µ, σ) generates Gaussian samples with mean µ and standard deviation σ. If X is an n d matrix consisting of independent samples generated using the uniform distribution, µ is a d-dimensional vector, and Σ is a d d matrix, then y = norminv(x, µ, Σ) generates Gaussian samples with mean µ and covariance matrix Σ.

Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

More information

Topic 4: Multivariate random variables. Multiple random variables

Topic 4: Multivariate random variables. Multiple random variables Topic 4: Multivariate random variables Joint, marginal, and conditional pmf Joint, marginal, and conditional pdf and cdf Independence Expectation, covariance, correlation Conditional expectation Two jointly

More information

Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.

Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015. Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x

More information

Chapter 3 RANDOM VARIATE GENERATION

Chapter 3 RANDOM VARIATE GENERATION Chapter 3 RANDOM VARIATE GENERATION In order to do a Monte Carlo simulation either by hand or by computer, techniques must be developed for generating values of random variables having known distributions.

More information

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i ) Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

More information

Covariance and Correlation. Consider the joint probability distribution f XY (x, y).

Covariance and Correlation. Consider the joint probability distribution f XY (x, y). Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 2: Section 5-2 Covariance and Correlation Consider the joint probability distribution f XY (x, y). Is there a relationship between X and Y? If so, what kind?

More information

4. Joint Distributions of Two Random Variables

4. Joint Distributions of Two Random Variables 4. Joint Distributions of Two Random Variables 4.1 Joint Distributions of Two Discrete Random Variables Suppose the discrete random variables X and Y have supports S X and S Y, respectively. The joint

More information

Some probability and statistics

Some probability and statistics Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,

More information

Generating Random Numbers Variance Reduction Quasi-Monte Carlo. Simulation Methods. Leonid Kogan. MIT, Sloan. 15.450, Fall 2010

Generating Random Numbers Variance Reduction Quasi-Monte Carlo. Simulation Methods. Leonid Kogan. MIT, Sloan. 15.450, Fall 2010 Simulation Methods Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Simulation Methods 15.450, Fall 2010 1 / 35 Outline 1 Generating Random Numbers 2 Variance Reduction 3 Quasi-Monte

More information

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2015 Timo Koski Matematisk statistik 24.09.2015 1 / 1 Learning outcomes Random vectors, mean vector, covariance matrix,

More information

Outline. Random Variables. Examples. Random Variable

Outline. Random Variables. Examples. Random Variable Outline Random Variables M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Random variables. CDF and pdf. Joint random variables. Correlated, independent, orthogonal. Correlation,

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

More information

Worked examples Multiple Random Variables

Worked examples Multiple Random Variables Worked eamples Multiple Random Variables Eample Let X and Y be random variables that take on values from the set,, } (a) Find a joint probability mass assignment for which X and Y are independent, and

More information

Joint Exam 1/P Sample Exam 1

Joint Exam 1/P Sample Exam 1 Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question

More information

Probability and Statistics

Probability and Statistics CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

More information

For a partition B 1,..., B n, where B i B j = for i. A = (A B 1 ) (A B 2 ),..., (A B n ) and thus. P (A) = P (A B i ) = P (A B i )P (B i )

For a partition B 1,..., B n, where B i B j = for i. A = (A B 1 ) (A B 2 ),..., (A B n ) and thus. P (A) = P (A B i ) = P (A B i )P (B i ) Probability Review 15.075 Cynthia Rudin A probability space, defined by Kolmogorov (1903-1987) consists of: A set of outcomes S, e.g., for the roll of a die, S = {1, 2, 3, 4, 5, 6}, 1 1 2 1 6 for the roll

More information

Chapters 5. Multivariate Probability Distributions

Chapters 5. Multivariate Probability Distributions Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are

More information

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4) Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

More information

Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

More information

Random Variables, Expectation, Distributions

Random Variables, Expectation, Distributions Random Variables, Expectation, Distributions CS 5960/6960: Nonparametric Methods Tom Fletcher January 21, 2009 Review Random Variables Definition A random variable is a function defined on a probability

More information

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution

SF2940: Probability theory Lecture 8: Multivariate Normal Distribution SF2940: Probability theory Lecture 8: Multivariate Normal Distribution Timo Koski 24.09.2014 Timo Koski () Mathematisk statistik 24.09.2014 1 / 75 Learning outcomes Random vectors, mean vector, covariance

More information

The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables

The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh The Monte Carlo Framework, Examples from Finance and Generating Correlated Random Variables 1 The Monte Carlo Framework Suppose we wish

More information

Errata and updates for ASM Exam C/Exam 4 Manual (Sixteenth Edition) sorted by page

Errata and updates for ASM Exam C/Exam 4 Manual (Sixteenth Edition) sorted by page Errata for ASM Exam C/4 Study Manual (Sixteenth Edition) Sorted by Page 1 Errata and updates for ASM Exam C/Exam 4 Manual (Sixteenth Edition) sorted by page Practice exam 1:9, 1:22, 1:29, 9:5, and 10:8

More information

Using pivots to construct confidence intervals. In Example 41 we used the fact that

Using pivots to construct confidence intervals. In Example 41 we used the fact that Using pivots to construct confidence intervals In Example 41 we used the fact that Q( X, µ) = X µ σ/ n N(0, 1) for all µ. We then said Q( X, µ) z α/2 with probability 1 α, and converted this into a statement

More information

Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab

Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?

More information

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur

Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Module No. #01 Lecture No. #15 Special Distributions-VI Today, I am going to introduce

More information

1 Sufficient statistics

1 Sufficient statistics 1 Sufficient statistics A statistic is a function T = rx 1, X 2,, X n of the random sample X 1, X 2,, X n. Examples are X n = 1 n s 2 = = X i, 1 n 1 the sample mean X i X n 2, the sample variance T 1 =

More information

Math 141. Lecture 7: Variance, Covariance, and Sums. Albyn Jones 1. 1 Library 304. jones/courses/141

Math 141. Lecture 7: Variance, Covariance, and Sums. Albyn Jones 1. 1 Library 304.  jones/courses/141 Math 141 Lecture 7: Variance, Covariance, and Sums Albyn Jones 1 1 Library 304 jones@reed.edu www.people.reed.edu/ jones/courses/141 Last Time Variance: expected squared deviation from the mean: Standard

More information

Math 431 An Introduction to Probability. Final Exam Solutions

Math 431 An Introduction to Probability. Final Exam Solutions Math 43 An Introduction to Probability Final Eam Solutions. A continuous random variable X has cdf a for 0, F () = for 0 <

More information

MATH4427 Notebook 2 Spring 2016. 2 MATH4427 Notebook 2 3. 2.1 Definitions and Examples... 3. 2.2 Performance Measures for Estimators...

MATH4427 Notebook 2 Spring 2016. 2 MATH4427 Notebook 2 3. 2.1 Definitions and Examples... 3. 2.2 Performance Measures for Estimators... MATH4427 Notebook 2 Spring 2016 prepared by Professor Jenny Baglivo c Copyright 2009-2016 by Jenny A. Baglivo. All Rights Reserved. Contents 2 MATH4427 Notebook 2 3 2.1 Definitions and Examples...................................

More information

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions...

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions... MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo c Copyright 2004-2012 by Jenny A. Baglivo. All Rights Reserved. Contents 3 MT426 Notebook 3 3 3.1 Definitions............................................

More information

October 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix

October 3rd, 2012. Linear Algebra & Properties of the Covariance Matrix Linear Algebra & Properties of the Covariance Matrix October 3rd, 2012 Estimation of r and C Let rn 1, rn, t..., rn T be the historical return rates on the n th asset. rn 1 rṇ 2 r n =. r T n n = 1, 2,...,

More information

Variances and covariances

Variances and covariances Chapter 4 Variances and covariances 4.1 Overview The expected value of a random variable gives a crude measure for the center of location of the distribution of that random variable. For instance, if the

More information

Covariance and Correlation

Covariance and Correlation Covariance and Correlation ( c Robert J. Serfling Not for reproduction or distribution) We have seen how to summarize a data-based relative frequency distribution by measures of location and spread, such

More information

Dongfeng Li. Autumn 2010

Dongfeng Li. Autumn 2010 Autumn 2010 Chapter Contents Some statistics background; ; Comparing means and proportions; variance. Students should master the basic concepts, descriptive statistics measures and graphs, basic hypothesis

More information

Lecture 6: Discrete & Continuous Probability and Random Variables

Lecture 6: Discrete & Continuous Probability and Random Variables Lecture 6: Discrete & Continuous Probability and Random Variables D. Alex Hughes Math Camp September 17, 2015 D. Alex Hughes (Math Camp) Lecture 6: Discrete & Continuous Probability and Random September

More information

Exact Confidence Intervals

Exact Confidence Intervals Math 541: Statistical Theory II Instructor: Songfeng Zheng Exact Confidence Intervals Confidence intervals provide an alternative to using an estimator ˆθ when we wish to estimate an unknown parameter

More information

Introduction to General and Generalized Linear Models

Introduction to General and Generalized Linear Models Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby

More information

5. Continuous Random Variables

5. Continuous Random Variables 5. Continuous Random Variables Continuous random variables can take any value in an interval. They are used to model physical characteristics such as time, length, position, etc. Examples (i) Let X be

More information

Random Vectors and the Variance Covariance Matrix

Random Vectors and the Variance Covariance Matrix Random Vectors and the Variance Covariance Matrix Definition 1. A random vector X is a vector (X 1, X 2,..., X p ) of jointly distributed random variables. As is customary in linear algebra, we will write

More information

3.6: General Hypothesis Tests

3.6: General Hypothesis Tests 3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

More information

Definition 6.1.1. A r.v. X has a normal distribution with mean µ and variance σ 2, where µ R, and σ > 0, if its density is f(x) = 1. 2σ 2.

Definition 6.1.1. A r.v. X has a normal distribution with mean µ and variance σ 2, where µ R, and σ > 0, if its density is f(x) = 1. 2σ 2. Chapter 6 Brownian Motion 6. Normal Distribution Definition 6... A r.v. X has a normal distribution with mean µ and variance σ, where µ R, and σ > 0, if its density is fx = πσ e x µ σ. The previous definition

More information

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

More information

15.062 Data Mining: Algorithms and Applications Matrix Math Review

15.062 Data Mining: Algorithms and Applications Matrix Math Review .6 Data Mining: Algorithms and Applications Matrix Math Review The purpose of this document is to give a brief review of selected linear algebra concepts that will be useful for the course and to develop

More information

Gaussian Conjugate Prior Cheat Sheet

Gaussian Conjugate Prior Cheat Sheet Gaussian Conjugate Prior Cheat Sheet Tom SF Haines 1 Purpose This document contains notes on how to handle the multivariate Gaussian 1 in a Bayesian setting. It focuses on the conjugate prior, its Bayesian

More information

Efficiency and the Cramér-Rao Inequality

Efficiency and the Cramér-Rao Inequality Chapter Efficiency and the Cramér-Rao Inequality Clearly we would like an unbiased estimator ˆφ (X of φ (θ to produce, in the long run, estimates which are fairly concentrated i.e. have high precision.

More information

The Bivariate Normal Distribution

The Bivariate Normal Distribution The Bivariate Normal Distribution This is Section 4.7 of the st edition (2002) of the book Introduction to Probability, by D. P. Bertsekas and J. N. Tsitsiklis. The material in this section was not included

More information

Master s Theory Exam Spring 2006

Master s Theory Exam Spring 2006 Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

More information

Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ).

Definition The covariance of X and Y, denoted by cov(x, Y ) is defined by. cov(x, Y ) = E(X µ 1 )(Y µ 2 ). Correlation Regression Bivariate Normal Suppose that X and Y are r.v. s with joint density f(x y) and suppose that the means of X and Y are respectively µ 1 µ 2 and the variances are 1 2. Definition The

More information

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20 Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

More information

Basics of Statistical Machine Learning

Basics of Statistical Machine Learning CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

More information

MULTIVARIATE PROBABILITY DISTRIBUTIONS

MULTIVARIATE PROBABILITY DISTRIBUTIONS MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined

More information

ST 371 (VIII): Theory of Joint Distributions

ST 371 (VIII): Theory of Joint Distributions ST 371 (VIII): Theory of Joint Distributions So far we have focused on probability distributions for single random variables. However, we are often interested in probability statements concerning two or

More information

Permutation Tests for Comparing Two Populations

Permutation Tests for Comparing Two Populations Permutation Tests for Comparing Two Populations Ferry Butar Butar, Ph.D. Jae-Wan Park Abstract Permutation tests for comparing two populations could be widely used in practice because of flexibility of

More information

Aggregate Loss Models

Aggregate Loss Models Aggregate Loss Models Chapter 9 Stat 477 - Loss Models Chapter 9 (Stat 477) Aggregate Loss Models Brian Hartman - BYU 1 / 22 Objectives Objectives Individual risk model Collective risk model Computing

More information

L10: Probability, statistics, and estimation theory

L10: Probability, statistics, and estimation theory L10: Probability, statistics, and estimation theory Review of probability theory Bayes theorem Statistics and the Normal distribution Least Squares Error estimation Maximum Likelihood estimation Bayesian

More information

Notes on Continuous Random Variables

Notes on Continuous Random Variables Notes on Continuous Random Variables Continuous random variables are random quantities that are measured on a continuous scale. They can usually take on any value over some interval, which distinguishes

More information

3. The Multivariate Normal Distribution

3. The Multivariate Normal Distribution 3. The Multivariate Normal Distribution 3.1 Introduction A generalization of the familiar bell shaped normal density to several dimensions plays a fundamental role in multivariate analysis While real data

More information

4. Continuous Random Variables, the Pareto and Normal Distributions

4. Continuous Random Variables, the Pareto and Normal Distributions 4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random

More information

MATH 201. Final ANSWERS August 12, 2016

MATH 201. Final ANSWERS August 12, 2016 MATH 01 Final ANSWERS August 1, 016 Part A 1. 17 points) A bag contains three different types of dice: four 6-sided dice, five 8-sided dice, and six 0-sided dice. A die is drawn from the bag and then rolled.

More information

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October 17, 2015 Outline

More information

UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

More information

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

More information

, for x = 0, 1, 2, 3,... (4.1) (1 + 1/n) n = 2.71828... b x /x! = e b, x=0

, for x = 0, 1, 2, 3,... (4.1) (1 + 1/n) n = 2.71828... b x /x! = e b, x=0 Chapter 4 The Poisson Distribution 4.1 The Fish Distribution? The Poisson distribution is named after Simeon-Denis Poisson (1781 1840). In addition, poisson is French for fish. In this chapter we will

More information

6. Distribution and Quantile Functions

6. Distribution and Quantile Functions Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 6. Distribution and Quantile Functions As usual, our starting point is a random experiment with probability measure P on an underlying sample spac

More information

Quadratic forms Cochran s theorem, degrees of freedom, and all that

Quadratic forms Cochran s theorem, degrees of freedom, and all that Quadratic forms Cochran s theorem, degrees of freedom, and all that Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Why We Care Cochran s theorem tells us

More information

Variance of OLS Estimators and Hypothesis Testing. Randomness in the model. GM assumptions. Notes. Notes. Notes. Charlie Gibbons ARE 212.

Variance of OLS Estimators and Hypothesis Testing. Randomness in the model. GM assumptions. Notes. Notes. Notes. Charlie Gibbons ARE 212. Variance of OLS Estimators and Hypothesis Testing Charlie Gibbons ARE 212 Spring 2011 Randomness in the model Considering the model what is random? Y = X β + ɛ, β is a parameter and not random, X may be

More information

Advanced 3G and 4G Wireless Communication Prof. Aditya K. Jagannatham Department of Electrical Engineering Indian Institute of Technology, Kanpur

Advanced 3G and 4G Wireless Communication Prof. Aditya K. Jagannatham Department of Electrical Engineering Indian Institute of Technology, Kanpur Advanced 3G and 4G Wireless Communication Prof. Aditya K. Jagannatham Department of Electrical Engineering Indian Institute of Technology, Kanpur Lecture - 3 Rayleigh Fading and BER of Wired Communication

More information

UNIT I: RANDOM VARIABLES PART- A -TWO MARKS

UNIT I: RANDOM VARIABLES PART- A -TWO MARKS UNIT I: RANDOM VARIABLES PART- A -TWO MARKS 1. Given the probability density function of a continuous random variable X as follows f(x) = 6x (1-x) 0

More information

Correlation in Random Variables

Correlation in Random Variables Correlation in Random Variables Lecture 11 Spring 2002 Correlation in Random Variables Suppose that an experiment produces two random variables, X and Y. What can we say about the relationship between

More information

Joint Probability Distributions and Random Samples. Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage

Joint Probability Distributions and Random Samples. Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage 5 Joint Probability Distributions and Random Samples Week 5, 2011 Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Two Discrete Random Variables The probability mass function (pmf) of a single

More information

ECE302 Spring 2006 HW7 Solutions March 11, 2006 1

ECE302 Spring 2006 HW7 Solutions March 11, 2006 1 ECE32 Spring 26 HW7 Solutions March, 26 Solutions to HW7 Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where

More information

1. χ 2 minimization 2. Fits in case of of systematic errors

1. χ 2 minimization 2. Fits in case of of systematic errors Data fitting Volker Blobel University of Hamburg March 2005 1. χ 2 minimization 2. Fits in case of of systematic errors Keys during display: enter = next page; = next page; = previous page; home = first

More information

Probability and Random Variables. Generation of random variables (r.v.)

Probability and Random Variables. Generation of random variables (r.v.) Probability and Random Variables Method for generating random variables with a specified probability distribution function. Gaussian And Markov Processes Characterization of Stationary Random Process Linearly

More information

Monte Carlo Simulation

Monte Carlo Simulation 1 Monte Carlo Simulation Stefan Weber Leibniz Universität Hannover email: sweber@stochastik.uni-hannover.de web: www.stochastik.uni-hannover.de/ sweber Monte Carlo Simulation 2 Quantifying and Hedging

More information

Variance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers

Variance Reduction. Pricing American Options. Monte Carlo Option Pricing. Delta and Common Random Numbers Variance Reduction The statistical efficiency of Monte Carlo simulation can be measured by the variance of its output If this variance can be lowered without changing the expected value, fewer replications

More information

Math 576: Quantitative Risk Management

Math 576: Quantitative Risk Management Math 576: Quantitative Risk Management Haijun Li lih@math.wsu.edu Department of Mathematics Washington State University Week 4 Haijun Li Math 576: Quantitative Risk Management Week 4 1 / 22 Outline 1 Basics

More information

Vector and Matrix Norms

Vector and Matrix Norms Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

More information

Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 18. A Brief Introduction to Continuous Probability

Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 18. A Brief Introduction to Continuous Probability CS 7 Discrete Mathematics and Probability Theory Fall 29 Satish Rao, David Tse Note 8 A Brief Introduction to Continuous Probability Up to now we have focused exclusively on discrete probability spaces

More information

Numerical Summarization of Data OPRE 6301

Numerical Summarization of Data OPRE 6301 Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

More information

Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23

Probability Theory. Elementary rules of probability Sum rule. Product rule. p. 23 Probability Theory Uncertainty is key concept in machine learning. Probability provides consistent framework for the quantification and manipulation of uncertainty. Probability of an event is the fraction

More information

Descriptive Statistics

Descriptive Statistics Y520 Robert S Michael Goal: Learn to calculate indicators and construct graphs that summarize and describe a large quantity of values. Using the textbook readings and other resources listed on the web

More information

Multivariate normal distribution and testing for means (see MKB Ch 3)

Multivariate normal distribution and testing for means (see MKB Ch 3) Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................

More information

Module 3: Correlation and Covariance

Module 3: Correlation and Covariance Using Statistical Data to Make Decisions Module 3: Correlation and Covariance Tom Ilvento Dr. Mugdim Pašiƒ University of Delaware Sarajevo Graduate School of Business O ften our interest in data analysis

More information

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling

Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data

More information

( ) = P Z > = P( Z > 1) = 1 Φ(1) = 1 0.8413 = 0.1587 P X > 17

( ) = P Z > = P( Z > 1) = 1 Φ(1) = 1 0.8413 = 0.1587 P X > 17 4.6 I company that manufactures and bottles of apple juice uses a machine that automatically fills 6 ounce bottles. There is some variation, however, in the amounts of liquid dispensed into the bottles

More information

The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].

The sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1]. Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real

More information

Chapter 4 - Lecture 1 Probability Density Functions and Cumul. Distribution Functions

Chapter 4 - Lecture 1 Probability Density Functions and Cumul. Distribution Functions Chapter 4 - Lecture 1 Probability Density Functions and Cumulative Distribution Functions October 21st, 2009 Review Probability distribution function Useful results Relationship between the pdf and the

More information

3 Random vectors and multivariate normal distribution

3 Random vectors and multivariate normal distribution 3 Random vectors and multivariate normal distribution As we saw in Chapter 1, a natural way to think about repeated measurement data is as a series of random vectors, one vector corresponding to each unit.

More information

Simple Linear Regression Inference

Simple Linear Regression Inference Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

More information

Lecture 8: More Continuous Random Variables

Lecture 8: More Continuous Random Variables Lecture 8: More Continuous Random Variables 26 September 2005 Last time: the eponential. Going from saying the density e λ, to f() λe λ, to the CDF F () e λ. Pictures of the pdf and CDF. Today: the Gaussian

More information

Hypothesis Testing COMP 245 STATISTICS. Dr N A Heard. 1 Hypothesis Testing 2 1.1 Introduction... 2 1.2 Error Rates and Power of a Test...

Hypothesis Testing COMP 245 STATISTICS. Dr N A Heard. 1 Hypothesis Testing 2 1.1 Introduction... 2 1.2 Error Rates and Power of a Test... Hypothesis Testing COMP 45 STATISTICS Dr N A Heard Contents 1 Hypothesis Testing 1.1 Introduction........................................ 1. Error Rates and Power of a Test.............................

More information

Statistical Foundations: Measures of Location and Central Tendency and Summation and Expectation

Statistical Foundations: Measures of Location and Central Tendency and Summation and Expectation Statistical Foundations: and Central Tendency and and Lecture 4 September 5, 2006 Psychology 790 Lecture #4-9/05/2006 Slide 1 of 26 Today s Lecture Today s Lecture Where this Fits central tendency/location

More information

Elliptical copulae. Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke

Elliptical copulae. Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke Elliptical copulae Dorota Kurowicka, Jolanta Misiewicz, Roger Cooke Abstract: In this paper we construct a copula, that is, a distribution with uniform marginals. This copula is continuous and can realize

More information

Point and Interval Estimates

Point and Interval Estimates Point and Interval Estimates Suppose we want to estimate a parameter, such as p or µ, based on a finite sample of data. There are two main methods: 1. Point estimate: Summarize the sample by a single number

More information

6.4 Normal Distribution

6.4 Normal Distribution Contents 6.4 Normal Distribution....................... 381 6.4.1 Characteristics of the Normal Distribution....... 381 6.4.2 The Standardized Normal Distribution......... 385 6.4.3 Meaning of Areas under

More information

Generalized Linear Models. Today: definition of GLM, maximum likelihood estimation. Involves choice of a link function (systematic component)

Generalized Linear Models. Today: definition of GLM, maximum likelihood estimation. Involves choice of a link function (systematic component) Generalized Linear Models Last time: definition of exponential family, derivation of mean and variance (memorize) Today: definition of GLM, maximum likelihood estimation Include predictors x i through

More information

Exercises with solutions (1)

Exercises with solutions (1) Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal

More information

1 Inner Products and Norms on Real Vector Spaces

1 Inner Products and Norms on Real Vector Spaces Math 373: Principles Techniques of Applied Mathematics Spring 29 The 2 Inner Product 1 Inner Products Norms on Real Vector Spaces Recall that an inner product on a real vector space V is a function from

More information

We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P. Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

More information