# Principle of Data Reduction

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Chapter 6 Principle of Data Reduction 6.1 Introduction An experimenter uses the information in a sample X 1,..., X n to make inferences about an unknown parameter θ. If the sample size n is large, then the observed sample x 1,..., x n is a long list of numbers that may be hard to interpret. Any statistic, T (X), defines a form of data reduction or data summary. For example, the sample mean, the sample variance, the largest observation, and the smallest observation are four statistics that might be used to summarize some key features of the sample. The statistic summarizes the data in that, rather than reporting the entire sample x, it reports only that T (x) = t. For example, two samples x and y will be treated as equal, if T (x) = T (y) is satisfied. The advantages and consequences of this type of data reduction are the topics of this chapter. We study three principles of data reduction. 1. The Sufficiency Principle promotes a method of data reduction that does not discard information about θ while achieving some summarization of the data. 2. The Likelihood Principle describes a function of the parameter determined by the observed sample, that contains all the information about θ that is available from the sample. 3. The Equivariance Principle prescribes yet another method of data reduction that still preserves some important features of the model. 11

2 12 CHAPTER 6. PRINCIPLE OF DATA REDUCTION 6.2 The Sufficiency Principle Sufficiency Principle: If T (X) is a sufficient statistic for θ, then any inference about θ should depend on the sample X only through the value T (X). That is, if x and y are two sample points such that T (x) = T (y), then the inference about θ should be the same whether X = x or Y = y is observed Sufficient Statistics Definition A statistic T (X) is a sufficient statistic for θ if the conditional distribution of the sample X given the value of T (X) does not depend on θ. To use this definition to verify that a statistic T (X) is a sufficient statistic for θ, we must verify that P (X = x T (X) = T (x)) does not depend on θ. Since {X = x} is a subset of {T (X) = T (x)}, P θ (X = x T (X) = T (x)) = P θ(x = x and T (X) = T (x)) P θ (T (X) = T (x)) P θ (X = x) = P θ (T (X) = T (x)) = p(x θ) q(t (x)) θ), where p(x θ) is the joint pmf/pdf of the sample X and q(t θ) is the pmf/pdf of T (X). Thus, T (X) is a sufficient statistic for θ if and only if, for every x, the above ratio is constant as a function of θ. Theorem If p(x θ) is the joint pdf or pmf of X and q(t θ) is the pdf or pmf of T (X), then T (X) is a sufficient statistic for θ if, for every x in the sample space, the ratio p(x θ)/q(t (x) θ) is constant as a function of θ. Example (Binomial sufficient statistic) Let X 1,..., X n be iid Bernoulli random variables with parameter θ, 0 < θ < 1. Then T (X) = X X n is a sufficient statistic for θ. Note that T (X) counts the number of X i s that equal 1, so T (X) Bi(n, θ). The ratio of pmfs is thus p(x θ) θ x q(t (x) θ) = i (1 θ) 1 x i ( n t) θ t (1 θ) n t (define t = n x i) = θp x i (1 θ) P (1 x i ) ( n ) t θ t (1 θ) n t = ( 1 n ) = ( 1 n t P xi ) Since this ratio does not depend on θ, by Theorem 6.2.1, T (x) is a sufficient statistic for θ.

3 6.2. THE SUFFICIENCY PRINCIPLE 13 Example (Normal sufficient statistic) Let X 1,..., X n be iid N(µ, σ 2 ), where σ 2 is known. We wish to show that the sample mean, T (X) = X, is a sufficient statistic for µ. n f(x µ) = (2πσ 2 ) 1/2 exp( (x i µ) 2 /(2σ 2 )) = (2πσ 2 ) n/2 exp{ [ (x i x) 2 + n( x µ) 2 ]/(2σ 2 )} Recall that the sample mean X N(µ, σ 2 /n). Thus, the ratio of pdf is p(x θ) q(t (x)) θ) = (2πσ2 ) n/2 exp{ [ n (x i x) 2 + n( x µ) 2 ]/(2σ 2 )} (2πσ 2 /n) 1/2 exp{ n( x µ) 2 /(2σ 2 )} = n 1/2 (2πσ 2 ) (n 1)/2 exp{ (x i x) 2 /(2σ 2 )}, which does not depend on µ. By Theorem 6.2.1, the sample mean is a sufficient statistic for µ. Example (Sufficient order statistics) Let X 1,..., X n be iid from a pdf f, where we are unable to specify any more information about the pdf. It then follows that n! n f X(1),...,X (n) (x) = f X(x i ), if x 1 <... < x n 0 otherwise where (x (1),..., x (n) is the order statistic. statistic. By Theorem 6.2.1, the order statistic is a sufficient Of course, this is not much of a reduction, but we should not expect more with so little information about the density f. However, even if we specify more about the density, we still may not be able to get much of a sufficient reduction. For example, suppose that f is the Cauchy pdf f(x θ) = 1 π(x θ) 2 or the logistic pdf f(x θ) = e (x θ) (1+e (x θ) ) 2. It turns out that outside of the exponential family of distributions, it is rare to have a sufficient statistic of smaller dimension than the size of the sample, so in many cases it will turn out that the order statistics are the best that we can do. Theorem (Factorization Theorem) Let f(x θ) denote the joint pdf or pmf of a sample X. A statistic T (X) is a sufficient statistic for θ if and only if there exist functions g(t θ) and h(x) such that, for all sample points x and all parameter points θ, f(x θ) = g(t (x) θ)h(x). (6.1)

4 14 CHAPTER 6. PRINCIPLE OF DATA REDUCTION Proof: We give the proof only for discrete distributions. Suppose T (X) is a sufficient statistic. Choose g(t θ) = P θ (T (X) = t) and h(x) = P (X = x T (X) = T (x)). Because T (X) is sufficient, the conditional probability h(x) does not depend on θ. Thus, f(x θ) = P θ (X = x) = P θ (X = x and T (X) = T (x)) = P θ (T (X) = T (x))p (X = x T (X) = T (x)) = g(t (x) θ)h(x) So factorization (6.1) has been exhibited. Also, the last two lines above imply that g(t (x) θ) is the pmf of T (X). Now assume the factorization (6.1) exists. Let q(t θ) be the pmf of T (X). Define A T (x) = {y : T (y) = T (x)}. Then f(x θ) g(t (x) θ)h(x) = q(t (x) θ) q(t (x) θ) g(t (x) θ)h(x) = A g(t (y) θ)h(y) (density transformation) T (x) = g(t (x) θ)h(x) g(t (x) θ) A T (x) h(y) h(x) = A h(y) T (x) (since T is a constant on A T (x)) Since the ratio does not depend on θ, by Theorem 6.2.1, T (X) is a sufficient statistic for θ. To use the Factorization Theorem to find a sufficient statistic, we factor the joint pdf of the sample into two parts. One part does not depend on θ and it constitutes the h(x) function. The other part depends on θ, and usually it depends on the sample x only through some function T (x) and this function is a sufficient statistic for θ. Example (Normal sufficient statistic) Let X 1,..., X n be iid N(µ, σ 2 ), where σ 2 is known. The pdf can be factored as f(x µ) = (2πσ 2 ) n/2 exp{ We can define (x i x) 2 /(2σ 2 )} exp{ n( x µ) 2 /(2σ 2 )}. g(t θ) = exp{ n(t µ) 2 /(2σ 2 )}

5 6.2. THE SUFFICIENCY PRINCIPLE 15 by defining T (x) = x, and h(x) = (2πσ 2 ) n/2 exp{ (x i x) 2 /(2σ 2 )}. Thus, by the Factorization Theorem, T (X) = X is a sufficient statistic for µ. Example (Uniform sufficient statistic) Let X 1,..., X n be iid observations from the discrete uniform distribution on 1,..., θ. That is, the unknown parameter, θ, is a positive integer and the pmf of X i is 1 θ x = 1, 2,..., θ f(x θ) = 0 otherwise. Thus, the joint pmf of X 1,..., X n is θ n x i {1,..., θ} for i = 1,..., n f(x θ) = 0 otherwise. Let I A (x) be the indicator function of the set A; that is, it is equal to 1 if x A and equal to 0 otherwise. Let N = {1, 2,...} be the set of positive integers and let N θ = {1, 2,..., θ}. Then the joint pmf of X 1,..., X n is n n f(x θ) = θ 1 I Nθ (x i ) = θ n I Nθ. Defining T (x) = max i x i, we see that n n I Nθ = ( I N (x i ))I Nθ (T (x)). Thus we have the factorization n f(x θ) = θ n I Nθ (T (x))( I N (x i )). By the factorization theorem, T (X) = max i X i is a sufficient statistic for θ. Example (Normal sufficient statistic, both parameters unknown) Assume that X 1,..., X n are iid N(µ, σ 2 ), and that both µ and σ 2 are unknown so the parameter vector is θ = (µ, σ 2 ). Let T 1 (x) = x and T 2 (x) = s 2 = n (x i x) 2 /(n 1). f(x θ) =(2πσ 2 ) n/2 exp{ [ (x i x) 2 + n( x µ) 2 ]/(2σ 2 )} = (2πσ 2 ) n/2 exp{ (n(t 1 µ) 2 + (n 1)t 2 )/(2σ 2 )}.

6 16 CHAPTER 6. PRINCIPLE OF DATA REDUCTION Let h(x) = 1. By the factorization theorem, T (X) = (T 1 (X), T 2 (X)) = ( X, S 2 ) is a sufficient statistic for (µ, σ 2 ). The results can be generalized to the exponential family of distributions. Theorem Let X 1,..., X n be iid observations from a pdf or pmf f(x θ) that belongs to an exponential family given by k f(x θ) = h(x)c(θ) exp{ w i (θ)t i (x)} where θ = (θ 1,..., θ d ), d k. Then T (X) = ( t 1 (X j ),..., t k (X j )) is a sufficient statistic for θ. j=1 j= Minimal Sufficient Statistics In any problem, there are many sufficient statistics. For example, 1. It is always true that the complete sample, X, is a sufficient statistic. 2. Any one-to-one function of a sufficient statistic is also a sufficient statistic. Recall that the purpose of a sufficient statistic is to achieve data reduction without loss of information about the parameter θ; thus, a statistic that achieves the most data reduction while still retaining all the information about θ might be considered preferable. Definition A sufficient statistic T (X) is called a minimal sufficient statistic if, for any other sufficient statistic T (X), T (x) is a function of T (x). To say that T (x) is a function of T (x) simply means that if T (x) = T (y), then T (x) = T (y). Let T = {t : t = T (x) for some x X } be the image of X under T (x). Then T x partitions the sample space into sets A t, t T, defined by A t = {x : T (x) = t}. If {B t : t T } are the partition sets for T (x) and {A t : t T } are the partition sets for T (x), then Definition states that every B t is a subset of some A t. Thus, the partition associates with a minimal sufficient statistic, is the coarsest possible partition for a sufficient statistic, and a minimal sufficient statistic achieves the greatest possible data reduction for a sufficient statistic.

7 6.2. THE SUFFICIENCY PRINCIPLE 17 Example (Two Normal sufficient statistics) Suppose that X 1,..., X n are observations from N(µ, σ 2 ) with σ 2 known. We know that T (X) = X is a sufficient statistic for µ. The factorization theorem shows that T (X) = ( X, S 2 ) is also a sufficient statistic for µ. T (X) can be written as a function of T (X) by defining the function r(a, b) = a. Then T (x) = x = r( x, S 2 ) = r(t (x)). Theorem let f(x θ) be the pmf or pdf of a sample X. Suppose there exists a function T (x) such that, for every two sample points x and y, the ratio f(x θ)/f(y θ) is a constant as a function of θ if and only if T (x) = T (y). Then T (x) is a minimal sufficient statistic for θ. Proof: To simply the proof, we assume f(x θ) > 0 for all x X and θ. First we show that T (X) is a sufficient statistic. Let T = {t : t = T (x) for some x X } be the image of X under T (x). Define the partition sets induced by T (x) as A t = {x : T (x) = t}. For each A t, choose and fix one element x t A t. For any x T, x T (x) is the fixed element that is in the same set, A t, as x. Since x and x T (x) are in the same set A t, T (x) = T (x T (x)) and, hence, f(x θ)/f(x T x θ) is constant as a function of θ. Thus, we can define a function on X by h(x) = f(x θ)/f(x T x θ) and h does not depend on θ. Define a function on T by g(t θ) = f(x t θ). Then it can be seen that f(x θ) = f(x T x θ)f(x θ) f(x T x θ) = g(t (x) θ)h(x) and, by the factorization theorem, T (X) is a sufficient statistic for θ. Now to show that T X is minimal, let T (X) be any other sufficient statistic. By the factorization theorem, there exist function g and h such that f(x θ) = g (T (x) θ)h (x). Let x and y be any two sample points with T (x) = T (y). Then f(x θ) f(y θ) = g (T (x) θ)h (x) g (T (y) θ)h (y) = h (x) h (y). Since this ratio does not depend on θ, the assumptions of the theorem imply that T (x) = T (y). Thus, T (x) is a function of T (x) and T (x) is minimal. Example (Normal minimal sufficient statistic) Let X 1,..., X n be iid N(µ, σ 2 ), both µ and σ 2 unknown. Let x and y denote two sample points, and let ( x, S 2 x) and (ȳ, S 2 y) be the sample means and variances corresponding to the x and y samples, respectively. Then the ratio of densities is f(x µ, σ 2 ) f(y µ, σ 2 ) = exp{[ n( x2 ȳ) 2 + 2nµ( x ȳ) (n 1)(s 2 x s 2 y)]/(2σ 2 )}.

8 18 CHAPTER 6. PRINCIPLE OF DATA REDUCTION This ratio will be constant as a function of µ and σ 2 if and only x = ȳ and s 2 x = s 2 y. Thus, by Theorem 6.2.4, ( X, S 2 ) is a minimal sufficient statistic for (µ, σ 2 ). Example (Uniform minimal sufficient statistic) Suppose X 1,..., X n are iid uniform observations on the interval (θ, θ + 1), < θ <. Then the joint pdf of X is 1 θ < x i < θ + 1, i = 1,..., n, f(x θ) = 0 otherwise, which can be written as 1 max i x i 1 < θ < min i x i f(x θ) = 0 otherwise Thus, for two sample points x and y, the numerator and denominator of the ratio f(x θ)/f(y θ) will be positive for the same values of θ if and only if min i x i = min i y i and max i x i = max i y i. Thus, we have that T (X) = (X (1), X (n) ) is a minimal sufficient statistic. This example shows the dimension of a minimal sufficient statistic may not match the dimension of the parameter. A minimal sufficient statistic is not unique. Any one-to-one function of a minimal sufficient statistic is also a minimal sufficient statistic. For example, T (X) = (X (n) X (1), (X (n) + X (1) )/2) is also a minimal sufficient statistic in Example 6.2.9, and T (X) = ( n X i, n X2 i ) is also a minimal sufficient statistic in Example Ancillary Statistics Definition A statistic S(X) whose distribution does not depend on the parameter θ is called an ancillary statistic. Alone, an ancillary statistic contains no information about θ. An ancillary statistic is an observation on a random variable whose distribution is fixed and known, unrelated to θ. Paradoxically, an ancillary statistic, when used in conjunction with other statistics, sometimes does contain valuable information for inferences about θ. Example (Uniform ancillary statistic) Let X 1,..., X n be iid uniform observations on the interval (θ, θ + 1), < θ <. The range statistic R = X (n) X (1) is an ancillary statistic.

9 6.2. THE SUFFICIENCY PRINCIPLE 19 Recall that the cdf of each X i is 0 x θ F (x θ) = x θ θ < x < θ θ + 1 x. Thus, the joint pdf of X (1) and X (n) is n(n 1)(x (n) x (1) ) n 2 θ < x (1) < x (n) < θ + 1 g(x (1), x (n) θ) = 0 otherwise Making the transformation R = X (n) X (1) and M = (X (n) + X (1) )/2, we see the joint pdf of R and M is n(n 1)r n 2 0 < r < 1, θ + r/2 < m < θ + 1 r/2 h(r, m θ) = 0 otherwise Thus, the pdf for R is h(r θ) = θ+1 r/2 θ+r/2 n(n 1)r n 2 dm = n(n 1)r n 2 (1 r), 0 < r < 1. This is a beta pdf with α = n 1 and β = 2. Thus, the distribution of R does not depend on θ, and R is ancillary. Example (Location and scale family ancillary statistic) Let X 1,..., X n be iid observations from a location family with cdf F (x θ), < θ <. Show that the range statistic R = X (n) X (1) is an ancillary statistic. Let X 1,..., X n be iid observations from a scale family with cdf F (x/σ), σ > 0. Show that any statistic that depends on the sample only through the n 1 values X 1 /X n,..., X n 1 /X n is an ancillary statistic Sufficient, Ancillary, and Complete Statistics A minimal sufficient statistic is a statistic that has achieved the maximal amount of data reduction possible while still retaining all the information about the parameter θ. Intuitively, a minimal sufficient statistic eliminates all the extraneous information in the sample, retaining only that piece with information about θ. Since the distribution of an ancillary statistic does not depend on θ, it

10 20 CHAPTER 6. PRINCIPLE OF DATA REDUCTION might be suspected that a minimal sufficient statistic is unrelated to an ancillary statistic. However, this is not necessarily the case. Recall Example in which X 1,..., X n were iid obs from a uniform(θ, θ + 1) distribution. We have pointed out that the statistic (X (n) X (1), (X (n) + X (1) )/2) is a minimal sufficient statistic, and in Example we showed that X (n) X (1) is an ancillary statistic. Thus, in this case, the ancillary statistic is an important component of the minimal sufficient statistic. Certainly, the ancillary statistic and the minimal sufficient statistic are not independent. The following example shows that an ancillary statistic can sometimes give important information for inference about θ. Example (Ancillary precision) Let X 1 and X 2 be iid obs from the discrete distribution that satisfies P (X = θ) = P (X = θ + 1) = P (θ + 2) = 1/3, where θ, the unknown parameter, is any integer. It can be shown with an argument similar to that in Example that (R, M), where R = X (n) X (1) and M = (X (n) + X (1) )/2, is a minimal sufficient statistic. To see how R might give information about θ, consider a sample point (r, m), where m is an integer. First we consider only m, θ must be one of three values, either θ = m or θ = m 1 or θ = m 2. With only the information that M = m, all three θ values are possible values. But now suppose we get the information that R = 2. Then it must be the case that X (1) = m 1 and X (2) = m + 1, and the only possible value for θ is θ = m 1. Thus, the knowledge of the value of the ancillary statistic R has increased our knowledge about θ. Of course, the knowledge of R alone would give us no information about θ. For many important situations, however, our intuition that a minimal sufficient statistic is independent of any ancillary statistic is correct. A description of situations in which this occurs relies on the next definition. Definition Let f(t θ) be a family of pdfs or pmfs for a statistic T (X). The family of probability distributions is called complete if E θ g(t ) = 0 for all θ implies P θ (g(t ) = 0) = 1 for all θ. Equivalently, T (X) is called a complete statistic.

11 6.2. THE SUFFICIENCY PRINCIPLE 21 Example (Binomial complete sufficient statistic) Suppose that T has a binomial (n, p) distribution, 0 < p < 1. Let g be a function such that E p g(t ) = 0. Then 0 = E p g(t ) = ( ) n g(t ) p t (1 p) n t = (1 p) n t t=0 ( ) n ( p g(t) t 1 p for all p, 0 < p < 1. Let r = p/(1 p), 0 < r <. The last expression is a polynomial of degree n in r, where the coefficient of r t is g(t) ( n t). For the polynomial to be 0 for all r, each coefficient must be 0. Thus, g(t) = 0 for t = 0, 1,..., n. Since T takes on the values 0, 1..., n with probability 1, this yields that P p (g(t ) = 0) = 1 for all p. Hence, T is a complete statistic. t=0 ) t Example Let X 1,..., X n be iid uniform (0, θ) observations, 0 < θ <. Show that T (X) = max i X i is a complete statistic. Using an argument similar to that used in one of previous examples, we can see that T (X) = max i X i is a sufficient statistic and the pdf of T (X) is nt n 1 θ n 0 < t < θ f(t θ) = 0 otherwise. Suppose g(t) is a function satisfying E θ g(t ) = 0 for all θ. Since E θ g(t ) is a constant of θ, its derivative with respect to θ is 0. Thus we have that 0 = d dθ E θg(t ) = d dθ θ 0 = (θ n ) d dθ g(t)nt n 1 θ n dt θ = θ n ng(θ)θ n = θ 1 ng(θ) 0 ng(t)t n 1 dt + ( d θ dθ θ n ) ng(t)t n 1 dt 0 Since θ 1 n 0, it must be g(θ) = 0. This is true for every θ > 0, hence, T is a complete statistic. Theorem (Basu s Theorem) If T (X) is a complete and minimal sufficient statistic, then T (X) is independent of every ancillary statistic. Proof: We give the proof only for discrete distributions. Let S(X) be any ancillary statistic. Then P (S(X) = s) does not depend on θ since S(X) is ancillary. Also the conditional probability, P (S(X) = s T (X) = t) = P (X {x : S(x) = s} T (X) = t),

12 22 CHAPTER 6. PRINCIPLE OF DATA REDUCTION does not depend on θ because T (X) is a sufficient statistic (recall the definition!). Thus, to show that S(X) and T (X) are independent, it suffices to show that P (S(X) = s T (X) = t) = P (S(X) = s) (6.2) for all possible values t T. Now, P (S(X) = s) = t T P (S(X) = s T (X) = t)p θ (T (X) = t). Furthermore, since t T P θ(t (X) = t) = 1, we can write P (S(X) = s) = t T P (S(X) = s)p θ (T (X) = t) Therefore, if we define the statistic g(t) = P (S(X) = s T (X) = t) P (S(X) = s), the above two equations show that E θ g(t ) = t T g(t)p θ (T (X) = t) = 0 for all θ. Since T (X) is a complete statistic, this implies that g(t) = 0 for all possible values t T. Hence, (6.2) is verified. It should be noted that the minimality of the sufficient statistic was not used in the proof of Basu s theorem. Indeed, the theorem is true with this word omitted, because a fundamental property of a complete statistic is that it is minimal. Theorem (Complete statistics in the exponential family) Let X 1,..., X n be iid observations from an exponential family with pdf or pmf of the form f(x θ) = h(x)c(θ) exp ( k w j (θ)t j (x) ), where θ = (θ 1, θ 2,..., θ k ). Then the statistic T (X) = ( t 1 (X i ), t 2 (X i ),..., t k (X i )) j=1 is complete if {w 1 (θ),..., w k (θ)) : θ Θ} contains an open set in R k.

13 6.3. THE LIKELIHOOD PRINCIPLE 23 Example (Using Basu s theorem) Let X 1,..., X n be iid exponential observations with parameter θ. Consider computing the expected value of g(x) = X n X X n. We first note that the exponential distributions form a scale parameter family and thus g(x) is an ancillary statistic. The exponential distributions also form an exponential family with t(x) = x and so, by Theorem 6.2.6, T (X) = is a complete statistic and by, Theorem 6.2.3, T (X) is a sufficient statistic. theorem, T (X) and g(x) are independent. Thus we have X i Hence, by Basu s θ = E θ X n = E θ T (X)g(X) = (E θ T (X))(E θ g(x)) = nθe θ g(x). Hence, for any θ, E θ g(x) = 1/n. 6.3 The likelihood Principle The likelihood function Definition Let f(x θ) denote the joint pdf or pmf of the sample X = (X 1,..., X n ). Then, given that X = x is observed, the function of θ defined by L(θ x) = f(x θ) is called the likelihood function. This definition almost seems to be defining the likelihood function to be the same as the pdf or pmf. The only distinction between these two functions is which variable is considered fixed and which is varying. When we consider the pdf or pmf f(x θ), we are considering θ as fixed and x as the variable; when we consider the likelihood function L(θ x), we are considering x to be the observed sample point and θ to be varying over all possible parameter values. If we compare the likelihood function at two parameter points and find that L(θ 1 x) > L(θ 2 x),

14 24 CHAPTER 6. PRINCIPLE OF DATA REDUCTION then the sample we actually observed is more likely to have occurred if θ = θ 1 than if θ = θ 2, which can be interpreted as saying that θ 1 is a more plausible value for the true value of θ than is θ 2. We carefully use the word plausible rather than probable because we often think of θ as a fixed value. Furthermore, although f(x θ), as a function of x, is a pdf, there is no guarantee that L(θ x), as a function of θ, is a pdf. LIKELIHOOD PRINCIPLE: If x and y are two sample points such that L(θ x) is proportional to L(θ y), that is, there exists a constant C(x, y) such that L(θ x) = C(x, y)l(θ y) for all θ, then the conclusions drawn from x and y should be identical. Note that C(x, y) may be different for different (x, y) pairs but C(x, y) does not depend on θ. 6.4 The Equivariance Principle The first type of equivariance might be called measurement equivariance. It prescribes that the inference made should not depend on the measurement scale that is used. The second type of equivariance, actually an invariance, might be called formal invariance. It states that if two inference problems have the same formal structure in terms of the mathematical model used, then the same inference procedure should be used in both problems. The elements of the model that must be the same are: Θ, the parameter space; {f(x θ) : θ Θ}, the set of pdfs or pmfs for the sample; and the set of allowable inferences and consequences of wrong inferences. For example, Θ may be Θ = {θ : θ > 0} in two problems. But in one problem θ may be the average price of a dozen eggs and in another problem θ may refer to the average height of giraffes. Yet, formal invariance equates these two parameter spaces since they both refer to the same set of real numbers. Equivariance Principle: If Y = g(x) is a change of measurement scale such that the model for Y has the same formal structure as the model for X, then an inference procedure should be both measurement equivariant and formally equivariant. Example (Binomial equivariance) Let X Bionmial(n, p) with known n and p. Let T (x) be the estimate of p that is used when X = x is observed. Rather than using the number of successes, X, to make an inference about p, we could use the number of failures, Y = n X. We

15 6.4. THE EQUIVARIANCE PRINCIPLE 25 can see that Y Binomial(n, q = 1 p). Let T (y) be the estimate of q that is used when Y = y is observed, so that 1 T (y) is the estimate of p when Y = y is observed. If x successes are observed, then the estimate of p is T (x). But if there are x successes, then there are n x failures and 1 T (n x) is also an estimate of p. Measurement equivariance requires that these two estimates be equal, that is, T (x) = 1 T (n x), since the change from X to Y is just a change in measurement scale. Furthermore, the formal structures of the inference problems based on X and Y are the same. X and Y both have binomial(n, θ) distributions, 0 θ 1. So formal invariance requires that T (z) = T (z) for all z = 0,..., n. Thus, measurement and formal invariance together require that T (x) = 1 T (n x) = 1 T (n x). If we consider only estimators satisfying the above equality, then we have greatly reduced and simplified the set of estimators we are wiling to consider. Whereas the specification of an arbitray estimator requires the specification of T (0),..., T (n), the specification of an estimator satisfying the above equality requires the specification only of T (0),..., T ([n/2]), where [n/2] is the greatest integer not larger than n/2. This is the type of data reduction that is always achieved by the equivariance principle. The inference to be made for some sample points determines the inference to be made for other sample points.

### Maximum Likelihood Estimation

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Maximum Likelihood Estimation 1 Maximum Likelihood Estimation Maximum likelihood is a relatively simple method of constructing an estimator for

### 1 Prior Probability and Posterior Probability

Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which

### 1 Sufficient statistics

1 Sufficient statistics A statistic is a function T = rx 1, X 2,, X n of the random sample X 1, X 2,, X n. Examples are X n = 1 n s 2 = = X i, 1 n 1 the sample mean X i X n 2, the sample variance T 1 =

### Using pivots to construct confidence intervals. In Example 41 we used the fact that

Using pivots to construct confidence intervals In Example 41 we used the fact that Q( X, µ) = X µ σ/ n N(0, 1) for all µ. We then said Q( X, µ) z α/2 with probability 1 α, and converted this into a statement

### Introduction to Hypothesis Testing. Point estimation and confidence intervals are useful statistical inference procedures.

Introduction to Hypothesis Testing Point estimation and confidence intervals are useful statistical inference procedures. Another type of inference is used frequently used concerns tests of hypotheses.

### Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation

Parametric Models Part I: Maximum Likelihood and Bayesian Density Estimation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2015 CS 551, Fall 2015

### Practice problems for Homework 11 - Point Estimation

Practice problems for Homework 11 - Point Estimation 1. (10 marks) Suppose we want to select a random sample of size 5 from the current CS 3341 students. Which of the following strategies is the best:

### Sufficient Statistics and Exponential Family. 1 Statistics and Sufficient Statistics. Math 541: Statistical Theory II. Lecturer: Songfeng Zheng

Math 541: Statistical Theory II Lecturer: Songfeng Zheng Sufficient Statistics and Exponential Family 1 Statistics and Sufficient Statistics Suppose we have a random sample X 1,, X n taken from a distribution

### Lecture 8. Confidence intervals and the central limit theorem

Lecture 8. Confidence intervals and the central limit theorem Mathematical Statistics and Discrete Mathematics November 25th, 2015 1 / 15 Central limit theorem Let X 1, X 2,... X n be a random sample of

### Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab

Monte Carlo Simulation: IEOR E4703 Fall 2004 c 2004 by Martin Haugh Overview of Monte Carlo Simulation, Probability Review and Introduction to Matlab 1 Overview of Monte Carlo Simulation 1.1 Why use simulation?

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### 5. Continuous Random Variables

5. Continuous Random Variables Continuous random variables can take any value in an interval. They are used to model physical characteristics such as time, length, position, etc. Examples (i) Let X be

### Efficiency and the Cramér-Rao Inequality

Chapter Efficiency and the Cramér-Rao Inequality Clearly we would like an unbiased estimator ˆφ (X of φ (θ to produce, in the long run, estimates which are fairly concentrated i.e. have high precision.

### Dongfeng Li. Autumn 2010

Autumn 2010 Chapter Contents Some statistics background; ; Comparing means and proportions; variance. Students should master the basic concepts, descriptive statistics measures and graphs, basic hypothesis

### Solution Using the geometric series a/(1 r) = x=1. x=1. Problem For each of the following distributions, compute

Math 472 Homework Assignment 1 Problem 1.9.2. Let p(x) 1/2 x, x 1, 2, 3,..., zero elsewhere, be the pmf of the random variable X. Find the mgf, the mean, and the variance of X. Solution 1.9.2. Using the

### Basics of Statistical Machine Learning

CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar

### MATH4427 Notebook 2 Spring 2016. 2 MATH4427 Notebook 2 3. 2.1 Definitions and Examples... 3. 2.2 Performance Measures for Estimators...

MATH4427 Notebook 2 Spring 2016 prepared by Professor Jenny Baglivo c Copyright 2009-2016 by Jenny A. Baglivo. All Rights Reserved. Contents 2 MATH4427 Notebook 2 3 2.1 Definitions and Examples...................................

### MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo. 3 MT426 Notebook 3 3. 3.1 Definitions... 3. 3.2 Joint Discrete Distributions...

MT426 Notebook 3 Fall 2012 prepared by Professor Jenny Baglivo c Copyright 2004-2012 by Jenny A. Baglivo. All Rights Reserved. Contents 3 MT426 Notebook 3 3 3.1 Definitions............................................

### Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment 2-3, Probability and Statistics, March 2015. Due:-March 25, 2015.

Department of Mathematics, Indian Institute of Technology, Kharagpur Assignment -3, Probability and Statistics, March 05. Due:-March 5, 05.. Show that the function 0 for x < x+ F (x) = 4 for x < for x

### Hypothesis Testing COMP 245 STATISTICS. Dr N A Heard. 1 Hypothesis Testing 2 1.1 Introduction... 2 1.2 Error Rates and Power of a Test...

Hypothesis Testing COMP 45 STATISTICS Dr N A Heard Contents 1 Hypothesis Testing 1.1 Introduction........................................ 1. Error Rates and Power of a Test.............................

### The Dirichlet-Multinomial and Dirichlet-Categorical models for Bayesian inference

The Dirichlet-Multinomial and Dirichlet-Categorical models for Bayesian inference Stephen Tu tu.stephenl@gmail.com 1 Introduction This document collects in one place various results for both the Dirichlet-multinomial

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

### Multivariate normal distribution and testing for means (see MKB Ch 3)

Multivariate normal distribution and testing for means (see MKB Ch 3) Where are we going? 2 One-sample t-test (univariate).................................................. 3 Two-sample t-test (univariate).................................................

### Normal distribution. ) 2 /2σ. 2π σ

Normal distribution The normal distribution is the most widely known and used of all distributions. Because the normal distribution approximates many natural phenomena so well, it has developed into a

### Jointly Distributed Random Variables

Jointly Distributed Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Jointly Distributed Random Variables 1 1.1 Definition......................................... 1 1.2 Joint cdfs..........................................

### CHAPTER 6: Continuous Uniform Distribution: 6.1. Definition: The density function of the continuous random variable X on the interval [A, B] is.

Some Continuous Probability Distributions CHAPTER 6: Continuous Uniform Distribution: 6. Definition: The density function of the continuous random variable X on the interval [A, B] is B A A x B f(x; A,

### 6. Jointly Distributed Random Variables

6. Jointly Distributed Random Variables We are often interested in the relationship between two or more random variables. Example: A randomly chosen person may be a smoker and/or may get cancer. Definition.

### Hypothesis Testing. 1 Introduction. 2 Hypotheses. 2.1 Null and Alternative Hypotheses. 2.2 Simple vs. Composite. 2.3 One-Sided and Two-Sided Tests

Hypothesis Testing 1 Introduction This document is a simple tutorial on hypothesis testing. It presents the basic concepts and definitions as well as some frequently asked questions associated with hypothesis

### Introduction to Finite Fields (cont.)

Chapter 6 Introduction to Finite Fields (cont.) 6.1 Recall Theorem. Z m is a field m is a prime number. Theorem (Subfield Isomorphic to Z p ). Every finite field has the order of a power of a prime number

### Practice with Proofs

Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using

### Joint Exam 1/P Sample Exam 1

Joint Exam 1/P Sample Exam 1 Take this practice exam under strict exam conditions: Set a timer for 3 hours; Do not stop the timer for restroom breaks; Do not look at your notes. If you believe a question

### Review for Calculus Rational Functions, Logarithms & Exponentials

Definition and Domain of Rational Functions A rational function is defined as the quotient of two polynomial functions. F(x) = P(x) / Q(x) The domain of F is the set of all real numbers except those for

### A Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails

12th International Congress on Insurance: Mathematics and Economics July 16-18, 2008 A Uniform Asymptotic Estimate for Discounted Aggregate Claims with Subexponential Tails XUEMIAO HAO (Based on a joint

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

### Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS

Chapter 3: DISCRETE RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS Part 4: Geometric Distribution Negative Binomial Distribution Hypergeometric Distribution Sections 3-7, 3-8 The remaining discrete random

### [Chapter 10. Hypothesis Testing]

[Chapter 10. Hypothesis Testing] 10.1 Introduction 10.2 Elements of a Statistical Test 10.3 Common Large-Sample Tests 10.4 Calculating Type II Error Probabilities and Finding the Sample Size for Z Tests

### MA107 Precalculus Algebra Exam 2 Review Solutions

MA107 Precalculus Algebra Exam 2 Review Solutions February 24, 2008 1. The following demand equation models the number of units sold, x, of a product as a function of price, p. x = 4p + 200 a. Please write

### Generalized Linear Models. Today: definition of GLM, maximum likelihood estimation. Involves choice of a link function (systematic component)

Generalized Linear Models Last time: definition of exponential family, derivation of mean and variance (memorize) Today: definition of GLM, maximum likelihood estimation Include predictors x i through

### P (x) 0. Discrete random variables Expected value. The expected value, mean or average of a random variable x is: xp (x) = v i P (v i )

Discrete random variables Probability mass function Given a discrete random variable X taking values in X = {v 1,..., v m }, its probability mass function P : X [0, 1] is defined as: P (v i ) = Pr[X =

### Limit processes are the basis of calculus. For example, the derivative. f f (x + h) f (x)

SEC. 4.1 TAYLOR SERIES AND CALCULATION OF FUNCTIONS 187 Taylor Series 4.1 Taylor Series and Calculation of Functions Limit processes are the basis of calculus. For example, the derivative f f (x + h) f

### Chapter Three. Functions. In this section, we study what is undoubtedly the most fundamental type of relation used in mathematics.

Chapter Three Functions 3.1 INTRODUCTION In this section, we study what is undoubtedly the most fundamental type of relation used in mathematics. Definition 3.1: Given sets X and Y, a function from X to

### Continuous Random Variables

Continuous Random Variables COMP 245 STATISTICS Dr N A Heard Contents 1 Continuous Random Variables 2 11 Introduction 2 12 Probability Density Functions 3 13 Transformations 5 2 Mean, Variance and Quantiles

### Math 120 Final Exam Practice Problems, Form: A

Math 120 Final Exam Practice Problems, Form: A Name: While every attempt was made to be complete in the types of problems given below, we make no guarantees about the completeness of the problems. Specifically,

### 3.6: General Hypothesis Tests

3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

### Chapter 6: Point Estimation. Fall 2011. - Probability & Statistics

STAT355 Chapter 6: Point Estimation Fall 2011 Chapter Fall 2011 6: Point1 Estimat / 18 Chap 6 - Point Estimation 1 6.1 Some general Concepts of Point Estimation Point Estimate Unbiasedness Principle of

### Nonparametric adaptive age replacement with a one-cycle criterion

Nonparametric adaptive age replacement with a one-cycle criterion P. Coolen-Schrijner, F.P.A. Coolen Department of Mathematical Sciences University of Durham, Durham, DH1 3LE, UK e-mail: Pauline.Schrijner@durham.ac.uk

### Chapter 4 Expected Values

Chapter 4 Expected Values 4. The Expected Value of a Random Variables Definition. Let X be a random variable having a pdf f(x). Also, suppose the the following conditions are satisfied: x f(x) converges

### Econometrics Simple Linear Regression

Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight

### ST 371 (VIII): Theory of Joint Distributions

ST 371 (VIII): Theory of Joint Distributions So far we have focused on probability distributions for single random variables. However, we are often interested in probability statements concerning two or

### 4. Continuous Random Variables, the Pareto and Normal Distributions

4. Continuous Random Variables, the Pareto and Normal Distributions A continuous random variable X can take any value in a given range (e.g. height, weight, age). The distribution of a continuous random

### Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01 Probability and Statistics for Engineers Winter 2010-2011 Contents 1 Joint Probability Distributions 1 1.1 Two Discrete

### Exponential Distribution

Exponential Distribution Definition: Exponential distribution with parameter λ: { λe λx x 0 f(x) = 0 x < 0 The cdf: F(x) = x Mean E(X) = 1/λ. f(x)dx = Moment generating function: φ(t) = E[e tx ] = { 1

### e.g. arrival of a customer to a service station or breakdown of a component in some system.

Poisson process Events occur at random instants of time at an average rate of λ events per second. e.g. arrival of a customer to a service station or breakdown of a component in some system. Let N(t) be

### Chapters 5. Multivariate Probability Distributions

Chapters 5. Multivariate Probability Distributions Random vectors are collection of random variables defined on the same sample space. Whenever a collection of random variables are mentioned, they are

### Definition of Random Variable A random variable is a function from a sample space S into the real numbers.

.4 Random Variable Motivation example In an opinion poll, we might decide to ask 50 people whether they agree or disagree with a certain issue. If we record a for agree and 0 for disagree, the sample space

### Introduction to polynomials

Worksheet 4.5 Polynomials Section 1 Introduction to polynomials A polynomial is an expression of the form p(x) = p 0 + p 1 x + p 2 x 2 + + p n x n, (n N) where p 0, p 1,..., p n are constants and x os

### 3.4 Statistical inference for 2 populations based on two samples

3.4 Statistical inference for 2 populations based on two samples Tests for a difference between two population means The first sample will be denoted as X 1, X 2,..., X m. The second sample will be denoted

### STAT 315: HOW TO CHOOSE A DISTRIBUTION FOR A RANDOM VARIABLE

STAT 315: HOW TO CHOOSE A DISTRIBUTION FOR A RANDOM VARIABLE TROY BUTLER 1. Random variables and distributions We are often presented with descriptions of problems involving some level of uncertainty about

### Part 2: One-parameter models

Part 2: One-parameter models Bernoilli/binomial models Return to iid Y 1,...,Y n Bin(1, θ). The sampling model/likelihood is p(y 1,...,y n θ) =θ P y i (1 θ) n P y i When combined with a prior p(θ), Bayes

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### The Normal Distribution. Alan T. Arnholt Department of Mathematical Sciences Appalachian State University

The Normal Distribution Alan T. Arnholt Department of Mathematical Sciences Appalachian State University arnholt@math.appstate.edu Spring 2006 R Notes 1 Copyright c 2006 Alan T. Arnholt 2 Continuous Random

### Statistics 100A Homework 8 Solutions

Part : Chapter 7 Statistics A Homework 8 Solutions Ryan Rosario. A player throws a fair die and simultaneously flips a fair coin. If the coin lands heads, then she wins twice, and if tails, the one-half

### STAT 830 Convergence in Distribution

STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2011 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2011 1 / 31

### THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok

THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan

### Chapter 3 Joint Distributions

Chapter 3 Joint Distributions 3.6 Functions of Jointly Distributed Random Variables Discrete Random Variables: Let f(x, y) denote the joint pdf of random variables X and Y with A denoting the two-dimensional

### Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.6 Comparing the

### Lecture 4: BK inequality 27th August and 6th September, 2007

CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound

### Stochastic Inventory Control

Chapter 3 Stochastic Inventory Control 1 In this chapter, we consider in much greater details certain dynamic inventory control problems of the type already encountered in section 1.3. In addition to the

### Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.5 The t Test 9.6

### Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections - we are still here Skip: 9.2 Testing Simple Hypotheses Skip: 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.5 The t Test 9.6 Comparing the

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### For a partition B 1,..., B n, where B i B j = for i. A = (A B 1 ) (A B 2 ),..., (A B n ) and thus. P (A) = P (A B i ) = P (A B i )P (B i )

Probability Review 15.075 Cynthia Rudin A probability space, defined by Kolmogorov (1903-1987) consists of: A set of outcomes S, e.g., for the roll of a die, S = {1, 2, 3, 4, 5, 6}, 1 1 2 1 6 for the roll

### Chapter 3 RANDOM VARIATE GENERATION

Chapter 3 RANDOM VARIATE GENERATION In order to do a Monte Carlo simulation either by hand or by computer, techniques must be developed for generating values of random variables having known distributions.

### Some probability and statistics

Appendix A Some probability and statistics A Probabilities, random variables and their distribution We summarize a few of the basic concepts of random variables, usually denoted by capital letters, X,Y,

### Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

### Lecture Notes 1. Brief Review of Basic Probability

Probability Review Lecture Notes Brief Review of Basic Probability I assume you know basic probability. Chapters -3 are a review. I will assume you have read and understood Chapters -3. Here is a very

### MTH4100 Calculus I. Lecture notes for Week 8. Thomas Calculus, Sections 4.1 to 4.4. Rainer Klages

MTH4100 Calculus I Lecture notes for Week 8 Thomas Calculus, Sections 4.1 to 4.4 Rainer Klages School of Mathematical Sciences Queen Mary University of London Autumn 2009 Theorem 1 (First Derivative Theorem

### Experimental Design. Power and Sample Size Determination. Proportions. Proportions. Confidence Interval for p. The Binomial Test

Experimental Design Power and Sample Size Determination Bret Hanlon and Bret Larget Department of Statistics University of Wisconsin Madison November 3 8, 2011 To this point in the semester, we have largely

### Properties of Future Lifetime Distributions and Estimation

Properties of Future Lifetime Distributions and Estimation Harmanpreet Singh Kapoor and Kanchan Jain Abstract Distributional properties of continuous future lifetime of an individual aged x have been studied.

### ST 371 (IV): Discrete Random Variables

ST 371 (IV): Discrete Random Variables 1 Random Variables A random variable (rv) is a function that is defined on the sample space of the experiment and that assigns a numerical variable to each possible

### The Delta Method and Applications

Chapter 5 The Delta Method and Applications 5.1 Linear approximations of functions In the simplest form of the central limit theorem, Theorem 4.18, we consider a sequence X 1, X,... of independent and

### E3: PROBABILITY AND STATISTICS lecture notes

E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................

### Lecture 6: Discrete & Continuous Probability and Random Variables

Lecture 6: Discrete & Continuous Probability and Random Variables D. Alex Hughes Math Camp September 17, 2015 D. Alex Hughes (Math Camp) Lecture 6: Discrete & Continuous Probability and Random September

### ( ) = P Z > = P( Z > 1) = 1 Φ(1) = 1 0.8413 = 0.1587 P X > 17

4.6 I company that manufactures and bottles of apple juice uses a machine that automatically fills 6 ounce bottles. There is some variation, however, in the amounts of liquid dispensed into the bottles

### Exercises with solutions (1)

Exercises with solutions (). Investigate the relationship between independence and correlation. (a) Two random variables X and Y are said to be correlated if and only if their covariance C XY is not equal

### Gambling and Data Compression

Gambling and Data Compression Gambling. Horse Race Definition The wealth relative S(X) = b(x)o(x) is the factor by which the gambler s wealth grows if horse X wins the race, where b(x) is the fraction

### Lecture 3: Continuous distributions, expected value & mean, variance, the normal distribution

Lecture 3: Continuous distributions, expected value & mean, variance, the normal distribution 8 October 2007 In this lecture we ll learn the following: 1. how continuous probability distributions differ

### An Introduction to Basic Statistics and Probability

An Introduction to Basic Statistics and Probability Shenek Heyward NCSU An Introduction to Basic Statistics and Probability p. 1/4 Outline Basic probability concepts Conditional probability Discrete Random

### Generating Random Numbers Variance Reduction Quasi-Monte Carlo. Simulation Methods. Leonid Kogan. MIT, Sloan. 15.450, Fall 2010

Simulation Methods Leonid Kogan MIT, Sloan 15.450, Fall 2010 c Leonid Kogan ( MIT, Sloan ) Simulation Methods 15.450, Fall 2010 1 / 35 Outline 1 Generating Random Numbers 2 Variance Reduction 3 Quasi-Monte

### Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 18. A Brief Introduction to Continuous Probability

CS 7 Discrete Mathematics and Probability Theory Fall 29 Satish Rao, David Tse Note 8 A Brief Introduction to Continuous Probability Up to now we have focused exclusively on discrete probability spaces

### Exact Confidence Intervals

Math 541: Statistical Theory II Instructor: Songfeng Zheng Exact Confidence Intervals Confidence intervals provide an alternative to using an estimator ˆθ when we wish to estimate an unknown parameter

### The Exponential Distribution

21 The Exponential Distribution From Discrete-Time to Continuous-Time: In Chapter 6 of the text we will be considering Markov processes in continuous time. In a sense, we already have a very good understanding

### 6 PROBABILITY GENERATING FUNCTIONS

6 PROBABILITY GENERATING FUNCTIONS Certain derivations presented in this course have been somewhat heavy on algebra. For example, determining the expectation of the Binomial distribution (page 5.1 turned

### ECE302 Spring 2006 HW7 Solutions March 11, 2006 1

ECE32 Spring 26 HW7 Solutions March, 26 Solutions to HW7 Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where

### M2S1 Lecture Notes. G. A. Young http://www2.imperial.ac.uk/ ayoung

M2S1 Lecture Notes G. A. Young http://www2.imperial.ac.uk/ ayoung September 2011 ii Contents 1 DEFINITIONS, TERMINOLOGY, NOTATION 1 1.1 EVENTS AND THE SAMPLE SPACE......................... 1 1.1.1 OPERATIONS

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

### 4.5 Linear Dependence and Linear Independence

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then