Probability Space. Sample space (denoted by ) by the problem we concern. Event space (denoted by a -field)

Transcription

1 4.1 Probability Space Sample space (denoted by ) by the problem we concern Event space (denoted by a -field) by a random variable preserving the structure Support (on the real line) with Copyright 013 Pearson Education Ch. 4-1

2 4.1 Event Space If I am interested in the numbers of heads, the event space is: E={{HH}, {HT, TH}, {TT}} HH, HT, TH, TT 0 1 Copyright 013 Pearson Education Ch. 4-

3 4.1 Review Random Variable is a real-valued set function that assigns one and only one real number to each outcome of a random experiment. Random variable preserves the structure of event space. Random variable maps from the event space to the real line, to construct the support of a probability distribution. The probabilities random variable assigned to the supports are consistent with the probability distribution over the event space. Copyright 013 Pearson Education Ch. 4-3

4 Discrete Random Variable Takes on no more than a countable number of values Examples: Roll a die twice Let X be the number of times 4 comes up (then X could be 0, 1, or times) Toss a coin 5 times. Let X be the number of heads (then X = 0, 1,, 3, 4, or 5) Copyright 013 Pearson Education Ch. 4-4

5 Continuous Random Variable Can take on any value in an interval (uncountable) Possible values are measured on a continuum Examples: Weight of packages filled by a mechanical filling process Temperature of a cleaning solution Time between failures of an electrical component Copyright 013 Pearson Education Ch. 4-5

6 4. Probability Distributions for Discrete Random Variables Let X be a discrete random variable and x be one of its possible values The probability that random variable X takes specific value x is denoted P(X = x) The probability distribution function of a random variable is a representation of the probabilities for all the possible outcomes. Can be shown algebraically, graphically, or with a table Copyright 013 Pearson Education Ch. 4-6

7 Probability Distributions for Discrete Random Variables Experiment: Toss Coins. Let X = # heads. Show P(x), i.e., P(X = x), for all values of x: 4 possible outcomes T T T H H T H H Probability Distribution x Value Probability 0 1/4 =.5 1 /4 =.50 1/4 =.5 Probability x Copyright 013 Pearson Education Ch. 4-7

8 Probability Distribution Required Properties 0 P(x) 1 for any value of x The individual probabilities sum to 1; x P(x) 1 (The notation indicates summation over all possible x values) Copyright 013 Pearson Education Ch. 4-8

9 Cumulative Probability Function The cumulative probability function, denoted F(x 0 ), shows the probability that X does not exceed the value x 0 F(x ) P(X x 0 0 ) Where the function is evaluated at all values of x 0 Copyright 013 Pearson Education Ch. 4-9

10 Cumulative Probability Function Example: Toss Coins. Let X = # heads. (continued) x Value P(x) F(x) Copyright 013 Pearson Education Ch. 4-10

11 Derived Relationship The derived relationship between the probability distribution and the cumulative probability distribution Let X be a random variable with probability distribution P(x) and cumulative probability distribution F(x 0 ). Then F(x ) P(x) 0 x x 0 (the notation implies that summation is over all possible values of x that are less than or equal to x 0 ) Copyright 010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 4-11

12 Derived Properties F(x 0 ) is a non-decreasing and right-continuous step function Copyright 010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 4-1

13 Derived Properties F(x 0 ) is a non-decreasing and right-continuous function Let X be a discrete random variable with cumulative probability distribution F(x 0 ). Then 1. 0 F(x 0 ) 1 for every number x 0. for x 0 < x 1, then F(x 0 ) F(x 1 ) Copyright 010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 4-13

14 4.3 Properties of Discrete Random Variables Expected Value (or mean) of a discrete random variable X is a weighted mean with P(x): E[X] μ xp(x) x Example: Toss coins, x=# of heads, compute expected value of x: x P(x) E(x) = (0 x.5) + (1 x.50) + ( x.5) = 1.0 Copyright 013 Pearson Education Ch. 4-14

15 Buffon s Needle Problem E[X]=n p Copyright 013 Pearson Education Ch. 4-15

16 Buffon s Needle Problem E[X]=n p Copyright 013 Pearson Education Ch. 4-16

17 Variance and Standard Deviation Variance of a discrete random variable X σ E[(X μ) Can also be expressed as ] (x μ) x P(x) σ E[X ] μ x x P(x) μ Standard Deviation of a discrete random variable X σ σ (x μ) x P(x) Copyright 013 Pearson Education Ch. 4-17

18 Standard Deviation Example Example: Toss coins, X = # heads, compute standard deviation (recall E[X] = 1) σ (x μ) P(x) x σ (0 1) (.5) (11) (.50) ( 1) (.5) Possible number of heads = 0, 1, or Copyright 013 Pearson Education Ch. 4-18

19 Functions of Random Variables If P(x) is the probability function of a discrete random variable X, and z(x) is some function of X, then the expected value of function g is E[z(X)] x z(x)p(x) Sometimes we don t care about X, but the reward for each realized x, i.e., z(x). Copyright 013 Pearson Education Ch. 4-19

20 Functions of Random Variables If P(x) is the probability function of a discrete random variable X, and z(x) is some function of X, then the expected value of function g is E[z(X)] x z(x)p(x) Notice that E[z(x)] is a weighted mean of z(x), x, where the weights are the probabilities P(x). Copyright 013 Pearson Education Ch. 4-0

21 Linear Functions of Random Variables Let random variable X have mean µ x and variance σ x Let a and b be any constants. Let Y = a + bx Then the mean and variance of Y are μ E(a bx) a Y bμ X (continued) σ Y Var(a bx) b σ X so that the standard deviation of Y is σ b Y σ X Copyright 013 Pearson Education Ch. 4-1

22 Properties of Linear Functions of Random Variables Let a and b be any constants. a) E(a) a and Var(a) 0 i.e., if a random variable always takes the value a, it will have mean a and variance 0 b) E(bX) bμ X and Var(bX) b σ X i.e., the expected value of b X is b E(x) Copyright 013 Pearson Education Ch. 4-

23 Moments Recall that E[z(X)] x z(x)p(x) When we consider we call E[X r ] the r-th moment: (r 級原動差 ) r z(x) X r E [X r ] x f ( x) z(x) (X - When we consider we call E[(X-b) r ] the r-th moment about b: (r 級概約動差 ) r xs Copyright 013 Pearson Education Ch. 4-3 b) r E [(X - b) ] ( x b) f ( x) xs r

24 Moments Recall that E[z(X)] x z(x)p(x) When we consider z(x) Xr X (X -1) (X - )... (X - r 1) we call E[X r ] the r-th factorial moment: (r 級階動差 ) Notice that E[X] E[ X ( X 1)] E[ X ] E[ X ] Copyright 013 Pearson Education Ch. 4-4

25 Moments So, how do we use moments? Now we can calculate the variance by: E[X ]- E[X (X -1)] E[X]- ( E[X]) In some cases, it is easier to have E[X(X-1)] than E[X ]. Copyright 013 Pearson Education Ch. 4-5

26 Moment-Generating Function Recall that E[z(X)] x z(x)p(x) z(x) e When we consider we call M(t)=E[e tx ] the moment generating function. ( 動差母函數 ) tx Copyright 013 Pearson Education Ch. 4-6

27 Moment-Generating Function So, how do we use moment-generating functions? Notice M (0)=E(X) and M (0)=E(X ), so we have E[X M"(0) ]- Ok, there are more applications of the momentgenerating functions in Econometrics, ex. Generalized Method of Moment (GMM). [ M '(0)] Copyright 013 Pearson Education Ch. 4-7

28 Probability Distributions Probability Distributions Ch. 4 Discrete Continuous Ch. 5 Probability Probability Distributions Distributions Binomial Poisson Hypergeometric Uniform Normal Exponential Copyright 013 Pearson Education Ch. 4-8

29 4.4 The Binomial Distribution Probability Distributions Discrete Probability Distributions Binomial Hypergeometric Poisson Copyright 013 Pearson Education Ch. 4-9

30 Bernoulli Distribution Consider only two outcomes: success or failure Let P denote the probability of success Let 1 P be the probability of failure Define random variable X: x = 1 if success, x = 0 if failure Then the Bernoulli probability distribution is P(0) (1P) and P(1) P Copyright 013 Pearson Education Ch. 4-30

31 Mean and Variance of a Bernoulli Random Variable The mean is µ x = P μ x E[X] X xp(x) (0)(1P) (1)P P The variance is σ x = P(1 P) σ x E[(X μ x ) ] X (x μ x ) P(x) (0 P) (1P) (1P) P P(1P) Copyright 013 Pearson Education Ch. 4-31

32 Developing the Binomial Distribution The number of sequences with x successes in n independent trials is: C n x n! x!(n x)! Where n! = n (n 1) (n )... 1 and 0! = 1 These sequences are mutually exclusive, since no two can occur at the same time Copyright 013 Pearson Education Ch. 4-3

33 Binomial Probability Distribution A fixed number of observations, n e.g., 15 tosses of a coin; ten light bulbs taken from a warehouse Two mutually exclusive and collectively exhaustive categories e.g., head or tail in each toss of a coin; defective or not defective light bulb Generally called success and failure Probability of success is P, probability of failure is 1 P Constant probability for each observation e.g., Probability of getting a tail is the same each time we toss the coin Observations are independent The outcome of one observation does not affect the outcome of the other Copyright 013 Pearson Education Ch. 4-33

34 Possible Binomial Distribution Settings A manufacturing plant labels items as either defective or acceptable A firm bidding for contracts will either get a contract or not A marketing research firm receives survey responses of yes I will buy or no I will not New job applicants either accept the offer or reject it Copyright 013 Pearson Education Ch. 4-34

35 The Binomial Distribution P(x) n! x! ( n x )! X n X P (1- P) P(x) = probability of x successes in n trials, with probability of success P on each trial x = number of successes in sample, (x = 0, 1,,..., n) n = sample size (number of independent trials or observations) P = probability of success Example: Flip a coin four times, let x = # heads: n = 4 P = P = (1-0.5) = 0.5 x = 0, 1,, 3, 4 Copyright 013 Pearson Education Ch. 4-35

36 Example: Calculating a Binomial Probability What is the probability of one success in five observations if the probability of success is 0.1? x = 1, n = 5, and P = 0.1 P(x 1) n! x!(n P x)! X 5! (0.1) 1!(5 1)! (1P) 1 nx (1 0.1) 51 (5)(0.1)(0.9) Copyright 013 Pearson Education Ch. 4-36

37 Shape of Binomial Distribution The shape of the binomial distribution depends on the values of P and n Mean Here, n = 5 and P = P(x) n = 5 P = x Here, n = 5 and P = 0.5 P(x) n = 5 P = x Copyright 013 Pearson Education Ch. 4-37

38 Mean and Variance of a Binomial Distribution Mean μ E(x) np Variance and Standard Deviation σ np(1-p) σ np(1- P) Where n = sample size P = probability of success (1 P) = probability of failure Copyright 013 Pearson Education Ch. 4-38

39 Binomial Characteristics Examples σ Mean μ np (5)(0.1) 0.5 np(1- P) (5)(0.1)(1 0.1) P(x) n = 5 P = x σ μ np (5)(0.5).5 np(1- P) (5)(0.5)(1 0.5) P(x) n = 5 P = x Copyright 013 Pearson Education Ch. 4-39

40 Using Binomial Tables N x p=.0 p=.5 p=.30 p=.35 p=.40 p=.45 p= Examples: n = 10, x = 3, P = 0.35: P(x = 3 n =10, p = 0.35) =.5 n = 10, x = 8, P = 0.45: P(x = 8 n =10, p = 0.45) =.09 Copyright 013 Pearson Education Ch. 4-40

41 4.5 The Poisson Distribution Probability Distributions Discrete Probability Distributions Binomial Poisson Hypergeometric Copyright 013 Pearson Education Ch. 4-41

42 The Poisson Distribution The Poisson distribution is used to determine the probability of a random variable which characterizes the number of occurrences or successes of a certain event in a given continuous interval (such as time, surface area, or length). Copyright 013 Pearson Education Ch. 4-4

43 The Poisson Distribution (continued) Assume an interval is divided into a very large number of equal subintervals where the probability of the occurrence of an event in any subinterval is very small. Poisson distribution assumptions 1. The probability of the occurrence of an event is constant for all subintervals.. There can be no more than one occurrence in each subinterval. 3. Occurrences are independent; that is, an occurrence in one interval does not influence the probability of an occurrence in another interval. Copyright 013 Pearson Education Ch. 4-43

44 Poisson Distribution Function The expected number of events per unit is the parameter (lambda), which is a constant that specifies the average number of occurrences (successes) for a particular time and/or space P(x) e where: P(x) = the probability of x successes over a given time or space, given = the expected number of successes per time or space unit, > 0 e = base of the natural logarithm system ( ) λ x! λ x Copyright 013 Pearson Education Ch. 4-44

45 Poisson Distribution Characteristics Mean and variance of the Poisson distribution Mean μ x E[X] λ Variance and Standard Deviation σ x E[(X μ x ) ] λ σ λ where = expected number of successes per time or space unit Copyright 013 Pearson Education Ch. 4-45

46 Using Poisson Tables X Example: Find P(X = ) if =.50 P(X ) e X! X e 0.50 (0.50)!.0758 Copyright 013 Pearson Education Ch. 4-46

47 Graph of Poisson Probabilities Graphically: = X 0 1 = P(x) x P(X = ) =.0758 Copyright 013 Pearson Education Ch. 4-47

48 Poisson Distribution Shape The shape of the Poisson Distribution depends on the parameter : 0.70 = 0.50 = P(x) P(x) x x Copyright 013 Pearson Education Ch. 4-48

49 Poisson Approximation to the Binomial Distribution Let X be the number of successes from n independent trials, each with probability of success P. The distribution of the number of successes, X, is binomial, with mean np. If the number of trials, n, is large and np is of only moderate size (preferably np 7 ), this distribution can be approximated by the Poisson distribution with = np. The probability distribution of the approximating distribution is P(x) e np (np) x! x for x 0,1,,... Copyright 013 Pearson Education Ch. 4-49

50 4.6 The Hypergeometric Distribution Probability Distributions Discrete Probability Distributions Binomial Poisson Hypergeometric Copyright 013 Pearson Education Ch. 4-50

51 The Hypergeometric Distribution n trials in a sample taken from a finite population of size N Sample taken without replacement Outcomes of trials are dependent Concerned with finding the probability of X successes in the sample where there are S successes in the population Copyright 013 Pearson Education Ch. 4-51

52 Hypergeometric Probability Distribution P(x) S CxC C NS nx N n S! (NS)! x!(sx)! (nx)!(nsn x)! N! n!(nn)! Where N = population size S = number of successes in the population N S = number of failures in the population n = sample size x = number of successes in the sample n x = number of failures in the sample Copyright 013 Pearson Education Ch. 4-5

53 Using the Hypergeometric Distribution Example: 3 different computers are checked from 10 in the department. 4 of the 10 computers have illegal software loaded. What is the probability that of the 3 selected computers have illegal software loaded? N = 10 n = 3 S = 4 x = S NS 4 CxCn x CC P(x ) N 10 C C n (6)(6) The probability that of the 3 selected computers have illegal software loaded is 0.30, or 30%. Copyright 013 Pearson Education Ch. 4-53

54 4.7 Jointly Distributed Discrete Random Variables A joint probability distribution is used to express the probability that simultaneously X takes the specific value x and Y takes the value y, as a function of x and y P(x, y) P(X x Y y) The marginal probability distributions are P(x) P(x,y) y P(y) x P(x,y) Copyright 013 Pearson Education Ch. 4-54

55 Properties of Joint Probability Distributions Properties of Joint Probability Distributions of Discrete Random Variables Let X and Y be discrete random variables with joint probability distribution P(x, y) 1. 0 P(x, y) 1 for any pair of values x and y. the sum of the joint probabilities P(x, y) over all possible pairs of values must be 1 Copyright 013 Pearson Education Ch. 4-55

56 Conditional Probability Distribution The conditional probability distribution of the random variable Y expresses the probability that Y takes the value y when the value x is specified for X. P(y x) P(x,y) P(x) Similarly, the conditional probability function of X, given Y = y is: P(x y) P(x,y) P(y) Copyright 013 Pearson Education Ch. 4-56

57 Independence The jointly distributed random variables X and Y are said to be independent if and only if their joint probability distribution is the product of their marginal probability functions: P(x, y) P(x)P(y) for all possible pairs of values x and y A set of k random variables are independent if and only if P(x1,x,,xk ) P(x1)P(x ) P(x k ) Copyright 013 Pearson Education Ch. 4-57

58 Conditional Mean and Variance The conditional mean is μ Y X E[Y X] Y (y x)p(y x) The conditional variance is σ Y X E[(Y μy X ) X] [(y μy X ) x]p(y x) Y Copyright 013 Pearson Education Ch. 4-58

59 Covariance Let X and Y be discrete random variables with means μ X and μ Y The expected value of (X - μ X )(Y - μ Y ) is called the covariance between X and Y For discrete random variables Cov(X, Y) E[(X An equivalent expression is Cov(X, Y) E(XY) μ μ μx )(Y μy )] (x μx )(y μy )P(x, y) x y x y xyp(x, y) μ x y x μ y Copyright 013 Pearson Education Ch. 4-59

60 Correlation The correlation between X and Y is: ρ Corr(X, Y) Cov(X, Y) σ σ X Y -1 ρ 1 ρ = 0 no linear relationship between X and Y ρ > 0 positive linear relationship between X and Y when X is high (low) then Y is likely to be high (low) ρ = +1 perfect positive linear dependency ρ < 0 negative linear relationship between X and Y when X is high (low) then Y is likely to be low (high) ρ = -1 perfect negative linear dependency Copyright 013 Pearson Education Ch. 4-60

61 Covariance and Independence The covariance measures the strength of the linear relationship between two variables If two random variables are statistically independent, the covariance between them is 0 The converse is not necessarily true Copyright 013 Pearson Education Ch. 4-61

62 Portfolio Analysis Let random variable X be the price for stock A Let random variable Y be the price for stock B The market value, W, for the portfolio is given by the linear function W ax by (a is the number of shares of stock A, b is the number of shares of stock B) Copyright 013 Pearson Education Ch. 4-6

63 Portfolio Analysis (continued) The mean value for W is μ W aμ The variance for W is E[W] E[aX X bμ Y by] σ W a σ b σ or using the correlation formula X Y abcov(x, Y) σ W a σ X b σ Y abcorr(x,y)σ X σ Y Copyright 013 Pearson Education Ch. 4-63

64 Example: Investment Returns Return per $1,000 for two types of investments Investment P(x i y i ) Economic condition Passive Fund X Aggressive Fund Y. Recession -$ 5 - $00.5 Stable Economy Expanding Economy E(x) = μ x = (-5)(.) +(50)(.5) + (100)(.3) = 50 E(y) = μ y = (-00)(.) +(60)(.5) + (350)(.3) = 95 Copyright 013 Pearson Education Ch. 4-64

65 Computing the Standard Deviation for Investment Returns Investment P(x i y i ) Economic condition Passive Fund X Aggressive Fund Y 0. Recession -$ 5 - $ Stable Economy Expanding Economy σ σ X y (-5 50) (-00 95) (0.) (50 50) (0.) (60 95) (0.5) (100 50) (0.5) (350 95) (0.3) (0.3) Copyright 013 Pearson Education Ch. 4-65

66 Covariance for Investment Returns Investment P(x i y i ) Economic condition Passive Fund X Aggressive Fund Y. Recession -$ 5 - $00.5 Stable Economy Expanding Economy Cov(X,Y) (-5 50)(-00 95)(.) (50 50)(60 95)(.5) (100 50)(350 95)(.3) 850 Copyright 013 Pearson Education Ch. 4-66

67 Portfolio Example Investment X: μ x = 50 σ x = Investment Y: μ y = 95 σ y = σ xy = 850 Suppose 40% of the portfolio (P) is in Investment X and 60% is in Investment Y: E(P).4(50) (.6)(95) 77 σ P (.4) (43.30) (.6) (193.1) (.4)(.6)(850) The portfolio return and portfolio variability are between the values for investments X and Y considered individually Copyright 013 Pearson Education Ch. 4-67

68 Interpreting the Results for Investment Returns The aggressive fund has a higher expected return, but much more risk μ y = 95 > μ x = 50 but σ y = > σ x = The Covariance of 850 indicates that the two investments are positively related and will vary in the same direction Copyright 013 Pearson Education Ch. 4-68

69 Chapter Summary Defined discrete random variables and probability distributions Discussed the Binomial distribution Reviewed the Poisson distribution Discussed the Hypergeometric distribution Defined covariance and the correlation between two random variables Examined application to portfolio investment Copyright 013 Pearson Education Ch. 4-69