Introduction to Random Variables (RVs)

Transcription

1 ECE270: Handout 5 Introduction to Random Variables (RVs) Outline: 1. informal definition of a RV, 2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV, 3. a general rule to find probability of events concerning a RV, 4. cumulative distribution function (CDF) of a RV, 5. formal definition of a RV using CDF, 6. discrete RV: probability mass function (pmf) and CDF, 7. continuous RV: probability density function (pdf) and CDF, 8. basic properties of the CDF. The outcome of a random experiment need not be a numbers. Examples are: coding the incoming patients in a hospital according to their insurance and health status, randomly selecting a committee from a group of people, randomly selecting balls from an urn, randomly selecting cards from a deck. Usually we are interested in some measurement or numerical attribute of the outcome. Examples are: counting the number of heads when tossing a coin 10 times, the number of re-transmission needed until the receiver receives the data packet correctly, the number of errors in erroneous received data packets, the lifetime of a memory chip, the number of packets arriving in t sec at a server, the number of queries arriving in t sec at a call center, the number of particles emitted by a radioactive mass during a fixed time period, the random thermal noise being added to the signal at the receiver of a communication system at a specific time. In these examples, we assign a real number to the outcome of the random experiment through measurement. Since the outcomes are random, the results of the measurements will be random too. Hence it makes sense to talk about the probabilities of the resulting numerical values. Informal Definition of a RV A RV X is a function that assigns a real valued number x = X(ξ) to each outcome ξ S (Recall: a function is a rule for assigning a numerical value to each element of a set). Sample space S is the domain of the RV X and the set of all real numbers taken on by X, S x, S x R, is the range of the RV. Three Types of a RV Three types of RVs: i) discrete, ii) continuous, iii) mixed.

2 i) The range of a discrete RV X is a countable set (either finite or infinite) S x = {x 1, x 2,...} or S x = {x 1, x 2,..., x n }. ii) The range of a continuous RV X is an uncountable set, e.g., S x = [0, ), S x = (, ), S x = [0, 1], S x = [a, b] for < a < b <. iii) The range of a mixed RV X is the union of an uncountable and a countable sets. Notation: capital letters X, Y, Z, V, U,... denote RVs, lowercase letters x, y, z, v, u,... denote possible values of RVs. Consider the RV X, the function or rule that assigns a real number x = X(ξ) to the outcome ξ S is fixed and deterministic, e.g., the rule of count the number of heads when we toss a coin 3 times. The randomness in the experiment is complete as soon as we toss the coin 3 times. The process of counting is deterministic. In some random experiments the outcome ξ is already the numerical value we are interested in, e.g., measure the lifetime of a chip under certain conditions, X(ξ) = ξ (identity function). The distribution of the values of a RV X is determined by the probabilities of the basic events of the underlying random experiment, i.e., we should be able to compute the probability of the observed value of X in terms of the probability of the underlying event. The specification of the measurements on the outcomes of a random experiment defines a function on S and hence a RV. A function of a RV is another RV. EXAMPLE 1 A fair coin is tossed 3 times and the sequence of heads and tails is noted. The sample space is S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T } and the outcomes are equally probable. a) Let X be the number of heads in the 3 tosses. S x = {0, 1, 2, 3}. b) A player pays 1.5$ to play the following game: the player receives 1$ if X = 2 and 8$ if X = 3, but nothing otherwise. Let Y be the reward to the player. S y = {0, 1, 8}. c) Let Z be a function of X such that Z = 0 if X {0, 1, 2} and Z = 1 if X = 3. S z = {0, 1}. ξ HHH HHT HTH THH HTT THT TTH TTT X(ξ) Y (ξ) Z(ξ) P (X = 0) = P ({T T T }) = 1/8 P (X = 1) = P ({T T H, T HT, HT T }) = 3/8 P (X = 2) = P ({T HH, HHT, HT H}) = 3/8 Page 2

3 P (X = 3) = P ({HHH}) = 1/8 P (Y = 0) = P ({T T T, T T H, T HT, HT T }) = 4/8 = 1/2 P (Y = 1) = P ({T HH, HT H, HHT }) = 3/8 P (Y = 8) = P ({HHH}) = 1/8 { P (Z = 0) = P (X {0, 1, 2}) = P (X = 0) + P (X = 1) + P (X = 2) = 1/8 + 3/8 + 3/8 = 7/8 P (Z = 1) = P (X = 3) = 1/8 or alternatively: { P (Z = 0) = P ({T T T, T T H, T HT, HT T, HHT, HT H, T HH}) = 7/8 P (Z = 1) = P ({HHH}) = 1/8 A General Rule to Find Probabilities of Events Concerning a RV X The example shows a general technique to find the probability of events involving RVs. To find the probability of X B (where B R) we need to find the set of outcomes A, A S that are mapped to B, i.e., the set A = {ξ : X(ξ) B}. If the experiment outcome ξ A then event A occurs. Hence X(ξ) B event B occurs. If event B occurs then X(ξ) B implies ξ A event A occurs. So: P (X B) = P (A) = P ({ξ : X(ξ) B}). We refer to A and B as equivalent events. In some random experiments the outcome ξ is already the numerical value we are interested in. In such cases we simply let X(ξ) = ξ, i.e., the identity function is used to obtain a random variable. Example: measure the received signal at a receive antenna. CDF of a RV X Regardless of X being discrete, continuous, or mixed, the cumulative distribution function (CDF) of a RV X is defined as the probability of the event B = {X x}: F X (x) = P (B) = P ({X x}) P (X x) for < x < The event B = {X x} and its probability vary as x is varied. Hence, F X (x) is a function of the variable x. In terms of the underlying random experiment F X (x) = P ({ξ : X(ξ) x} ). }{{} an event Formal Definition of a RV Using CDF We are ready for a formal definition of a RV: Consider a random experiment with sample space S. The RV X is a function from S to real line R with the property that the set A = {ξ : X(ξ) b} is an event for every b R. This definition simply requires that every set A has a well defined probability in the underlying random experiment. Page 3

4 Discrete RV: pmf and CDF A discrete RV X has the CDF F X (x) that is right continuous, non-decreasing staircase function of x, with jumps at points x 1, x 2,... where x i S x, and it grows from 0 to 1: F X (x) = P (X = x i ) u(x x }{{} i ) i=1 pmf p X (x) = P (X = x) = P ({ξ : X(ξ) = x}), for x a real number, is called the probability mass function (pmf) of X, and is the magnitude of the jump in CDF at the point x. p X (x) is a function of x over the real line and is nonzero only at the points x 1, x 2,... S x. pmf p X (x) for all x S x provides all information required to calculate probabilities of any event involving a discrete RV X. We can obtain CDF from pmf and vice versa. We can forget about the underlying random experiment, its sample space S and its associated probability law and just work with S x and the pmf of X, or equivalently the CDF of X. The pmf of X is the probability of all elementary events from S x. The pmf p X (x) satisfies three properties: 1) 0 p X (x) 1, 2) x S x p X (x) = 1, 3) P (X B) = x B p X(x) where B R. EXAMPLE 2 (CDF of a discrete RV, back to example 1) Find and plot F X (x) in example 1. Continuous RV: pdf and CDF A continuous RV X has the CDF F X (x) that is continuous everywhere and is a non-decreasing function of x that grows from 0 to 1 as x ranges from its minimum to its maximum values. Also, F X (x) is smooth enough that it can be written as an integral of some nonnegative function f X (x): F X (x) = x f X (x) dx }{{} pdf Considering F X (x) we calculate probabilities as integral of probability density function (pdf) over an interval of the real line (, x]. We can obtain CDF from pdf and vice versa. For a continuous RV X we have P (X = x) = 0. EXAMPLE 3 (CDF of a continuous RV) We spin an arrow attached to the center of a circular board. Let θ be the angle of the arrow where 0 < θ 2π. The probability that θ falls in a subinterval of (0, 2π] is equal to the length of the subinterval divided by 2π. We define the RV X(θ) = θ/2π. What is F X (x)? Page 4

5 EXAMPLE 4 (CDF of a mixed RV) The waiting time X of a customer at a taxi stand is zero if the customer finds a taxi parked at the stand, and a uniformly distributed random length of time in the interval [0, 1] (in hour) if no taxi is found upon arrival. The probability that a taxi is at the stand when the customer arrives is p. What is the CDF of X? SOLUTION: using the total probability theorem we have F X (x) = P (X x) = P (X x find taxi)p + P (X x no taxi)(1 p) P (X x find taxi) = { 1 x 0 0 x < 0 1 x > 1 P (X x no taxi) = x 0 x 1 0 x < 0 1 p + 1 (1 p) = 1 x > 1 F X (x) = P (X x) = 1 p + x(1 p) = p + x(1 p) 0 x 1 0 x < 0 Basic Properties of the CDF The three axioms of probability and the corresponding properties imply that the CDF has the following properties: (i) 0 F X (x) 1 (ii) lim x F X (x) = 1 (iii) lim x F X (x) = 0 (iv) F X (x) is a nondecreasing function of x, that is, if a < b then F X (a) F X (b) (v) F X (x) is continuous from the right, i.e., for h > 0, F X (b) = lim h 0 F X (b + h) = F X (b + ). This property implies that at points of discontinuity, the CDF is equal to the limit from the right. These five properties confirm that, in general, CDF is a nondecreasing function of x that grows from 0 to 1 as x increases from to. More properties follow: (vi) P (a < X b) = F X (b) F X (a) (vii) P (X = b) = F X (b) F X (b ) = F X (b + ) F X (b ). This property says that the probability that X = b is equal to the magnitude of the jump of the CDF at the point b. So if the CDF is continuous at a point b, then P (X = b) = 0. (viii) P (X > x) = 1 F X (x) Page 5

6 Based on above properties we can compute P (a X b), P (a X < b), P (a < X < b). Since {a X b} = {a} {a < X b}, we have: P (a X b) = P (X = a) + P (a < X b) = F X (a) F X (a ) + F X (b) F X (a) = F X (b) F X (a ) Since {a X b} = {b} {a X < b}, we have: P (a X < b) = P (a X b) P (X = b) = F X (b ) F X (a ) Since {a X < b} = {a} {a < X < b}, we have: P (a < X < b) = P (a X < b) P (X = a) = F X (b ) F X (a) NOTE: if the CDF is continuous at the endpoints of an interval, then the endpoints have zero probabilities, and therefore, they can be included in, or excluded from, the interval without affecting the probabilities. EXAMPLE 5 (back to example 1) Consider example 1 and let A = {1 < X 2}, B = {0.5 X < 2.5}, C = {1 X < 2}. Find P (A), P (B), P (C). EXAMPLE 6 (back to example 3) Find P ( 0.5 < X < 0.25), P (0.3 < X < 0.65), P ( X 0.4 > 0.2). Page 6