Discrete Random Variables

Transcription

1 Discrete Random Variables A dichotomous random variable takes only the values 0 and 1. Let X be such a random variable, with Pr(X=1) = p and Pr(X=0) = 1-p. Then E[X] = p, and Var[X] = p(1-p). Consider a sequence of n independent experiments, each of which has probability p of being a success. Let X k = 1 if the k-th experiment is a success, and 0 otherwise. Then the total number of successes in n trials is X = X X n ; X is a binomial random variable, and Pr(X=k) = n k k n-k p (1 - p ). E[X] = np, and Var[X] = np(1-p). (These results follow from the properties of the expected value and variance of sums of independent random variables.) Next, consider a sequence of independent experiments, and let Y be the number of trials up to (and including) the first success. Y is a geometric random variable, and k-1 Pr(Y = k) = (1- p) p. E[Y] = 1/p, and Var[Y] = (1-p)/p 2. (These results follow from the evaluation of infinite sums.) A hypergeometric random variable Z results from drawing a sample of size n from a population of size N containing g good members, and then counting the number of good members in the sample: Pr(Z = z) = g N g z n z. N n (This formula was used to compute the relevant probabilities in the Bag R vs. Bag B example.)

2 Continuous Random Variables A continuous random variable is a random variable which can take any value in some interval. A continuous random variable is characterized by its probability density function, a graph which has a total area of 1 beneath it: The probability of the random variable taking values in any interval is simply the area under the curve over that interval. The normal distribution: This most-familiar of continuous probability distributions has the classic bell shape (see the left-hand graph below). The peak occurs at the mean of the distribution, i.e., at the expected value of the normally-distributed random variable with this distribution, and the standard deviation (the square root of the variance) indicates the spread of the bell, with roughly 68% of the area within 1 standard deviation of the peak. The normal distribution arises so frequently in applications due to an amazing fact: If you take a bunch of independent random variables (with comparable variances) and average them, the result will be roughly normally distributed, no matter what the distributions of the separate variables might be. (This is known as the Central Limit Theorem.) Many interesting quantities (ranging from IQ scores, to demand for a retail product, to lengths of shoelaces) are actually a composite of many separate random variables, and hence are roughly normally distributed. If X is normal, and Y = ax+b, then Y is also normal, with E[Y] = a E[X] + b and StdDev[Y] = a StdDev[X]. If X and Y are normal (independent or not), then X+Y and X-Y = X+(-Y) are also normal (intuition: the sum of two bunches is a bunch). Any normallydistributed random variable can be transformed into a standard normal random variable (with mean 0 and standard deviation 1) by subtracting off its mean and dividing by its standard deviation. Hence, a single tabulation of the cumulative distribution for a standard normal random variable (attached) can be used to do probabilistic calculations for any normally-distributed random variable. The exponential distribution: Consider the time between successive incoming calls at a switchboard, or between successive patrons entering a store. These interarrival times are typically exponentially distributed. If the mean interarrival time is 1/λ (so λ is the mean arrival rate per unit time), then the variance will be 1/λ 2 (and the standard deviation will be 1/λ ). The right-hand graph above displays the graph of the exponential density function when λ = 1. Generally, if X is exponentially distributed, then Pr(s < s < X t) = e -λs - e -λt (where e ).

3 The exponential distribution fits the examples cited above because it is the only distribution with the lack-of-memory property: If X is exponentially distributed, then Pr(X s+t X > s) = Pr(X t). (After waiting a minute without a call, the probability of a call arriving in the next two minutes is the same as was the probability (a minute ago) of getting a call in the following two minutes. As you continue to wait, the chance of something happening soon neither increases nor decreases.) Note that, among discrete distributions, the geometric distribution is the only one with the lack-of-memory property; indeed, the exponential and geometric distributions are analogues of one another. Let the time between successive arrivals into some system be exponentially distributed, and let N be the number of arrivals in a fixed interval of time of length t. Then N (a discrete random variable) has the Poisson distribution, and Pr(N=k) = e -λ t ( λ t) k! E[N] = λt, and Var[N] = λt as well. The exponential and Poisson distributions arise frequently in the study of queuing, and of process quality. An interesting (and sometimes useful) fact is that the minimum of two independent, identically-distributed exponential random variables is a new random variable, also exponentially distributed and with a mean precisely half as large as the original mean(s). Connections: The geometric distribution deals with the time between successes in a series of independent trials, and the binomial distribution deals with the number of successes in a fixed number of trials. The exponential distribution deals with the time between occurrences of successive events, and the Poisson distribution deals with the number of occurrences in a fixed period of time. Obviously, there's a relationship here. Rule of thumb: If n > 20 and p < 0.05, then a binomial random variable with parameters (n, p) has a probability distribution very similar to that of a Poisson random variable with parameters λ = np and t = 1. (Think of dividing one interval of time into n subintervals, and having a probability p of an arrival in each subinterval. That's very much like having a rate of np arrivals (on average) per unit time.) The uniform distribution: A random variable U is uniformly distributed on an interval [a,b] if its density function is flat over that interval. (The uniform distribution is what one typically has in mind when one thinks of picking a number at random over some range.) The expected value of U is (a+b)/2, and Var[U] = (b-a) 2 /12. If n independent, identically distributed random variables are all uniformly distributed on [0,1] (i.e., if we pick n numbers at random between 0 and 1), then the expected values of the largest, second-largest,..., smallest are n/(n+1), (n- 1)/(n+1),..., 1/(n+1) (i.e., the expected values divide the interval into n+1 equally-large subintervals). Given that n individuals enter a system (with exponential interarrival times) in a fixed interval of time, the n actual arrival times will look as if they were drawn uniformly over that time interval. The beta distribution: A random variable X has the beta distribution (with parameters α > 0 and β > 0) on the interval [0, 1] if its density has the form k x α 1 (1-x) β 1 (where k is a scale factor which makes the area under the curve 1). E[X] = α/(α+β), and Var[X] = αβ/[(α+β) 2 (α+β+1)]. k.

4 The main use of the beta distribution is to fit it to observed data when building a model of a real-world phenomenon: The beta distribution can take a wide variety of shapes, as seen in the graphs below. The lefthand graph arises when α = 5 and β = 3, the center when α = 1.5 and β = 3, and the right when α = 0.5 and β = 0.5. Generally, if α > 2, the density has a slope of 0 at x = 0, if 2 > α > 1, the density is near-vertical near x = 0, and if 1 > α > 0, the density rises as x approaches 0. (Analogous properties involving β hold when x is near 1.) When α = β = 1, the beta distribution is uniform on [0,1].

5 Right-Tail Probabilities of the Normal Distribution

6 Simulating Random Variables Sometimes one wishes to simulate a random process (so as to compare alternative policies on a computer before trying them out in real life). Simulations require the generation of random variables fitting particular distributions. Most computer languages and spreadsheets can provide (pseudo-) random numbers, uniformly distributed between 0 and 1. Fortunately, random variables with any probability distribution can be generated from these uniform random variables: Consider any random variable X. Simply take a uniformly-distributed observation u, solve the equation Pr(X x) = u for x, and let the resulting x be your observation of X. For example, in Excel the function RAND() returns a random value uniformly distributed between 0 and 1. In addition, the function NORMINV(.,.,.) returns the inverse of the cumulative normal probability distribution. Therefore, to generate a random observation from a normal distribution with a particular mean and standard deviation, one can simply enter the formula =NORMINV(RAND(),mean,stddev) in a cell of the spreadsheet.