Random variables. TMS-062: Lecture 3 Random Variables. Discrete random variables

Transcription

1 Random variables TMS-062: Lecture 3 Random Variables A Random variable X is a variable whose value is the outcome of a random experiment, so it is just a function X(ω) on the sample space Ω. We will consider 3 types of R.V. s (mixtures are possible) Categorical/Qualitative R.V. s : non-numerical, e. g. Colour, Sex, Blood type, etc. Example: 60% of people have blood of group O, 20% A, 15% B and 5% AB. An individual is chosen at random and its blood group X is noted. X is categorical R.V. Its Distribution is just the following table (X, P(X)) listing probabilities of each possible outcome: X O A B AB (total) P(X) Discrete random variables The distribution of X is a similar table (X, P(X)). Events of interest include X = x i or x m < X x n. We then have X is discrete R.V. if it takes values on a discrete (finite or countable) set, i. e. its values can (in principal) be listed: {x 1,..., x n,... }. If we write p i = P(X = x i ) = P{ω : X(ω) = x i }, i = 1, 2,... then we should have i p i = 1 as in any outcome the value of X lies in Ω. P(X = x i ) = x i } = p i n P(x m < X x n ) = i=m+1 In general, for any set B R, P(X B) = sum of all p i s such that x i B. p i.

2 Distribution Function A general way to describe the distribution of a numerical R.V. s is via the Cumulative Distribution Function (c.d.f) which is F X (t) = P(X t). Properties: 0 F X (t) 1; F X (t) is non-decreasing and F X ( ) = 0, F X (+ ) = 1; F X (t) is continuous from the right: F X (t+) = F X (t); P(a < X b) = F X (b) F X (a) P(a X b) = F X (b) F X (a ) P(a < X < b) = F X (b ) F X (a) P(a X < b) = F X (b ) F X (a ). For discrete X, c.d.f. is a step function with jumps p i at x i : 1 0 p 1 p 2 x 1 x 2 x 3 F X (t) t Indeed, if t < x 1, then P(X t) = 0, as x 1 is the smallest possible value. If x 1 t < x 2 then P(X t) = P(X = x 1 ) = p 1 for all such t. When t = x 2 we have P(X x 2 ) = P(X = x 1 ) + P(X = x 2 ) = p 1 + p 2, so F X (t) has a jump of size p 2 there, etc. F X (t) = P(X t) = x i t p i and P(X = x k ) = F X (x k ) F X (x k ) Continuous random variables Continuous R.V. s take values on a non-countable sets, e. g. height, weight, length, time, etc. We must find a way to describe how the unit of probability is distributed over this infinite set. For continuous R.V. s, the c.d.f. is a continuous function and often it has a derivative f X (t) = F X (t) such that P(a < X b) = F X (b) F X (a) = b a f X (t) dt f X (t) is then called the Probability Density Function (p.d.f). Such R.V. s are called absolutely continuous. We then have f X (t) 0 and + f X (t) dt = 1.

3 Discrete vs. Continuous R.V. s When F X (t) is continuous, P(X = a) = F X (a) F X (a ) = 0, so that prob. that X takes is any particular value is zero! Since P(X (t dt/2, t + dt/2)) f X (t)dt, p.d.f. in each point t shows how densely the probability is packed there, much the same as the particles making a stick weigh nothing, but not the stick itself which weight is the integral of the particles density. Thus no probability is associated with specific values of X but in Discrete case all probabilities are associated with specific values: important when dealing with inequalities. Example: Discrete R.V. with Ω = {0, 1, 2, 3, 4}: P(X 2) = p 0 + p 1 + p 2, but P(X < 2) = p 0 + p 1. For continuous X with Ω = (0, 4) we have if a density exists. P(X 2) = P(X < 2) = F X (2) = 2 0 f X (t)dt Density and Histogram Histogram of relative frequencies show how often the values from each bin occur. But f X (t) also shows the relative frequencies of values around each t to occur. So when bin size diminishes and the number of realisations of X grows, the histogram should resemble the density function

4 Expected value of a R.V. The expected value or expectation or the average or the mean value of a R.V. X is k E X = x kp(x = x k ) (Discrete) + tf X (t)dt (Abs. Continuous) For any function g(x) one has similarly k E g(x) = g(x k)p(x = x k ) (Discrete) + g(t)f X (t)dt (Abs. Continuous) One needs to distinguish the theoretical mean µ = E X and its sample counterpart X the sample mean. In general, X for population X = {X 1, X 2,..., X N } is an estimate of the random variable X s mean µ which may be unknown. Variance Variance of a R.V. X is a theoretical analogue of the sample variance the average square deviation from the mean: σ 2 = var X = E(X E X) 2 = E X 2 (E X) 2 The standard deviation is σ = var X. Example: Exam results: 2 students got 5, 2 4, 5 3 and 1 2 (failed). A student is picked at random and his class X is written. Find distribution of X and its first 2 moments (E X and var X). Probabilistic model: X P E X = = 3.5 (compare with X = 1 2/ / = E X here the way we have chosen the model!) E X 2 = = 13.1, thus var X = 13.1 (3.5) 2 = 0.85 and σ =

5 Example: A machine is overhauled. Suppose time until breakdown T (in hours) has p.d.f. f T (t) = e t/100, t > 0 Find the probabilities that the machine (a) breaks down within 10 hours; (b) runs for at least 50 hours. What is the c.d.f. and E T, σ? 10 (a) P(T 10) = 0 = e t/ e t/100 dt 0 = 1 e 10/ (b) P(T > 50) = 1 P(T 50)... = e 50/ t E T = e t/100 dt = int. by parts = 100; E T 2 t 2 = e t/100 dt = int. by parts = ; σ = = 100 Independence Two R.V. s X, Y are independent if for any A, B R one has P(X A, Y B) = P(X A) P(Y B) Example: Tossing two symmetric distinguishable coins. X i what shows i-th coin. Then X 1, X 2 are independent. Indeed P(X 1 = H, X 2 = H) = P(HH) = 1/4 = 1/2 1/2 = P(X 1 = H) P(X 2 = H) Similarly, for other combinations of H and T. Properties of E and var Suppose X, Y are R.V. s and a, b, c are constants. E(aX) = a E X E(b) = b E(X + Y ) = E X + E Y E(XY ) = E X E Y if X, Y are independent As consequence E(aX + by + c) = a E X + b E Y + c and if X, Y are, in addition, independent, then var(ax + by + c) = a 2 var X + b 2 var Y. var(ax) = a 2 var X var(b) = 0 var(x + Y ) = var X + var Y if X, Y are independent

6 Some important distributions: Binomial A trial has two possible outcomes: success (S) or Failure (F ) with probs. p = P(S) = 1 P(F ). The R.V. X the No. of successes in n independent such trials has Binomial Bin(n, p) distribution: P(X = k) = ( n k ) p k (1 p) n k, k = 0,..., n Indeed, on {X = k} an outcome has form (FFSFS... FS) with exactly k S s and n k F s. Each such outcome has prob. p k (1 p) n k (independence!) and there are ( n k) such outcomes since there are that many ways to distribute k S s over n places. E X = np and var X = np(1 p) (1) Let ξ i be R.V. equal 1 if there is a success in the i-th trial and 0 otherwise. We have E ξ i = 1 p + 0 (1 p) = p = E ξi 2 and var ξ i = E ξi 2 (E ξ i ) 2 = p p 2 = p(1 p). But X = n i=1 ξ i and ξ i are independent, thus we have (1) by properties 3 and 7 above. Example: Suppose that 20% of a large batch of resistors are defective and 5 are chosen at random. What are probs. that there are (i) exactly 4 are good; (ii) No more than 4 are good? (iii) What are the mean and the st.dev. of the number of good resistors in the sample? ( ) 5 (i) P(X = 4) = = (ii) P(X 4) = 1 P(X > 4) = 1 P(X = 5) = = (iii) E X = = 4, var X = = 0.8 Note the Binomial distr. is appropriate here because the sample is small and the batch is large: the proportion of defective transistors still may be assumed to stay at 20%. In general, when sampling without replacement from a finite population we use the Hypergeometric distribution hence σ =

7 Hypergeometric Distribution Population consists of M good and N bad and we select n items at random without replacement. Denote also p = M/(N + M) the proportion of good items. X is the number of good items in a sample of size n. Then ( M )( N ) k P(X = k) = Indeed, all outcomes are equiprobable and there are n k ( M+N n ) k M,. ( ) M+N of them. The first term is the number of ways to n choose k good among M good, while the second term to choose the rest n k bad. To any choice of good items Example: the same but the batch has 10 resistors, 2 of them are defective. Compare this: ( )( ) ( ) (i) P(X = 4) = / ( ) ( ) 8 10 (ii) P(X 4) = 1 / there may correspond any choice of bad items, so there are as many as their product choices corresponding to X = k. ( E X = np and var X = np(1 p) 1 n 1 ) N + M 1 Poisson Distribution Consider a fixed finite length (time) interval divided into n (v. large but very short) slots. During each such slot an event may either occur with (small) prob. p n or not. Slots are so short that no more than 1 occurrence is virtually possible. Then the No. X of occurrences of the event follows Binomial Bin(n, p n ) distribution. Let now n and p n 0, but so that np n λ > 0 (e. g. time slots have length 1/n and p n = λ/n). Then in the limit Very useful property is that E X = λ = var X. P(X = k) = λk k! e λ, k = 0, 1,... and we say that X follows Poisson distribution Po(λ).

8 Uniform distribution X has a Uniform Unif(a, b) distribution on [a, b] if X falls with the same probability in any subset (c, d) (a, b) provided its length d c is constant. This implies that p.d.f. is constant { 1/(b a) t (a, b) f X (t) = 0 otherwise 1 0 a F X (t) b thus P(X (c, d)) = (d c)/(b a) E X = (a + b)/2 var X = (b a)/12 0 t < a F X (t) = (t a)/(b a) t [a, b] 1 t > b Exponential distribution A random variable T has Exponential Exp(λ) distribution with parameter (or rate) λ if for t 0 and 0 otherwise. F T (t) = 1 e λt, f T (t) = λe λt E T = 1/λ and var T = 1/λ 2

9 Normal distribution One of the most important distribution Normal: N (m, σ 2 ) has bell-shaped density f X (t) = 1 (t } m)2 exp { 2πσ σ 2, t (, + ) E X = m and var X = σ 2. Standartisation Important property: Normal distribution is stable and preserved by linear transformations: if X 1 N (m 1, σ 2 1 ), X 2 N (m 2, σ 2 2 ) are independent, then X 1 + X 2 N (m 1 + m 2, σ σ2 2 ) ax + b N (am + b, a 2 σ 2 ) for any const. a, b If X N (m, σ 2 ) then, by above, Z = (X m)/σ has N (0, 1) distribution. Therefore P(X < t) = P((X m)/σ < (t m)/σ) = P(Z < (t m)/σ) = Φ((t m)/σ) There is no explicit expression for integrals of f X (t), so computer codes are used. The c.d.f. Φ(t) = P(Z < t) for the standard normal distribution Z N (0, 1) is a special Laplace function. found by software, e.g. Matlab. Note that the symmetry of the p.d.f. ϕ(t) = e t2 /2 2π Φ(t) = 1 Φ( t). implies that

10 Poisson Point Process Example: Assuming that the weight of screws is normally distr. with mean 2.10 gm and st.dev gm, find the proportion of screws weighing more than 2.55 gm. wt = X N (2.10, ); denote Z = (X 2.10)/0.15 P(X > 2.55) = P(Z > ( )/0.15) = P(Z > 3) = 1 P(Z 3) = Poisson distribution is useful to describe events occurring independently in some continuous medium, e. g. time: calls or accidents in some time interval, length: faults in isolation of a cable, area: position of trees in tundra, etc. Poisson Point Process Superposition property In this context we refer to the underlying Poisson point process (PPP): flow of events occurring with a constant rate λ in such a way that 1 No. of events in any interval of length (area) t follows Poisson Po(λt) distribution; and 2 No. events in disjoint intervals (sets) are independent. By the last property we cannot observe clustering of events, nor uniform spreading, i. e. almost the same spacing between events. Therefore Poisson process is also called completely random as the occurrence of events are independent. If X 1 and X 2 are independent Poisson R.V. s with parameters λ 1, λ 2, resp, then X 1 + X 2 also Po(λ 1 + λ 2 ). This is known as stability of the Poisson distribution. In particular, superposition of two Poisson processes with rates λ 1, λ 2 is again a Poisson process with intensity λ 1 + λ 2.

11 Example: On average, 3 cars/min and 1 lorry/min pass across a particular junction. If cars and lorries may be considered to be indep. Poisson processes, find the prob. that there we no vehicle during 30 seconds. Cars: PPP(λ 1 ) Lorries: PPP(λ 2 ) All vehicles: PPP(λ 1 + λ 2 ) Memorylessness of Poisson process Let T be the time passed from t 0 to the next event in a Poisson process of rate λ. P(T > t) = P{no events during time t} = (λt)0 e λt = e λt 0! i. e. T has exponential Exp(λ) distribution. Independence of events in Poisson process implies that the distribution is the same irrespective of whether t 0 was fixed (say, time 0) or it was a point of a Poisson process! Moreover, the intervals between events are not only Exp(λ) distributed, but also independent. No. of vehicle crossings during 0.5 min. follows Pois((3 + 1) 0.5), so prob.= 20 0! e 2 = e S T