Quiz I Review Probabilistic Systems Analysis

Transcription

1 Quiz I Review Probabilistic Systems Analysis 6.041SC Massachusetts Institute of Technology Quiz I Review 1 / 26

2 Quiz Information Content: Chapters 1-2, Lecture 1-7, Recitations 1-7, Psets 1-4, Tutorials 1-3 Quiz I Review 2 / 26

3 A Probabilistic Model: Sample Space: The set of all possible outcomes of an experiment. Probability Law: An assignment of a nonnegative number P(E) to each event E. Sample Space Probability Law Experiment Event A Event B P(A) P(B) A B Events Quiz I Review 3 / 26

4 Probability Axioms Given a sample space Ω: 1. Nonnegativity: P(A) 0 for each event A 2. Additivity: If A and B are disjoint events, then P(A B) = P(A) + P(B) If A 1, A 2,..., is a sequence of disjoint events, P(A 1 A 2 ) = P(A 1 ) + P(A 2 ) + 3. Normalization P(Ω) = 1 Quiz I Review 4 / 26

5 Properties of Probability Laws Given events A, B and C : 1. If A B, then P(A) P(B) 2. P(A B) = P(A) + P(B) P(A B) 3. P(A B) P(A) + P(B) 4. P(A B C ) = P(A) + P(A c B) + P(A c B c C ) Quiz I Review 5 / 26

6 Discrete Models Discrete Probability Law: If Ω is finite, then each event A Ω can be expressed as A = {s 1, s 2,..., s n } s i Ω Therefore the probability of the event A is given as P(A) = P(s 1 ) + P(s 2 ) + + P(s n ) Discrete Uniform Probability Law: If all outcomes are equally likely, P(A) = A Ω Quiz I Review 6 / 26

7 Conditional Probability Given an event B with P(B) > 0, the conditional probability of an event A Ω is given as P(A B) = P( A B) P ( B) P(A B) is a valid probability law on Ω, satisfying 1. P(A B) 0 2. P(Ω B) = 1 3. P(A 1 A 2 B) = P(A 1 B) + P(A 2 B) +, where {A i } i is a set of disjoint events P(A B) can also be viewed as a probability law on the restricted universe B. Quiz I Review 7 / 26

8 Multiplication Rule Let A 1,..., A n be a set of events such that ( ) n 1 P A i > 0. i=1 Then the joint probability of all events is ( ) n n 1 P A i = P(A 1 )P(A 2 A 1 )P(A 3 A 1 A 2 ) P(A n A i ) i=1 i =1 A A A P(A A 1 A 2 A 3 2 A 1 ) P(A 3 A 1 A 2 ) P(A 1 ) Quiz I Review 8 / 26

9 Total Probability Theorem Let A 1,..., A n be disjoint events that partition Ω. If P(A i ) > 0 for each i, then for any event B, n n P(B) = P(B A i ) = P(B A i )P(A i ) i=1 i=1 B A 1 B A 1 A 2 B A 1A2 B c B B c A 2 B A 3 A 3 B B c A 3 B Quiz I Review 9 / 26

10 Bayes Rule Given a finite partition A 1,..., A n of Ω with P(A i ) > 0, then for each event B with P(B) > 0 P(A i B) = P(B A i )P(A i ) P (B A = i )P(A i ) P(B) n i =1 P(B A i )P( A i ) B A 1 B A 1 A 2 B A 1A2 B c B B c A 2 B A 3 A 3 B B c A 3 B Quiz I Review 10 / 26

11 Independence of Events Events A and B are independent if and only if or P(A B) = P(A)P(B) P(A B) = P(A) if P(B) > 0 Events A and B are conditionally independent given an event C if and only if or P(A B C ) = P(A C )P(B C ) P(A B C ) = P(A C ) if P(B C ) > 0 Independence Conditional Independence. Quiz I Review 11 / 26

12 Independence of a Set of Events The events A 1,..., A n are pairwise independent if for each i = j P(A i A j ) = P(A i )P(A j ) The events A 1,..., A n are independent if ( ) P A i = P(A i ) S {1, 2,..., n} i S i S Pairwise independence independence, but independence pairwise independence. Quiz I Review 12 / 26

13 Counting Techniques Basic Counting Principle: For an m-stage process with n i choices at stage i, # Choices = n 1 n 2 n m Permutations: k-length sequences drawn from n distinct items without replacement (order is important): n! # Sequences = n(n 1) (n k + 1) = (n k)! Combinations: Sets with k elements drawn from n distinct items (order within sets is not important): ( ) # Sets = n n! k = k!(n k)! Quiz I Review 13 / 26

14 Counting Techniques-contd Partitions: The number of ways to partition an n-element set into r disjoint subsets, with n k elements in the k th subset: ( ) ( )( ) ( ) n n n n 1 n n 1 n r 1 = n 1, n 2,..., n r n 1 n 2 n r n! = n1!n 2!,, n r! where ( ) n n! = k k!(n k)! r n i = n i=1 Quiz I Review 14 / 26

15 Discrete Random Variables A random variable is a real-valued function defined on the sample space: X : Ω R The notation {X = x} denotes an event: {X = x} = {ω Ω X (ω) = x} Ω The probability mass function (PMF) for the random variable X assigns a probability to each event {X = x}: ( ) p X (x) = P({X = x}) = P {ω Ω X (ω) = x} Quiz I Review 15 / 26

16 PMF Properties Let X be a random variable and S a countable subset of the real line The axioms of probability hold: 1. p X (x) 0 2. P(X S) = x S p X (x) 3. x p X (x) = 1 If g is a real-valued function, then Y = g(x ) is a random variable: ω X X (ω) g g(x (ω)) = Y (ω) with PMF p Y (y) = P X (x) x g(x)=y Quiz I Review 16 / 26

17 Expectation Given a random variable X with PMF p X (x): E[X ] = x xp X (x) Given a derived random variable Y = g(x ): E[g(X )] = g(x)p X (x) = yp Y (y) = E [Y ] x E[X n ] = x n p X (x) x Linearity of Expectation: E[aX + b] = ae[x ] + b. y Quiz I Review 17 / 26

18 Variance The expected value of a derived random variable g(x ) is E[g(X )] = g(x)p X (x) The variance of X is calculated as var(x ) = E[(X E[X ]) 2 ] = x(x E[X ]) 2 p X (x) var(x ) = E[X 2 ] E[X ] 2 var(ax + b) = a 2 var(x ) Note that var(x) 0 x Quiz I Review 18 / 26

19 Multiple Random Variables Let X and Y denote random variables defined on a sample space Ω. The joint PMF of X and Y is denoted by ( ) p X,Y (x, y) = P {X = x} {Y = y} The marginal PMFs of X and Y are given respectively as p X (x) = p X,Y (x, y) y p Y (y) = p X,Y (x, y) x Quiz I Review 19 / 26

20 Functions of Multiple Random Variables Let Z = g(x, Y ) be a function of two random variables PMF: p Z (z) = p X,Y (x, y) (x,y) g(x,y)=z Expectation: E[Z ] = g(x, y)p X,Y (x, y) x,y Linearity: Suppose g(x, Y ) = ax + by + c. E[g(X, Y )] = ae[x ] + be[y ] + c Quiz I Review 20 / 26

21 Conditioned Random Variables Conditioning X on an event A with P(A) > 0 results in the PMF: ( ) p X A (x) = P ( {X = x} A ) = P {X = x } A P (A) Conditioning X on the event Y = y with P Y (y) > 0 results in the PMF: ( ) p X Y (x y) = P {X = ( x} {Y ) = y} = p X,Y (x, y) P {Y = y} p Y (y) Quiz I Review 21 / 26

22 Conditioned RV - contd Multiplication Rule: p X,Y (x, y) = p X Y (x y)p Y (y) Total Probability Theorem: n p X (x) = P(A i )p X Ai (x) i=1 p X (x) = p X Y (x y)p Y (y) y Quiz I Review 22 / 26

23 Conditional Expectation Let X and Y be random variables on a sample space Ω. Given an event A with P(A) > 0 E[X A] = xp X A (x) If P Y (y) > 0, then E[X {Y = y}] = xp X Y (x y) Total Expectation Theorem: Let A 1,..., A n be a partition of Ω. If P(A i ) > 0 i, then n E[X ] = P(A i )E[X A i ] i=1 x x Quiz I Review 23 / 26

24 Independence Let X and Y be random variables defined on Ω and let A be an event with P(A) > 0. X is independent of A if either of the following hold: p X A (x) = p X (x) x p X,A (x) = p X (x)p(a) x X and Y are independent if either of the following hold: p X Y (x y) = p X (x) x y p X,Y (x, y) = p X (x)p Y (y) x y Quiz I Review 24 / 26

25 Independence If X and Y are independent, then the following hold: If g and h are real-valued functions, then g(x ) and h(y ) are independent. E[XY ] = E[X ]E[Y ] (inverse is not true) var(x + Y ) = var(x ) + var(y ) Given independent random variables X 1,..., X n, var(x 1 +X 2 + +X n ) = var(x 1 )+var(x 2 )+ +var(x n ) Quiz I Review 25 / 26

26 Some Discrete Distributions Bernoulli Binomial Geometric Uniform { X 1 success 0 failure Number of successes in n Bernoulli trials Number of trials until first success An integer in the interval [a,b] { p X (k) p k = 1 ( ) 1 p k = 0 n pk (1 p) n k k = 0, 1,..., n (1 p) k 1 p { k = 1, 2,... 1 b a+1 k = a,..., b 0 otherwise E[X ] var(x ) p p(1 p) np 1 p a+b 2 np(1-p) 1 p p 2 (b a)(b a+2) 12 Quiz I Review 26 / 26

27 MIT OpenCourseWare SC Probabilistic Systems Analysis and Applied Probability Fall 2013 For information about citing these materials or our Terms of Use, visit: