Lecture 01: Mathematical Preliminaries and Probability

Transcription

1 Lecture 01: Mathematical Preliminaries and Probability Ba Chu ba Web: bchu (Note that this is a lecture note. Please refer to the textbooks suggested in the course outline for details. Examples will be given and explained in the class) 1 Objectives The first part of this lecture collects fundamental mathematical concepts that we will use in this course. Most of these concepts should seem familiar for 4th year undergraduate students, although my presentation of them may be a bit more formal than you have previously seen. This formalism will be quite useful as you study probability and statistics. The second part of this lecture provides you with some probability tools that will be needed when you study statistics, I also hope to impart some of the beauty of those tools. 2 Mathematical Preliminaries 2.1 Sets Suppose that there exists a designated universe of possible objects. In the course, we will often denote the universe by S. By a set, we mean a collection of objects with the property that each object in the S either does or does not belong to the collection. We will tend to denote sets by uppercase Roman letters, e.g., A, B, C, etc. The set of objects that do not belong to a designated set 1

2 A is called the complement of A. We will denote complements by A c, B c, C c, etc. The complement of the universe is the empty set, denoted by S c =. An object that belongs to a designated set is called an element or member of that set. We will denote elements by lowercase Roman letters and write expressions such as x A, pronounced x is an element of the set A. In this course, of course, we usually will be concerned with sets defined by certain mathematical properties. Some familiar sets to which we will refer repeatedly include: The set of natural numbers, N = {1, 2, 3,..., }. The set of integers, Z = {..., 3, 2, 1, 0, 1, 2, 3,... }. The set of real numbers, R = (, + ). If A and B are sets and each element of A is also an element of B, then we say that A is a subset of B and write A B. Given sets A and B, there are several important sets that can be constructed from them. The union of A and B is the set: A B = {x S : x A and x B}; the intersection of A and B is the set: A B = {x S : x A or x B}. If A B = A and B have no element in common, then A and B are disjoint. 2.2 Permutations and combinations We now consider two more concepts that are often employed when counting the elements of finite sets. We motivate these concepts with an example Example 1. A buffet restaurant offers a single entree that comes with a choice of 3 side dishes from a total of 15. To address the perception that it serves only one dinner, the restaurant conceives an advertisement that identifies each choice of side dishes as a distinct dinner. Assuming that each entree must be accompanied by 3 distinct dishes, e.g., {chicken wings, barbecued pork, fried vegetables} is permitted by {chicken wings, chicken wings, fried vegetables} is not, how many distinct dinners are available? Answer 1: there are = 2730 possible dinners. 2

3 Answer 2: there are 2730/(3 2 1) = 455 possible dinners. In the first answer, 2730 is the number of permutations (ordered choices) of 3 from 15. In the second answer, 455 is the number of combinations (unordered choices) of 3 from 15. From this example, we abstract the following definitions: Definition 1. The number of permutations of r objects from n objects is P (n, r) = n (n 1) (n r +1); and the number of combinations of r objects from n objects is C(n, r) = P (n, r)/p (r, r). Example 2. In a class of 40 students, how many ways can one choose 20 students to receive exam A? Assuming that the class comprises 30 freshmen and 10 non-freshmen, how many ways can one choose 15 freshmen and 5 non-freshmen to receive exam A? The answer to the first question is C(40, 20) = 137, 846, 528, 820, and the answer to the second question is C(30, 15) C(10, 5) = 39, 089, 615, Functions A function is a rule that assigns a unique element of a set B to each element of another set A. A familiar example is the rule that assign each real number x the real number y = x 2, e.g., that assign y = 4 to x = 2. The set A is the function s domain and the set B is the function s range. We use a variety of letters to denote various types of functions. Examples include f, g, φ. If φ is a function with domain A and range B, then we write φ : A B, often pronounced φ maps A into B. If φ assigns b B to a A, then we say that b is the value of φ at a and we write b = φ(a). If φ : A B, then, for each b B, there is a subset of A comprising those elements of A at which φ has value b. We denote this set by φ 1 (b) = {a A : φ(a) = b}. 2.4 Limits Let {y n } denote a sequence of real numbers. We say that {y n } converges to a constant value c R if, for every arbitrarily small ɛ > 0, there exists a natural number N such that y n (c ɛ, c + ɛ) for each n N. 3

4 3 Probability 3.1 What does probability mean? Probabilistic statement can be interpreted in different ways. For instance, how would you interpret the following statement? There is a 40 percent chance of rain today. Your interpretation varies depending on the context in which the statement is made. If the statement was made as part of a forecast by the National Weather Service, then something like the following might be appropriate: In the recent history, of all days on which present atmospheric conditions have been experienced, rain has occurred on approximately 40 percent of them. This is an example of the frequentist interpretation of probability. With this interpretation, a probability is a long-run average proportion of occurrence. Suppose that you had just peered out a window and seen dark clouds, then something like the following might be appropriate: I believe that it might very well rain, but that is a little less likely to rain than not. This is an example of the subjectivist interpretation of probability. With this interpretation, a probability expresses the strength of one s belief. Dual notions of probability frequentist and subjectivist have co-existed throughout history. But, however, we decide to interpret probabilities, we will need a formal mathematical description of probability to which we can appeal for insight and guidance. 3.2 Axioms of probability The mathematical model that dominated the study of probability was formulated by A. N. Kolmogorov (1933). The central concept in this model is a probability space, which is assumed to have 3 components: 4

5 S a sample space, a universe of possible outcomes for the experiment in question. C a designated collection of observable subsets (called events) of the sample space. P a probability measure, a function that assigns real numbers (called probabilities) to events. Example 3. A coin tossed twice. A plausible sample space for this experiment comprises 4 outcomes, i.e., S = {HH, T H, HT, T T }. If we designated all subsets of S as events, then we obtain: C{S, {HH, HT, T H}, {HT, T H, T T }, {HH, T T }, {T T }, }. Once the collection of events has been designated, each event E C can be assigned a probability P (E). This must be done according to specific rules; in particular, the probability measure P must satisfy the following properties: 1. If E is an event, then 0 P (E) P (S) = If {E 1, E 2, E 3,... } is a countable collection of pairwise disjoint events, then P { i=1e i } = P (E i ). i=1 The third property is called countably additivity. All of the facts about probability that we will use in studying statistical inference are consequences of the assumptions of the Kolmogorov probability model. We state 3 elementary (and useful) propositions: Theorem 1. If E is an event, then P (E c ) = 1 P (E). Theorem 2. If A and B are events and A B, then P (A) P (B). Theorem 3. If A and B are events, then P (A B) = P (A) + P (B) P (A B). Proofs of these theorems will be explained in the class. 5

6 3.3 Finite sample space Let S = {s 1,..., s N } denote a sample space that contains N outcomes and suppose that every subset of S is an event. For notational convenience, let p i = P ({s i }) denote the probability of the outcome i, for i = 1,..., N. Then, for any event A, we have P (A) = P ( si A{s i }) = s i A P ({s i }) = s i A p i. If the outcomes are equally likely, i.e., p i = 1, then we have N P (A) = s i A 1 N = s i A 1 N = #(A) #(S), where #(A) is the number of outcomes in the set A, and #(S) is the number of outcomes in the sample space S. This equation reveals that, when the outcomes in a finite sample space are equally likely, calculating probabilities is just a matter of counting. Example 4. A fair coin is tossed twice, the probability of observing exactly one Head is P (A) = 2/4 = 0.5, where A = {HT, T H}. Example 5. Five fair dice are tossed simultaneously. The probability that the top faces of the dice all show the same number of dots is P (A) = 6/6 5 = Example 6. In a class of 70 students, the probability that at least two students share a common birthday is P (A) = #(A) #(S) = 1 P (Ac ) = 1 #(Ac ) #(S) = (365 70) , where #(A c ) is the number of outcomes in which each student has a different birthday by observing that 365 possible birthdays are available for the oldest student, after which 364 possible birthdays for the next oldest students, after which 363 possible birthdays remain for the next, etc. 6

7 3.4 Conditional probability The conditional probability of the event A given the occurrence of the event B is given as P (A B) = #(A B) = #(A B)/#(S) #(S B) #(B)/#(S) = P (A B) P (B). Two events are said to be independent if the occurrence of either is unaffected by the occurrence of the other. This notion can be expressed mathematically using the concept of conditional probability. The events A and B are regarded as independent, then P (A B) = P (A) or P (B A) = P (B). This implies P (A B) = P (A)P (B). Example 7. Consider the population of Carleton undergraduate students, let A denote the event that the student is male and let B denote the event that the student is concentrating in art. I was told that P(B A)=0.2 and P(A)=0.45, thus A and B are dependent. Definition 2. Let A i be an arbitrary collection of events. These events are mutually independent if and only if, for every finite choice of events A 1,..., A n, P (A 1 A 2 A n ) = P (A 1 )... P (A n ). 3.5 Random variables A random variable is a rule for assigning real numbers to experimental outcomes. By convention, random variables are usually denoted by uppercase Roman letters, e.g., X, Y, Z. Example 8. A coin is tossed once and Heads (H) or Tails (T) is observed. The sample space for this experiment is S = {H, T }. For obvious reasons, it is often convenient to assign the real number 1 to H and the real number 0 to T. This assignment, which we denote by the random variable X, can be depicted as follows: (H, T ) X (1, 0). In functional notation, X : S R; and the rule of assignment is defined by X(H) = 1 and X(T ) = 0. 7

8 Example 9. A coin is tossed twice, and the number of H is counted. The rule of assignments is (HH, HT, T H, T T ) X (2, 1, 1, 0). The primary reason that we construct a random variable, X, is to replace the probability space that is naturally suggested by the experiment in question with a familiar probability space in which the possible outcomes are real numbers. Thus, we replace the original sample space, S, with the familiar number line, R, thus the collection of events, C, contains every intervals of the form (, y]. Definition 3. A function X : S R is a random variable if and only if P ({s S : X(s) y}) = P (X y) exists for all y R. We will denote the probability measure induced by the random variable X by P X. Definition 4. The cumulative distribution function (cdf) of a random variable X is the function F : R R defined by F (y) = P (X y). Note that if y 2 > y 1 then F (y 2 ) > F (y 1 ) and that lim y F (y) = 1 and lim y F (y) = 0. Example 10. A typical coin is tossed. For P (H) = P (T ) = 0.5, given that (H, T ) following values of the cdf are easily obtained: X (1, 0), the If y < 0, then F (y) = P (X y) = 0. F (0) = P (X 0) = 0.5. If y (0, 1), then F (y) = P (X y) = 0.5. F (1) = P (X 1) = 1. If y > 1, then F (y) = P (X y) = 1. Example 11. Suppose that a fair coin is tossed twice and each of the 4 possible outcomes in S is equally likely, the cdf can be easily determined. ( derive it yourself). 8

9 4 Exercises 1. For n = 0, 1, 2,..., let y n = n k=0 2 k = n. (a) Compute y 0, y 1, y 2, y 3, and y 4. (b) The sequence {y 0, y 1, y 2,... } is an example of a partial sum. Guess the value of its limit, usually written lim y n = lim n n n 2 k = k=0 2 k. k=0 2. Suppose that 4 fair dice are thrown simultaneously. (a) How many outcomes are possible? (b) What is the probability that the top face shows a different number? (c) What is the probability that the top faces show 4 numbers that sum to five? (d) What is the probability that at least one of the top faces shows an odd number? (e) What is the probability that 3 of the top faces show the same odd number and the other top face shows an even number? 3. Suppose that you buy a share and hold it for 2 periods. The probabilities that the share price goes up and down are 0.5 and 0.3 respectively. (a) What is the probability that the share price does not change till the end of the first period? (b) What is the probability that the share price does not change till the end of the second period? (c) What is the probability that the share price remain at least the same till the end of the first period? (d) What is the conditional probability that the share price goes up by the end of the second period given that it has gone down by the end of the first period? 9

10 4. Suppose that P (A) = 0.7, P (B) = 0.6, and P (A c B) = 0.2. (a) Draw a Venn diagram that describes this experiment. (b) Is it possible for A and B to be disjoint events? Why or why not? (c) What is the probability of A B c? (d) Is it possible for A and B to be independent events? Why or why not? (e) What is the conditional probability of A given B? 5. Two undergraduate students are renting a house. Before leaving town for Winter break, each writes a check for his/her share of the rent. John writes his check on Dec. 16. By chance, it happens that the number of his check ends with the digits 16. Anne writes her check on Dec. 18. By chance, it happens that the number of her check ends with the digits 18. What is the probability of such a coincidence, i.e., that both students would use checks with numbers that end in the same two digits as the date? 6. Suppose that X is a random variable with cdf: 0, if y 0, y/3, if y [0, 1], F (y) = 2/3, if y [1, 2], y/3, if y [2, 3], 1, if y 3. Graph F (y) and compute the following probabilities: P (X > 0.5), P (2 < X 3), P (0.5 < X 2.5), and P (X = 1). 10