Introduction to Randomized Algorithms: QuickSort and QuickSelect
|
|
|
- Annice Armstrong
- 7 years ago
- Views:
Transcription
1 CS 473: Fundamental Algorithms, Fall 2014 Introduction to Randomized Algorithms: QuickSort and QuickSelect Lecture 13 October 14, 2014 Alexandra (UIUC) CS473 1 Fall / 54
2 Part I Introduction to Randomized Algorithms Alexandra (UIUC) CS473 2 Fall / 54
3 Randomized Algorithms Input x Deterministic Algorithm Output y random bits r Input x Randomized Algorithm Output y r Alexandra (UIUC) CS473 3 Fall / 54
4 Randomized Algorithms Input x Deterministic Algorithm Output y random bits r Input x Randomized Algorithm Output y r Alexandra (UIUC) CS473 3 Fall / 54
5 Example: Randomized QuickSort QuickSort Hoare [1962] 1 Pick a pivot element from array 2 Split array into 3 subarrays: those smaller than pivot, those larger than pivot, and the pivot itself. 3 Recursively sort the subarrays, and concatenate them. Randomized QuickSort 1 Pick a pivot element uniformly at random from the array 2 Split array into 3 subarrays: those smaller than pivot, those larger than pivot, and the pivot itself. 3 Recursively sort the subarrays, and concatenate them. Alexandra (UIUC) CS473 4 Fall / 54
6 Example: Randomized Quicksort Recall: QuickSort can take Ω(n 2 ) time to sort array of size n. Theorem Randomized QuickSort sorts a given array of length n in O(n log n) expected time. Note: On every input randomized QuickSort takes O(n log n) time in expectation. On every input it may take Ω(n 2 ) time with some small probability. Alexandra (UIUC) CS473 5 Fall / 54
7 Example: Randomized Quicksort Recall: QuickSort can take Ω(n 2 ) time to sort array of size n. Theorem Randomized QuickSort sorts a given array of length n in O(n log n) expected time. Note: On every input randomized QuickSort takes O(n log n) time in expectation. On every input it may take Ω(n 2 ) time with some small probability. Alexandra (UIUC) CS473 5 Fall / 54
8 Example: Randomized Quicksort Recall: QuickSort can take Ω(n 2 ) time to sort array of size n. Theorem Randomized QuickSort sorts a given array of length n in O(n log n) expected time. Note: On every input randomized QuickSort takes O(n log n) time in expectation. On every input it may take Ω(n 2 ) time with some small probability. Alexandra (UIUC) CS473 5 Fall / 54
9 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Deterministic algorithm: 1 Multiply A and B and check if equal to C. 2 Running time? O(n 3 ) by straight forward approach. O(n 2.37 ) with fast matrix multiplication (complicated and impractical). Alexandra (UIUC) CS473 6 Fall / 54
10 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Deterministic algorithm: 1 Multiply A and B and check if equal to C. 2 Running time? O(n 3 ) by straight forward approach. O(n 2.37 ) with fast matrix multiplication (complicated and impractical). Alexandra (UIUC) CS473 6 Fall / 54
11 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Deterministic algorithm: 1 Multiply A and B and check if equal to C. 2 Running time? O(n 3 ) by straight forward approach. O(n 2.37 ) with fast matrix multiplication (complicated and impractical). Alexandra (UIUC) CS473 6 Fall / 54
12 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Randomized algorithm: 1 Pick a random n 1 vector r. 2 Return the answer of the equality ABr = Cr. 3 Running time? O(n 2 )! Theorem If AB = C then the algorithm will always say YES. If AB C then the algorithm will say YES with probability at most 1/2. Can repeat the algorithm 100 times independently to reduce the probability of a false positive to 1/ Alexandra (UIUC) CS473 7 Fall / 54
13 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Randomized algorithm: 1 Pick a random n 1 vector r. 2 Return the answer of the equality ABr = Cr. 3 Running time? O(n 2 )! Theorem If AB = C then the algorithm will always say YES. If AB C then the algorithm will say YES with probability at most 1/2. Can repeat the algorithm 100 times independently to reduce the probability of a false positive to 1/ Alexandra (UIUC) CS473 7 Fall / 54
14 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Randomized algorithm: 1 Pick a random n 1 vector r. 2 Return the answer of the equality ABr = Cr. 3 Running time? O(n 2 )! Theorem If AB = C then the algorithm will always say YES. If AB C then the algorithm will say YES with probability at most 1/2. Can repeat the algorithm 100 times independently to reduce the probability of a false positive to 1/ Alexandra (UIUC) CS473 7 Fall / 54
15 Example: Verifying Matrix Multiplication Problem Given three n n matrices A, B, C is AB = C? Randomized algorithm: 1 Pick a random n 1 vector r. 2 Return the answer of the equality ABr = Cr. 3 Running time? O(n 2 )! Theorem If AB = C then the algorithm will always say YES. If AB C then the algorithm will say YES with probability at most 1/2. Can repeat the algorithm 100 times independently to reduce the probability of a false positive to 1/ Alexandra (UIUC) CS473 7 Fall / 54
16 Why randomized algorithms? 1 Many many applications in algorithms, data structures and computer science! 2 In some cases only known algorithms are randomized or randomness is provably necessary. 3 Often randomized algorithms are (much) simpler and/or more efficient. 4 Several deep connections to mathematics, physics etc Lots of fun! Alexandra (UIUC) CS473 8 Fall / 54
17 Where do I get random bits? Question: Are true random bits available in practice? 1 Buy them! 2 CPUs use physical phenomena to generate random bits. 3 Can use pseudo-random bits or semi-random bits from nature. Several fundamental unresolved questions in complexity theory on this topic. Beyond the scope of this course. 4 In practice pseudo-random generators work quite well in many applications. 5 The model is interesting to think in the abstract and is very useful even as a theoretical construct. One can derandomize randomized algorithms to obtain deterministic algorithms. Alexandra (UIUC) CS473 9 Fall / 54
18 Average case analysis vs Randomized algorithms Average case analysis: 1 Fix a deterministic algorithm. 2 Assume inputs comes from a probability distribution. 3 Analyze the algorithm s average performance over the distribution over inputs. Randomized algorithms: 1 Algorithm uses random bits in addition to input. 2 Analyze algorithms average performance over the given input where the average is over the random bits that the algorithm uses. 3 On each input behaviour of algorithm is random. Analyze worst-case over all inputs of the (average) performance. Alexandra (UIUC) CS Fall / 54
19 Discrete Probability We restrict attention to finite probability spaces. Definition A discrete probability space is a pair (Ω, Pr) consists of finite set Ω of elementary events and function p : Ω [0, 1] which assigns a probability Pr[ω] for each ω Ω such that ω Ω Pr[ω] = 1. Example An unbiased coin. Ω = {H, T} and Pr[H] = Pr[T] = 1/2. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. Alexandra (UIUC) CS Fall / 54
20 Discrete Probability We restrict attention to finite probability spaces. Definition A discrete probability space is a pair (Ω, Pr) consists of finite set Ω of elementary events and function p : Ω [0, 1] which assigns a probability Pr[ω] for each ω Ω such that ω Ω Pr[ω] = 1. Example An unbiased coin. Ω = {H, T} and Pr[H] = Pr[T] = 1/2. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. Alexandra (UIUC) CS Fall / 54
21 Discrete Probability And more examples Example A biased coin. Ω = {H, T} and Pr[H] = 2/3, Pr[T] = 1/3. Example Two independent unbiased coins. Ω = {HH, TT, HT, TH} and Pr[HH] = Pr[TT] = Pr[HT] = Pr[TH] = 1/4. Example A pair of (highly) correlated dice. Ω = {(i, j) 1 i 6, 1 j 6}. Pr[i, i] = 1/6 for 1 i 6 and Pr[i, j] = 0 if i j. Alexandra (UIUC) CS Fall / 54
22 Events Definition Given a probability space (Ω, Pr) an event is a subset of Ω. In other words an event is a collection of elementary events. The probability of an event A, denoted by Pr[A], is ω A Pr[ω]. The complement event of an event A Ω is the event Ω \ A frequently denoted by Ā. Alexandra (UIUC) CS Fall / 54
23 Events Examples Example A pair of independent dice. Ω = {(i, j) 1 i 6, 1 j 6}. 1 Let A be the event that the sum of the two numbers on the dice is even. { } Then A = (i, j) Ω (i + j) is even. Pr[A] = A /36 = 1/2. 2 Let B{ be the event that the first die has 1. Then} B = (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6). Pr[B] = 6/36 = 1/6. Alexandra (UIUC) CS Fall / 54
24 Independent Events Definition Given a probability space (Ω, Pr) and two events A, B are independent if and only if Pr[A B] = Pr[A] Pr[B]. Otherwise they are dependent. In other words A, B independent implies one does not affect the other. Example Two coins. Ω = {HH, TT, HT, TH} and Pr[HH] = Pr[TT] = Pr[HT] = Pr[TH] = 1/4. 1 A is the event that the first coin is heads and B is the event that second coin is tails. A, B are independent. 2 A is the event that the two coins are different. B is the event that the second coin is heads. A, B independent. Alexandra (UIUC) CS Fall / 54
25 Independent Events Definition Given a probability space (Ω, Pr) and two events A, B are independent if and only if Pr[A B] = Pr[A] Pr[B]. Otherwise they are dependent. In other words A, B independent implies one does not affect the other. Example Two coins. Ω = {HH, TT, HT, TH} and Pr[HH] = Pr[TT] = Pr[HT] = Pr[TH] = 1/4. 1 A is the event that the first coin is heads and B is the event that second coin is tails. A, B are independent. 2 A is the event that the two coins are different. B is the event that the second coin is heads. A, B independent. Alexandra (UIUC) CS Fall / 54
26 Independent Events Examples Example A is the event that both are not tails and B is event that second coin is heads. A, B are dependent. Alexandra (UIUC) CS Fall / 54
27 Dependent or independent? Consider two rolls of the dice. 1 A = the event that the first roll is odd. 2 B = the event that the sum of the two rolls is odd. The events A and B are (A) dependent. (B) independent. Alexandra (UIUC) CS Fall / 54
28 Union bound The probability of the union of two events, is no bigger than the probability of the sum of their probabilities. Lemma For[ any two ] events[ E] and F, [ we ] have that Pr E F Pr E + Pr F. Proof. Consider E and F to be a collection of elmentery events (which they are). We have [ ] Pr E F = Pr[x] x E F x E Pr[x] + [ ] Pr[x] = Pr E x F [ ] + Pr F. Alexandra (UIUC) CS Fall / 54
29 Random Variables Definition Given a probability space (Ω, Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω R. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. 1 X : Ω R where X(i) = i mod 2. 2 Y : Ω R where Y(i) = i 2. Definition A binary random variable is one that takes on values in {0, 1}. Alexandra (UIUC) CS Fall / 54
30 Random Variables Definition Given a probability space (Ω, Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω R. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. 1 X : Ω R where X(i) = i mod 2. 2 Y : Ω R where Y(i) = i 2. Definition A binary random variable is one that takes on values in {0, 1}. Alexandra (UIUC) CS Fall / 54
31 Random Variables Definition Given a probability space (Ω, Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω R. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. 1 X : Ω R where X(i) = i mod 2. 2 Y : Ω R where Y(i) = i 2. Definition A binary random variable is one that takes on values in {0, 1}. Alexandra (UIUC) CS Fall / 54
32 Indicator Random Variables Special type of random variables that are quite useful. Definition Given a probability space (Ω, Pr) and an event A Ω the indicator random variable X A is a binary random variable where X A (ω) = 1 if ω A and X A (ω) = 0 if ω A. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. Let A be the even that i is divisible by 3. Then X A (i) = 1 if i = 3, 6 and 0 otherwise. Alexandra (UIUC) CS Fall / 54
33 Indicator Random Variables Special type of random variables that are quite useful. Definition Given a probability space (Ω, Pr) and an event A Ω the indicator random variable X A is a binary random variable where X A (ω) = 1 if ω A and X A (ω) = 0 if ω A. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. Let A be the even that i is divisible by 3. Then X A (i) = 1 if i = 3, 6 and 0 otherwise. Alexandra (UIUC) CS Fall / 54
34 Expectation Definition For a random variable X over a probability space (Ω, Pr) the expectation of X is defined as ω Ω Pr[ω] X(ω). In other words, the expectation is the average value of X according to the probabilities given by Pr[ ]. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. 1 X : Ω R where X(i) = i mod 2. Then E[X] = 1/2. 2 Y : Ω R where Y(i) = i 2. Then E[Y] = 6 1 i=1 6 i2 = 91/6. Alexandra (UIUC) CS Fall / 54
35 Expectation Definition For a random variable X over a probability space (Ω, Pr) the expectation of X is defined as ω Ω Pr[ω] X(ω). In other words, the expectation is the average value of X according to the probabilities given by Pr[ ]. Example A 6-sided unbiased die. Ω = {1, 2, 3, 4, 5, 6} and Pr[i] = 1/6 for 1 i 6. 1 X : Ω R where X(i) = i mod 2. Then E[X] = 1/2. 2 Y : Ω R where Y(i) = i 2. Then E[Y] = 6 1 i=1 6 i2 = 91/6. Alexandra (UIUC) CS Fall / 54
36 Expected number of vertices? Let G = (V, E) be a graph with n vertices and m edges. Let H be the graph resulting from deleting every vertex of G with probability 1/2. The expected number of vertices in H is (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Alexandra (UIUC) CS Fall / 54
37 Expectation Proposition For an indicator variable X A, E[X A ] = Pr[A]. Proof. E[X A ] = X A (y) Pr[y] y Ω = 1 Pr[y] + 0 Pr[y] y A = Pr[y] y A = Pr[A]. y Ω\A Alexandra (UIUC) CS Fall / 54
38 Dude, where is my car? Suppose you re on a game show, and you re given the choice of three doors: Behind one door is a car; behind the others, vampires. You pick a door, say No. 1, and the host, who knows what s behind the doors, opens another door, say No. 3, which has a vampire. He then says to you, Do you want to pick door No. 2? Is it to your advantage to switch your choice? (A) Yep. Changing doors is a good idea. (B) Nope. The probability that you are on the right door is the same, whether you change the door, or stay with your current selection. Alexandra (UIUC) CS Fall / 54
39 Linearity of Expectation Lemma Let X, Y be two random variables (not necessarily independent) over a probability space (Ω, Pr). Then E[X + Y] = E[X] + E[Y]. Proof. E[X + Y] = ω Ω Pr[ω] (X(ω) + Y(ω)) = ω Ω Pr[ω] X(ω) + ω Ω Pr[ω] Y(ω) = E[X] + E[Y]. Corollary E[a 1 X 1 + a 2 X a n X n ] = n i=1 a i E[X i ]. Alexandra (UIUC) CS Fall / 54
40 Linearity of Expectation Lemma Let X, Y be two random variables (not necessarily independent) over a probability space (Ω, Pr). Then E[X + Y] = E[X] + E[Y]. Proof. E[X + Y] = ω Ω Pr[ω] (X(ω) + Y(ω)) = ω Ω Pr[ω] X(ω) + ω Ω Pr[ω] Y(ω) = E[X] + E[Y]. Corollary E[a 1 X 1 + a 2 X a n X n ] = n i=1 a i E[X i ]. Alexandra (UIUC) CS Fall / 54
41 Expected number of edges? Let G = (V, E) be a graph with n vertices and m edges. Let H be the graph resulting from deleting every vertex of G with probability 1/2. The expected number of edges in H is (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Alexandra (UIUC) CS Fall / 54
42 Expected number of triangles? Let G = (V, E) be a graph with n vertices and m edges. Assume G has t triangles (i.e., a triangle is a simple cycle with three vertices). Let H be the graph resulting from deleting every vertex of G with probability 1/2. The expected number of triangles in H is (A) t/2. (B) t/4. (C) t/8. (D) t/16. (E) none of the above. Alexandra (UIUC) CS Fall / 54
43 Types of Randomized Algorithms Typically one encounters the following types: 1 Las Vegas randomized algorithms: for a given input x output of algorithm is always correct but the running time is a random variable. In this case we are interested in analyzing the expected running time. 2 Monte Carlo randomized algorithms: for a given input x the running time is deterministic but the output is random; correct with some probability. In this case we are interested in analyzing the probability of the correct output (and also the running time). 3 Algorithms whose running time and output may both be random. Alexandra (UIUC) CS Fall / 54
44 Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π: 1 Let Q(x) be the time for Q to run on input x of length x. 2 Worst-case analysis: run time on worst input for a given size n. T wc (n) = max Q(x). x: x =n Randomized algorithm R for a problem Π: 1 Let R(x) be the time for Q to run on input x of length x. 2 R(x) is a random variable: depends on random bits used by R. 3 E[R(x)] is the expected running time for R on x 4 Worst-case analysis: expected time on worst input of size n T rand wc (n) = max x: x =n E [Q(x)]. Alexandra (UIUC) CS Fall / 54
45 Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π: 1 Let Q(x) be the time for Q to run on input x of length x. 2 Worst-case analysis: run time on worst input for a given size n. T wc (n) = max Q(x). x: x =n Randomized algorithm R for a problem Π: 1 Let R(x) be the time for Q to run on input x of length x. 2 R(x) is a random variable: depends on random bits used by R. 3 E[R(x)] is the expected running time for R on x 4 Worst-case analysis: expected time on worst input of size n T rand wc (n) = max x: x =n E [Q(x)]. Alexandra (UIUC) CS Fall / 54
46 Analyzing Monte Carlo Algorithms Randomized algorithm M for a problem Π: 1 Let M(x) be the time for M to run on input x of length x. For Monte Carlo, assumption is that run time is deterministic. 2 Let Pr[x] be the probability that M is correct on x. 3 Pr[x] is a random variable: depends on random bits used by M. 4 Worst-case analysis: success probability on worst input P rand wc (n) = min x: x =n Pr[x]. Alexandra (UIUC) CS Fall / 54
47 Part II Why does randomization help? Alexandra (UIUC) CS Fall / 54
48 Ping and find. Consider a deterministic algorithm A that is trying to find an element in an array X of size n. At every step it is allowed to ask the value of one cell in the array, and the adversary is allowed after each such ping, to shuffle elements around in the array in any way it seems fit. For the best possible deterministic algorithm the number of rounds it has to play this game till it finds the required element is (A) O(1) (B) O(n) (C) O(n log n) (D) O(n 2 ) (E). Alexandra (UIUC) CS Fall / 54
49 Ping and find randomized. Consider an algorithm randfind that is trying to find an element in an array X of size n. At every step it asks the value of one random cell in the array, and the adversary is allowed after each such ping, to shuffle elements around in the array in any way it seems fit. This algorithm would stop in expectation after steps. (A) O(1) (B) O(log n) (C) O(n) (D) O(n 2 ) (E). Alexandra (UIUC) CS Fall / 54
50 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
51 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
52 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
53 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
54 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
55 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
56 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
57 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
58 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
59 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
60 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
61 Massive randomness.. Is not that random. Consider flipping a fair coin n times independently, head given 1, tail gives zero. How many heads?...we get a binomial distribution. Alexandra (UIUC) CS Fall / 54
62 Massive randomness.. Is not that random. This is known as concentration of mass. This is a very special case of the law of large numbers. Alexandra (UIUC) CS Fall / 54
63 Side note... Law of large numbers (weakest form)... Informal statement of law of large numbers For n large enough, the middle portion of the binomial distribution looks like (converges to) the normal/gaussian distribution. Alexandra (UIUC) CS Fall / 54
64 Massive randomness.. Is not that random. Intuitive conclusion Randomized algorithm are unpredictable in the tactical level, but very predictable in the strategic level. Alexandra (UIUC) CS Fall / 54
65 Binomial distribution X n = numbers of heads when flipping a coin n times. Claim [ ] Pr X n = i = (n i) 2 n. Where: ( ) n k = n!. (n k)!k! Indeed, ( n i) is the number of ways to choose i elements out of n elements (i.e., pick which i coin flip come up heads). Each specific such possibility (say ) had probability 1/2 n. We are interested in the bad event Pr[X n n/4] (way too few heads). We are going to prove this probability is tiny. Alexandra (UIUC) CS Fall / 54
66 Binomial distribution Playing around with binomial coefficients Lemma n! (n/e) n. Proof. n n n! i=0 n i i! = en, by the Taylor expansion of e x = x i i=0. This implies that i! (n/e) n n!, as required. Alexandra (UIUC) CS Fall / 54
67 Binomial distribution Playing around with binomial coefficients Lemma For any k n, we have ( ) ( n k ne ) k. k Proof. ( ) n n! n(n 1)(n 2)... (n k + 1) = = k (n k)!k! k! ( ) nk k! ( nk ne k k k =. k e) since k! (k/e) k (by previous lemma). Alexandra (UIUC) CS Fall / 54
68 Binomial distribution Playing around with binomial coefficients [ Pr X n n ] = 4 n/4 2 n k=0 ( ) 1 n k 1 ( ) n 2 2 n n/4 For k n/4 the above sequence behave like a geometric variable. ( ) ( ) n n n! / = k + 1 k (k + 1)!(n k 1)! / n! (k)!(n k)! = n k k + 1 (3/4)n n/ Alexandra (UIUC) CS Fall / 54
69 Binomial distribution Playing around with binomial coefficients [ Pr X n n ] 1 ( ) n ( ) ne n/4 ( 4e n n/ n n/ n/4. ) n/4 We just proved the following theorem. Theorem Let X n be the random variable which is the number of heads when flipping an unbiased coin independently n times. Then [ Pr X n n ] [ n/4 and Pr X n 3n ] n/ Alexandra (UIUC) CS Fall / 54
70 Flipping a coin. If you flip an unbiased coin 1000 times, the probability you get at most 250 heads is at most (A) (B) (C) (D) (E) 0.1. Alexandra (UIUC) CS Fall / 54
71 Part III Randomized Quick Sort and Selection Alexandra (UIUC) CS Fall / 54
72 Randomized QuickSort Randomized QuickSort 1 Pick a pivot element uniformly at random from the array. 2 Split array into 3 subarrays: those smaller than pivot, those larger than pivot, and the pivot itself. 3 Recursively sort the subarrays, and concatenate them. Alexandra (UIUC) CS Fall / 54
73 Example 1 array: 16, 12, 14, 20, 5, 3, 18, 19, 1 Alexandra (UIUC) CS Fall / 54
74 Analysis via Recurrence 1 Given array A of size n, let Q(A) be number of comparisons of randomized QuickSort on A. 2 Note that Q(A) is a random variable. 3 Let A i left and Ai right be the left and right arrays obtained if: Q(A) = n + n i=1 pivot is of rank i in A. [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ). Since each element of A has probability exactly of 1/n of being chosen: n 1 ) Q(A) = n + (Q(A ileft n ) + Q(Airight ). i=1 Alexandra (UIUC) CS Fall / 54
75 Analysis via Recurrence 1 Given array A of size n, let Q(A) be number of comparisons of randomized QuickSort on A. 2 Note that Q(A) is a random variable. 3 Let A i left and Ai right be the left and right arrays obtained if: Q(A) = n + n i=1 pivot is of rank i in A. [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ). Since each element of A has probability exactly of 1/n of being chosen: n 1 ) Q(A) = n + (Q(A ileft n ) + Q(Airight ). i=1 Alexandra (UIUC) CS Fall / 54
76 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We have, for any A: n Q(A) = n + i=1 [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ) Therefore, by linearity of expectation: [ ] n [ ] pivot is ( ] E Q(A) = n + Pr of rank i E[Q(A ileft ) i=1 ]) + E [Q(A iright ). [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
77 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We have, for any A: n Q(A) = n + i=1 [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ) Therefore, by linearity of expectation: [ ] n [ ] pivot is ( ] E Q(A) = n + Pr of rank i E[Q(A ileft ) i=1 ]) + E [Q(A iright ). [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
78 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We have, for any A: n Q(A) = n + i=1 [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ) Therefore, by linearity of expectation: [ ] n [ ] pivot is ( ] E Q(A) = n + Pr of rank i E[Q(A ileft ) i=1 ]) + E [Q(A iright ). [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
79 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We have, for any A: n Q(A) = n + i=1 [ ] ) Pr pivot has rank i (Q(A ileft ) + Q(Airight ) Therefore, by linearity of expectation: [ ] n [ ] pivot is ( ] E Q(A) = n + Pr of rank i E[Q(A ileft ) i=1 ]) + E [Q(A iright ). [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
80 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We derived: [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Note that above holds for any A of size n. Therefore max E [Q(A)] = T(n) n + A: A =n n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
81 Analysis via Recurrence Let T(n) = max A: A =n E[Q(A)] be the worst-case expected running time of randomized QuickSort on arrays of size n. We derived: [ ] E Q(A) n + n 1 (T(i 1) + T(n i)). n i=1 Note that above holds for any A of size n. Therefore max E [Q(A)] = T(n) n + A: A =n n 1 (T(i 1) + T(n i)). n i=1 Alexandra (UIUC) CS Fall / 54
82 Solving the Recurrence T(n) n + with base case T(1) = 0. Lemma T(n) = O(n log n). n 1 (T(i 1) + T(n i)) n i=1 Proof. (Guess and) Verify by induction. Alexandra (UIUC) CS Fall / 54
83 Solving the Recurrence T(n) n + with base case T(1) = 0. Lemma T(n) = O(n log n). n 1 (T(i 1) + T(n i)) n i=1 Proof. (Guess and) Verify by induction. Alexandra (UIUC) CS Fall / 54
84 Solving the Recurrence T(n) n + with base case T(1) = 0. Lemma T(n) = O(n log n). n 1 (T(i 1) + T(n i)) n i=1 Proof. (Guess and) Verify by induction. Alexandra (UIUC) CS Fall / 54
85 Notes Alexandra (UIUC) CS Fall / 54
86 Notes Alexandra (UIUC) CS Fall / 54
87 Notes Alexandra (UIUC) CS Fall / 54
88 Notes Alexandra (UIUC) CS Fall / 54
89 Hoare, C. A. R. (1962). Quicksort. Comput. J., 5(1): Alexandra (UIUC) CS Fall / 54
Randomized algorithms
Randomized algorithms March 10, 2005 1 What are randomized algorithms? Algorithms which use random numbers to make decisions during the executions of the algorithm. Why would we want to do this?? Deterministic
Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 10 Introduction to Discrete Probability Probability theory has its origins in gambling analyzing card games, dice,
The Binomial Distribution
The Binomial Distribution James H. Steiger November 10, 00 1 Topics for this Module 1. The Binomial Process. The Binomial Random Variable. The Binomial Distribution (a) Computing the Binomial pdf (b) Computing
6.3 Conditional Probability and Independence
222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted
Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80)
Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you
Question: What is the probability that a five-card poker hand contains a flush, that is, five cards of the same suit?
ECS20 Discrete Mathematics Quarter: Spring 2007 Instructor: John Steinberger Assistant: Sophie Engle (prepared by Sophie Engle) Homework 8 Hints Due Wednesday June 6 th 2007 Section 6.1 #16 What is the
Lecture 13 - Basic Number Theory.
Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted
Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 13. Random Variables: Distribution and Expectation
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 3 Random Variables: Distribution and Expectation Random Variables Question: The homeworks of 20 students are collected
Lecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
E3: PROBABILITY AND STATISTICS lecture notes
E3: PROBABILITY AND STATISTICS lecture notes 2 Contents 1 PROBABILITY THEORY 7 1.1 Experiments and random events............................ 7 1.2 Certain event. Impossible event............................
Mathematical Induction. Lecture 10-11
Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach
2.1 Complexity Classes
15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look
CSE 326, Data Structures. Sample Final Exam. Problem Max Points Score 1 14 (2x7) 2 18 (3x6) 3 4 4 7 5 9 6 16 7 8 8 4 9 8 10 4 Total 92.
Name: Email ID: CSE 326, Data Structures Section: Sample Final Exam Instructions: The exam is closed book, closed notes. Unless otherwise stated, N denotes the number of elements in the data structure
Continued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs
CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like
FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL
FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint
SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH
31 Kragujevac J. Math. 25 (2003) 31 49. SHARP BOUNDS FOR THE SUM OF THE SQUARES OF THE DEGREES OF A GRAPH Kinkar Ch. Das Department of Mathematics, Indian Institute of Technology, Kharagpur 721302, W.B.,
CSC2420 Fall 2012: Algorithm Design, Analysis and Theory
CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms
Reading 13 : Finite State Automata and Regular Expressions
CS/Math 24: Introduction to Discrete Mathematics Fall 25 Reading 3 : Finite State Automata and Regular Expressions Instructors: Beck Hasti, Gautam Prakriya In this reading we study a mathematical model
Math 55: Discrete Mathematics
Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What
1 Review of Newton Polynomials
cs: introduction to numerical analysis 0/0/0 Lecture 8: Polynomial Interpolation: Using Newton Polynomials and Error Analysis Instructor: Professor Amos Ron Scribes: Giordano Fusco, Mark Cowlishaw, Nathanael
Lecture 1: Course overview, circuits, and formulas
Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik
Triangle deletion. Ernie Croot. February 3, 2010
Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,
MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
Notes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
ALGEBRA. sequence, term, nth term, consecutive, rule, relationship, generate, predict, continue increase, decrease finite, infinite
ALGEBRA Pupils should be taught to: Generate and describe sequences As outcomes, Year 7 pupils should, for example: Use, read and write, spelling correctly: sequence, term, nth term, consecutive, rule,
Mathematical goals. Starting points. Materials required. Time needed
Level S2 of challenge: B/C S2 Mathematical goals Starting points Materials required Time needed Evaluating probability statements To help learners to: discuss and clarify some common misconceptions about
NOTES ON LINEAR TRANSFORMATIONS
NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all
The positive minimum degree game on sparse graphs
The positive minimum degree game on sparse graphs József Balogh Department of Mathematical Sciences University of Illinois, USA jobal@math.uiuc.edu András Pluhár Department of Computer Science University
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010. Class 4 Nancy Lynch
6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 4 Nancy Lynch Today Two more models of computation: Nondeterministic Finite Automata (NFAs)
Lecture 22: November 10
CS271 Randomness & Computation Fall 2011 Lecture 22: November 10 Lecturer: Alistair Sinclair Based on scribe notes by Rafael Frongillo Disclaimer: These notes have not been subjected to the usual scrutiny
Bayesian Tutorial (Sheet Updated 20 March)
Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that
Chapter 6: Episode discovery process
Chapter 6: Episode discovery process Algorithmic Methods of Data Mining, Fall 2005, Chapter 6: Episode discovery process 1 6. Episode discovery process The knowledge discovery process KDD process of analyzing
Lecture 2: Universality
CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal
SECTION 10-2 Mathematical Induction
73 0 Sequences and Series 6. Approximate e 0. using the first five terms of the series. Compare this approximation with your calculator evaluation of e 0.. 6. Approximate e 0.5 using the first five terms
9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1
9th Max-Planck Advanced Course on the Foundations of Computer Science (ADFOCS) Primal-Dual Algorithms for Online Optimization: Lecture 1 Seffi Naor Computer Science Dept. Technion Haifa, Israel Introduction
6.042/18.062J Mathematics for Computer Science. Expected Value I
6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you
Introduction to Algorithms. Part 3: P, NP Hard Problems
Introduction to Algorithms Part 3: P, NP Hard Problems 1) Polynomial Time: P and NP 2) NP-Completeness 3) Dealing with Hard Problems 4) Lower Bounds 5) Books c Wayne Goddard, Clemson University, 2004 Chapter
1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
Guessing Game: NP-Complete?
Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple
Quotient Rings and Field Extensions
Chapter 5 Quotient Rings and Field Extensions In this chapter we describe a method for producing field extension of a given field. If F is a field, then a field extension is a field K that contains F.
Notes from Week 1: Algorithms for sequential prediction
CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking
Econometrics Simple Linear Regression
Econometrics Simple Linear Regression Burcu Eke UC3M Linear equations with one variable Recall what a linear equation is: y = b 0 + b 1 x is a linear equation with one variable, or equivalently, a straight
14.1 Rent-or-buy problem
CS787: Advanced Algorithms Lecture 14: Online algorithms We now shift focus to a different kind of algorithmic problem where we need to perform some optimization without knowing the input in advance. Algorithms
Lecture Note 1 Set and Probability Theory. MIT 14.30 Spring 2006 Herman Bennett
Lecture Note 1 Set and Probability Theory MIT 14.30 Spring 2006 Herman Bennett 1 Set Theory 1.1 Definitions and Theorems 1. Experiment: any action or process whose outcome is subject to uncertainty. 2.
Data Structures Fibonacci Heaps, Amortized Analysis
Chapter 4 Data Structures Fibonacci Heaps, Amortized Analysis Algorithm Theory WS 2012/13 Fabian Kuhn Fibonacci Heaps Lacy merge variant of binomial heaps: Do not merge trees as long as possible Structure:
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints
Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2
CS 70 Discrete Mathematics and Probability Theory Fall 2009 Satish Rao, David Tse Note 2 Proofs Intuitively, the concept of proof should already be familiar We all like to assert things, and few of us
11 Multivariate Polynomials
CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6
Mathematical Induction
Mathematical Induction (Handout March 8, 01) The Principle of Mathematical Induction provides a means to prove infinitely many statements all at once The principle is logical rather than strictly mathematical,
Mathematical Induction. Mary Barnes Sue Gordon
Mathematics Learning Centre Mathematical Induction Mary Barnes Sue Gordon c 1987 University of Sydney Contents 1 Mathematical Induction 1 1.1 Why do we need proof by induction?.... 1 1. What is proof by
Probabilistic Strategies: Solutions
Probability Victor Xu Probabilistic Strategies: Solutions Western PA ARML Practice April 3, 2016 1 Problems 1. You roll two 6-sided dice. What s the probability of rolling at least one 6? There is a 1
I. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
Analysis of Algorithms, I
Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications
MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform
MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish
COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012
Binary numbers The reason humans represent numbers using decimal (the ten digits from 0,1,... 9) is that we have ten fingers. There is no other reason than that. There is nothing special otherwise about
(67902) Topics in Theory and Complexity Nov 2, 2006. Lecture 7
(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition
Chapter 3. Cartesian Products and Relations. 3.1 Cartesian Products
Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing
4.5 Linear Dependence and Linear Independence
4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system
1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables
THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE. Alexander Barvinok
THE NUMBER OF GRAPHS AND A RANDOM GRAPH WITH A GIVEN DEGREE SEQUENCE Alexer Barvinok Papers are available at http://www.math.lsa.umich.edu/ barvinok/papers.html This is a joint work with J.A. Hartigan
6.207/14.15: Networks Lecture 15: Repeated Games and Cooperation
6.207/14.15: Networks Lecture 15: Repeated Games and Cooperation Daron Acemoglu and Asu Ozdaglar MIT November 2, 2009 1 Introduction Outline The problem of cooperation Finitely-repeated prisoner s dilemma
INCIDENCE-BETWEENNESS GEOMETRY
INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full
WRITING PROOFS. Christopher Heil Georgia Institute of Technology
WRITING PROOFS Christopher Heil Georgia Institute of Technology A theorem is just a statement of fact A proof of the theorem is a logical explanation of why the theorem is true Many theorems have this
Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Probability: Terminology and Examples Class 2, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom 1 Learning Goals 1. Know the definitions of sample space, event and probability function. 2. Be able to
Connectivity and cuts
Math 104, Graph Theory February 19, 2013 Measure of connectivity How connected are each of these graphs? > increasing connectivity > I G 1 is a tree, so it is a connected graph w/minimum # of edges. Every
Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain
Lectures notes on orthogonal matrices (with exercises) 92.222 - Linear Algebra II - Spring 2004 by D. Klain 1. Orthogonal matrices and orthonormal sets An n n real-valued matrix A is said to be an orthogonal
Just the Factors, Ma am
1 Introduction Just the Factors, Ma am The purpose of this note is to find and study a method for determining and counting all the positive integer divisors of a positive integer Let N be a given positive
. 0 1 10 2 100 11 1000 3 20 1 2 3 4 5 6 7 8 9
Introduction The purpose of this note is to find and study a method for determining and counting all the positive integer divisors of a positive integer Let N be a given positive integer We say d is a
Random variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.
Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()
Applied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
Polynomial Invariants
Polynomial Invariants Dylan Wilson October 9, 2014 (1) Today we will be interested in the following Question 1.1. What are all the possible polynomials in two variables f(x, y) such that f(x, y) = f(y,
Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics
Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights
1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let
Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as
1 The Line vs Point Test
6.875 PCP and Hardness of Approximation MIT, Fall 2010 Lecture 5: Low Degree Testing Lecturer: Dana Moshkovitz Scribe: Gregory Minton and Dana Moshkovitz Having seen a probabilistic verifier for linearity
160 CHAPTER 4. VECTOR SPACES
160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results
ST 371 (IV): Discrete Random Variables
ST 371 (IV): Discrete Random Variables 1 Random Variables A random variable (rv) is a function that is defined on the sample space of the experiment and that assigns a numerical variable to each possible
COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH. 1. Introduction
COMBINATORIAL PROPERTIES OF THE HIGMAN-SIMS GRAPH ZACHARY ABEL 1. Introduction In this survey we discuss properties of the Higman-Sims graph, which has 100 vertices, 1100 edges, and is 22 regular. In fact
Solving Linear Systems, Continued and The Inverse of a Matrix
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing
SYMMETRIC ENCRYPTION. Mihir Bellare UCSD 1
SYMMETRIC ENCRYPTION Mihir Bellare UCSD 1 Syntax A symmetric encryption scheme SE = (K,E,D) consists of three algorithms: K and E may be randomized, but D must be deterministic. Mihir Bellare UCSD 2 Correct
SCORE SETS IN ORIENTED GRAPHS
Applicable Analysis and Discrete Mathematics, 2 (2008), 107 113. Available electronically at http://pefmath.etf.bg.ac.yu SCORE SETS IN ORIENTED GRAPHS S. Pirzada, T. A. Naikoo The score of a vertex v in
Chapter 6. 1. What is the probability that a card chosen from an ordinary deck of 52 cards is an ace? Ans: 4/52.
Chapter 6 1. What is the probability that a card chosen from an ordinary deck of 52 cards is an ace? 4/52. 2. What is the probability that a randomly selected integer chosen from the first 100 positive
Systems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
Why? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
Chapter 4 Lecture Notes
Chapter 4 Lecture Notes Random Variables October 27, 2015 1 Section 4.1 Random Variables A random variable is typically a real-valued function defined on the sample space of some experiment. For instance,
MATH 289 PROBLEM SET 4: NUMBER THEORY
MATH 289 PROBLEM SET 4: NUMBER THEORY 1. The greatest common divisor If d and n are integers, then we say that d divides n if and only if there exists an integer q such that n = qd. Notice that if d divides
Near Optimal Solutions
Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.
THE DIMENSION OF A VECTOR SPACE
THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field
Math 4310 Handout - Quotient Vector Spaces
Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
CSE 135: Introduction to Theory of Computation Decidability and Recognizability
CSE 135: Introduction to Theory of Computation Decidability and Recognizability Sungjin Im University of California, Merced 04-28, 30-2014 High-Level Descriptions of Computation Instead of giving a Turing
Feb 7 Homework Solutions Math 151, Winter 2012. Chapter 4 Problems (pages 172-179)
Feb 7 Homework Solutions Math 151, Winter 2012 Chapter Problems (pages 172-179) Problem 3 Three dice are rolled. By assuming that each of the 6 3 216 possible outcomes is equally likely, find the probabilities
8.1 Min Degree Spanning Tree
CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree
HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!
Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following
SOLUTIONS TO EXERCISES FOR. MATHEMATICS 205A Part 3. Spaces with special properties
SOLUTIONS TO EXERCISES FOR MATHEMATICS 205A Part 3 Fall 2008 III. Spaces with special properties III.1 : Compact spaces I Problems from Munkres, 26, pp. 170 172 3. Show that a finite union of compact subspaces
7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
arxiv:1112.0829v1 [math.pr] 5 Dec 2011
![arxiv:1112.0829v1 [math.pr] 5 Dec 2011](/thumbs/39/18839218.jpg "arxiv:1112.0829v1 [math.pr] 5 Dec 2011")
How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly
Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)
Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse