# Lecture 1: Course overview, circuits, and formulas

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik Kopparty Website: sk1233/courses/topics-s13/ Good reference books worth flipping through: Jukna: Extremal Combinatorics Jukna: Boolean Function Complexity Wegener: Complexity of Boolean Functions Vadhan: Pseudorandomness Luby and Wigderson: Pairwise Independent Hashing and Applications 2 Course Overview 2.1 Circuits and Formulas A typical setting is that there is a function f : {0, 1} n {0, 1} we want to compute. resources. We will be studying whether this function can be computed using little Definition 1. A boolean circuit is a directed acyclic graph with wires carrying some value in {0, 1}, and these values are processed by gates computing certain prespecified boolean operations. Example 2. For example, using the gates AND ( ), OR ( ) and NOT ( ), the function: f(x 1, x 2, x 3 ) = (x 1 x 2 ) x 3 is representable by a circuit with 3 gates. Definition 3. A formula is a circuit where each gate has fan-out at most 1. Formulas represent computation where the results of subcomputations cannot used more than once. 1

2 2.2 Branching Programs Branching programs are used to model low-space computation. We will see some interesting results about the power of branching programs. We will also come across them when we study pseudorandomness. 2.3 Data Structures Data Structures are schemes for storing data in a way that we can answer questions about the data quickly. A sample problem is to design a data structure for the SET MEMBERSHIP problem: We are given a set S [m] with S = n. We want to store something compactly in memory so that we can answer questions of the form Is i S? (where i [m]) by probing very few locations in memory. There is a trivial data structure that uses m bits of memory and allows us to answer questions with just 1 probe to memory: Let the i th bit of memory equal 1 iff i S. Then to answer whether i S, we check the value of the i th bit. This algorithm achieves minimum runtime, but requires a lot of space (m bits). What if we want to save as much space as possible? If we use s bits of space, then what is stored in memory can take on at most 2 s possible values. Now since what is stored in memory should suffice to answer all the questions is i S?, what is stored in memory should determine S. Thus 2 s ( ( m n) and so the space required is at least m log2 n) bits. This is because the query determines whether i S for all i [m], so the query actually determines the subset S. The lower bound on the space requirement then follows from the fact that there are ( m n) n-element subsets of [m]. A simple algorithm ( that achieves the lower bound on space maps each choice of S to a unique sequence of log m ) n bits. The algorithm simply reads off the bits and uses the map to find the corresponding set S and answer the query. This algorithm uses the minimal amount of space ( log ( m n) bits), but has a large query time of log ( m n). These algorithms seem to imply that there is a tradoff between space and query time. However, we will see an algorithm that uses log ( m n) (1 + o(1)) bits with a query time of O(1). 2.4 Pseudorandomness The other major theme of the class is pseudorandomness, or the generation of randomness. What is randomness? How do we get it? How do we know when we ve gotten it? The big insight that starts the theory is that when we want randomness, it is for some purpose. We then define pseudorandomness to be whatever suffices for that purpose. Definition 4. A sequence of bits is pseudorandom if no algorithm of interest can distinguish between it and truly random bits. We will be much more concrete when the time comes, but this is the gist of it. A good example of a randomized algorithms comes from graph connectedness. Given an undirected graph G on n vertices, is G connected? Suppose only O(log n) space is allowed. It is not obvious 2

3 how to do this at all. The classical breadth-first-search (BFS) takes Ω(n) space. There is a beautiful classical low-space algorithm for this problem: For every pair of vertices (s, t), do: 1. Start a random walk on G starting at s. 2. Run it for n 10 steps. 3. If you never see t, say DISCONNECTED. If you never said DISCONNECTED, say CONNECTED. On every graph, this randomized algorithm outputs the correct answer with probability greater than An amazing result of Reingold shows how to make this algorithm deterministic. It involves taking a pseudorandom walk. We will prove this result in the course. 2.5 Hardness vs Randomness If a problem is hard, then you can t guess the answer, and so the answer is random to you. Perhaps you can then use the answer to that problem as the random bits to feed into a randomized algorithm. The theory of hardness versus randomness makes this rigorous. Roughly: if you can give me f that is hard for some computational model C, then you can get pseudorandom generators that generate bits that look random to C. We will prove versions of this and use it to get actual pseudorandom generators for some interesting computational models. 2.6 Concrete pseudorandomness I will just mention some words: Expander graphs ɛ-biased sets k-wise independent sets randomness extractors 3 Circuits and Formulas Let f : {0, 1} n {0, 1}. Recall that a circuit is built from wires carrying 0 or 1 and gates. There are two common bases for the gates: the full binary basis and the de Morgan basis. The de Morgan basis consists of (AND), (OR), and (NOT). The full binary basis consists of gates computing all 16 functions b : {0, 1} 2 {0, 1}, and includes the de Morgan basis. Note that all basis elements have fan-in 2 (number of inputs into gate). The fan-out, the number of outputs out of a gate, is unrestricted in circuits. In formulas, the fan-out for each gate is 1. 3

4 Observation 5. Every function has a formula. To see this, note that for each y {0, 1} n, there is a δ y : {0, 1} n {0, 1}, which is expressible as a of n variables and complements of variables, such that δ y (x) = 1 iff x = y. Writing completes the proof. f(x) = y {0,1} n f(y)=1 δ y (x) We can use this proof to get an upper bound on the number of gates of f: gates(f) (n 1)(2 n 1) O(n 2 n ). 3.1 Circuit size basics Maybe every function has a circuit of size poly(n)? This is ruled out by the following fundamental result. Theorem 6 (Shannon). ɛ > 0, most functions f : {0, 1} n {0, 1} require circuits of size at least 2 n n (1 ɛ). Proof. In 1 line: There are lots of functions, but not too many small circuits. The total number of functions is 2 2n. Suppose we consider circuits of size (number of gates) at most s. Each gate takes exactly two inputs that are either other gates or input bits. So there are at most ( ) s+n 2 ways to choose inputs for each gate. Since there are 16 binary functions in the full binary basis, this gives at most 16 ( ) s+n 2 ways to choose a gate function and its inputs. Since there are at most s gates, the number of circuits of size s is at most 16 s( ) s+n s. 2 Set s = 2n n (1 ɛ). We want to show that the number of circuits of size at most s is much less than the total number of functions. ( ) s + n s 16 s 2 16 s (s + n) s 16 s (2s) s = 32 s s s ) 2 n n (1 ɛ) (1 ɛ) ( = 32 2n 2 n n (1 ɛ) n ( ) 2 n 32(1 ɛ) n (1 ɛ) = 2 2n (1 ɛ) n < 2 2n (1 ɛ) 2 2n. 4

5 Theorem 7 (Shannon and Lupanov). ɛ > 0, functions f : {0, 1} n {0, 1}, circuits of size at most 2n n (1 + ɛ). We will only prove an upper bound of O( 2n n ). Proof. First, we find a circuit of size O(2 n ) using the recurrence: f(x 1,..., x n ) = (x 1 f(1, x 2,..., x n )) (x 1 f(0, x 2,..., x n )). size(n) size(n 1) size(n) = O(2 n ). More explicitly, this gives us a sequence of representations of the function, in terms of residual functions on fewer bits which we obtain by fixing the first few input bits to certain values: as f(x 1,..., x n ) = (x 1 f(1, x 2,..., x n )) (x 1 f(0, x 2,..., x n )). f(x 1,..., x n ) =(x 1 x 2 f(1, 1, x 3,..., x n )) (x 1 x 2 f(0, 1, x 3,..., x n )) (x 1 x 2 f(1, 0, x 3,..., x n )) (x 1 x 2 f(0, 0, x 3,..., x n )). To get a circuit of size O( 2n n ), we stop the recursion after t steps (t to be determined later). At this stage of the recurrence, for each (b 1,..., b t ) {0, 1} t we need to compute f(b 1,..., b t, x t+1,..., x n ), thought of as a function on the last n t bits. Now the total number of functions on the variables x t+1,..., x n is 2 2n t. The idea is this: if 2 2n t 2 t, then it is more economical to compute all the 2 2n t functions once beforehand, rather than to compute the residual function for each of the 2 t branches of the recursion. So we compute all 2 2n t functions on x t+1,..., x n and feed them to the appropriate gates of the representation of f using t steps of the recursion. Call this circuit C. Then size(c) 3 2 t + size required to compute all functions on n t bits. Claim 8. Size required to compute all functions on k bits 2 2k (1 + o(1)). We prove the claim by induction on k. We first compute all functions on k 1 bits, which takes size 2 2k 1. For each f 1 and f 2 in this class, compute (x k f 1 (1, x 2,..., x k 1 )) (x k f 2 (0, x 2,..., x k 1 )). This gives all functions on k bits. Hence, size(k) This completes the proof of the claim. ( 2 2k 1) k k 1 = 2 2k (1 + o(1)). So size(c) 3 2 t + 2 2n t. Set t = n log(n log n). Then 5

6 size(c) n log(n log n) 2log(n log n) + 2 = 3 2 n n log n + 2n log n 2 n = 3 n log n + 2n n ( ) 2 n = O. n The best known lower bound on circuit size for an explicit function is only 5n. Yet we know that there are functions out there (in fact most functions out there) whose smallest circuit has exponential size. One of the biggest problems of complexity theory is to find an explicit such function. 4 Formula size basics We saw earlier that every function of size n can be computed by a circuit of size (1 + ɛ)2 n /n, and that relatively few functions can be computed by a circuit of size less than (1 ɛ)2 n /n. The corresponding tight lower bound on the size of formulas for most functions is 2 n / log n. Theorem 9 (Shannon). Most boolean functions on n variables cannot be computed by a formula of size less than (1 ɛ)2 n / log n, for any ɛ > 0. Theorem 10 (Lyupanov). Every boolean function on n variables can be computed by a formula of size O(2 n / log n). These theorems hold in both the full binary basis and the de Morgan basis. Theorem 9 can be proved by the following counting argument; we won t see a proof of 10. Proof. We will compare a lower bound on the number of formulas of size s to the number of functions of size n. A formula of size s can be written as a binary tree with s internal nodes. Each internal node is labeled by a gate, and each leaf is labeled by a variable, the complement of a variable, 0, or 1. The number of binary trees with s internal nodes is given by the Catalan number C s 4 s. Each internal node is labeled with one of the gates in our basis; if there are b gates in the basis, the number of ways to do this labeling is b s ; for any basis, this is at most 16 s, for the full binary basis. There are (2n + 2) ways to label each leaf in the tree, and there are s + 1 leaves. Thus, the total number of formulas of size s is less than (2n + 2) s+1 4 s 16 s n s c s for some constant c > 1. The number of functions from (0, 1) n to (0, 1) is 2 2n. 6

7 If s (1 ɛ)2 n / log n for any ɛ > 0, then n s c s << 2 2n. Thus, only a small fraction of all the boolean functions on n variables can be computed by any formula of size (1 ɛ)2 n / log n. 5 The complexity class NC 1 The class of functions computable by a formula of size poly(n) is the same as the class of functions computable by a formula of size poly(n) and depth O(log n). Theorem 11. Every formula of size s can be converted into an equivalent formula of depth O(log s). Proof. To prove this theorem, we need the following lemma: Lemma 12. In any rooted binary tree with s vertices, there is a vertex g such that the tree rooted at g has between s/3 and 2s/3 vertices. Proof. A vertex that is the root of a subtree with more than 2s/3 vertices has a child that is the root of a subtree with at least s/3 vertices. We can use this fact to design a simple algorithm to find a vertex that is the root of a subtree with between s/3 and 2s/3 vertices. Start with the root as the current vertex. Select the child of the current vertex that has at least s/3 vertices in its subtree. If this vertex has at most 2s/3 vertices, we re done. Otherwise, set this vertex as the current vertex, and repeat the process. Now we ll use this lemma to prove theorem 11 The proof will be by induction; the base case is trivial. The inductive hypothesis is that for any s < s every formula of size s can be expressed as a formula of depth c log s + d (for constants c, d). Given a formula F, we can use the lemma to find a sub-formula F g with size in the range [s/3, 2s/3]. We can also find two formulas F 0 and F 1, which are F with the sub-formula F g replaced by 0 and 1, respectively. These formulas also have size in the range [s/3, 2s/3]. It is easy to check that F = (F g F 1 ) ( F g F 0 ). Thus, by applying the inductive hypothesis, we can replace F with an equivalent formula of depth 2 + c log(2s/3) + d c log(s) + d (for suitable c, d). The largest possible size of a formula of depth d is 2 d. Thus, a formula with logarithmic depth must have polynomial size. Theorem 11 shows that all formulas with size O(n c ) have equivalent formulas with size O(n c ) and depth O(log n). Thus, there is a single complexity class that captures all functions that have formulas with polynomial size, and all functions that have formulas with polynomial size and logarithmic depth. This is the class NC 1. If we begin with a circuit with the fan-in bounded by 2 and depth d, an upper bound on the size of the formula computing the function would be 2 d. To do this, simply replace any sub-circuit having more than one output with several copies of that sub-circuit, each having a single output. 7

8 Thus, NC 1 also equals the class of all functions that have circuits of polynomial size and logarithmic depth. 6 Upper bounds on formula size for specific functions Several specific problems are known to be in NC 1. PARITY is the family of boolean functions that evalute to 0 if the sum of the inputs is even and to 1 if the sum of inputs is odd. Theorem 13. PARITY NC 1. Proof. In the full binary basis, we can calculate the parity function on {0, 1} n as PARITY{0, 1} a PARITY{0, 1} n a. Thus, we can use binary divide and conquer to calculate the parity function with a formula of size O(n). In the de Morgan basis, x 1 x 2 can be expressed as (x 1 x 2 ) (x 1 x 2 ). Using this substitution, we can calculate the parity function with a circuit of size O(n). In order to convert this circuit to a formula, we need to duplicate each input to the xor gate so that no gate has more than one output. Let F (n) be the minimal size of a formula, counting only or and and gates, for PARITY of n bits in the de Morgan basis. Using the construction described in the previous paragraph, F (n) 3 + 2F ( n/2 ) + 2F ( n/2 ). Thus, F (n) O(n 2 ). We can also do operations on integers with small formulas. All integers are assumed to be nonnegative and written in base 2. We first show that we can add two n bit integers using a formula of size poly(n). Theorem 14. Integer addition is in NC 1. Proof. We ll assume binary inputs and use the full binary basis for the formula we build. If we can compute the i th carry bit c i, then we can compute the i th bit of the solution as s i = x i y i c i, where x i and y i are the i th bits of the input. The last bit of the solution will be s n+1 = c n+1. The i th bit of the carry will be 1 iff there exists a j < i such that x j = y j = 1, and for all j < k < i it is not the case that x k = y k = 0. This condition is directly expressible as a formula: c i = x j y j (x j y j ). j<i j<k<i 8

9 i 1 The number of gates needed to compute the carry is thus n+1 i=1 j=1 (3 + i 1 k=j+1 2) O(n3 ). The number of additional gates required to calculate the solution from the carry and the inputs is O(n), so addition of two n bit numbers can be performed by a formula in the full binary basis with O(n 3 ) gates. It is also possible to add n numbers, each n bits long, using a formula of size poly(n). Theorem 15. Iterated integer addition is in NC 1. Proof. For any three integers, X,Y, and Z, we will show how to find two numbers S and C such that X + Y + Z = S + C using a formula with constant depth. Using this procedure, we can reduce the problem of adding n integers to the problem of adding 2 integers with a formula with depth O(log 3/2 n). Finally we can add two integers using the previous theorem. For an integer A, let A i denote its i th bit (corresponding to the coefficient of 2 i ). The trick is the following: define S i = X i Y i Z i, and define C i+1 = Majority(X i, Y i, Z i ). These are the sum and carry respectively when we add the three bits X i, Y i, Z i. It is easy to check that S + C = X + Y + Z. Corollary 16. Integer multiplication is in NC 1. MAJ is the family of boolean functions on an odd number of input bits that outputs the majority value of its input bits. Theorem 17. MAJ NC 1. This follows immediately from theorem 16, since we can compute the sum of the bits of the input (treated as 1-bit integers) and compare it to n/2 using a formula of polynomial size. 7 Lower Bounds for Formulas A few specific, non-trivial lower bounds are known for formulas. Krapchenko proved that parity requires a formula of size Ω(n 2 ) in the de Morgan basis. Subbotovskaya and Andreev found explicit functions having no formulas of size o(n 2.5 ) in the de Morgan basis. Nechiporuk proved an Ω(n 2 ) lower bound in the full binary basis. Håstad proved an Ω(n 3 ) lower bound for an explicit function; we will not cover this result in detail. 7.1 Krapchenko s Lower Bound Krapchenko s method uses formal complexity measures. A formal complexity measure F C assigns real numbers to boolean functions such that it has the following properties: 1. F C(x i ) = F C(f) = F C( f). 9

10 3. F C(f g) F C(f) + F C(g). Properties 2 and 3 together imply 4. F C(f g) F C(f) + F C(g). Formal complexity measures are a powerful tool to prove lower bounds in the de Morgan basis. Let L(f) be the formula complexity of f; that is, the size of the smallest formula in the demorgan basis for the function f. Theorem 18. Any formal complexity measure F C satisfies F C(f) L(f). Proof. We will proceed by induction on L(f). The base case L(f) = 1 follows from property 1. Consider the smallest formula for f. If f = f 1 f 2, then L(f) = L(f 1 ) + L(f 2 ) F C(f 1 ) + F C(f 2 ) F C(f 1 f 2 ) = F C(f). The case f = f 1 f 2 follows in the same manner, using property 4 rather than property 3. Theorem 19 (Krapchenko). PARITY requires a formula of size Ω(n 2 ) in the de Morgan basis. We will prove this theorem in the next class. The main tool is the following formal complexity measure: F C(f) = max A f 1 (0) B f 1 (1) e(a, B) 2, A B where e(a, B) is the number of pairs (x, y) A B such that x and y differ in exactly one coordinate. In the next class we will show that it is indeed a formal complexity measure. For PARITY, let us show that F C(P ARIT Y ) Ω(n 2 ). Consider A = f 1 (0) and B = f 1 (1). Then A = B = 2 n 1, and e(a, B) = n2 n 1. So F C(P ARIT Y ) n 2, and so L(P ARIT Y ) n 2. 10

### Lecture 4: AC 0 lower bounds and pseudorandomness

Lecture 4: AC 0 lower bounds and pseudorandomness Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: Jason Perry and Brian Garnett In this lecture,

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

### Outline Introduction Circuits PRGs Uniform Derandomization Refs. Derandomization. A Basic Introduction. Antonis Antonopoulos.

Derandomization A Basic Introduction Antonis Antonopoulos CoReLab Seminar National Technical University of Athens 21/3/2011 1 Introduction History & Frame Basic Results 2 Circuits Definitions Basic Properties

### Chapter 4. Trees. 4.1 Basics

Chapter 4 Trees 4.1 Basics A tree is a connected graph with no cycles. A forest is a collection of trees. A vertex of degree one, particularly in a tree, is called a leaf. Trees arise in a variety of applications.

### GRAPH THEORY LECTURE 4: TREES

GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection

### Recurrence Relations

Recurrence Relations Introduction Determining the running time of a recursive algorithm often requires one to determine the big-o growth of a function T (n) that is defined in terms of a recurrence relation.

### Lecture 2: The Complexity of Some Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

### CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992

CS268: Geometric Algorithms Handout #5 Design and Analysis Original Handout #15 Stanford University Tuesday, 25 February 1992 Original Lecture #6: 28 January 1991 Topics: Triangulating Simple Polygons

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### 4.4 The recursion-tree method

4.4 The recursion-tree method Let us see how a recursion tree would provide a good guess for the recurrence = 3 4 ) Start by nding an upper bound Floors and ceilings usually do not matter when solving

### 8.1 Min Degree Spanning Tree

CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

Answers to Homework 1 p. 13 1.2 3 What is the smallest value of n such that an algorithm whose running time is 100n 2 runs faster than an algorithm whose running time is 2 n on the same machine? Find the

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Sample Problems in Discrete Mathematics

Sample Problems in Discrete Mathematics This handout lists some sample problems that you should be able to solve as a pre-requisite to Computer Algorithms Try to solve all of them You should also read

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### Depth-Independent Lower Bounds on the Communication Complexity of Read-Once Boolean Formulas

Depth-Independent Lower Bounds on the Communication Complexity of Read-Once Boolean Formulas Rahul Jain 1, Hartmut Klauck 2, and Shengyu Zhang 3 1 Centre for Quantum Technologies and Department of Computer

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### 1. What s wrong with the following proofs by induction?

ArsDigita University Month : Discrete Mathematics - Professor Shai Simonson Problem Set 4 Induction and Recurrence Equations Thanks to Jeffrey Radcliffe and Joe Rizzo for many of the solutions. Pasted

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### Boolean Representations and Combinatorial Equivalence

Chapter 2 Boolean Representations and Combinatorial Equivalence This chapter introduces different representations of Boolean functions. It then discuss the applications of these representations for proving

### Algorithms complexity of recursive algorithms. Jiří Vyskočil, Marko Genyk-Berezovskyj

complexity of recursive algorithms Jiří Vyskočil, Marko Genyk-Berezovskyj 2010-2014 Recurrences A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.

### Lecture 5 - CPA security, Pseudorandom functions

Lecture 5 - CPA security, Pseudorandom functions Boaz Barak October 2, 2007 Reading Pages 82 93 and 221 225 of KL (sections 3.5, 3.6.1, 3.6.2 and 6.5). See also Goldreich (Vol I) for proof of PRF construction.

### Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### Lecture 7: NP-Complete Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit

### 8.1 Makespan Scheduling

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programing: Min-Makespan and Bin Packing Date: 2/19/15 Scribe: Gabriel Kaptchuk 8.1 Makespan Scheduling Consider an instance

### Chapters 1 and 5.2 of GT

Subject 2 Spring 2014 Growth of Functions and Recurrence Relations Handouts 3 and 4 contain more examples;the Formulae Collection handout might also help. Chapters 1 and 5.2 of GT Disclaimer: These abbreviated

### Arithmetic complexity in algebraic extensions

Arithmetic complexity in algebraic extensions Pavel Hrubeš Amir Yehudayoff Abstract Given a polynomial f with coefficients from a field F, is it easier to compute f over an extension R of F than over F?

### Solutions to Exercises 8

Discrete Mathematics Lent 2009 MA210 Solutions to Exercises 8 (1) Suppose that G is a graph in which every vertex has degree at least k, where k 1, and in which every cycle contains at least 4 vertices.

### Influences in low-degree polynomials

Influences in low-degree polynomials Artūrs Bačkurs December 12, 2012 1 Introduction In 3] it is conjectured that every bounded real polynomial has a highly influential variable The conjecture is known

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### MATH 105: PRACTICE PROBLEMS FOR SERIES: SPRING Problems

MATH 05: PRACTICE PROBLEMS FOR SERIES: SPRING 20 INSTRUCTOR: STEVEN MILLER (SJM@WILLIAMS.EDU).. Sequences and Series.. Problems Question : Let a n =. Does the series +n+n 2 n= a n converge or diverge?

### Lecture 14: Fourier Transform

Machine Learning: Foundations Fall Semester, 2010/2011 Lecture 14: Fourier Transform Lecturer: Yishay Mansour Scribes: Avi Goren & Oron Navon 1 14.1 Introduction In this lecture, we examine the Fourier

### Introduction to finite fields

Introduction to finite fields Topics in Finite Fields (Fall 2013) Rutgers University Swastik Kopparty Last modified: Monday 16 th September, 2013 Welcome to the course on finite fields! This is aimed at

### CS 561, Lecture 3 - Recurrences. Jared Saia University of New Mexico

CS 561, Lecture 3 - Recurrences Jared Saia University of New Mexico Recurrence Relations Oh how should I not lust after eternity and after the nuptial ring of rings, the ring of recurrence - Friedrich

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### Basic Notions on Graphs. Planar Graphs and Vertex Colourings. Joe Ryan. Presented by

Basic Notions on Graphs Planar Graphs and Vertex Colourings Presented by Joe Ryan School of Electrical Engineering and Computer Science University of Newcastle, Australia Planar graphs Graphs may be drawn

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### Mathematical Induction. Lecture 10-11

Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### CSL851: Algorithmic Graph Theory Semester I Lecture 1: July 24

CSL851: Algorithmic Graph Theory Semester I 2013-2014 Lecture 1: July 24 Lecturer: Naveen Garg Scribes: Suyash Roongta Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

### ORDERS OF ELEMENTS IN A GROUP

ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since

### Counting Falsifying Assignments of a 2-CF via Recurrence Equations

Counting Falsifying Assignments of a 2-CF via Recurrence Equations Guillermo De Ita Luna, J. Raymundo Marcial Romero, Fernando Zacarias Flores, Meliza Contreras González and Pedro Bello López Abstract

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### Partitioning edge-coloured complete graphs into monochromatic cycles and paths

arxiv:1205.5492v1 [math.co] 24 May 2012 Partitioning edge-coloured complete graphs into monochromatic cycles and paths Alexey Pokrovskiy Departement of Mathematics, London School of Economics and Political

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Mathematical Induction

Chapter 2 Mathematical Induction 2.1 First Examples Suppose we want to find a simple formula for the sum of the first n odd numbers: 1 + 3 + 5 +... + (2n 1) = n (2k 1). How might we proceed? The most natural

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### The sum of d small-bias generators fools polynomials of degree d

The sum of d small-bias generators fools polynomials of degree d Emanuele Viola April 15, 2009 Abstract We prove that the sum of d small-bias generators L : F s F n fools degree-d polynomials in n variables

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### 11 Multivariate Polynomials

CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6

### 6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III

6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these

### Lecture 2: Universality

CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### CMSC 451: Graph Properties, DFS, BFS, etc.

CMSC 451: Graph Properties, DFS, BFS, etc. Slides By: Carl Kingsford Department of Computer Science University of Maryland, College Park Based on Chapter 3 of Algorithm Design by Kleinberg & Tardos. Graphs

### Mathematical Induction

Mathematical Induction Victor Adamchik Fall of 2005 Lecture 2 (out of three) Plan 1. Strong Induction 2. Faulty Inductions 3. Induction and the Least Element Principal Strong Induction Fibonacci Numbers

### 1 Formulating The Low Degree Testing Problem

6.895 PCP and Hardness of Approximation MIT, Fall 2010 Lecture 5: Linearity Testing Lecturer: Dana Moshkovitz Scribe: Gregory Minton and Dana Moshkovitz In the last lecture, we proved a weak PCP Theorem,

### Minimum cost maximum flow, Minimum cost circulation, Cost/Capacity scaling

6.854 Advanced Algorithms Lecture 16: 10/11/2006 Lecturer: David Karger Scribe: Kermin Fleming and Chris Crutchfield, based on notes by Wendy Chu and Tudor Leu Minimum cost maximum flow, Minimum cost circulation,

### Induction. Margaret M. Fleck. 10 October These notes cover mathematical induction and recursive definition

Induction Margaret M. Fleck 10 October 011 These notes cover mathematical induction and recursive definition 1 Introduction to induction At the start of the term, we saw the following formula for computing

### Polynomial Degree and Lower Bounds in Quantum Complexity: Collision and Element Distinctness with Small Range

THEORY OF COMPUTING, Volume 1 (2005), pp. 37 46 http://theoryofcomputing.org Polynomial Degree and Lower Bounds in Quantum Complexity: Collision and Element Distinctness with Small Range Andris Ambainis

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

### Prime Numbers. Chapter Primes and Composites

Chapter 2 Prime Numbers The term factoring or factorization refers to the process of expressing an integer as the product of two or more integers in a nontrivial way, e.g., 42 = 6 7. Prime numbers are

### MATHEMATICAL INDUCTION. Mathematical Induction. This is a powerful method to prove properties of positive integers.

MATHEMATICAL INDUCTION MIGUEL A LERMA (Last updated: February 8, 003) Mathematical Induction This is a powerful method to prove properties of positive integers Principle of Mathematical Induction Let P

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### Triangle deletion. Ernie Croot. February 3, 2010

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

### APPLICATIONS OF THE ORDER FUNCTION

APPLICATIONS OF THE ORDER FUNCTION LECTURE NOTES: MATH 432, CSUSM, SPRING 2009. PROF. WAYNE AITKEN In this lecture we will explore several applications of order functions including formulas for GCDs and

### COT5405 Analysis of Algorithms Homework 3 Solutions

COT0 Analysis of Algorithms Homework 3 Solutions. Prove or give a counter example: (a) In the textbook, we have two routines for graph traversal - DFS(G) and BFS(G,s) - where G is a graph and s is any

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### Principle of (Weak) Mathematical Induction. P(1) ( n 1)(P(n) P(n + 1)) ( n 1)(P(n))

Outline We will cover (over the next few weeks) Mathematical Induction (or Weak Induction) Strong (Mathematical) Induction Constructive Induction Structural Induction Principle of (Weak) Mathematical Induction

### Math 55: Discrete Mathematics

Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What

### Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that. a = bq + r and 0 r < b.

Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that a = bq + r and 0 r < b. We re dividing a by b: q is the quotient and r is the remainder,

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### x if x 0, x if x < 0.

Chapter 3 Sequences In this chapter, we discuss sequences. We say what it means for a sequence to converge, and define the limit of a convergent sequence. We begin with some preliminary results about the

### Lecture 11: The Goldreich-Levin Theorem

COM S 687 Introduction to Cryptography September 28, 2006 Lecture 11: The Goldreich-Levin Theorem Instructor: Rafael Pass Scribe: Krishnaprasad Vikram Hard-Core Bits Definition: A predicate b : {0, 1}

### Integer multiplication

Integer multiplication Suppose we have two unsigned integers, A and B, and we wish to compute their product. Let A be the multiplicand and B the multiplier: A n 1... A 1 A 0 multiplicand B n 1... B 1 B

### COMP 250 Fall Mathematical induction Sept. 26, (n 1) + n = n + (n 1)

COMP 50 Fall 016 9 - Mathematical induction Sept 6, 016 You will see many examples in this course and upcoming courses of algorithms for solving various problems It many cases, it will be obvious that

### Two General Methods to Reduce Delay and Change of Enumeration Algorithms

ISSN 1346-5597 NII Technical Report Two General Methods to Reduce Delay and Change of Enumeration Algorithms Takeaki Uno NII-2003-004E Apr.2003 Two General Methods to Reduce Delay and Change of Enumeration

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### Cartesian Products and Relations

Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special

### Notes from Week 1: Algorithms for sequential prediction

CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking

### 13 Infinite Sets. 13.1 Injections, Surjections, and Bijections. mcs-ftl 2010/9/8 0:40 page 379 #385

mcs-ftl 2010/9/8 0:40 page 379 #385 13 Infinite Sets So you might be wondering how much is there to say about an infinite set other than, well, it has an infinite number of elements. Of course, an infinite

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### R u t c o r Research R e p o r t. Boolean functions with a simple certificate for CNF complexity. Ondřej Čepeka Petr Kučera b Petr Savický c

R u t c o r Research R e p o r t Boolean functions with a simple certificate for CNF complexity Ondřej Čepeka Petr Kučera b Petr Savický c RRR 2-2010, January 24, 2010 RUTCOR Rutgers Center for Operations

### Michal Forišek: Early beta verzia skrípt z ADŠ

Časová zložitosť II Michal Forišek: Early beta verzia skrípt z ADŠ In this part of the article we will focus on estimating the time complexity for recursive programs. In essence, this will lead to finding

### Proof: A logical argument establishing the truth of the theorem given the truth of the axioms and any previously proven theorems.

Math 232 - Discrete Math 2.1 Direct Proofs and Counterexamples Notes Axiom: Proposition that is assumed to be true. Proof: A logical argument establishing the truth of the theorem given the truth of the

### C H A P T E R. Logic Circuits

C H A P T E R Logic Circuits Many important functions are naturally computed with straight-line programs, programs without loops or branches. Such computations are conveniently described with circuits,

### p 2 1 (mod 6) Adding 2 to both sides gives p (mod 6)

.9. Problems P10 Try small prime numbers first. p p + 6 3 11 5 7 7 51 11 13 Among the primes in this table, only the prime 3 has the property that (p + ) is also a prime. We try to prove that no other

### Steven Skiena. skiena

Lecture 3: Recurrence Relations (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Argue the solution to T(n) =

### Balanced search trees

Lecture 8 Balanced search trees 8.1 Overview In this lecture we discuss search trees as a method for storing data in a way that supports fast insert, lookup, and delete operations. (Data structures handling

### Discrete Mathematics (2009 Spring) Induction and Recursion (Chapter 4, 3 hours)

Discrete Mathematics (2009 Spring) Induction and Recursion (Chapter 4, 3 hours) Chih-Wei Yi Dept. of Computer Science National Chiao Tung University April 17, 2009 4.1 Mathematical Induction 4.1 Mathematical

### CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are