# Lecture 1: Course overview, circuits, and formulas

Size: px
Start display at page:

Transcription

1 Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik Kopparty Website: sk1233/courses/topics-s13/ Good reference books worth flipping through: Jukna: Extremal Combinatorics Jukna: Boolean Function Complexity Wegener: Complexity of Boolean Functions Vadhan: Pseudorandomness Luby and Wigderson: Pairwise Independent Hashing and Applications 2 Course Overview 2.1 Circuits and Formulas A typical setting is that there is a function f : {0, 1} n {0, 1} we want to compute. resources. We will be studying whether this function can be computed using little Definition 1. A boolean circuit is a directed acyclic graph with wires carrying some value in {0, 1}, and these values are processed by gates computing certain prespecified boolean operations. Example 2. For example, using the gates AND ( ), OR ( ) and NOT ( ), the function: f(x 1, x 2, x 3 ) = (x 1 x 2 ) x 3 is representable by a circuit with 3 gates. Definition 3. A formula is a circuit where each gate has fan-out at most 1. Formulas represent computation where the results of subcomputations cannot used more than once. 1

2 2.2 Branching Programs Branching programs are used to model low-space computation. We will see some interesting results about the power of branching programs. We will also come across them when we study pseudorandomness. 2.3 Data Structures Data Structures are schemes for storing data in a way that we can answer questions about the data quickly. A sample problem is to design a data structure for the SET MEMBERSHIP problem: We are given a set S [m] with S = n. We want to store something compactly in memory so that we can answer questions of the form Is i S? (where i [m]) by probing very few locations in memory. There is a trivial data structure that uses m bits of memory and allows us to answer questions with just 1 probe to memory: Let the i th bit of memory equal 1 iff i S. Then to answer whether i S, we check the value of the i th bit. This algorithm achieves minimum runtime, but requires a lot of space (m bits). What if we want to save as much space as possible? If we use s bits of space, then what is stored in memory can take on at most 2 s possible values. Now since what is stored in memory should suffice to answer all the questions is i S?, what is stored in memory should determine S. Thus 2 s ( ( m n) and so the space required is at least m log2 n) bits. This is because the query determines whether i S for all i [m], so the query actually determines the subset S. The lower bound on the space requirement then follows from the fact that there are ( m n) n-element subsets of [m]. A simple algorithm ( that achieves the lower bound on space maps each choice of S to a unique sequence of log m ) n bits. The algorithm simply reads off the bits and uses the map to find the corresponding set S and answer the query. This algorithm uses the minimal amount of space ( log ( m n) bits), but has a large query time of log ( m n). These algorithms seem to imply that there is a tradoff between space and query time. However, we will see an algorithm that uses log ( m n) (1 + o(1)) bits with a query time of O(1). 2.4 Pseudorandomness The other major theme of the class is pseudorandomness, or the generation of randomness. What is randomness? How do we get it? How do we know when we ve gotten it? The big insight that starts the theory is that when we want randomness, it is for some purpose. We then define pseudorandomness to be whatever suffices for that purpose. Definition 4. A sequence of bits is pseudorandom if no algorithm of interest can distinguish between it and truly random bits. We will be much more concrete when the time comes, but this is the gist of it. A good example of a randomized algorithms comes from graph connectedness. Given an undirected graph G on n vertices, is G connected? Suppose only O(log n) space is allowed. It is not obvious 2

3 how to do this at all. The classical breadth-first-search (BFS) takes Ω(n) space. There is a beautiful classical low-space algorithm for this problem: For every pair of vertices (s, t), do: 1. Start a random walk on G starting at s. 2. Run it for n 10 steps. 3. If you never see t, say DISCONNECTED. If you never said DISCONNECTED, say CONNECTED. On every graph, this randomized algorithm outputs the correct answer with probability greater than An amazing result of Reingold shows how to make this algorithm deterministic. It involves taking a pseudorandom walk. We will prove this result in the course. 2.5 Hardness vs Randomness If a problem is hard, then you can t guess the answer, and so the answer is random to you. Perhaps you can then use the answer to that problem as the random bits to feed into a randomized algorithm. The theory of hardness versus randomness makes this rigorous. Roughly: if you can give me f that is hard for some computational model C, then you can get pseudorandom generators that generate bits that look random to C. We will prove versions of this and use it to get actual pseudorandom generators for some interesting computational models. 2.6 Concrete pseudorandomness I will just mention some words: Expander graphs ɛ-biased sets k-wise independent sets randomness extractors 3 Circuits and Formulas Let f : {0, 1} n {0, 1}. Recall that a circuit is built from wires carrying 0 or 1 and gates. There are two common bases for the gates: the full binary basis and the de Morgan basis. The de Morgan basis consists of (AND), (OR), and (NOT). The full binary basis consists of gates computing all 16 functions b : {0, 1} 2 {0, 1}, and includes the de Morgan basis. Note that all basis elements have fan-in 2 (number of inputs into gate). The fan-out, the number of outputs out of a gate, is unrestricted in circuits. In formulas, the fan-out for each gate is 1. 3

4 Observation 5. Every function has a formula. To see this, note that for each y {0, 1} n, there is a δ y : {0, 1} n {0, 1}, which is expressible as a of n variables and complements of variables, such that δ y (x) = 1 iff x = y. Writing completes the proof. f(x) = y {0,1} n f(y)=1 δ y (x) We can use this proof to get an upper bound on the number of gates of f: gates(f) (n 1)(2 n 1) O(n 2 n ). 3.1 Circuit size basics Maybe every function has a circuit of size poly(n)? This is ruled out by the following fundamental result. Theorem 6 (Shannon). ɛ > 0, most functions f : {0, 1} n {0, 1} require circuits of size at least 2 n n (1 ɛ). Proof. In 1 line: There are lots of functions, but not too many small circuits. The total number of functions is 2 2n. Suppose we consider circuits of size (number of gates) at most s. Each gate takes exactly two inputs that are either other gates or input bits. So there are at most ( ) s+n 2 ways to choose inputs for each gate. Since there are 16 binary functions in the full binary basis, this gives at most 16 ( ) s+n 2 ways to choose a gate function and its inputs. Since there are at most s gates, the number of circuits of size s is at most 16 s( ) s+n s. 2 Set s = 2n n (1 ɛ). We want to show that the number of circuits of size at most s is much less than the total number of functions. ( ) s + n s 16 s 2 16 s (s + n) s 16 s (2s) s = 32 s s s ) 2 n n (1 ɛ) (1 ɛ) ( = 32 2n 2 n n (1 ɛ) n ( ) 2 n 32(1 ɛ) n (1 ɛ) = 2 2n (1 ɛ) n < 2 2n (1 ɛ) 2 2n. 4

5 Theorem 7 (Shannon and Lupanov). ɛ > 0, functions f : {0, 1} n {0, 1}, circuits of size at most 2n n (1 + ɛ). We will only prove an upper bound of O( 2n n ). Proof. First, we find a circuit of size O(2 n ) using the recurrence: f(x 1,..., x n ) = (x 1 f(1, x 2,..., x n )) (x 1 f(0, x 2,..., x n )). size(n) size(n 1) size(n) = O(2 n ). More explicitly, this gives us a sequence of representations of the function, in terms of residual functions on fewer bits which we obtain by fixing the first few input bits to certain values: as f(x 1,..., x n ) = (x 1 f(1, x 2,..., x n )) (x 1 f(0, x 2,..., x n )). f(x 1,..., x n ) =(x 1 x 2 f(1, 1, x 3,..., x n )) (x 1 x 2 f(0, 1, x 3,..., x n )) (x 1 x 2 f(1, 0, x 3,..., x n )) (x 1 x 2 f(0, 0, x 3,..., x n )). To get a circuit of size O( 2n n ), we stop the recursion after t steps (t to be determined later). At this stage of the recurrence, for each (b 1,..., b t ) {0, 1} t we need to compute f(b 1,..., b t, x t+1,..., x n ), thought of as a function on the last n t bits. Now the total number of functions on the variables x t+1,..., x n is 2 2n t. The idea is this: if 2 2n t 2 t, then it is more economical to compute all the 2 2n t functions once beforehand, rather than to compute the residual function for each of the 2 t branches of the recursion. So we compute all 2 2n t functions on x t+1,..., x n and feed them to the appropriate gates of the representation of f using t steps of the recursion. Call this circuit C. Then size(c) 3 2 t + size required to compute all functions on n t bits. Claim 8. Size required to compute all functions on k bits 2 2k (1 + o(1)). We prove the claim by induction on k. We first compute all functions on k 1 bits, which takes size 2 2k 1. For each f 1 and f 2 in this class, compute (x k f 1 (1, x 2,..., x k 1 )) (x k f 2 (0, x 2,..., x k 1 )). This gives all functions on k bits. Hence, size(k) This completes the proof of the claim. ( 2 2k 1) k k 1 = 2 2k (1 + o(1)). So size(c) 3 2 t + 2 2n t. Set t = n log(n log n). Then 5

6 size(c) n log(n log n) 2log(n log n) + 2 = 3 2 n n log n + 2n log n 2 n = 3 n log n + 2n n ( ) 2 n = O. n The best known lower bound on circuit size for an explicit function is only 5n. Yet we know that there are functions out there (in fact most functions out there) whose smallest circuit has exponential size. One of the biggest problems of complexity theory is to find an explicit such function. 4 Formula size basics We saw earlier that every function of size n can be computed by a circuit of size (1 + ɛ)2 n /n, and that relatively few functions can be computed by a circuit of size less than (1 ɛ)2 n /n. The corresponding tight lower bound on the size of formulas for most functions is 2 n / log n. Theorem 9 (Shannon). Most boolean functions on n variables cannot be computed by a formula of size less than (1 ɛ)2 n / log n, for any ɛ > 0. Theorem 10 (Lyupanov). Every boolean function on n variables can be computed by a formula of size O(2 n / log n). These theorems hold in both the full binary basis and the de Morgan basis. Theorem 9 can be proved by the following counting argument; we won t see a proof of 10. Proof. We will compare a lower bound on the number of formulas of size s to the number of functions of size n. A formula of size s can be written as a binary tree with s internal nodes. Each internal node is labeled by a gate, and each leaf is labeled by a variable, the complement of a variable, 0, or 1. The number of binary trees with s internal nodes is given by the Catalan number C s 4 s. Each internal node is labeled with one of the gates in our basis; if there are b gates in the basis, the number of ways to do this labeling is b s ; for any basis, this is at most 16 s, for the full binary basis. There are (2n + 2) ways to label each leaf in the tree, and there are s + 1 leaves. Thus, the total number of formulas of size s is less than (2n + 2) s+1 4 s 16 s n s c s for some constant c > 1. The number of functions from (0, 1) n to (0, 1) is 2 2n. 6

7 If s (1 ɛ)2 n / log n for any ɛ > 0, then n s c s << 2 2n. Thus, only a small fraction of all the boolean functions on n variables can be computed by any formula of size (1 ɛ)2 n / log n. 5 The complexity class NC 1 The class of functions computable by a formula of size poly(n) is the same as the class of functions computable by a formula of size poly(n) and depth O(log n). Theorem 11. Every formula of size s can be converted into an equivalent formula of depth O(log s). Proof. To prove this theorem, we need the following lemma: Lemma 12. In any rooted binary tree with s vertices, there is a vertex g such that the tree rooted at g has between s/3 and 2s/3 vertices. Proof. A vertex that is the root of a subtree with more than 2s/3 vertices has a child that is the root of a subtree with at least s/3 vertices. We can use this fact to design a simple algorithm to find a vertex that is the root of a subtree with between s/3 and 2s/3 vertices. Start with the root as the current vertex. Select the child of the current vertex that has at least s/3 vertices in its subtree. If this vertex has at most 2s/3 vertices, we re done. Otherwise, set this vertex as the current vertex, and repeat the process. Now we ll use this lemma to prove theorem 11 The proof will be by induction; the base case is trivial. The inductive hypothesis is that for any s < s every formula of size s can be expressed as a formula of depth c log s + d (for constants c, d). Given a formula F, we can use the lemma to find a sub-formula F g with size in the range [s/3, 2s/3]. We can also find two formulas F 0 and F 1, which are F with the sub-formula F g replaced by 0 and 1, respectively. These formulas also have size in the range [s/3, 2s/3]. It is easy to check that F = (F g F 1 ) ( F g F 0 ). Thus, by applying the inductive hypothesis, we can replace F with an equivalent formula of depth 2 + c log(2s/3) + d c log(s) + d (for suitable c, d). The largest possible size of a formula of depth d is 2 d. Thus, a formula with logarithmic depth must have polynomial size. Theorem 11 shows that all formulas with size O(n c ) have equivalent formulas with size O(n c ) and depth O(log n). Thus, there is a single complexity class that captures all functions that have formulas with polynomial size, and all functions that have formulas with polynomial size and logarithmic depth. This is the class NC 1. If we begin with a circuit with the fan-in bounded by 2 and depth d, an upper bound on the size of the formula computing the function would be 2 d. To do this, simply replace any sub-circuit having more than one output with several copies of that sub-circuit, each having a single output. 7

8 Thus, NC 1 also equals the class of all functions that have circuits of polynomial size and logarithmic depth. 6 Upper bounds on formula size for specific functions Several specific problems are known to be in NC 1. PARITY is the family of boolean functions that evalute to 0 if the sum of the inputs is even and to 1 if the sum of inputs is odd. Theorem 13. PARITY NC 1. Proof. In the full binary basis, we can calculate the parity function on {0, 1} n as PARITY{0, 1} a PARITY{0, 1} n a. Thus, we can use binary divide and conquer to calculate the parity function with a formula of size O(n). In the de Morgan basis, x 1 x 2 can be expressed as (x 1 x 2 ) (x 1 x 2 ). Using this substitution, we can calculate the parity function with a circuit of size O(n). In order to convert this circuit to a formula, we need to duplicate each input to the xor gate so that no gate has more than one output. Let F (n) be the minimal size of a formula, counting only or and and gates, for PARITY of n bits in the de Morgan basis. Using the construction described in the previous paragraph, F (n) 3 + 2F ( n/2 ) + 2F ( n/2 ). Thus, F (n) O(n 2 ). We can also do operations on integers with small formulas. All integers are assumed to be nonnegative and written in base 2. We first show that we can add two n bit integers using a formula of size poly(n). Theorem 14. Integer addition is in NC 1. Proof. We ll assume binary inputs and use the full binary basis for the formula we build. If we can compute the i th carry bit c i, then we can compute the i th bit of the solution as s i = x i y i c i, where x i and y i are the i th bits of the input. The last bit of the solution will be s n+1 = c n+1. The i th bit of the carry will be 1 iff there exists a j < i such that x j = y j = 1, and for all j < k < i it is not the case that x k = y k = 0. This condition is directly expressible as a formula: c i = x j y j (x j y j ). j<i j<k<i 8

9 i 1 The number of gates needed to compute the carry is thus n+1 i=1 j=1 (3 + i 1 k=j+1 2) O(n3 ). The number of additional gates required to calculate the solution from the carry and the inputs is O(n), so addition of two n bit numbers can be performed by a formula in the full binary basis with O(n 3 ) gates. It is also possible to add n numbers, each n bits long, using a formula of size poly(n). Theorem 15. Iterated integer addition is in NC 1. Proof. For any three integers, X,Y, and Z, we will show how to find two numbers S and C such that X + Y + Z = S + C using a formula with constant depth. Using this procedure, we can reduce the problem of adding n integers to the problem of adding 2 integers with a formula with depth O(log 3/2 n). Finally we can add two integers using the previous theorem. For an integer A, let A i denote its i th bit (corresponding to the coefficient of 2 i ). The trick is the following: define S i = X i Y i Z i, and define C i+1 = Majority(X i, Y i, Z i ). These are the sum and carry respectively when we add the three bits X i, Y i, Z i. It is easy to check that S + C = X + Y + Z. Corollary 16. Integer multiplication is in NC 1. MAJ is the family of boolean functions on an odd number of input bits that outputs the majority value of its input bits. Theorem 17. MAJ NC 1. This follows immediately from theorem 16, since we can compute the sum of the bits of the input (treated as 1-bit integers) and compare it to n/2 using a formula of polynomial size. 7 Lower Bounds for Formulas A few specific, non-trivial lower bounds are known for formulas. Krapchenko proved that parity requires a formula of size Ω(n 2 ) in the de Morgan basis. Subbotovskaya and Andreev found explicit functions having no formulas of size o(n 2.5 ) in the de Morgan basis. Nechiporuk proved an Ω(n 2 ) lower bound in the full binary basis. Håstad proved an Ω(n 3 ) lower bound for an explicit function; we will not cover this result in detail. 7.1 Krapchenko s Lower Bound Krapchenko s method uses formal complexity measures. A formal complexity measure F C assigns real numbers to boolean functions such that it has the following properties: 1. F C(x i ) = F C(f) = F C( f). 9

10 3. F C(f g) F C(f) + F C(g). Properties 2 and 3 together imply 4. F C(f g) F C(f) + F C(g). Formal complexity measures are a powerful tool to prove lower bounds in the de Morgan basis. Let L(f) be the formula complexity of f; that is, the size of the smallest formula in the demorgan basis for the function f. Theorem 18. Any formal complexity measure F C satisfies F C(f) L(f). Proof. We will proceed by induction on L(f). The base case L(f) = 1 follows from property 1. Consider the smallest formula for f. If f = f 1 f 2, then L(f) = L(f 1 ) + L(f 2 ) F C(f 1 ) + F C(f 2 ) F C(f 1 f 2 ) = F C(f). The case f = f 1 f 2 follows in the same manner, using property 4 rather than property 3. Theorem 19 (Krapchenko). PARITY requires a formula of size Ω(n 2 ) in the de Morgan basis. We will prove this theorem in the next class. The main tool is the following formal complexity measure: F C(f) = max A f 1 (0) B f 1 (1) e(a, B) 2, A B where e(a, B) is the number of pairs (x, y) A B such that x and y differ in exactly one coordinate. In the next class we will show that it is indeed a formal complexity measure. For PARITY, let us show that F C(P ARIT Y ) Ω(n 2 ). Consider A = f 1 (0) and B = f 1 (1). Then A = B = 2 n 1, and e(a, B) = n2 n 1. So F C(P ARIT Y ) n 2, and so L(P ARIT Y ) n 2. 10

### Lecture 4: AC 0 lower bounds and pseudorandomness

Lecture 4: AC 0 lower bounds and pseudorandomness Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: Jason Perry and Brian Garnett In this lecture,

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

### GRAPH THEORY LECTURE 4: TREES

GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection

### Outline Introduction Circuits PRGs Uniform Derandomization Refs. Derandomization. A Basic Introduction. Antonis Antonopoulos.

Derandomization A Basic Introduction Antonis Antonopoulos CoReLab Seminar National Technical University of Athens 21/3/2011 1 Introduction History & Frame Basic Results 2 Circuits Definitions Basic Properties

### Full and Complete Binary Trees

Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### 8.1 Min Degree Spanning Tree

CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Lecture 5 - CPA security, Pseudorandom functions

Lecture 5 - CPA security, Pseudorandom functions Boaz Barak October 2, 2007 Reading Pages 82 93 and 221 225 of KL (sections 3.5, 3.6.1, 3.6.2 and 6.5). See also Goldreich (Vol I) for proof of PRF construction.

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Arithmetic complexity in algebraic extensions

Arithmetic complexity in algebraic extensions Pavel Hrubeš Amir Yehudayoff Abstract Given a polynomial f with coefficients from a field F, is it easier to compute f over an extension R of F than over F?

### Lecture 7: NP-Complete Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit

### Lecture 2: Universality

CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

### Definition 11.1. Given a graph G on n vertices, we define the following quantities:

Lecture 11 The Lovász ϑ Function 11.1 Perfect graphs We begin with some background on perfect graphs. graphs. First, we define some quantities on Definition 11.1. Given a graph G on n vertices, we define

### Recursive Algorithms. Recursion. Motivating Example Factorial Recall the factorial function. { 1 if n = 1 n! = n (n 1)! if n > 1

Recursion Slides by Christopher M Bourke Instructor: Berthe Y Choueiry Fall 007 Computer Science & Engineering 35 Introduction to Discrete Mathematics Sections 71-7 of Rosen cse35@cseunledu Recursive Algorithms

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### Outline BST Operations Worst case Average case Balancing AVL Red-black B-trees. Binary Search Trees. Lecturer: Georgy Gimel farb

Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worst-case time complexity of BST operations

### Influences in low-degree polynomials

Influences in low-degree polynomials Artūrs Bačkurs December 12, 2012 1 Introduction In 3] it is conjectured that every bounded real polynomial has a highly influential variable The conjecture is known

### Cartesian Products and Relations

Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special

### Guessing Game: NP-Complete?

Guessing Game: NP-Complete? 1. LONGEST-PATH: Given a graph G = (V, E), does there exists a simple path of length at least k edges? YES 2. SHORTEST-PATH: Given a graph G = (V, E), does there exists a simple

### Math 55: Discrete Mathematics

Math 55: Discrete Mathematics UC Berkeley, Fall 2011 Homework # 5, due Wednesday, February 22 5.1.4 Let P (n) be the statement that 1 3 + 2 3 + + n 3 = (n(n + 1)/2) 2 for the positive integer n. a) What

### Sample Induction Proofs

Math 3 Worksheet: Induction Proofs III, Sample Proofs A.J. Hildebrand Sample Induction Proofs Below are model solutions to some of the practice problems on the induction worksheets. The solutions given

### Minimum cost maximum flow, Minimum cost circulation, Cost/Capacity scaling

6.854 Advanced Algorithms Lecture 16: 10/11/2006 Lecturer: David Karger Scribe: Kermin Fleming and Chris Crutchfield, based on notes by Wendy Chu and Tudor Leu Minimum cost maximum flow, Minimum cost circulation,

### Math 55: Discrete Mathematics

Math 55: Discrete Mathematics UC Berkeley, Spring 2012 Homework # 9, due Wednesday, April 11 8.1.5 How many ways are there to pay a bill of 17 pesos using a currency with coins of values of 1 peso, 2 pesos,

### C H A P T E R. Logic Circuits

C H A P T E R Logic Circuits Many important functions are naturally computed with straight-line programs, programs without loops or branches. Such computations are conveniently described with circuits,

### CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis. Linda Shapiro Winter 2015

CSE373: Data Structures and Algorithms Lecture 3: Math Review; Algorithm Analysis Linda Shapiro Today Registration should be done. Homework 1 due 11:59 pm next Wednesday, January 14 Review math essential

### 11 Multivariate Polynomials

CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6

### Mathematical Induction. Lecture 10-11

Mathematical Induction Lecture 10-11 Menu Mathematical Induction Strong Induction Recursive Definitions Structural Induction Climbing an Infinite Ladder Suppose we have an infinite ladder: 1. We can reach

### Triangle deletion. Ernie Croot. February 3, 2010

Triangle deletion Ernie Croot February 3, 2010 1 Introduction The purpose of this note is to give an intuitive outline of the triangle deletion theorem of Ruzsa and Szemerédi, which says that if G = (V,

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### Polynomial Degree and Lower Bounds in Quantum Complexity: Collision and Element Distinctness with Small Range

THEORY OF COMPUTING, Volume 1 (2005), pp. 37 46 http://theoryofcomputing.org Polynomial Degree and Lower Bounds in Quantum Complexity: Collision and Element Distinctness with Small Range Andris Ambainis

### HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

### Lecture 22: November 10

CS271 Randomness & Computation Fall 2011 Lecture 22: November 10 Lecturer: Alistair Sinclair Based on scribe notes by Rafael Frongillo Disclaimer: These notes have not been subjected to the usual scrutiny

### MATH10040 Chapter 2: Prime and relatively prime numbers

MATH10040 Chapter 2: Prime and relatively prime numbers Recall the basic definition: 1. Prime numbers Definition 1.1. Recall that a positive integer is said to be prime if it has precisely two positive

### CMPSCI611: Approximating MAX-CUT Lecture 20

CMPSCI611: Approximating MAX-CUT Lecture 20 For the next two lectures we ll be seeing examples of approximation algorithms for interesting NP-hard problems. Today we consider MAX-CUT, which we proved to

### Lecture 16 : Relations and Functions DRAFT

CS/Math 240: Introduction to Discrete Mathematics 3/29/2011 Lecture 16 : Relations and Functions Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT In Lecture 3, we described a correspondence

### MATHEMATICAL INDUCTION. Mathematical Induction. This is a powerful method to prove properties of positive integers.

MATHEMATICAL INDUCTION MIGUEL A LERMA (Last updated: February 8, 003) Mathematical Induction This is a powerful method to prove properties of positive integers Principle of Mathematical Induction Let P

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### Midterm Practice Problems

6.042/8.062J Mathematics for Computer Science October 2, 200 Tom Leighton, Marten van Dijk, and Brooke Cowan Midterm Practice Problems Problem. [0 points] In problem set you showed that the nand operator

### k, then n = p2α 1 1 pα k

Powers of Integers An integer n is a perfect square if n = m for some integer m. Taking into account the prime factorization, if m = p α 1 1 pα k k, then n = pα 1 1 p α k k. That is, n is a perfect square

### 1 Formulating The Low Degree Testing Problem

6.895 PCP and Hardness of Approximation MIT, Fall 2010 Lecture 5: Linearity Testing Lecturer: Dana Moshkovitz Scribe: Gregory Minton and Dana Moshkovitz In the last lecture, we proved a weak PCP Theorem,

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

### 1 The Line vs Point Test

6.875 PCP and Hardness of Approximation MIT, Fall 2010 Lecture 5: Low Degree Testing Lecturer: Dana Moshkovitz Scribe: Gregory Minton and Dana Moshkovitz Having seen a probabilistic verifier for linearity

### COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012

Binary numbers The reason humans represent numbers using decimal (the ten digits from 0,1,... 9) is that we have ten fingers. There is no other reason than that. There is nothing special otherwise about

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal

Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal ABSTACT Mohit Singh Tepper School of Business Carnegie Mellon University Pittsburgh, PA USA mohits@andrew.cmu.edu In the MINIMUM

### Lecture 11: The Goldreich-Levin Theorem

COM S 687 Introduction to Cryptography September 28, 2006 Lecture 11: The Goldreich-Levin Theorem Instructor: Rafael Pass Scribe: Krishnaprasad Vikram Hard-Core Bits Definition: A predicate b : {0, 1}

### Notes from Week 1: Algorithms for sequential prediction

CS 683 Learning, Games, and Electronic Markets Spring 2007 Notes from Week 1: Algorithms for sequential prediction Instructor: Robert Kleinberg 22-26 Jan 2007 1 Introduction In this course we will be looking

### x a x 2 (1 + x 2 ) n.

Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Sorting revisited. Build the binary search tree: O(n^2) Traverse the binary tree: O(n) Total: O(n^2) + O(n) = O(n^2)

Sorting revisited How did we use a binary search tree to sort an array of elements? Tree Sort Algorithm Given: An array of elements to sort 1. Build a binary search tree out of the elements 2. Traverse

### Notes on Complexity Theory Last updated: August, 2011. Lecture 1

Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

### Answer: (a) Since we cannot repeat men on the committee, and the order we select them in does not matter, ( )

1. (Chapter 1 supplementary, problem 7): There are 12 men at a dance. (a) In how many ways can eight of them be selected to form a cleanup crew? (b) How many ways are there to pair off eight women at the

### Practice with Proofs

Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

### Tree-representation of set families and applications to combinatorial decompositions

Tree-representation of set families and applications to combinatorial decompositions Binh-Minh Bui-Xuan a, Michel Habib b Michaël Rao c a Department of Informatics, University of Bergen, Norway. buixuan@ii.uib.no

### Generating models of a matched formula with a polynomial delay

Generating models of a matched formula with a polynomial delay Petr Savicky Institute of Computer Science, Academy of Sciences of Czech Republic, Pod Vodárenskou Věží 2, 182 07 Praha 8, Czech Republic

### The positive minimum degree game on sparse graphs

The positive minimum degree game on sparse graphs József Balogh Department of Mathematical Sciences University of Illinois, USA jobal@math.uiuc.edu András Pluhár Department of Computer Science University

### Borne inférieure de circuit : une application des expanders

Borne inférieure de circuit : une application des expanders Simone Bova 1 Florent Capelli 2 Friedrich Slivovski 1 Stefan Mengel 3 1 TU Wien, 2 IMJ-PRG, Paris 7, 3 CRIL, Lens 6 Novembre 2015 JGA 2015 Motivation

### CS 598CSC: Combinatorial Optimization Lecture date: 2/4/2010

CS 598CSC: Combinatorial Optimization Lecture date: /4/010 Instructor: Chandra Chekuri Scribe: David Morrison Gomory-Hu Trees (The work in this section closely follows [3]) Let G = (V, E) be an undirected

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### Scheduling Shop Scheduling. Tim Nieberg

Scheduling Shop Scheduling Tim Nieberg Shop models: General Introduction Remark: Consider non preemptive problems with regular objectives Notation Shop Problems: m machines, n jobs 1,..., n operations

### The Taxman Game. Robert K. Moniot September 5, 2003

The Taxman Game Robert K. Moniot September 5, 2003 1 Introduction Want to know how to beat the taxman? Legally, that is? Read on, and we will explore this cute little mathematical game. The taxman game

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new

Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

### Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new GAP (2) Graph Accessibility Problem (GAP) (1)

Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of

### THE DIMENSION OF A VECTOR SPACE

THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field

### CHAPTER 5. Number Theory. 1. Integers and Division. Discussion

CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a

### We can express this in decimal notation (in contrast to the underline notation we have been using) as follows: 9081 + 900b + 90c = 9001 + 100c + 10b

In this session, we ll learn how to solve problems related to place value. This is one of the fundamental concepts in arithmetic, something every elementary and middle school mathematics teacher should

### Polynomial Invariants

Polynomial Invariants Dylan Wilson October 9, 2014 (1) Today we will be interested in the following Question 1.1. What are all the possible polynomials in two variables f(x, y) such that f(x, y) = f(y,

### 6.080/6.089 GITCS Feb 12, 2008. Lecture 3

6.8/6.89 GITCS Feb 2, 28 Lecturer: Scott Aaronson Lecture 3 Scribe: Adam Rogal Administrivia. Scribe notes The purpose of scribe notes is to transcribe our lectures. Although I have formal notes of my

### Section IV.1: Recursive Algorithms and Recursion Trees

Section IV.1: Recursive Algorithms and Recursion Trees Definition IV.1.1: A recursive algorithm is an algorithm that solves a problem by (1) reducing it to an instance of the same problem with smaller

### 6.3 Conditional Probability and Independence

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

### Quantum Computing Lecture 7. Quantum Factoring. Anuj Dawar

Quantum Computing Lecture 7 Quantum Factoring Anuj Dawar Quantum Factoring A polynomial time quantum algorithm for factoring numbers was published by Peter Shor in 1994. polynomial time here means that

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

### Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010. Chapter 7: Digraphs

MCS-236: Graph Theory Handout #Ch7 San Skulrattanakulchai Gustavus Adolphus College Dec 6, 2010 Chapter 7: Digraphs Strong Digraphs Definitions. A digraph is an ordered pair (V, E), where V is the set

### Dynamic TCP Acknowledgement: Penalizing Long Delays

Dynamic TCP Acknowledgement: Penalizing Long Delays Karousatou Christina Network Algorithms June 8, 2010 Karousatou Christina (Network Algorithms) Dynamic TCP Acknowledgement June 8, 2010 1 / 63 Layout

### Chapter 3. Distribution Problems. 3.1 The idea of a distribution. 3.1.1 The twenty-fold way

Chapter 3 Distribution Problems 3.1 The idea of a distribution Many of the problems we solved in Chapter 1 may be thought of as problems of distributing objects (such as pieces of fruit or ping-pong balls)

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Lemma 5.2. Let S be a set. (1) Let f and g be two permutations of S. Then the composition of f and g is a permutation of S.

Definition 51 Let S be a set bijection f : S S 5 Permutation groups A permutation of S is simply a Lemma 52 Let S be a set (1) Let f and g be two permutations of S Then the composition of f and g is a

### A Sublinear Bipartiteness Tester for Bounded Degree Graphs

A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite

### Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

### Reading 13 : Finite State Automata and Regular Expressions

CS/Math 24: Introduction to Discrete Mathematics Fall 25 Reading 3 : Finite State Automata and Regular Expressions Instructors: Beck Hasti, Gautam Prakriya In this reading we study a mathematical model

### 8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

### Catalan Numbers. Thomas A. Dowling, Department of Mathematics, Ohio State Uni- versity.

7 Catalan Numbers Thomas A. Dowling, Department of Mathematics, Ohio State Uni- Author: versity. Prerequisites: The prerequisites for this chapter are recursive definitions, basic counting principles,

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### 1. Prove that the empty set is a subset of every set.

1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since

### Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

### Connectivity and cuts

Math 104, Graph Theory February 19, 2013 Measure of connectivity How connected are each of these graphs? > increasing connectivity > I G 1 is a tree, so it is a connected graph w/minimum # of edges. Every

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### CS 103X: Discrete Structures Homework Assignment 3 Solutions

CS 103X: Discrete Structures Homework Assignment 3 s Exercise 1 (20 points). On well-ordering and induction: (a) Prove the induction principle from the well-ordering principle. (b) Prove the well-ordering