CS 154 Time Complexity: P and NP Complexity Theory Study of what can and can t be computed with limited resources on computation, such as time, space, and so on We ll start with: Time complexity Measuring Time Complexity We measure time complexity by counting the elementary steps required for a machine to halt Consider the language A = { 0 k 1 k k 0 } On input of length n: 1. Scan across the tape and reject if the ~n string is not of the form 0 i 1 j ~n 2 ~n 2. Repeat the following if both 0s and 1s remain on the tape: Scan across the tape, crossing off a single 0 and a single 1 3. If 0s remain after all 1s have been crossed off, or vice-versa, reject. Otherwise accept. Let M be a TM that halts on all inputs. Definition: The running time or time complexity of M is the function f : N N such that f(n) = maximum number of steps taken by M over any input of length n. Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L =L(M) } = {L L is a language decided by a Turing Machine with O(t(n)) running time } We showed: A = { 0 k 1 k k 0 } TIME(n 2 ) A = { 0 k 1 k k 0 } TIME(n log n) M(w) := If w is not of the form 0*1*, reject. Repeat until all bits of w are crossed out: If the parity of 0 s parity of 1 s, reject. Cross out every other 0. Cross out every other 1. Otherwise, accept. 00000000000001111111111111 x0x0x0x0x0x0xx1x1x1x1x1x1x xxx0xxx0xxx0xxxx1xxx1xxx1x xxxxxxx0xxxxxxxxxxxx1xxxxx xxxxxxxxxxxxxxxxxxxxxxxxxx 1
Theorem: A = { 0 k 1 k k 0 } can be decided in O(n) time with a two-tape TM. Proof Idea: Scan all 0s, copy them to the second tape. Scan all 1s. For each 1 scanned, cross off a 0 from the second tape. Different models of computation can yield different running times for the same language! Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine Theorem: Let t(n) be a function such that t(n) n. Then every t(n) time multi-tape TM has an equivalent O(t(n) 2 ) single tape TM Our simulation of multitape TMs achieved this! T(n) time FINITE STATE CONTROL 1 0 0 T(n) 2 time FINITE STATE CONTROL... 1 0 0 # # # Time Hierarchy Theorem Intuition: If you get more time to work, then you can solve strictly more problems Theorem: For reasonable f, g : N N where f(n) = O(g(n) 1/3 ), TIME(f(n)) TIME(g(n)) Proof Idea: Diagonalization! Make a TM N that does the opposite of all g(n) 1/3 time machines on at least one input, and runs in O(g(n)) time So L(N) is in TIME(g(n)) but not in TIME(f(n)) Time Hierarchy Theorem Theorem: For reasonable f, g where f(n) = O(g(n) 1/3 ) TIME(f(n)) TIME(g(n)) Proof Sketch: Define a machine N as follows. N(w) = If w is not equal to M10 i for some M, i, reject Simulate M on w for f( w ) steps (This simulation itself uses f( w ) 2 time) If M tries to take more time, reject Accept if and only if M has rejected Eventually, w is large enough that f( w ) < g( w ) Simulation can be carried out using multitape sim. Need to compute f( w ) using only O(g( w )) time 2
Time Hierarchy Theorem Theorem: For reasonable f, g where f(n) = O(g(n) 1/3 ) TIME(f(n)) TIME(g(n)) Corollary: TIME(n) TIME(n 2 ) TIME(n 4 ) There is an infinite hierarchy of increasingly more time-consuming problems P = TIME(n k ) k N Question: Are there important everyday problems that are high up in the hierarchy? A natural problem that needs exactly n 10 time? THIS IS AN OPEN QUESTION! The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion = Polynomial-Time of Efficient Turing Machines Algorithms Nondeterminism and NP Much more controversial. Potential counterexamples include n 100 time algorithms, Quantum & Randomized algorithms Nondeterministic Turing Machines read write move 0 0, R are just like standard TMs, except: 0 0, R, R q accept 1. The machine may proceed according to several possibilities, R q reject 0 0, R 2. The machine accepts a string if there exists a path from start configuration to an accepting configuration 3
Definition: A nondeterministic TM is a 7-tuple T = (Q, Σ, Γ, δ, q 0, q accept, q reject ), where: Q is a finite set of states Σ is the input alphabet, where Σ Γ is the tape alphabet, where Γand Σ Γ δ : Q Γ 2 (Q Γ {L,R}) q 0 Q is the start state q accept Q is the accept state q reject Q is the reject state, and q reject q accept Deterministic Computation accept or reject Nondeterministic Computation accept reject Defining Acceptance for NTMs Let N be a nondeterministic Turing machine An accepting computation history for N on w is a sequence of configurations C 0,C 1,,C t where 1. C 0 is the start configuration q 0 w, 2. C t is an accepting configuration, 3. Each configuration C i yields C i+1 N(w) accepts in t time iff such a history exists. Accepting computation histories for N on w do not have to be unique! Definition: NTIME(t(n)) = { L L is decided by a O(t(n)) time nondeterministic Turing machine } TIME(t(n)) NTIME(t(n)) Is this a proper containment for all t(n)? THIS IS AN OPEN QUESTION! A Boolean formula is satisfiable if there exists a satisfying assignment for it A 3cnf-formula has the form: (x 1 x 2 x 3 ) (x 4 x 2 x 5 ) (x 3 x 2 x 1 ) YES NO a b c d (x y) x SAT = { φ φ is a satisfiable Boolean formula } literals clauses (x 1 x 2 x 1 ) (x 3 x 1 ) (x 3 x 2 x 1 ) (x 1 x 2 x 3 ) ( x 4 x 2 x 1 ) (x 3 x 1 x 1 ) (x 1 x 2 x 3 ) (x 3 x 2 x 1 ) 3SAT = { φ φ is a satisfiable 3cnf-formula } 4
3SAT = { φ φ is a satisfiable 3cnf-formula } Theorem: 3SAT NTIME(n 2 ) On input φ: 1. Check if the formula is in 3cnf 2. For each variable, nondeterministically substitute it with 0 or 1 ( x y x ) NP = NTIME(n k ) k N ( 0 y 0 ) ( 1 y 1 ) ( 0 0 0 ) ( 0 1 0 ) 3. Test if the assignment satisfies φ Theorem: L NP there is a poly-time Turing machine V and k such that we can define L as: L = { x y [ y x k and V(x,y) accepts ] } Proof: (1) If L = { x y y x k and V(x,y) accepts } then L NP Nondeterministically guess y and then run V(x,y) (2) If L NP then L = { x y y x k and V(x,y) accepts } 3SAT = { φ y such that y is a satisfying assignment to φ and φ is in 3cnf } SAT = { φ y such that y is a satisfying assignment to φ } Let N be a nondeterministic poly-time TM that decides L. Define V(x,y) to accept iff y is an accepting computation history of N on x The Hamiltonian Path Problem A language is in NP if and only if there are polynomial-length proofs for membership to the language b a e d f g SAT is in NP because a satisfying assignment is a polynomial-length proof that a formula is satisfiable i A Hamiltonian path traverses through each node exactly once c h 5
Assume a reasonable encoding of graphs (example: the adjacency matrix is reasonable) HAMPATH = { (G,s,t) G is a directed graph with a Hamiltonian path from s to t } Theorem: HAMPATH NP Next Episode: NP-Complete Problems The Hamiltonian path is a proof that (G,s,t) is a member of HAMPATH 6