Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology email: bvogt@ist.tugraz.at office hour: Tuesday 10:30 11:30 slides: http://www.ist.tugraz.at/pact.html Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 1
Last Time NP-completeness exercises: HAMPATH P UHAMPATH HAMCYCLE = P UHAMCYCLE strong NP-completess of TSP Summary: - picture - proof + reduction checklist Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 2
Last Time NP-completeness exercises: L bold = strongly NPC Arrows HAMPATH indicate P UHAMPATH HAMCYCLE = P UHAMCYCLE SAT ksat for k > 3 presented reductions strong NP-completess of TSP 3SAT Summary: - picture MAX2SAT SUBSET SUM - proof + reduction checklist HAMPATH INDEP. SET KNAPSACK HAMCYCLE UHAMPATH CLIQUE VERTEX COVER UHAMCYCLE TSP Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 2
Last Time New complexity classes via complements: UNSAT: unsatisfiability of boolean formulas Definition of co-classes including conp Discussion P - NP - conp Picture of one possibility Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 2
Last Time New complexity classes via complements: UNSAT: unsatisfiability of boolean formulas Definition of co-classes NP including conp conp Discussion P - NP - conp P Picture of one possibility NPC SAT conpc UNSAT Picture: one possibility! Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 2
This Time: Space Complexity recall definitions: SPACE(f(n)), NSPACE(f(n)), L, NL, PSPACE clarify computation model and #configurations revisit containment relations of complexity classes two examples: SAT, PATH recall log-space reductions L consider closeness under L define completeness w.r.t. L prove Savitch s Theorem define conl Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 3
Definition: Space Complexity: Definition Let s : N N be a complexity function. Deterministic Space: SPACE(s(n)) := {L L is a language decidable by an O(s(n))-space det. TM } Nondeterministic Space: NSPACE(s(n)) := {L L is a language decidable by an O(s(n))-space nondet. TM } Det. Polynomial Space: PSPACE := k 1 SPACE(nk ) Det. Logarithmic Space: L := SPACE(log n) Nondet. Logarithmic Space: NL := NSPACE(log n) Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 4
Space Complexity: Definition Definition: Let s : N N be a complexity function. Deterministic Space: SPACE(s(n)) := {L L is a language decidable PSPACEby an O(s(n))-space det. TM } Nondeterministic Space: NSPACE(s(n)) := {L L is a language decidable NL by an Input uses n cells How can O(s(n))-space nondet. TM } L a TM use only log n space? Det. Polynomial Space: PSPACE := k 1 SPACE(nk ) Det. Logarithmic Space: L := SPACE(log n) Nondet. Logarithmic Space: NL := NSPACE(log n) Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 4
Input / Work / Output TM Model of computation: special 3-tape DTM / NTM Input tape: read only! 1 0 1 1 Work tape: only one counted 1 0 0 0 0 1 Output tape: write only! no going back Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 5
Input / Work / Output TM Model of computation: special 3-tape DTM / NTM 1. Q: finite set of states, 2. Σ: finite input alphabet not containing the blank symbol and the start symbol, 3. Γ: finite work alphabet with, Γ and Σ Γ, 4. transition function δ : Q Σ Γ Q {L,, R} in Γ {L,, R} work Γ {, R} out, 5. q 0 Q: start state, 6. q accept Q: accept state, and 7. q reject Q: reject state, where q reject q accept. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 5
Input / Work / Output TM Model of computation: special 3-tape DTM / NTM 1. Q: finite set of states, 2. Σ: finite input alphabet not containing the blank symbol and the start symbol, 3. Γ: finite work alphabet with, Γ and Σ Γ, 4. transition relation : Q Σ Γ P (Q {L,, R} in Γ {L,, R} work Γ {, R} out ), 5. q 0 Q: start state, 6. q accept Q: accept state, and 7. q reject Q: reject state, where q reject q accept. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 5
Input / Work / Output TM Model of computation: special 3-tape DTM / NTM Question: How many different configurations for given input of size N and work-tape size S? 1. input tape head position: N 2. work tape content: Γ S 3. work tape head position: S 4. the machines state: Q in total: N Γ S S Q input unchangable 1 instead of Σ N! output unreadable need not be counted! Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 5
Input / Work / Output TM Model of computation: special 3-tape DTM / NTM Question: How many different configurations for given input of size N and work-tape size S? 1. input tape head position: N 2. work tape content: Γ S 3. work tape head position: S 4. the machines state: Q in total: Corollary: N Γ S S Q L P, NL NP. Proof: Space bound S = O(log N) N Γ log N log N Q = O(p(N)). Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 5
Containment Relations L NL P NP PSPACE EXP L NL, P NP: DTM = special case of NTM PSPACE EXP, L P: # configurations of a DTM P PSPACE: O(f(n))-time O(f(n))-space EXP PSPACE NP P NL NP PSPACE: simulate O(f(n))-time NTM with O(f(n))-space DTM NL P: later in this course L Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 6
Containment Relations L NL P NP PSPACE EXP L NL, P NP: DTM = special case of NTM PSPACE EXP, L P: # configurations of a DTM P PSPACE: O(f(n))-time O(f(n))-space EXP PSPACE NP P NL NP PSPACE: simulate O(f(n))-time NTM with O(f(n))-space DTM NL P: later in this course lesson L Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 6
Example: SAT Claim: SAT SPACE(n). M = On input φ, where φ is a Boolean formula: 1. For each truth assignment T of the variables x 1,... x m of φ: 2. Evaluate φ on T 3. If φ(t ) evaluates to true then accept φ 4. reject φ In one iteration of the loop, M needs to store only the truth assignment T O(m) = O(n) space. Each iteration can use the same portion of the work tape M runs in O(n) = linear space in total. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 7
Example: PATH In a directed graph G = (V, E), a (simple) path from s V to t V is a directed path that starts at s, ends at t, and visits every vertex of V at most once. Decision problem PATH: Does a given directed graph G contain a path from s to t? t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 8
Example: PATH In a directed graph G = (V, E), a (simple) path from s V to t V is a directed path that starts at s, ends at t, and visits every vertex of V at most once. Decision problem PATH: Does a given directed graph G contain a path from s to t? Instance: A directed graph G = (V, E) and s, t V. t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 8
Claim: Example: PATH PATH is in NL. nondeterministic algorithm for G = (V, E), s, t: 1. Start with u = s 2. for i = 1,..., V : 3. let u be a nondeterministic neighbor of u 4. if u = t then acccept 5. reject needed Space: O(log V ) for current u O(log V ) for counting to V O(log V ) = O(log n) total Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 8 t s
Input / Work / Witness TM Alternative formulation for space-bounded NTM: Input tape: read only! 1 0 1 1 Work tape: only one counted 1 0 0 0 0 1 Witness tape: read only! no going back witness for PATH: a path s t Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 9
Definition: Log-Space Reductions A language A is log-space (Karp) reducible to a language B (denoted by A L B), if there exists a log-space computable function f : Σ A Σ B w A f(w) B. (called reduction-function), such that log-space DTM that outputs f(w) on input w. A f B Σ A \A Σ B \B Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 10
Log-Space Reductions Theorem: L is closed under log-space (Karp) reductions: Claim: If f is a log-space reduction from A to B and B L, then A L. Proof for polynomial time reductions: w a polynomial time algorithm for A polynomial-time algorithm for f f(w) polynomial-time algorithm for B f(w) B? w A? Question: Why does this reasoning not work for L and L? f(w) too large for inside A! Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 10
Log-Space Reductions Theorem: L is closed under log-space (Karp) reductions: Claim: If f is a log-space reduction from A to B and B L, then A L. Proof: Make a machine M A that on input w for A... simulates B s machine M B on input f(w) whenever M B would read the i th symbol of its input f(w), runs f s machine M f on w and waits for the i th symbol to be outputted. Theorem: The complexity classes L, NL, PSPACE, P, NP,and EXP are closed under log-space (Karp) reductions. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 10
Log-Space Reductions Theorem: L is closed under log-space (Karp) reductions: Claim: If f is a log-space reduction from A to B and B L, then A L. Proof: Make a machine M A that on input w for A... simulates B s machine M B on input f(w) whenever M B would read the i th symbol of its input f(w), runs f s machine M f on w and waits for the i th symbol to be outputted. Definition: A decision problem B NL is called NL-complete if A L B for all A NL. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 10
Theorem: PATH is NL-complete PATH is NL-complete. Proof for NL-hardness: Given an arbitrary log-space NTM N and an input w, find a log-space computable function f that maps N, w to an instance f(n, w) = (G, s, t) of PATH such that G has path from s to t N accepts w N, w f t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
PATH is NL-complete Configurations graph G N,w = (V, E) for (N, w): vertex set V : all configurations of (N, w) edge (u, v) E iff transition u v possible in (N, w) start vertex s: start configuration of (N, w) target vertex t: accepting configuration of (N, w) Modify N to have a unique accepting configuration s G has path from s to t N accepts w t Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
Theorem: PATH is NL-complete PATH is NL-complete. Proof for NL-hardness: Given an arbitrary log-space NTM N and an input w, find a log-space computable function f that maps N, w to an instance f(n, w) = (G, s, t) of PATH such that G has path from s to t N accepts w N, w f t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
PATH is NL-complete Log-space computability of G N,w for (N, w): V : any configuration can be represented with c log w space (c const.) for all strings s of length s c log w : test whether s is a legal configuration of (N, w) output s iff yes E: try all pairs (c 1, c 2 ) of configurations: test whether with the tape contents and head locations given in c 1, c 2 is a possible next configuration (by the transition relation ) output (c 1, c 2 ) iff yes Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
Theorem: PATH is NL-complete PATH is NL-complete. Proof for NL-hardness: Given an arbitrary log-space NTM N and an input w, find a log-space computable function f that maps N, w to an instance f(n, w) = (G, s, t) of PATH such that G has path from s to t N accepts w N, w f t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
PATH is NL-complete Theorem: Corollary: NL P. PATH is NL-complete. Proof: PATH is NL-complete. reduction f from proof in space O(log n). f in time n2 O(log n) = O(p(n)). For any A NL: A P PATH. PATH P For any A NL: A P. t s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 11
Containment Relations L NL P NP PSPACE EXP L NL, P NP: DTM = special case of NTM PSPACE EXP, L P: # configurations of a DTM P PSPACE: O(f(n))-time O(f(n))-space EXP PSPACE NP P NL NP PSPACE: simulate O(f(n))-time NTM with O(f(n))-space DTM NL P: later in this lesson PATH NL-complete and P. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 12 L
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Proof: 1. We construct an algorithm path(g, x, y, i) that returns whether there exists a path in G from x to y that has length at most 2 i. path(g, x, y, i): if i = 0: if x = y or (x, y) E: accept if i > 0: for each z V : if path(g, x, z, i 1) and path(g, z, y, i 1): accept reject Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13 t s
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Proof: 2. path(g, x, y, log n ) answers the decision question PATH(G, x, y): If there exists a path P from x to y then log n P has length P n 2 P can be split into two halves, both of length log n 1 at most P /2 n/2 2 Two paths P 1 : x z and P 2 : z y can be combined into one path P : x y with length P P 1 + P 2 Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Proof: 3. path(g, x, y, log n ) can be computed in O((log n) 2 ) space: use stack for recursion with triplets (x, y, i). For each vertex z: write (x, z, i 1) to stack if path(x, z, i 1) accepts replace (x, z, i 1) by (z, y, i 1). else replace (x, z, i 1) by (x, z, i 1). each stack element needs at most space 3 log n as i log n, the height of the stack is at most log n the total stack space is O((log n) 2 ). Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Corollary: NL = NSPACE(log n) SPACE((log n) 2 ). Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Corollary (Savitch s Theorem): f Ω(log n) : NSPACE(f(n)) SPACE((f(n)) 2 ). Proof: 1. simulate f(n)-space NTM N with input w by running path on its configurations graph G N,w. 2. G N,w is represented only implicitly. It suffices to read w and the description of N for i = 0. 3. G N,w has c f(n) vertices (for some constant c). path needs at most O((log(c f(n) )) 2 ) = O((f(n)) 2 ) space. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13
Savitch s Theorem Theorem (Savitch 1970): PATH SPACE((log n) 2 ). Corollary (Savitch s Theorem): f Ω(log n) : NSPACE(f(n)) SPACE((f(n)) 2 ). Exercise: Perform path with the shown graph path(g, x, y, i): if i=0: if x = y or (x, y) E: accept t if i > 0: for each z V : if path(g, x, z, i 1) and path(g, z, y, i 1): accept reject Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 13 s
Discussion Space complexity classes: especially L, NL log-space reductions L closeness under L NL P: PATH is NL-complete NL SPACE((log n) 2 ): PATH SPACE((log n) 2 ) Savitch s Theorem: NSPACE(f(n)) SPACE(f 2 (n)) deterministic simulation of nondeterministic space-bounded computation does not incur that large overhead! Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 14
Next More about space complexity classes: NL vs. conl PSPACE Motivation: Better understanding of space complexity Differences between time complexity and space complexity Start with NOPATH: How complicated is it to decide if a given graph G does not contain a path from s to t? Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 15
NOPATH NOPATH: Nonexistence of a path NOPATH = {(G, s, t) G contains no path from s to t} Decision Problem: Does a given directed graph G not contain a path from s to t? Instance: A directed graph G = (V, E) and s, t V. Question: In which complexity classes is NOPATH for sure? Definition: conl= {A A c NL} s t Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 16
NOPATH NOPATH: Nonexistence of a path NOPATH = {(G, s, t) G contains no path from s to t} Decision Problem: Does a given directed graph G not contain a path from s to t? Instance: A directed graph G = (V, E) and s, t V. Question: In which complexity classes is NOPATH for sure? Answers: P, SPACE(log 2 (n)), conl; NOPATH is conl-complete s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 17 t
NOPATH NOPATH: Nonexistence of a path NOPATH = {(G, s, t) G contains no path from s to t} Decision Problem: Does a given directed graph G not contain a path from s to t? Instance: A directed graph G = (V, E) and s, t V. Question: What if we can prove that NOPATH is in NL? Answer: Then we have proven NL= conl! s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 17 t
REACHABILITY REACHABILITY: Given a directed graph G and a vertex s of G, how many vertices are reachable from s? Instance: A directed graph G = (V, E) and s V. Output: The number of vertices reachable from s. Example: How many in the shown graph? Question: How complicated is it to compute REACHABILITY? s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY REACHABILITY: Given a directed graph G and a vertex s of G, how many vertices are reachable from s? Instance: A directed graph G = (V, E) and s V. Output: The number of vertices reachable from s. Example: How many in the shown graph? Question: How complicated is it to compute REACHABILITY? s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY Theorem (Immerman-Szelepscényi 1988): Given a directed graph G with n vertices and a vertex s of G, the number of vertices that are reachable from s can be computed in NSPACE(log n) = NL. Proof: Let S(k) be the set of vertices of G that are reachable from s on a path of length at most k. S(n 1) is the total number of vertices that are reachable from s. We iteratively compute S(1), S(2),..., S(n 1) with a nondeterministic algorithm using space O(log n) Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY countreachablenodes(g, s) c prev = 1 for k = 1 to n 1 /* c prev : # of vertices reachable in k 1 steps */ c curr = 0 for each v V /* Is v reachable from s in k steps? */ if iselement(s, v, G, k, c prev ): c curr = c curr + 1 c prev = c curr output c prev Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY iselement(s, v, G, k, c prev ) sc = 0, reply = FALSE find path s k 1 w 1 v for each w V goodp ath = TRUE, u curr = s for j = 1 to k 1 /* Guess a k 1 long path */ u prev = u curr, guess u curr if (u prev, u curr ) E: goodp ath = FALSE if u curr == w: break /* check if path is good and ends in w */ if goodp ath and u curr == w sc = sc + 1 if (w, v) E: reply = TRUE if sc < c prev : nondeterministic abort else return reply Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY iselement(s, v, G, k, c prev ): guesses a path of length k 1 for each vertex w. counts vertices w for which it guessed a path checks that it found all (sc == c prev ): some computation branch finds all (first) checks for all w with path whether it can be extended with an edge to v extendible for one w implies answer TRUE countreachablenodes(g, s) iterative computation of c prev enables sanity-check for iselement(s, v, G, k, c prev ) stores only constantly many variables of size log n runs in NSPACE(log n) Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
REACHABILITY Theorem (Immerman-Szelepscényi 1988): Given a directed graph G with n vertices and a vertex s of G, the number of vertices that are reachable from s can be computed in NSPACE(log n) = NL. Proof: Let S(k) be the set of vertices of G that are reachable from s on a path of length at most k. S(n 1) is the total number of vertices that are reachable from s. We iteratively compute S(1), S(2),..., S(n 1) with a nondeterministic algorithm using space O(log n) Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 18
NOPATH is in NL NOPATH: Nonexistence of a path NOPATH = {(G, s, t) G contains no path from s to t} Decision Problem: Does a given directed graph G not contain a path from s to t? Instance: A directed graph G = (V, E) and s, t V. Question: What if we can prove that NOPATH is in NL? Answer: Then we have proven NL= conl! s Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 19 t
NOPATH is in NL Corollary: from Immerman-Szelepscényi it follows that NOPATH is a member of NL Proof by algorithm: unreachability(g, s, t) c prev = 1 for k = 1 to n 1 /* c prev : # of vertices reachable in k 1 steps */ c curr = 0 for each v V /* Is v reachable from s in k steps? */ if iselement(s, v, G, k, c prev ): c curr = c curr + 1 if v == t: REJECT c prev = c curr ACCEPT Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 19
NOPATH is in NL Corollary: Corollary: Corollary: Proof: from Immerman-Szelepscényi it follows that NOPATH is a member of NL NL= conl the Immerman-Szelepscényi theorem: Let f(n) be an arbitrary complexity function with f(n) = Ω(log n). Then it holds that NSPACE(f(n)) = conspace(f(n)). Exercise. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 19
Remarks on L, NL, and conl PATH and NOPATH are both NL-complete. 2SAT (and 2UNSAT) are NL-complete as well. Exercise: Prove that 2UNSAT NL. It is generally believed that NP conp and the same also appeared to hold for NL and conl. The Immerman-Szelepscényi theorem is one of the most surprising ones concerning the relationships of complexity classes, showing that the intuition about computation has still many gaps! It still remains open whether L NL. Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 20
Remarks on L, NL, and conl PATH and NOPATH are both NL-complete. 2SAT (and 2UNSAT) are NL-complete as well. Exercise: Prove that 2UNSAT NL. It is generally believed that NP conp and the same also appeared to hold for NL and conl. The Immerman-Szelepscényi theorem is one of the most surprising ones concerning the relationships of complexity classes, showing that the intuition about computation has still many gaps! It still remains open whether L NL. Thank you for your attention! Birgit Vogtenhuber Problem Analysis and Complexity Theory, 716.054, winter term 2014/2015 20