Computability and computational complexity

Transcription

1 Computability and computational complexity Lecture 2: Turing machines Ion Petre Computer Science, Åbo Akademi University Fall October 26,

2 Content Algorithms finite vs. infinite Turing machine (TM) basics TM as algorithms TM with multiple tapes Convention: we denote the empty word by 1, the blank space by # October 26,

3 ALGORITHMS: FINITE VS. INFINITE October 26,

4 A model for algorithms To discuss about algorithmic solutions (or lack of them) for a given problem we need a formalization for algorithm Turing machines will be our formal model for algorithms in this course What is an algorithm? A natural list of requirements: i. it consists of a finite number of simple operations ii. the instructions are applied deterministically iii. the instructions are universal they apply to each input instance iv. the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations Without loss of generality we can assume that all algorithms have nonnegative integers as input and output any string can be seen as a nonnegative integer the set S* of strings over the finite alphabet S is isomorphic with N, for any S an algorithm A calculates a function f A :N N October 26,

5 Formalizing the notion of algorithm Assume that we have an arbitrary formalization for algorithm, that satisfies conditions i-iv this means to have a precise definition for an algorithm it also means they can be effectively enumerated: A 1, A 2, A 3,, where effective means that for a given i, algorithm A i can be found Consider now the following algorithm D: its input is a nonnegative integer for each input n, define D(n)=f An (n)+1 in other words: for an input n, select algorithm A n, calculate its output on n and give its successor as an output of D Recall our natural requirements of an algorithm: i. it consists of a finite number of simple operations ii. the instructions are applied deterministically iii. the instructions are universal they apply to each input instance iv. the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations October 26,

6 The diagonalization argument: defining the output of D(n) D(n) A 1 A 2 A 3 A n 1 f A1 (1)+1 2 f A2 (2)+1 3 f A3 (3)+1 n f An (n)+1 Note: algorithm D is not in our list! if D=A r, for some r, then we should have f D (r)=f Ar (r), but by definition, we have f D (r)=f Ar (r)+1 October 26,

7 Discussion Note: the technique above is called a diagonalization argument Step ii. is natural (because of practical considerations) but nondeterministic algorithms are also useful, at least for theoretical considerations more on this later in connection to NP The escape is in considering also algorithms that do not satisfy step iv. Conclusion: in any formalization of the notion of algorithm, we must allow partial, rather than total functions allow algorithms that do not terminate for all legal inputs whenever they terminate, they give the correct answer indeed, this is what we have in Turing machines Recall our natural requirements of an algorithm: i. it consists of a finite number of simple operations ii. the instructions are applied deterministically iii. the instructions are universal they apply to each input instance iv. the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations October 26,

8 TURING MACHINES October 26,

9 Turing machines: basic definition Definition: A Turing machine is a quadruple M=(K,Σ, δ,s) K is a finite set of states s K is the initial state Σ is a finite set of symbols, called the alphabet of M; K Σ= ; there are two special characters in Σ: the blank symbol # and the first symbol > δ : KxΣ (K {h, yes, no }) x Σ x {,,-} is the transition function, where: h is a halting state, yes is an accepting state, no is a rejecting state {,,-} are cursor directions assume that {h, yes, no,,,-} (K Σ) = if δ(q,a)=(q,a,d), then we usually write (q,a) (q,a,d) δ can be thought of as the program of the machine (q,>) (p,>, ) Intuitively: the first symbol is never erased and the cursor move on it is always to the right October 26,

10 Computation of a Turing machine Start in the initial state s Input string is >, followed by a finite sequence x of symbols from Σ-{#} we say that x is the input of the machine the cursor is pointing on > From this initial configuration, the machine takes a step according to the transition function enters a new state, rewrites the current symbol, moves the cursor It can never fall-off the left-end of the tape It can go the right arbitrarily much it will read a blank space and handle it according to δ The machine can only stop if it reaches h, yes, or no we say in this case that the machine has halted if the state is yes we say that the machine accepts its input; output is yes if the state is no we say that the machine rejects its input; output is no if the state is h, then its output is the current string of M if >y### is the current string of M, where last(y) #, then M(x)=y; if the machine does not halt, then we write M(x)= October 26,

11 Discussion Turing machines allow non-terminating computations TM defined with strings as inputs can easily consider a number as input through its n-aric representation (decimal, binary, etc) October 26,

12 Example: right transfer machine Input: aba p K σ Σ δ(p,σ) s > (q,>, ) q a (q a,#, ) q a b (q b,a, ) q a # (h,a,-) 1. s, >aba 2. q, >aba 3. q a, >#ba 4. q b, >#aa 5. q a, >#ab# 6. h, >#aba Output: #aba Useful trick to remember: using the states as a finite memory October 26,

13 Example: binary successor function p K σ Σ δ(p,σ) s > (s,>, ) s 0 (s,0, ) s 1 (s,1, ) s # (q,#, ) q 0 (h,1,-) q 1 (q,0, ) q > (t,>, ) t 0 (t,1, ) t 0 (t,0, ) t # (h,0,-) t # (h,#,-) Input: s, > s, > s, > s, > s, >010 # 6. q, >010 # 7. h, >011# Output: 011 Input: s, >11 2. s, >11 3. s, >11 4. s, >11# 5. q, >11# 6. q, >10# 7. q, >00# 8. t, >00# 9. t, 10# 10. t, 10# 11. h, 100 Output: 100 October 26,

14 Configuration Definition: A configuration is (q,w,u) where: q K, w is the string to the left of the cursor, including the symbol scanned by the cursor, u is the string to the right of the cursor Definition: Configuration (q,w,u) yields configuration (q,w,u ) in one step, denoted (q,w,u) (q,w,u ) if: δ(q,a)=(q,a, ) and w=xa, u=by, w =xa b, u =y δ(q,a)=(q,a, ) and w=xa, u=1, w =xa #, u =1 δ(q,a)=(q,a, ) and w=xba, w =xb, u =a u δ(q,a)=(q,a,-) and w=xa, w =xa, u =u Define yields as a transitive closure of yields in one step : configuration (q,w,u) yields (q,w,u ) if there is n 0 and configurations (q i,w i,u i ), 1 i n such that (q,w,u)=(q 0,w 0,u 0 ), (q,w,u )=(q n,w n,u n ) (q i,w i,u i ) (q i+1,w i+1,u i+1 ), for all 1 i n-1 October 26,

15 Example: palindrome checker A string is a palindrome if it consists of the same sequence of letters left-to-right and right-to-left A Turing machine for deciding whether its input is a palindrome or not: remember the first symbol, transform it into #, travel to the right end and compare it to the last symbol if they do not coincide, halt with answer no if they coincide, change the last symbol into # and travel to the left searching for the first non-blank letter repeat the procedure above until a string of length 0 or 1 is left, in which case halt with answer yes Skip the detailed formulation of the machine October 26,

16 Turing machines as algorithms Let L (Σ-{#})* A Turing machine decides L iff for every x (Σ-{#})*, if x L, then M(x)= yes if x L, then M(x)= no In this case, we say that L is a recursive language A Turing machine accepts L iff for every x (Σ-{#})*, x L iff M(x)= yes if x L, M might not terminate on x we could ask that M(x)= for all x L In this case we say that L is a recursively enumerable language Note: any recursive language is recursively enumerable A Turing machine computes a function f : (Σ-{#})* Σ* iff for every x (Σ-{#})*, M halts on x and M(x)=f(x) In this case we call f a recursive function October 26,

17 Solving problems with Turing machines The instance of the problem (e.g., a graph) is given a string representation An algorithm for a decision problem is a TM that decides the corresponding language accept if the input represents a yes instance of the problem, reject otherwise An algorithm for an optimization problem is a TM that computes the corresponding function from strings to strings the output is similarly represented as a string October 26,

18 Discussion: representations Any finite mathematical object can be represented as a finite string elements of a finite set: integers in binary tuples of elements: integers separated by commas sets: use commas and parenthesis graphs: adjacency matrix, with rows separated by a special character Note: representation of numbers unary representation of a number requires exponential space with respect to the binary representation of the same number we always assume in this course binary representations of numbers We always assume a reasonably succinct representation of input important in judging the complexity of a problem October 26,

19 MULTI-TAPE TURING MACHINES October 26,

20 Turing machines with multiple tapes/strings Definition: A k-tape Turing machine is a quadruple M=(K,Σ, δ,s) K is a finite set of states s K is the initial state Σ is a finite set of symbols, called the alphabet of M; K Σ= ; there are two special characters in Σ: the blank symbol # and the first symbol > δ : KxΣ k (K {h, yes, no }) x (Σ x {,,-}) k is the transition function Intuitively: the machine has now k tapes, with a string and an own cursor on each, the cursors can move independently of each other Similarly as for 1-tape TMs, on each tape the first symbol is > the first symbol on each tape is never erased and the cursor move on it is always to the right October 26,

21 Example: more efficient palindrome check Two-tape TM copy the input to the second tape move the cursor of the second tape on the right-most symbol move the two cursors in opposite directions and check the equality October 26,

22 Configurations, transitions of k-tape TM Configuration: a (2k+1)-tuple (q,w 1,u 1,,w k,u k ), where q is the current state w i u i is the current content of the i-th tape with the last symbol of w i holding the cursor Transition: (q,w 1,u 1,,w k,u k ) (q,w 1,u 1,,w k,u k ) defined as follows: denote a i the last symbol of w i, for all 1 i k assume δ(q,a 1,, a k )=(p,a 1,D 1,,a k,d k ) D i = and w i =x i a i, u i =b i y i, w i =x i a i b i, u =y i D i = and w i =x i a i, u i =1, w i =x i a i *, u i =1 D i = and w i =x i b i a i, w i =x i b i, u i =a i u i D i =- and w i =x i a i, w i =x i a i, u i =u i The yields in n steps, denoted n, and yields, denoted *, defined as before October 26,

23 Computations with k-tape TM Start of the computation: (s,>,x,>,1,,>,1) the first tape holds the input with the symbol > in front of it; all other tapes hold just the first symbol > If (s,>,x,>,1,,>,1) *( yes,w 1,u 1,,w k,u k ), then we say that M(x)= yes If (s,>,x,>,1,,>,1) *( no,w 1,u 1,,w k,u k ), then we say that M(x)= no If (s,>,x,>,1,,>,1) *(h,w 1,u 1,,w k,u k ), then we say that M(x)=y, where y is obtained from w k u k by removing the first symbol >, as well as the largest (infinite) suffix formed just by # With these definitions, acceptance, decision, and computation are defined as before October 26,

24 Time If (s,>,x,>,1,,>,1) n (H,w 1,u 1,,w k,u k ), for some H {h, yes, no }, then the time required by M on input x is n If M(x)=, then the time required by M on x is Time bounds for a machine M Let f:n N. We say that M operates within time f(n) if for any input string x, the time required by M is at most f( x ) If a language L is decided by a multi-tape TM operating in time f(n), then we say that L TIME(f(n)) TIME(f(n)) is called a complexity class It is a set of languages October 26,

25 Example Recall the 1-tape TM for checking whether a string of length n is a palindrome it needs n/2 stages first stage: 2n+1 steps (transitions) to compare the first and the last symbol and come back to the beginning of the string second stage: same thing for a string of length n-2 third stage: same for a string of length n-4, etc. Calculate (2n+1)+(2n-3)+(2n-7)+ =(n+1)(n+2)/2 Conclusion: the language of all palindromes is in TIME((n+1)(n+2)/2) Note: the computation may halt much faster, for example for input 01 n-1 Note: the 2-tape TM for checking palindromes takes time at most f (n)=3n+3 à a quadratic speed-up Question: How much speed-up can be gained through multi-tapes? October 26,

26 Multi-tape vs. single-tape TMs Theorem. For any k-tape Turing machine M operating within time f(n) we can construct a (single-tape) TM M operating within time O(f 2 (n)) such that for any input x, M(x)=M (x) Proof: M=(K,Σ,δ,s). We build M =(K,Σ,δ,s) as follows maintain in the unique string of M the concatenation of the k strings of M, delimited by some markers remember the position of the k cursors of M simulate the functioning of M October 26,

27 Multi-tape vs single-tape TM: proof (continued) Let Σ = Σ Σ {>,>, <}, where Σ is a copy of Σ and >, >, < are new symbols M represents a configuration (q,w 1,u 1,,w k,u k ) of M through a configuration (q,>, w 1 u 1 <w 2 u 2 < w k,u k <<), where w i is obtained from w i by replacing the first symbol > with > and the last symbol a i with a i the last two << encode the end of the simulated string Simulating M through M start the computation by shifting its input one position to the right to make space for the symbol >, then add <(> <) k-1 at the end of the string to encode the empty k-1 tapes of M simulate a move of M through two scans of the current string, left-to-right and back October 26,

28 Multi-tape vs single-tape: proof (continued) First scan: gather information about the k currently scanned symbols in M find the k underlined symbols remember them through new states in this way it will know what transition to trigger Second scan: from left to right, stop at each underlined symbol and do the change dictated by M on that string rewrite the current symbol move the cursor by changing an underlined symbol into a normal one and a normal one into an underlined one one complication possible: if in M the move was to the right beyond the limit of the current string on that tape, M will have to create space for a # symbol: move the rest of the string to the right with one position and come back to the current position October 26,

29 Multi-tape vs. single-tape - proof (continued) Proceed with the simulation until M halts. Erase all strings of M except for the last one and halt M October 26,

30 Multi-tape vs. single-tape - proof (continued) How long does M work on input x? recall that M halts within f( x ) steps and so, none of its tapes can become larger than f( x ) hence, total length of the string of M is at most k(f( x )+1)+1 to account for the separators < Simulating one move takes first scan: 2k(f x )+1)+2 because of the scan left to right and back second scan: same, plus the possible shifting to the right (implemented through another double scan left-to-right-to-left), in the worst case to be done for each string of M: at most 2k(f x )+1)+2 +k(2k(f x )+1)+2 ) All in all: O(f( x ) Conclusion: M takes O(f 2 ( x )) steps October 26,

31 Multi-tape vs. single-tape Turing machines Conclusion: strong evidence of the power and stability of Turing machines as a model of computation adding a finite number of tapes does not increase their computational capabilities and only improves their efficiency polynomially We will see later that other additions to the model of basic Turing machines also fail to significantly alter their computing power Thesis: there are no realistic improvements on the Turing machine that will increase the domain of the languages such machines decide, or will affect their speed more than polynomially October 26,

32 Learning objectives An understanding of why (non-halting) Turing machines make up a good model of computing Decision problems, optimization problems as Turing machines Time-complexity Equivalence of multi-tape and single-tape TMs October 26,