4. Turing Machines and Unrestricted Languages

Transcription

1 4. Turing Machines and Unrestricted Languages AI & CV Lab, SNU 0

2 4.1 Turing Machines AI & CV Lab, SNU 1

3 Definition of a Turing Machine Definition 4.1.1: A Turing machine M is defined by M = (Q,,,δ,q 0,,F) where Q is the set of internal states, is the input alphabet, is a finite set of symbols called the tape alphabet, δ: Q x Q x x {L, R} is the transition (partial) function where { }, is a special symbol called the blank, q 0 Q is the initial state, and F Q is the set of final states. AI & CV Lab, SNU 2

4 Definition of a Turing Machine (cont.) Example 4.1.1: Turing machine M = ( Q,,,, q 0,, F ) Q = {q 0, q 1 }, and = {a, b}, = {a, b, }, F = {q 1 }, (q 0, a) = (q 0, b, R), (q 0, b) = (q 0, b, R), (q 0, ) = (q 1,, L) Example 4.1.2: a, a, L; b, b, L;,, L q 0 q 1 (q 0, a) = (q 1, a, R), (q 1, a) = (q 0, a, L), (q 0, b) = (q 1, b, R), (q 1, b) = (q 0, b, L), (q 0, ) = (q 1,, R), (q 1, ) = (q 0,, L) a, a, R; b, b, R;,, R AI & CV Lab, SNU 3

5 Standard Turing Machine: Definition of a Turing Machine (cont.) 1. The Turing machine has a tape that is bounded in both directions allowing any number of left and right moves. 2. The Turing machine is deterministic in the sense that δ defines at most one move for each configuration. 3. There is no special input file. We assume that at the initial time the tape has some specified content. Some of this may be considered as input. Similarly, there is no special output device. Whenever the machine halts, some or all of the contents of the tape may be viewed as output. An instantaneous description (a configuration) of a Turing machine is given by x 1 qx 2 where q is the state of the control unit, x 1 x 2 is the tape contents, and the position of the read-write head between x 1 and x 2. x 1 x 2 input b b AI & CV Lab, SNU 4 control unit

6 Definition of a Turing Machine (cont.) Example 4.1.3: q 0 q 0 q 0 q 1 a a b a b b b b The above pictures correspond to the sequence of instantaneous descriptions q 0 aa, bq 0 a, bbq 0, and bq 1 b. A move from one configuration to another will be denoted by. Thus, if (q 1, c) = (q 2, e, R), then the move abq 1 cd abeq 2 d is made whenever the internal state is q 1, the tape contains abcd, and the read-write head is on the c. * The symbol has the usual meaning of an arbitrary number of moves. Subscripts, such as M, are used in arguments to distinguish between several machines. AI & CV Lab, SNU 5

7 Definition of a Turing Machine (cont.) Definition 4.1.2: Given a Turing machine M, a move a 1 a k-1 q 1 a k a n a 1 a k-1 bq 2 a k+1 a n is possible if and only if δ(q 1, a k ) = (q 2, b, R). M is said to halt starting from some initial configuration x 1 q i x 2 if x 1 q i x 2 * y 1 q j ay 2 for any q j and a for which (q j, a) is undefined. The sequence of configurations leading to a halt state is called a computation. (Note that x 1 qx 2 * indicates that the machine never halts, starting from the initial configuration x 1 qx 2.) AI & CV Lab, SNU 6

8 Turing Machines as Language Accepters Definition 4.1.3: Given a Turing machine M, the language accepted by M is L(M) = {w + : q 0 w * x 1 q f x 2 for some q f F, x 1, x 2 * }. Example 4.1.4: For = {a, b}, design a Turing machine that accepts L = {a n b n : n 1}. M = (Q,,,, q 0,, F), Q={q 0, q 1, q 2, q 3, q 4 }, F= {q 4 }, ={a, b} (q 0, a) = (q 1, x, R) (q 2, y) = (q 2, y, L) (q 0, y) = (q 3, y, R) (q 1, a) = (q 1, a, R) (q 2, a) = (q 2, a, L) (q 3, y) = (q 3, y, R) (q 1, y) = (q 1, y, R) (q 2, x) = (q 0, x, R) (q 3, ) = (q 4,, R) (q 1, b) = (q 2, y, L) Example 4.1.5: Design a Turing machine that accepts L ={a n b n c n : n 1}. AI & CV Lab, SNU 7

9 Turing Machines as Transducers Definition 4.1.4: A function f with domain D is said to be Turing-computable (computable) if there exists some Turing machine M such that Example 4.1.6: q 0 w * M q f f(w), q f F, for all w D. Given two integers x and y, compute f(x+y) = x+y. x: represented by w(x) {1} +, w(x) = x (ex. x=3, w(3)=111) Design a Turing machine for performing the computation q 0 (x)0 (y) q f (x+y)0 M = (Q,,,, q 0,, F), Q={q 0, q 1, q 2, q 3, q 4 }, F= {q 4 } (q 0, 1) = (q 0, 1, R) (q 2, 1) = (q 3, 0, L) (q 0, 0) = (q 1, 1, R) (q 3, 1) = (q 3, 1, L) (q 1, 1) = (q 1, 1, R) (q 3, ) = (q 4,, R) (q 1, ) = (q 2,, L) * AI & CV Lab, SNU 8

10 Turing Machines as Transducers Example 4.1.7: Design a Turing machine that performs the computation * q 0 w q f ww for any w {1} + Example 4.1.8: Let x and y be two positive integers represented in unary notation. Construct a Turing machine that will halt in a final state q y if x y and will halt in a nonfinal state q n if x y. More specifically, the machine is to perform the computation * q 0 w(x)0w(y) q y w(x)0w(y) if x y, * q 0 w(x)0w(y) q n w(x)0w(y) if x y. AI & CV Lab, SNU 9

11 Combining Turing Machines for Complicated Tasks Example 4.1.9: Design a Turing machine that computes the function. f (x, y) = x + y if x y 0 otherwise if x y q C0 w(x)0w(y) q A0 w(x)0w(y) q Af w(x+y)0 if x < y q C0 w(x)0w(y) q E0 w(x)0w(y) q Ef 0 * * * * Example : Design a Turing machine that multiplies two positive integers in unary notation. AI & CV Lab, SNU 10

12 Turing s Thesis Any computation that can be carried out by mechanical means can be performed by some Turing machine. (Not theorem because it cannot be proven. To prove we would have to define precisely the terms mechanical means.) Turing thesis is more properly viewed as a definition of what constitutes a mechanical computation: a computation is mechanical if and only if it can be performed by some Turing machine. AI & CV Lab, SNU 11

13 Turing s Thesis (cont.) Some arguments for accepting the Turing thesis as the definition of a mechanical computation are 1. Anything that can be done on any existing digital computer can also be done by a Turing machine. 2. No one has yet been able to suggest a problem, solvable by what we intuitively consider an algorithm, for which a Turing machine program cannot be written. 3. Alternative models have been proposed for mechanical computation, but none of them are more powerful than the Turing machine model. AI & CV Lab, SNU 12

14 Turing s Thesis (cont.) Definition 4.1.5: An algorithm for a function f: D R is a Turing machine M, which given as input any d D on its tape, eventually halts with the correct answer f (d) R on its tape. Specifically, we can require that * q 0 d M q f f(d), q f F, for all d D. AI & CV Lab, SNU 13

15 4.2 Other Models of Turing Machines AI & CV Lab, SNU 14

16 Equivalence of Classes of Automata Definition 4.2.1: Two automata are equivalent if they accept the same languages. Consider two classes of automata C 1 and C 2. If for every automaton M 1 in C 1 there is an automaton M 2 in C 2 such that L(M 1 ) = L(M 2 ), we say that C 2 is at least as powerful as C 1. If the converse also holds and for every M 2 in C 2 there is an M 1 in C 1 such that L(M 1 ) = L(M 2 ), we say that C 1 and C 2 are equivalent. AI & CV Lab, SNU 15

17 Turing Machines with a Stay-Option Theorem 4.2.1: 1. The class of Turing machines with stay-options is equivalent to the class of standard Turing machines. 2. The class of Turing machines with semi-infinite tape is equivalent to the class of standard Turing machines. 3. The class of off-line Turing machines (Turing machines with read-only input file) is equivalent to the class of standard Turing machines. 4. The class of multi-tape Turing machines is equivalent to the class of standard Turing machines. 5. The class of multidimensional Turing machines is equivalent to the class of standard Turing machines. AI & CV Lab, SNU 16

18 Nondeterministic Turing Machines Theorem 4.2.2: The class of deterministic Turing machines and the class of nondeterministic Turing machines are equivalent. Definition 4.2.2: A universal Turing machine M u is an automaton that, given as input the description of any Turing machine M and a string w, can simulate the computation of M on w. AI & CV Lab, SNU 17

19 Universal Turing Machine How to construct a universal Turing machine? Let Q={q 1,q 2,,q n } where q 1 is the initial state, q 2 is the single final state, and ={a 1,a 2,,a m } where a 1 represents the blank. We then select an encoding in which q 1 is represented by 1, q 2 is represented by 11, and so on. Similarly, a 1 is encoded as 1, a 2 as 11, etc. The symbol 0 will be used as a separator between the 1 s. With the initial and the final state and the blank defined by this convention, an Turing machine can be described completely with δ only. The transition function is encoded according to this scheme, with the arguments and result in some prescribed sequence. For example, δ(q 1, a 2 ) = (q 2, a 3, L) might appear as AI & CV Lab, SNU 18

20 Control unit of M u Description of M Tape contents of M Internal state of M AI & CV Lab, SNU 19

21 Definition 4.2.3: Universal Turing Machine (cont.) Let S be a set of strings on some alphabet. Then an enumeration procedure for S is a Turing machine that can carry out the sequence of steps * * q 0 q s x 1 #s 1 q s x 2 #s 2, with x i * {#}, s i S, in such a way that any s in S is produced in a finite number of steps. The state q s is a state signifying membership in S; that is, whenever q s is entered, the string following # must be in S. Note that a set is countable if there exists an enumeration procedure that produces its elements in some order: Proper order: String elements are ordered in length as the first criterion, followed by an alphabetical ordering of all equal length strings. AI & CV Lab, SNU 20

22 Universal Turing Machine (cont.) Theorem 4.2.3: The set of all Turing machines, although infinite, is countable. AI & CV Lab, SNU 21

23 Definition 4.2.4: Universal Turing Machine (cont.) A linear bounded automaton is a nondeterministic Turing machine, subject to the restriction that must contain two special symbols [ and ], such that δ(q i, [) can contain only elements of the form (q j, [, R) and δ(q i, ]) can contain only elements of the form (q j, ], L). Definition 4.2.5: A string w is accepted by a linear bounded automaton if there is a possible * sequence of moves q 0 [w] [x 1 q f x 2 ] for some q f F, x 1, x 2 *. The language accepted by the linear bounded automaton is the set of all such accepted strings. AI & CV Lab, SNU 22

24 4.3 Unrestricted Grammars and Associated Languages AI & CV Lab, SNU 23

25 Recursive and Recursively Enumerable Languages Definition 4.3.1: A language L is said to be recursively enumerable if there exists a Turing machine that accepts it. Definition 4.3.2: A language L is said to be recursive if there exists a Turing machine M that accepts L and that halts on every w in +. In other words, a language is recursive if and only if there exists a membership algorithm for it. Lemmas: 1. If a language is recursive, then there exists an enumeration procedure. 2. If a language is recursively enumerable, then there exists an enumeration procedure. 3. Every language for which an enumeration procedure exists is recursively enumerable. AI & CV Lab, SNU 24

26 Languages that are not Recursively Enumerable Theorem 4.3.1: Let S be an infinite countable set. Then its powerset 2 S is not countable. Theorem 4.3.2: For any nonempty, there exist languages that are not recursively enumerable. Proof: A language is a subset of *, and every subset is a language. Therefore the set of all languages is exactly 2 *. Since * is infinite, Theorem tells us that the set of all languages on is not countable. But the set of all Turing machines can be enumerated, so the set of all recursively enumerable languages is countable. AI & CV Lab, SNU 25

27 Languages that are not Recursively Enumerable (cont.) Theorem 4.3.3: There exists a recursively enumerable language whose complement is not recursively enumerable. Proof: Let = {a}, and consider the set of all Turing machines with this input alphabet. By Theorem 4.2.3, this set is countable, so we can associate an order M 1, M 2, with its elements. For each Turing machine M i, there is an associated recursively enumerable language L(M i ). Conversely, for each recursively enumerable language on, there is some Turing machine that accepts it. We now consider a new language L defined as follows. For each i 1, the string a i is in L if and only if a i L(M i ). It is clear that the language L is well defined, since the statement a i L(M i ), and hence a i L, must either be true or false. AI & CV Lab, SNU 26

28 Languages that are not Recursively Enumerable (cont.) Proof (cont.): Next, we consider the complement of L, L = {a i : a i L(M i )}, (1) which is also well defined but, as we will show, is not recursively enumerable. We will show this by contradiction, starting from the assumption that L is recursively enumerable. If this is so, then there must be some Turing machine, say M k, such that L = L(M k ). (2) Consider the string a k. Is it in L or in L? Suppose that a k L. By (2), this implies that a k L(M k ). But (1) now implies that a k L. Conversely, if we assume that a k is in L, then a k L and (2) implies that a k L(M k ). AI & CV Lab, SNU 27

29 Languages that are not Recursively Enumerable (cont.) Proof (cont.): But then from (1) we get that a k L. The contradiction is inescapable, and we must conclude that our assumption that L is recursively enumerable is false. To complete the proof of the theorem as stated, we must still show that L is recursively enumerable. We can use for this the known enumeration procedure for Turing machines. Given a i, we first find i by counting the number of a s. We then use the enumeration procedure for Turing machines to find M i. Finally, we give its description along with a i to a universal Turing machine M u that simulates the action of M on a i. If a i is in L, the computation carried out by M u will eventually halt. The combined effect of this is a Turing machine that accepts every a i L. Therefore, by Definition 4.3.1, L is recursively enumerable. AI & CV Lab, SNU 28

30 Language that is Recursively Enumerable but not Recursive Theorem 4.3.4: If a language L and its complement L are both recursively enumerable, then both languages are recursive. If L is recursive, then L is also recursive, and consequently both are recursively enumerable. Proof: If L and L are both recursively enumerable, then there exist Turing machines M and M that serve as enumeration procedures for L and L, respectively. The first will produce w 1, w 2,, in L, the second, w 1, w 2, in L. Suppose now we are given any w +. We first let M generate w 1 and compare it with w. If they are not the same, we let M generate w 1 and compare again. If we need to continue, we next let M generate w 2, then M generate w, and so on. Any w + 2 will be generated either by M or M, so eventually we will get a match. AI & CV Lab, SNU 29

31 Language that is Recursively Enumerable but not Recursive (cont.) Proof (cont.): If the matching string is produced by M, w belongs to L, otherwise it is in L. The process is a membership algorithm for both L and L, so they are both recursive. For the converse, assume that L is recursive. Then there exists a membership algorithm for it. But this becomes a membership algorithm for L by simply complementing its conclusion. Therefore L is recursive. Since any recursive language is recursively enumerable, the proof is completed. AI & CV Lab, SNU 30

32 Theorem 4.3.5: Language that is Recursively Enumerable but not Recursive (cont.) There exists a recursively enumerable language that is not recursive; that is, the family of recursive languages is a proper subset of the family of recursively enumerable languages. Proof: Consider the language L of Theorem This language is recursively enumerable, but its complement is not. Therefore, by Theorem 4.3.4, it is not recursive, giving us the looked-for example. AI & CV Lab, SNU 31

33 Unrestricted Grammars Definition 4.3.3: A grammar G=(V, T, S, P) is called unrestricted if all the productions are of the form u v, where u is in (V T) + and v is in (V T) *. Theorem 4.3.6: Any language generated by an unrestricted grammar is recursively enumerable. Proof: The grammar in effect defines a procedure for enumerating all strings in the language systematically. For example, we can list all w in L such that S w, that is, w is derived in one step. Since the set of the productions of the grammar is finite, there will be a finite number of such strings. Next, we list all w in L that can be derived in two steps S x w and so on. We can simulate these derivations on a Turing machine and, therefore, have an enumeration procedure for the language. Hence, it is recursively enumerable. AI & CV Lab, SNU 32

34 Theorem 4.3.7: Unrestricted Grammars (cont.) For every recursively enumerable language L, there exists an unrestricted grammar G such that L = L(G). Proof: By definition of recursively enumerable language L, there exists a Turing machine M such that L(M) = L. The grammar G=(S,, V, P) is then constructed from M to meet the following: 1. S can derive q 0 w for all w Productions are such that q 0 w xq f y in M if and only if q 0 w xq f y in G. 3. When a string xq f y with q f F is generated, the grammar transforms this string into the original w. The complete sequence of derivations is then S q 0 w xq f y w. To remember w in the third step, the coded version originally has two copies of w. The first is saved, while the second is used in the second step. The first step: S V S SV T, * T TV aa V a0a, for all a. * * * * AI & CV Lab, SNU 33

35 Unrestricted Grammars (cont.) Proof (cont.): The second step: M = (Q,,,δ,q 0,,F) For each transition δ(q i, c) = (q j, d, R) of M, put the productions V aic V pq V ad V pjq, for all a, p { }, q. For each transition δ(q i,c) = (q j, d, L) of M, put the productions V pq V aic V pjq V ad, for all a, p { }, q. The third step: For every q j F, put the productions V ajb a for all a { }, b, which creates the first terminal in the string and then causes a rewriting in the rest by cv ab ca, V ab c ac, for all a, c { }, b. One more special production is added: λ. With the grammar construction above, show that L(M)=L(G) AI & CV Lab, SNU 34

36 Context-Sensitive Grammars and Languages Definition 4.3.4: A grammar G=(V, T, S, P) is said to be context-sensitive if all productions are of the form x y where x, y (V T) + and x y. Definition 4.3.5: A language L is said to be context-sensitive if there exists a context-sensitive grammar G, such that L = L(G) or L = L(G) {λ}. AI & CV Lab, SNU 35

37 Context-Sensitive Grammars and Languages (cont.) Theorem 4.3.8: For every context-sensitive language L not including λ, there exists some linear bounded automaton M such that L = L(M). Proof: If L is context-sensitive, then there exists a context-sensitive grammar for L {λ}. We show that derivations in this grammar can be simulated by a linear bounded automaton. The linear bounded automaton will have two tracks, one containing the input string w, the other containing the sentential forms derived using G. A key point of this argument is that no possible sentential form can have length greater than w. Another point to notice is that a linear bounded automaton is, by definition, nondeterministic. This is necessary in the argument, since we can claim that the correct production can always be guessed and that no unproductive alternatives have to be pursued. Therefore, the computation described in Theorem can be carried out without using space except that originally occupied by w; that is, it can be done by a linear bounded automaton. AI & CV Lab, SNU 36

38 Context-Sensitive Grammars and Languages (cont.) Theorem 4.3.9: If a language L is accepted by some linear bounded automaton M, then there exists a context-sensitive grammar that generates L. Proof: The construction here is similar to that in Theorem All productions generated in Theorem are noncontracting except λ. But this production can be omitted. It is necessary only when the Turing machine moves outside the bounds of the original input, which is not the case here. The grammar obtained by the construction without this unnecessary production is noncontracting, completing the argument. AI & CV Lab, SNU 37

39 Theorem : Every context-sensitive language L is recursive. Relation between Recursive and Context-Sensitive Languages Proof: Consider the context-sensitive language L with an associated context-sensitive grammar G, and look at a derivation of w S x 1 x 2 x n w. We can assume without any loss of generality that all sentential forms in a single derivation are different; that is, x i x j for all i j. The crux of our argument is that the number of steps in any derivation is a bounded function of w. We know that x j x j+1 because G is noncontracting. The only thing we need to add is that there exist some m, depending only on G and w, such that x j < x j+m for all j, with m = m( w ) a bounded function of V T and w. AI & CV Lab, SNU 38

40 Proof (cont.): Relation between Recursive and Context-Sensitive Languages (cont.) This follows because the finiteness of V T implies that there are only a finite number of strings of a given length. Therefore, the length of a derivation of w L is at most w m( w ). This observation gives us immediately a membership algorithm for L. We check all derivations of length up to w m( w ). Since the set of productions of G is finite, there are only a finite number of these. If any of them give w, then w L, otherwise it is not. AI & CV Lab, SNU 39

41 Relation between Recursive and Context-Sensitive Languages (cont.) Theorem : There exists a recursive language that is not context-sensitive. Proof: Consider the set of all context-sensitive grammars on T= {a, b}. We can use a convention in which each grammar has a variable set of the form V={V 0, V 1, V 2, }. Every context-sensitive grammar is completely specified by its productions; we can think of them as written as a single string x 1 y 1 ; x 2 y 2 ; ; x m y m. To this string we now apply the homomorphism h(a) = 010, h(b) = , h( ) = , h(;) = , h(v i ) = 01 i+5 0. Thus, any context-sensitive grammar can be represented uniquely by a string from L((011*0)*). Furthermore, the representation is invertible in the sense that, given any such string, there is at most one context-sensitive grammar corresponding to it. AI & CV Lab, SNU 40

42 Proof (cont.): Let us introduce a proper ordering on {0, 1}+, so we can write strings in the order w 1, w 2, etc. A given string w j may not define a context-sensitive grammar; if it does, call the grammar G j. Next, we define a language L by L = {w i :w i defines a context-sensitive grammar G i and w i L(G i )}, L is well defined and is in fact recursive. To see this, we construct a member-ship algorithm. Given w i, we check it to see if it defines a context-sensitive grammar G i. If not, then w i L. If the string does define a grammar, then L(G i ) is recursive, and we can use the membership algorithm of Theorem to find out if w i L(G i ). If it is not, then w i belongs to L. Relation between Recursive and Context-Sensitive Languages (cont.) But L is not context-sensitive. If it were, there would exist some w j such that L=L(G j ), then by definition w j is not in L. But L=L(G j ), so we have a contradiction. Conversely, if we assume that w j L(G j ), then by definition w j L and we have another contradiction,. We must therefore conclude that L is not context-sensitive. AI & CV Lab, SNU 41

43 Chomsky Hierarchy L RE L RE L CS L CF L REG L REC L CS L CF L DCF L REG Figure 4.1 L CF Figure 4.2 L LIN L REG L DCF Figure 4.3 (Not known whether the family of languages accepted by deterministic linear bounded automata is a proper subset of the context-sensitive language defined to be the language accepted by nondeterministic linear bounded automata.) AI & CV Lab, SNU 42

44 4.4 Limits of Algorithmic Computations AI & CV Lab, SNU 43

45 Computability and Decidability The Turing Machine Halting Problem: Given the description of a Turing machine M and an input w, does M, when started in the initial configuration q 0 w, perform a computation that eventually halts? AI & CV Lab, SNU 44

46 Definition 4.4.1: Turing Machine Halting Problem Let w M be a string that describes a Turing machine M and let w be a string in M s alphabet. We will assume that w M and w are encoded as a string of 0 s and 1 s, as suggested in Section 4.2. A solution of the halting problem is a Turing machine H, which for any w M and w performs the computation * q 0 w M w x 1 q y x 2 if M applied to w halts, and * q 0 w M w y 1 q n y 2 if M applied to w does not halt. Here q y and q n are both final states of H, and q 0 is the initial state of H. AI & CV Lab, SNU 45

47 Theorem 4.4.1: Turing Machine Halting Problem (cont.) There does not exist any Turing machine H that behaves as required by Definition The halting problem is therefore undecidable. Proof: Assume that there exists an algorithm, and consequently some Turing machine H that solves the halting problem. The input to H will be the string w M w. Then the Turing machine H will halt in one of two corresponding final states, say, q y or q n as shown in Figure 4.4. That is, * q 0 w M w H x 1 q y x 2 if M applied to w halts, and * q 0 w M w H y 1 q n y 2 if M applied to w does not halt. < Figure 4.4 > w M w q 0 q y q n H AI & CV Lab, SNU 46

48 Turing Machine Halting Problem (cont.) Proof (cont.): Next, we modify H to produce a Turing machine H' with the structure shown in Figure 4.5. Then the H' operates as follows: * q 0 w M w H' if M applied to w halts, and * q 0 w M w H' y 1 q n y 2 if M applied to w does not halt. < Figure 4.5 > w M w q 0 q y q a q b q n H From H' we construct another Turing machine H''. This new machine H'' takes as input w M and copies it, ending in its initial state q 0. After that, it behaves exactly like H'. Then H'' is such that * * q 0 w M H'' q 0 w M w M H'' if M applied to w M halts, and * * q 0 w M H'' q 0 w M w M H'' y 1 q n y 2 if M applied to w M does not halt. AI & CV Lab, SNU 47

49 Turing Machine Halting Problem (cont.) Proof (cont.): Since H" is a Turing machine, it has a description in {0,1} *, say, w". This string also can be used as input string identifying M with H". Then * q 0 w" H" if H" applied to w" halts, and * q 0 w" H" y 1 q n y 2 if H" applied to w" does not halt. This is clearly nonsense. The contradiction tells that our assumption of the existence of H, and hence the assumption of decidability of the halting problem, must be false. q a q b w M w M w M q 0 q 0 q y q n AI & CV Lab, SNU 48 H H < Figure 4.6 >

50 Turing Machine Halting Problem (cont.) Theorem 4.4.2: If the halting problem were decidable, then every recursively enumerable language would be recursive. Consequently, the halting problem is undecidable. Proof: To see this, let L be a recursively enumerable language on, and let M be a Truing machine that accepts L. Let H be the Turing machine that solves the halting problem. We construct from this the following procedure: 1. Apply H to w M w. If H says no, then by definition w is not in L. 2. If H says yes, then apply M to w. But M must halt, so it will eventually tell us whether w is in L or not. This constitutes a membership algorithm, making L recursive. But we already know that there are recursively enumerable languages that are not recursive. The contradiction implies that H cannot exist, that is, that the halting problem is undecidable. (Reducing One Undecidable Problem to Another) A problem A is reduced to a problem B if the decidability of A follows from the decidability of B. Then if we know that A is undecidable, we can conclude that B is also undecidable. AI & CV Lab, SNU 49

51 Reducing One Undecidable Problem to Another Example 4.4.1: (The state-entry problem) Given any Turing machine M = (Q,,,δ,q 0,,F) and any q Q, w +, decide whether or not the state q is ever entered when M is applied to w. This problem is undecidable. To reduce the halting problem to the state entry problem, suppose that we have an algorithm A that solves the state entry problem: Given any M and w, we first construct ˆM such that Qˆ Q {}. qˆ where ˆq Q. ˆ { ( q ˆ i, a) ( q, a, R) for every halting state in M where is the halting state in M ˆ }. Then M halts on w if and only if ˆM halts in state qˆ. q i ˆq AI & CV Lab, SNU 50

52 Reducing One Undecidable Problem to Another (cont.) Example 4.4.2: (The blank-tape halting problem) Given a Turing machine M, determine whether or not M halts if started with a blank tape. This is undecidable. To show how this reduction is accomplished, assume that we are given some M and some w. We first construct from M a new machine M w that starts with a blank tape, writes w on it, then positions it self in a configuration q 0 w. After that, M w acts like M. Clearly M w will halt on a blank tape if and only if M halts on w. * q 0 ' v q 0 ' q 0 w M... M w * * M w AI & CV Lab, SNU 51

53 Reducing One Undecidable Problem to Another (cont.) Suppose now that the blank-tape halting problem were decidable. Given any (M, w), we first construct M w, then apply the blank tape halting problem algorithm to it. The conclusion tells whether M applied to w will halt. Since this can be done for any M and w, an algorithm for the blank-tape halting problem can be converted into an algorithm for the halting problem. Since the latter is known to be undecidable, the same must be true for the blank-tape halting problem. M, w Generate M w M w Blank-tape halting algorithm A M w halts M w does not halt (M, w) halts (M, w) does not halt Figure 4.7 AI & CV Lab, SNU 52

54 Undecidable Problems for Recursively Enumerable Languages There is no membership algorithm for recursively enumerable languages. To use this theorem, we can reduce the halting problem to the recursively enumerable language related problem. Theorem 4.4.3: Let G be an unrestricted grammar. Then the problem of determining whether or not L(G) = Ø is undecidable. Proof: We will reduce the membership problem for recursively enumerable languages to this problem. Given a Turing machine M and some string w, we modify M to generate M w as follows: M first saves its input, whatever it is, on some special part of its tape. Then whenever it enters a final state, it checks its saved input and accepts it if and only if it is w. Then L(M w ) = L(M) {w}. From M w, the corresponding grammar G w can be constructed such that L(M w ) = L(G w ). It is clear that L(G w ) is nonempty if and only if w L(M). AI & CV Lab, SNU 53

55 Undecidable Problems for Recursively Enumerable Languages (cont.) Suppose there exists an algorithm A for deciding whether or not L(G)=Ø. Then as shown in the following figure, we construct a Turing machine M which for any M and w tells us whether or not w is in L(M). If such a Turing machine existed, we would have a membership algorithm for any recursively enumerable language, in direct contradiction to the previous result. We thus conclude that the problem of L(G)=Ø is not decidable. M, w Construct G w G w Emptiness algorithm A L(G w ) not empty L(G w ) empty w L(M) w L(M) Figure 4.8 AI & CV Lab, SNU 54

56 Theorem 4.4.4: Let M be any Turing machine. Then the question of whether or not L(M) is finite is undecidable. Proof: Consider the halting problem (M, w). From M we construct another Turing machine M' that does the following: First, the halting states of M are changed so that if any one is reached, all input is accepted by M'. This can be done by having any halting configuration go to a final state. Second, the original machine is modified so that the new machine M' first generates w on its tape, then performs the same computations as M, using the newly created w and some otherwise unused space. If M halts in any configuration, then M' will halt in a final state. That is, M' is such that for any v +, q 0 ' v M' q 0 w M x 1 q h x 2 M' x 1 q f 'x 2 where q h is the halting state in M, q 0 ' is the initial state in M', and q f ' is the final state in M'. Undecidable Problems for Recursively Enumerable Languages (cont.) * * AI & CV Lab, SNU 55

57 Undecidable Problems for Recursively Enumerable Languages (cont.) Then if (M,w) halts, M' will reach a final state for all input. If (M,w) does not halt, then M' will not halt either and so will accept nothing. In other words, M' either accepts the infinite language + or the finite language Ø. If we assume the existence of an algorithm A that tells us whether or not L(M') is finite, we can construct a solution to the halting problem as shown in Figure 4.9. Thus no algorithm for deciding whether or not L(M) is finite can exist. M, w Generate M' M' Finiteness algorithm A L(M') is finite L(M') is not finite (M, w) Does not halt (M, w) Halts Figure 4.9 AI & CV Lab, SNU 56

58 Post Correspondence Problem The Post Correspondence Problem: Given two sequences of n strings on some alphabet, say A=w 1, w 2,, w n and B=v 1, v 2,, v n, we say that there exists a Post correspondence solution (PC-solution) for pair (A, B) if there is a nonempty sequence of integers i,j,,k, such that w i w j w k = v i v j v k. The Modified Post Correspondence Problem: The pair (A, B) has a modified Post correspondence solution (MPC-solution) if there exists a sequence of integers i,j,,k, such that w 1 w i w j w k = v 1 v i v j v k. (Note that if there exists an MPC-solution, then there is also a PC-solution, but the converse is not true.) AI & CV Lab, SNU 57

59 Post Correspondence Problem (cont.) Theorem 4.4.5: Let G = (V, T, S, P) be any unrestricted grammar, with w any string in T +. Let (A, B) be the correspondence pair constructed from G and w be the process exhibited in Figure Then the pair (A, B) permits an MPC-solution if and only if w L(G). Proof: (Reasoning Outline) In Figure 4.10, the string FS is to be taken as w 1 and the string F as v 1. The procedure to show that w L(G) if and only if the sets A and B in Figure 4.10 have an MPC-solution is illustrated with the following example: A B FS F F is a symbol not in V T a a for every a T v i v i for every v i V E we E is a symbol not in V T y i x i For every x i y i in P Figure 4.10 AI & CV Lab, SNU 58

60 Post Correspondence Problem (cont.) Example: Let G=({A, B, C}, {a, b, c}, S, P) with productions S aabb Bbb Bb C, AC aac and take w=aaac. The sequences A and B obtained from the suggested construction are given in Figure The derivation of w=aaac is then S aabb aac aaac AI & CV Lab, SNU 59 Figure 4.11 i w i v i 1 FS F 2 a a 3 b b 4 c c 5 A A 6 B B 7 C C 8 S S 9 E aaace 10 aabb S 11 Bbb S 12 C Bb 13 aac AC 14

61 Post Correspondence Problem (cont.) From the following Figure 4.12 and 4.13, the string w 1 w i w j is always longer than the corresponding string v 1 v i v j and the first is exactly one step a head in the derivation. The only exception is the last step, where w 9 must be applied to let the v-string catch up. w 1 w 10 F S a A B b w 1 w 10 F S a A B b w 14 w 2 w 5 w 12 a A C v 1 v 10 v 1 v 10 v 14 v 2 v 5 v 12 Figure 4.12 w 1 w 10 w 14 w 2 w 5 w 12 w 14 w 2 w 13 w 9 F S a A B b a A C a a a c E v 1 v 10 v 14 v 2 v 5 v 12 v 14 v 2 v 13 v 9 Figure 4.13 AI & CV Lab, SNU 60

62 Theorem 4.4.6: Post Correspondence Problem (cont.) The modified Post correspondence problem is undecidable. G, w Construct A and B as in Figure 12.8 A, B MPC algorithm MPC-solution No MPC-solution w L(G) w L(G) Figure 4.14 Theorem 4.4.7: The Post correspondence problem is undecidable. A, B Construct C, D C, D PC algorithm PC-solution No PC-solution MPC-solution No MPC-solution Figure 4.15 AI & CV Lab, SNU 61

63 A,B Post Correspondence Problem (cont.) Proof: Let A = w 1,w 2,,w n and B = v 1,v 2, v n on some alphabet Σ. Using new symbols, and, the new sequences C = y 0,y 1,,y n+1 and D = z 0,z 1,,z n+1 are defined where for i=1,2,,n, y i = w i1 w i2 w imi and z i = v i1 v i2 v iri with w ij and v ij denoting the jth letter of w i and v i, respectively, and m i = w i, r i = v i. In addition, y 0 = y 1, y n+1 =, z 0 =z 1, z n+1 =. Suppose the pair (C,D) has a PC-solution. Then it must look like w 11 w 12 w j1 w k1 = v 11 v 12 v j1 v k1. Ignoring the characters and, we see that this implies w 1 w j w k = v 1 v j v k so that the pair (A,B) permits an MPC-solution. By turning this argument around, it is claimed that if there is an MPC-solution for the pair (A,B), then there is a PC-solution for the pair (C,D). Assume now that the Post correspondence problem is decidable. Then the machine in Fig 4.15 can be constructed, from which we can conclude that the Post correspondence problem is undecidable. AI & CV Lab, SNU 62

64 Theorem 4.4.8: Undecidable Problems for Context-Free Languages There exists no algorithm for deciding whether any given context-free grammar is ambiguous. Proof: Consider two sequences of strings A=(w 1, w 2,, w n ) and B=(v 1, v 2,, v n ) over some alphabet. Choose a new set of distinct symbols a 1, a 2,, a n such that {a 1, a 2,, a n } =, and consider the two languages L A = {w i w j w l w k a k a l a j a i } and L B = {v i v j v l v k a k a l a j a i }. Now look at the context free grammar G=({S, S A, S B }, {a 1, a 2,, a n }, P, S) AI & CV Lab, SNU 63

65 Undecidable Problems for Context-Free Languages (cont.) where the set of productions P is the union of the two subsets: the first set P A consists of S S A, S A w i S A a i w i a i, i = 1, 2,, n the second set P B has the productions S S B, S B v i S B a i v i a i, i = 1, 2,, n Now take G A = ({S, S A }, {a 1, a 2,, a n }, P A, S) and G B = ({S, S B }, {a 1, a 2,, a n }, P B, S). Then clearly L A = L(G A ), L B = L(G B ), and L(G) = L A L B. AI & CV Lab, SNU 64

66 Undecidable Problems for Context-Free Languages (cont.) It is easy to see that G A and G B by themselves are unambiguous. If a given string in L(G) ends with a i, then its derivation with grammar G A must have started with S w i Sa i. Similarly, we can tell at any later stage which rule has to be applied. Thus, if G is ambiguous it must be because there is a w for which there are two derivations and * S S A w i S A a i w i w j w k a k a j a i = w * S S B v i S B a i v i v j v k a k a j a i = w Consequently, if G is ambiguous, then the Post correspondence problem with the pair (A, B) has a solution. Conversely, if G is unambiguous, then the Post correspondence problem cannot have a solution. AI & CV Lab, SNU 65

67 Undecidable Problems for Context-Free Languages (cont.) If there existed an algorithm for solving the ambiguity problem, we could adapt it to solve the Post correspondence problem as shown in Figure But since there is no algorithm for the Post correspondence problem, we conclude that the ambiguity problem is undecidable A, B Construct G G Ambiguity algorithm G is ambiguous G is not ambiguous PC-solution No PC-solution Figure 4.16 AI & CV Lab, SNU 66

68 Theorem 4.4.9: Undecidable Problems for Context-Free Languages (cont.) There exists no algorithm for deciding whether or not L(G 1 ) L(G 2 )=Ø for arbitrary context-free grammar G 1 and G 2. Proof: Take as G 1 the grammar G A and as G 2 the grammar G B as defined in the proof of Theorem Suppose that L(G A ) and L(G B ) have a common element, that is and * S A w i w j w k a k a j a i * S B v i v j v k a k a j a i Then the pair (A,B) has a PC-solution. Conversely, if the pair does not have a PC-solution, then L(G A ) and L(G B ) cannot have a common element. We conclude that L(G A ) L(G B ) is nonempty if and only if (A, B) has a PC-solution. This reduction proves the theorem. AI & CV Lab, SNU 67

69 Undecidable Problems for Context-Free Languages (cont.) A, B Construct G A and G B Algorithm to check if L(G A ) L(G B ) L(G A ) L(G B ) L(G A ) L(G B ) = PC-solution No PC-solution Figure 4.17 AI & CV Lab, SNU 68

70 Theorems related to Unsolvable Problems Assume for the given two sequences of n strings A=(w 1,w 2,..,w n ) and B=(v 1,v 2,,v n ), the associated context-free grammars G A and G B are constructed as in Theorem 12.8 (page 318). Theorem 1: The problem of deciding, for arbitrary context-free languages L A and L B, whether L A L B is infinite is unsolvable. Proof: If the PC solution for two context-free grammars G A and G B exists, then L(G A ) L(G B ) is infinite because the solution is to be pumped. Otherwise, it is empty. Thus the procedure for deciding if it is infinite can determine if the PC solution for the pair (A,B) exists, which contradicts to previously established theorem that the PC problem is unsolvable. AI & CV Lab, SNU 69

71 Theorems related to Unsolvable Problems Theorem 2: The problem of deciding, for an arbitrary context-free language L, whether L c is empty is unsolvable. Proof: Since L(G A ) and L(G B ) are deterministic, their complements are deterministic and context-free. The union of L c (G A ) and L c (G B ) is also context-free, L=L c (G A ) L c (G B ). Then L c = L(G A ) L(G B ). If the PC solution for two context-free grammars G A and G B does not exist, then L c is empty. Otherwise, it is infinite. Thus the procedure for deciding whether the complement of an arbitrary context-free language is empty can determine whether L(G A ) L(G B ) is infinite (empty), which contradicts to Theorem 1. AI & CV Lab, SNU 70

72 Theorems related to Unsolvable Problems Theorem 3: The problem of deciding whether a given context-free language is regular is unsolvable. Proof: Let a context-free language L= L c (G A ) L c (G B ). If L is regular, then L c is also regular. Let L c = L(G A ) L(G B ). If L c is empty, it is regular. Otherwise, it is infinite but not regular because it violates the pumping lemma. Thus the procedure for deciding whether L is regular can determine whether L c is empty (due to regular), which contradicts to Theorem 2. AI & CV Lab, SNU 71

73 Theorems related to Unsolvable Problems Theorem 4: The problem of deciding whether a given context-free language is deterministic is unsolvable. Proof: Suppose there exists a procedure for deciding whether a given context-free language is deterministic. Then apply that procedure to the given context-free language. If deterministic, apply Stearn s procedure to see if it is regular. Otherwise, it is not regular (because the regular language is always deterministic). Thus this procedure can determine whether the given context-free language is regular, which contradicts to Theorem 3. AI & CV Lab, SNU 72

74 Theorems related to Unsolvable Problems Theorem 5: The problem of deciding whether L(G 1 )=L(G 2 ) for arbitrary contextfree grammars G 1 and G 2 (two arbitrary context-free languages are same) is unsolvable. Proof: Let T be the set of terminal letters on which G 1 and G 2 are defined. Let G 2 be defined such that L(G 2 )=T*. If L(G 1 )=L(G 2 ), then L c (G 1 ) is empty. Thus the procedure for deciding if L(G 1 )=L(G 2 ) can also determine if the complement L c (G 1 ) of an arbitrary context-free language L(G 1 ) is empty, which contradicts to Theorem 2. AI & CV Lab, SNU 73