Outline 2 CDM Turing Machines Turing Machines 2 Coding and Universality Klaus Sutner Carnegie Mellon University 3-turing-mach 205/8/30 9:46 3 The Busy Beaver Problem 4 Wolfram Prize 5 Church-Turing Thesis Intuitive Computability 3 Alan Turing 4 How can we capture the intuitive notion of computability? Historically, the first notions of computability suggested around 930 were Gödel-Herbrand general recursive functions, and Church s λ-calculus. Gödel was not entirely convinced that these proposals properly capture computability. Soundness and Completeness 5 Turing s Machines 6 We need to identify a formal notion of computability that arguably does not go beyond what a human computor could do (in principle), and arguably captures everything a human computor can do. Note that qualifier arguably, there won t be a theorem. Basic Idea: Observe the computor. We all agree that mathematicians compute (among other things such as drinking coffee or proving theorems). So we could try to define an abstract machine that can perform any calculation whatsoever that could be performed in principle by a mathematician, and only those. To this end we need to formalize what a computor is doing. Mathematicians can always write down their calculations on paper. This amounts simply to bookkeeping of all intermediate results. We need to formalize what can be written down and the rules that control the process of writing things down. Also note that Turing had terrible handwriting, he was always interested in typewriters. His machines are, in a sense, just glorified typewriters.
The Abstraction 7 Digression 8 Mathematicians use two-dimensional notation, but could all be flattened out into one-dimensional notation. Only finitely many symbols are allowed. Can think of having a strip of paper subdivided into cells, each cell containing exactly one symbol (possibly the blank symbol). 2345 * 6789 74070 8645 98760 05 8380205 Flattening out notation is standard when dealing with keyboard input. Instead of 0 2 + x one has to deal with something like arctan( dx = 2 2 2 ) Integrate[/(2 + x^2), {x,0,}] ==> ArcTan[/Sqrt[2]]/Sqrt[2] is flattened out into the linear string 2345 * 6789 74070 8645 98760 05 = 8380205 The Rules 9 Justification 0 The input is written on an otherwise empty tape. The computor s mind has finitely many possible states and is always in one particular of these states. At each step, the computor focuses on one particular cell, the current cell. The computor inspects the current cell and takes action depending on his own state of mind and the symbol in the cell. She may overwrite the symbol in the current cell. She may shift attention to the cell on the left or right. She may switch into a new state of mind. Initially, she is in state start and looks at the first empty cell to the left of the input. The computor keeps doing this until a state of mind finished is reached. These assumptions are quite robust. It would take the computor infinitely long to learn the meaning of infinitely many symbols. Likewise, the mind can assume only finitely many configurations (quantum physics, digital physics). A human mind cannot pay attention to infinitely many symbols. But finitely many symbols is the same as, say, 2. Likewise a step cannot depend on infinitely many symbols. But using finitely many symbols is the same as just using one. All but finitely many cells will always contain a special blank symbol. Otherwise someone would already have had to spend an infinite amount of time writing down the initial tape contents. What is written on the tape at this point constitutes the result of the computation. Finite Tapes A Turing Machine 2 It should be noted that the last condition is a bit harsh. It is certainly justified to insist on a very simple initial tape inscription, but all blanks on either side of the input pushes things a bit. For example, an inscription of the form work tape b a c a b a a a read/write head... 000000000000x x 2... x n x n000000000000... is surely harmless. For Turing machines, this makes no difference whatsoever, but there are other models of computation such as cellular automata where the more general starting configurations are of great interest. finite state control
The Pieces 3 Formalization 4 A tape: a bi-infinite strip of paper, subdivided into cells. Each cell contains a single letter; all but finitely many contain just a blank. So we have a tape inscription. A read/write head that is positioned at a particular cell. That head can move left and right. A finite state control that directs the head: symbols are read and written, the head is moved around and the internal state of the FSC changes. alphabet Σ: finite set of allowed symbols, special blank symbol state set Q: finite set of possible mind configurations δ : Q Σ Q Σ {, 0, } : transition function a special initial state q 0 Q a special halting state q H Q. Definition A Turing machine is a structure M = Q, Σ, δ, q 0, q H. Σ Example: Successor Function 5 Simulation 6 Tape alphabet Σ = {, } States Q = {0,, 2, 3} Initial state q 0 = 0, final state q H = 3. The transition function δ is given be the following table: p σ δ(p, σ) 0 + 2 0 2 3 0 2 2 No other transitions are needed. It is not hard to write a program that simulates a Turing machine. Hence, do some computational experiments on these machines. The following visualization method is sometimes useful to understand behavior of small Turing machines. In the pictures: The tape is represented by a row of colored squares. Time flows from top to bottom (usually; sometimes from left to right). The position of the tape head is indicated by a black frame around a square. The rightmost column indicates the state of the machine. Sample Run 7 Addition 8 We are using unary notation, n is represented by n... } {{ } n+ Hence we have to erase two s at the end: 0 is the initial state, 6 is the final state 0 2 2 3 2 2 3 4 4 5 5 6 0 5 5
Sample Run 9 A Longer Run 20 Correctness 2 Palindromes 22 Note that for very simple tasks such as addition in unary one can read off a correctness proof from these pictures. What would a complete correctness proof for the Turing machine that performs unary addition look like? What is difficult about the proof? Construct a Turing machine that performs addition when the input is given in binary. What should be pictures look like in this case? How hard is a correctness proof? Comments 23 Formalizing Turing Computation 24 In the pictures, time flows from left to right and the states are in the top row. The first picture represents a unsuccessful computation. The second picture represents a successful computation. Figure out how this machine works. (Hard) Show that any one-tape Turing machine requires quadratic time to recognize palindromes. We need to describe a snapshot during a computation; we need the current tape inscription the current head position the current state For the following, assume for simplicity that Q Σ =. Definition A configuration or instantaneous description (ID) is a word b p a where a, b Σ and p Q: b mb m... b p a a 2... a n means that the read/write head is positioned at a and the whole tape inscription is b m... b a... a n.
The Step Relation 25 Many Steps 26 We need to explain the one-step relation bpa M b qa Let δ(p, a ) = (q, c, ) Then the next configuration is defined by b m... q b ca 2... a n = b m...b q ca 2... a n = 0 b m...b c q a 2... a n = + Here we assume that a and b are non-empty, otherwise we can set b = a =. As always, we extend the one-step relation to multiple steps by iteration: C M t C M C M C : one step C : exactly t steps C : any finite number of steps The last notion is problematic: there is no bound on the number of steps. Input and Output 27 Computations 28 Given any input x, x 2,..., x n Σ the initial configuration is C x = q 0 A final configuration is of the form x x 2... x n C H y = q H y y 2... y m For input we have chosen to place the head at the last blank to the left of the input symbol, there are lots of other possibilities (e.g, q 0x... x n). Also, we require our machines to erase the tape (except for the output) and return the tape head to the initial position before halting; nothing important changes if we drop this condition. If C x M C H y then y = y, y 2,..., y m is the output of the computation of machine M on input x. M computes f : N k N if, for all x N k, If f is defined on x then C x M C H y and f(x) = y. If f is undefined on x then the computation of M on x does not halt. Here x is a k-tuple of natural numbers and x Σ is the corresponding tape-inscription. Halting 29 Computable Functions 30 Note that a Turing machine may very well fail to reach the halting state q H during a computation. For example, the machine could ignore its input and just move the head to right at each step. Or it could go into an infinite loop where the same sequence of configurations repeats over and over again. As it turns out, this is not a bug but a feature: we will see shortly that failure to halt is a problem that cannot be eliminated. There are restricted types of Turing machines that a guaranteed to halt on all inputs, but unfortunately they do not capture the full notion of computability. Definition A function f : N k N is (Turing) computable if there is a Turing machine M that computes f. Note that these functions are potentially partial: on any particular input the TM may fail to halt. In fact, the machine may fail to halt on all inputs. Historically these functions are also called partial recursive functions. The ones that are in addition total are called (total) recursive functions.
Relations 3 Semidecidable Relations 32 We can use the characteristic function of a relation R { x R char R(x) = 0 otherwise. to translate relations into functions. Definition A relation is (Turing) decidable if its characteristic function is Turing computable. Historically these relations are also called recursive relations. Since Turing machines may fail to halt one and use halting to characterize a class of relations. Definition A relation R is (Turing) semidecidable if there is a Turing machine M that halts precisely on all x such that R(x). We will see shortly that a relation R is decidable iff R and its complement are both semidecidable. Semidecidable sets are also called recursively enumerable (r.e.). Turing Computability and Robustness 33 Input/Output Tapes 34 So we have a notion of a function being computable and a relation being decidable (or semidecidable). b a work tape c a b a a a These notions do not change if we modify our definitions slightly: one-way infinite tapes multiple tapes multiple heads different input/output conventions 0 0 0 input tape 0 output tape Note that without this kind of robustness our model would be essentially useless. M Input/Output Tapes 35 The Important Ideas 36 In this model, the input tape is read-only, one-way infinite output tape is write-only, one-way infinite work tape is read/write, two-way infinite As we will, see this is just the right model for space complexity. We can specify a model of computation by defining a space C of possible configurations (snapshots), a next configuration relation, an input and output convention. The details vary, but it s always the same pattern. Details about input/output conventions become really important in low complexity classes; higher up they are all interchangeable.
Models of Computation 37 Models of Computation, II 38 C C C C C C N k N Σ Σ The number-theoretic scenario (input and output are natural numbers). The string scenario (input and output are words over some alphabet). s 39 Explain how a multi-tape TM can be simulated by a single-tape TM. Determine how long it takes to recognize palindromes on two tapes versus one tape. Turing Machines 2 Coding and Universality The Busy Beaver Problem Determine the general slow-down caused by switching from two tapes to one. Show that it does not matter whether the tape head is required to return to the standard left position at the end of a computation. Wolfram Prize Church-Turing Thesis TMs and Data Structures 4 Leopold Kronecker 42 Since Turing machines operate only on strings it is not entirely clear how powerful they are, compared to, say, C programs. For example, could one concoct a TM that computes shortest paths in a graph? The key issue is that we need to code the graph as a string. Plus all other needed data structures: lists, trees, matrices,... This is not particularly difficult, but it is worth spelling out the details once and for all. In fact, we will not use strings but an more elementary setting: natural numbers (this approach is more interesting in the context of provability and systems of arithmetic). Die ganzen Zahlen hat der liebe Gott gemacht, alles andere ist Menschenwerk. Dear god made the integers, everything else is the work of men.
Sequence Numbers 43 Length 44 We would like to express a sequence a, a 2,..., a n of natural numbers as a single number a, a 2,..., a n. So we need a coding function, a polyadic map of the form. : N N and a decoding function that extracts the components of a coded sequence dec : N N N Note that dec(x, i) need not be defined when i is too large or when x is not of the form a 0, a, a 2,..., a n. One usually insists that the function is total and that there is a length function len( a 0, a, a 2,..., a n ) = n The numbers that appear as codes of sequences are called sequence numbers. so that for i = 0,..., n. a i = dec( a 0, a, a 2,..., a n, i) Show how to check if a number is a sequence number given dec and len. Pairs 45 In Binary 46 The first step is to select a pairing function, an injective map N N N. There are many possibilities, here is a particularly simple choice: π(x, y) = 2 x (2y + ) So π(5, 27) = 760 = 000000 2 Note that the binary expansion of π(x, y) looks like so: y ky k... y 0 00... 0 } {{ } x where y ky k... y 0 is the binary expansion of y with y k being the most significant digit. This makes it easy to find the corresponding unpairing functions: x = π (π(x, y)) y = π 2(π(x, y)). Extending to Sequences 47 That s It! 48 nil = 0 a,..., a n = π(a, a 2,..., a n ) We can now code any discrete structure as an integer by expressing it as a nested list of natural numbers, and then applying the coding function. For example, the so-called Petersen graph on the left is given by the nested list on the right. Here are some sequence numbers for this particular coding function: 0 = 024 0, 0, 0 = 7, 2, 3, 4, 5 = 532754 ((, 3), (, 4), (2, 4), (2, 5), (3, 5), (6, 7), (7, 8), (8, 9), (9, 0), (6, 0), (, 6), (2, 7), (3, 8), (4, 9), (5, 0))
Not So Fast 49 Coding TMs 50 Of course, we also need to be able to operate on sequence numbers. Construct a TM that computes the length of a sequence, given the code number as input. Construct a TM that computes the two unpairing functions x = π (π(x, y)) and y = π 2(π(x, y)) Construct a TM that implements the join of two lists given as sequence numbers. A Turing machine is essentially a n 5 table of numbers: The transition function has the form δ : Q Σ Q Σ and we can identify the set of states Q, the alphabet Σ and the displacement with natural numbers. We can further assume without loss of generality that Q is the initial state and, for Q = {, 2,..., n}, that n is the halting state. But the table can be expressed by a single number using our coding machinery. We will write M for the number associated with TM M. Universal Turing Machines 5 Comments 52 The fact that we can code a Turing machine as an integer combined with the observation that Turing machine naturally can compute on integers has an interesting consequence: We can build a Turing machine U that takes as input 2 numbers e and x, interprets e as the code number of a Turing machine M, and then executes M on input x. U will halt on e and x iff M halts on x; moreover, U will return the same output in this case. Any such machine U is called a universal Turing machine. The details of the construction of U are very tedious, but it s clear that this can be done. If U is given a number e as input that fails to code a Turing machine we assume that U fails to halt. Clearly the argument also works for Turing machines with multiple inputs (which we can either keep separate or code into a single input). Note that U is by no means uniquely determined, there are lots and lots of choices. The simulation by a universal TM is even efficient in the sense that the computation of U won t take much longer than the computation of M. A very interesting question is how large a universal Turing machine needs to be. Amazingly, there is a 2-state, 5-symbol UTM (see the lecture on cellular automata). Enumerating Computable Functions 53 Convergence and Divergence 54 Fix some universal Turing machine U. Each TM M translates into an integer e = M, the index for M. Similarly we can think of e as describing a particular computable function, often written ϕ e or {e} Hence, ({e}) e is a complete listing of all computable functions. We won t deal with arity issues here, they are purely technical. When dealing with partial recursive functions it is convenient to have some shorthand notation for convergence and divergence: Similarly one writes {e} (x) versus {e} (x) {e} (x) y to mean that the left hand side converges with output y, or both sides are undefined. Note that {e} (x) is semidecidable.
Partial versus Total 55 The Halting Problem 56 It might be tempting to think of partial recursive functions as total recursive functions restricted to some smaller domain of definition. Tempting, but very wrong. Proposition There is a partial recursive function g that is not the restriction of any total recursive function. Proof. Let g(e) = { {e} (e) + if {e} (e), otherwise. It is easy to see that g cannot be of the form g f D for any total recursive function f. Theorem Problem: Halting Problem Instance: An index e, an input x. Question: Does {e} (x) converge? The Halting Problem is undecidable. Proof. Suppose otherwise and define a function g by { {e} (e) + if {e} (e), g(e) = 0 otherwise. Then g is computable, so g {e} for some e. Contradiction via input e. Nota Bene 57 Turing Machines A definition of the form g(x) = { f(x) x A, otherwise. where f is computable and A is semidecidable produces a computable function. But g(x) = { f (x) x A, f 2(x) otherwise. Coding and Universality 3 The Busy Beaver Problem Wolfram Prize does not in general. It is OK when A is decidable, though. Church-Turing Thesis Busy Beaver Problem 59 Variants 60 Here is a famous problem due to Tibor Rado (962). Consider only TMs on tape alphabet Σ = {0, } (where 0 is the blank symbol) and n states. Question: What is the largest number of s any such machine can write on an initially empty tape, and then halt? We write β(n) for this number and refer to β : N N as the Busy Beaver function. Halting is crucial, otherwise we could trivially write infinitely many s. We will not insist that the s are contiguous (that is a variant of the problem with much the same properties). It is standard practice to ignore the halting state in the count, so n means n ordinary states plus one halting state. There are several variants of the BBP in the literature: What is the largest number written in unary (one contiguous block of s) that can be written by a halting n-state machine? What is the largest number of moves a halting n-state machine can make (time complexity)? What is the largest number of tape cells a halting n-state machine can use (space complexity)? These are all closely related, we will only discuss the total number of s version. Establish a hierarchy of corresponding Busy Beaver functions.
Busy Beaver n = 6 Busy Beaver n = 2 62 Amazingly, the answer is no longer obvious: β(2) = 4. OK, this is nearly trivial, but still... β() = : if we were to write a second we would never halt. 0 p (q,,r) (q,,l) q (p,,l) halt p0 q0 p q0 p0 q Busy Beaver n = 3 63 How bad can it be? 64 n 2 3 4 5 6 β(n) 4 6 3 4098 4.6 0 439 Already for n = 5 we only have a lower bound, not the exact value. The champion machine due to Marxen and Buntrock is a small miracle: there are 89 628 286 980 80 possible machines (this is a brute-force count, one can trim this number a bit). The Marxen-Buntrock Machine 65 Marxen-Buntrock 800 Steps 66 Entry (q, b, X) in position (p, a) means: in state p, scanning an a, go into state q, write b and move X. 0 (2,,R) (3,,L) 2 (3,,R) (2,,R) 3 (4,,R) (5,0,L) 4 (,,L) (4,,L) 5 halt (,0,L) Currently this machine is the 5-state champion: no other halting 5-state machine is known that produces more ones.
Misleading Pictures 67 Why is this Hard? 68 Looking at a run of the Marxen-Buntrock machine for a few hundred or even a few thousand steps one invariably becomes convinced that the machine never halts: the machine zig-zags back and forth, sometimes building solid blocks of s, sometimes a striped pattern. Whatever the details, the machine seems to be in a loop (not a an easy concept for Turing machines). Bear in mind: there are only 5 states, there is no obvious method to code an instruction such as do some zig-zag move million times, then stop. Still, this machine stops after 47 76 870 steps. There are several fundamental obstructions to computing β(n). No one has a strong theory of the behavior of Turing machines (or any other model of computation). Hence, one has to use brute-force search, at least to some degree. But the number of Turing machines on n states grows wildly exponentially. Most important is the last problem: Even if we could somehow consider all machines on, say, 0 states, there is the problem that we don t know if a machine will ever halt it might just keep running forever. The Halting Problem is directly responsible for BB being so hard. Small Machines can be Complicated 69 Busy Beaver s 70 Define a (n, k)-turing machine to be a TM that has n states and a tape alphabet of size k. Clearly, there is a Busy Beaver problem for (n, k) TMs, the standard problem is just the special case (n +, 2). Very little is known about the general case. In a similar spirit, one can ask for small values of n and k if there is a universal (n, k) machine. One would expect a trade-off between n and k. Some values where universal machines are known to exist are (24, 2), (0, 3), (7, 4), (5, 5), (4, 6), (3, 0), (2, 8), (2, 5) Derive the transition table of the 3-state Busy Beaver machine from the last picture. Give an intuitive explanation of how this machine works. Prove that the last machine is indeed the champion: no other halting 3-state machine writes more than 6 ones. (Hard) Find the Busy Beaver champion for n = 4. (Extremely Hard) Organize a search for the Busy Beaver champion for n = 5. A Prize Question 72 Turing Machines Coding and Universality In May 2007, Stephen Wolfram posed the following challenge question: The Busy Beaver Problem 4 Wolfram Prize Is the following (2,3)-Turing machine universal? 0 2 p (p,,l) (p,0,l) (q,,r) q (p,2,r) (q,0,r) (p,0,l) Church-Turing Thesis Prize money: $25,000.
A Run 73 Another 74 Head Movement 75 Compressed Computation 76 0 5 0 5 0 50 00 50 200 250 Compressed Computation with Different Initial Condition 77 The Big Difference 78 We saw how to construct a universal universal Turing machine. But the prize machine is not designed to do any particular computation, much less to be universal. The problem here is to show that this tiny little machine can simulate arbitrary computations given the right initial configuration (presumably a rather complicated initial configuration). Alas, that s not so easy.
The Big Controversy 79 Turing Machines In the Fall of 2007, Alex Smith, an undergraduate at Birmingham, submitted a proof that the machine is indeed universal. Coding and Universality The proof is painfully informal, fails to define crucial notions and drifts into chaos in several places. The Busy Beaver Problem A particularly annoying feature is that it uses infinite configurations: the tape inscription is not just a finite word surrounded by blanks. Wolfram Prize At this point, it is not clear what exactly Smith s argument shows. 5 Church-Turing Thesis Models of Computation 8 Turing s Contribution 82 There are many other models of computation: general recursive functions, λ calculus, Turing machines, Post systems, Markov algorithms, register machines, random access machines,... Historically, Gödel-Herbrand general recursive functions and Church s λ-calculus were the first fully developed models. Once their equivalence was known, Church unsuccessfully tried to persuade Gödel to accept the following proposition: Claim A function is computable in either model iff it is computable in the intuitive sense. But Gödel was convinced when Turing published his paper in 936. Interestingly, he always gave full credit to Turing for having settled the question of what computability really is but none for Church and himself. The ideas that all these models of computation reflect the true meaning of computation has become known as the Church-Turing thesis. June 23, 202 83 Summary 84 In 202 there were lots of events commemorating Turing s 00th birthday. The most bizarre I ve seen: Some colleagues in Israel have come up with the exciting suggestion of organizing a worldwide Turing night along the lines of the events honoring Yuri Gagarin; see: http://en.wikipedia.org/wiki/yuri s_night Maybe all of us should band together to construct a framework for similar local casual events and parties for June 23 next year and perhaps thereafter. What do you think? Turing machines are a particularly useful model of computation; important historically and the standard model in complexity theory. The Busy Beaver function is a good example of computational hardness that appears at very low levels of a hierarchy. There are several models of computation that are all equivalent in the sense that they produce the same notion of computability and decidability. The Church-Turing thesis states that various formal notions of computability precisely capture the intuitive notion of computability.