Mathematics for Computer Science

Transcription

1 Mathematics for Computer Science Lecture 3: Turing Machines Areces, Blackburn and Figueira TALARIS team INRIA Nancy Grand Est Contact: Course website:

2 Today s lecture The main purpose of today s lecture is to introduce Turing machines, a simple, abstract and (as we shall see) general model of computation. But although Turing machines are abstract, they reflect certain very concrete ideas about what it is to compute. Indeed, such ideas have long inspired others to create machines of various kinds to compute, as we shall now see. One of the most important skills you will learn today is how to write simple Turing machines. This skill is important for at least two reasons. First, it will teach you some basic programming skills. Second, it will help you to understand and appreciate just how simple and general Turing s model of computation is.

3 Roman abacus It is unclear whether the Roman abacus influenced the Chinese abacus or vice-versa (Rome and China were in contact via trade at the time). What is clear is that the Roman abacus was far slower than the Chinese one and much faster than hand calculation with Roman numerals.

4 Chinese Abacus above (Japanese below) The Chinese abacus was first mentioned around 190 AD. They became widely used around 1200 AD. Note the two rows of count 5 beads.

5 Japanese Abacus (Soroban) The Japanese adopted the abacus from the Chinese around the fifteenth or sixteenth century. They dropped one of the row of the count 5 beads, yielding the modern Soroban abacus, still used in modern Japan.

6 Effectiveness of the abacus On November 12, 1946, a contest was held in Tokyo to compare the Japanese soroban with an electric calculator. The soroban was operated by Kiyoshi Matsuzaki, the calculator by US Army Private Thomas Nathan Wood. The contest compared speed and accuracy for all four basic arithmetic operations (that is, addition, subtraction, division and multiplication) and one problem combined all four operations. The soroban won 4 to 1 (the electric calculator won the multiplication context). This contest formed the basis for a science fiction short story by Arthur C. Clarke in which teams of abacus operators successfully take over the functions of a broken computer on a spacecraft!

7 Inca abacus Inca ( ) calculator. There are several ideas about how it worked, but no real certainty.

8 Gottfried Leibniz Gottfried Leibniz ( ), philosopher, inventor of calculus, pioneer of symbolic logic, and general all-round genius, was also interested in the idea of mechanical calculation. As he put it: It is unworthy of excellent men to lose hours like slaves in the labor of calculation, which could be safely relegated to anyone else if machines were used.

9 Stepped Reckoner (I) In 1671, Leibniz started working on a machine, the Stepped Reckoner, that could perform the four basic arithmetical operations. He gradually improved over the years. The idea attracted a lot of interest, and led to Leibniz being elected a member of the Royal Society in 1673.

10 Stepped Reckoner (II) A number of versions of the machine were constructed in Leibniz s lifetime by a Hanover-based craftsman. However the Leibniz s ideas pushed the technology of the time to its limits: the machine was prone to jamming and malfunctioning.

11 The Turk The idea of computing over other domains than numbers also seems to have been in the air. Chess was an obvious choice. Between , the Turk, a mechanical chess player, took on all comers. Famous victims include Napoleon Bonaparte and Benjamin Franklin. In later incarnations it had a voice box, and uttered the word Check!. The Turk was destroyed by fire in Sadly, three years later, it was revealed to have been a fake; a human being concealed inside operated it.

12 Charles Babbage English pioneer of computer science and sadly, too far in advance of his time to realise his ideas,

13 A forerunner of more than mere computer science...! Here s what he wrote to Tennyson (the English poet) about Tennyson s poem The vision of sin poem: In your otherwise beautiful poem, one verse reads, Every moment dies a man, Every moment one is born... If this was true, the population of the world would be at a standstill. In truth, the rate of birth is slightly in excess of that of death. I would suggest: Every moment dies a man, Every moment 1 1/16 is born. Strictly speaking, the actual figure is so long I cannot get it into a line, but I believe the figure 1 1/16 will be sufficiently accurate for poetry. Geeks rule!

14 The Difference Engine (I) Babbage s first project was the Difference Engine. He conceived of it as a special-purpose mechanical digital calculator, designed to tabulate polynomial functions; he proposed it in an 1822 paper to the Royal Astronomical Society, entitled Note on the application of machinery to the computation of very big mathematical tables. Such tables were of immense important at the time, for example for navigation and gunnery.

15 The Difference Engine (II) This Difference Engine was designed to use the ordinary base 10 number system (that is, ordinary decimal arithmetic) and to be powered by turning a handle. The British government initially financed the project, but stopped when Babbage repeatedly asked for more money when he had made no apparent progress. The Difference Engine was never finished, but was later reconstructed by the Science Museum in London (the picture on the previous slide is of this reconstruction). The reason for Babbage s failure to finish the machine was his all-consuming enthusiasm for a more general project (closer to a modern computer) that he called the Analytical Engine.

16 The Analytical Engine The analytical engine was to have been powered by a steam engine; if it had been built, would have been over 30 metres long and 10 metres wide. The input was to have been via punched cards (such cards were already being used in mechanical looms). There were to be a variety of possible outputs: a printer, a curve plotter, a bell(!) and a card punch. It was designed for ordinary base 10 arithmetic. It could be programmed. Indeed, the Analytical Engine seems to have been comparable in power with the programmable calculators marketed by Texas Instruments and Hewlett Packard in the 1980s.

17 The Mill of the Analytical Engine This is the only part of the Analytical Engine which was ever built. Called the mill, which did the four basic arithmetic operations, comparisons, and square roots.

18 Ada Lovelace the first programmer Ada Lovelace ( ), daughter of Lord Byron, was a was a passionate advocate for the analytical engine and the potential of computing. A mathematician, in 1842 she translated an Italian treatise on the Analytical Engine into English, and added a program for calculating Bernoulli numbers. Babbage called her The enchantress of numbers. The programming language Ada is named after her.

19 Practical computing Leibniz s Stepped Reckoner and Babbage s Analytical Engine were visionary and outstripped the technology of their time. But much existing technology (for example, the punched card) certainly was ready to be exploited for more down-to-earth computing tasks. One such task was tabulation counting and classifying large amounts of data for statistical purposes. The need for speedy methods of doing this was clear by the end of the 19th century. Hollerith designed and built a number of machines for this purposes, building on and extending existing technology.

20 Hollerith s fundamental patent The herein-described method of compiling statistics, which consists in recording separate statistical items pertaining to the individual by holes or combinations of holes punched in sheets of electrically non-conducting material, and bearing a specific relation to each other and to a standard, and then counting or tallying such statistical items separately or in combination by means of mechanical counters operated by electro-magnets the circuits through which are controlled by the perforated sheets, substantially as and for the purpose set forth.

21 The Hollerith Machine Hollerith s company, the Tabulating Machine Company, built and leased a number of such machines to the US Government. His company was one four that eventually merged to form IBM.

22 Columbia Machine Such work led eventually to other impressive technical developments. For example, the Columbia machine combined fast tabulation with the ability to perform arithmetic calculations.

23 Journalistic description of the Columbia Machine Indeed, the Columbia-machine seemed like science fiction to some writers at the time. But behind the journalistic exaggerations, it was clear that technology needed to exploit the theoretical work of Turing and his colleagues was almost in place.

24 How we calculate So what is a Turing machine, and how did the concept arise? Turing thought hard about what was involved in carrying out simple calculations, such as adding a column of numbers, or multiplying two large numbers together. He saw clearly that when performing such calculations, in essence we are carrying out simple algorithms. That is, we are following, step by step, a series of rules. For example, when adding two columns of numbers we learn to add them from the rightmost digit to leftmost, and to carry numbers. When multiplying two numbers greater than 10 we learn to do it digit by digit, to add in extra zeroes, and to obtain the final result by addition.

25 Algorithms are everywhere Following the steps in a recipe book, or following a knitting pattern, are two good examples of everyday algorithms. Indeed, much of the mathematics we learn in school isn t what a mathematician would regard as mathematics at all; rather, it s learning certain algorithms for certain tasks. Early in school we learn algorithms for carrying out addition, multiplication, division, or finding greatest common divisors (using the famous Euclidean algorithm); only much later, probably at university, do we learn the deeper concepts of number theory. Later in school we may learn simple calculus. But at school this is mostly about learning simple algorithms for differentiating and integrating simple functions such as polynomials; only later (at university) so we learn the underlying ideas of continuity, differentiability, and so on.

26 From algorithms to machines The idea of an algorithm, the step by step following of a sequence of precise rules, is one of the fundamental ideas of computer science. But how are algorithms actually realised in the physical world? How can we design machines to really carry them out? Turing answered this question by thinking carefully about what is involved when a person carries out calculations.

27 Turing s analysis Turing saw that performing an algorithm required some sort of workspace a place to write the input, the output, and the intermediate results. (For a human, such a workspace might be a sheet of paper.) He also saw that there needed to be clearly-defined finite number of simple operations that could be performed; for example, how to add two digits, or how to multiply two digits. Last but not least, he abstracted the notion of state. Internal states are the simple model of mind that lie behind Turing s design.

28 Turing machines Putting all together, a Turing machine has three components: An unending tape divided into squares. The tape is the workspace. Input, output, and intermediate calculations are written on the tape. We think of the tape as infinite both to the right and to the left (if you prefer: imagine that we always have the possibility of adding more squares, at either end, should we need them). There is a read/write head (or simple, head) which moves along the tape. The head is restricted in its movement; it can only move one square left or right at a time. The head can read the symbol (if any) that is written on the tape, and write one of a finite number of symbols on the tape. A Turing machine can be in one of a fixed finite number of internal states. The ability of a Turing machine to enter different internal states is the real key to its power.

29 Writing Turing machines A Turing machine starts by scanning the input symbols which are written on the tape. We say a Turing machine is in standard configuration if it is scanning the leftmost symbol of all the input symbols. We write a Turing machine by saying what action the Turing machine is to perform when scanning a certain symbol in a certain state, and by saying what state it goes into after it has performed that action.

30 What actions can Turing machines perform? There are three basic things a Turing machine can do: it can move one square to the left, move one square to the right, or write a new symbol on the square it is reading (overwriting what is already there; note that we regard the blank square as a symbol). We also need to say what state the machine moves into after performing the action. This might be the state it was already in, or a new state. A Turing machine halts if no action is defined for the state it is in. Such a state is called a final state, or a halting state. If a Turing machine halts scanning the leftmost symbol of its answer, we say it halts in standard configuration.

31 An example Turing machine... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3 Here s an example machine. We see a blank tape, the read/write head (the red arrow), and the table which says what the machine is to do when in a given state scanning a given symbol. Note: to prevent ambiguity, in the table we represent the symbol blank square by #.

32 What does this machine do? This machine is designed to start in standard configuration scanning a tape containing a mixture of the symbols $ and x. The head moves along the tape from left to right, and changes every x it finds to a $ (that is, it overwrites every x with a $ symbol). When it reaches the first blank square to the right of the input, it moves left all the way back to the first blank to the left of the input. It then moves one square to the right (so it is scanning the left most dollar symbol) and halts in standard configuration. Fine but how exactly does it do this? Let s work through very carefully and see...

33 Turn Xs to Dollars!... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3 Here s the machine again...

34 Example input... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3 And here s some typical input. What happens next? Well, let s assume we start in State 1

35 Configuration 1... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

36 What to do...?... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

37 There!... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

38 Configuration 2... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

39 What to do...?... $ x $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

40 There...!... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

41 Configuration 3... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

42 What to do...?... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

43 There!... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

44 Configuration 4... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

45 What to do...?... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

46 There!... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

47 Configuration 5... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

48 What to do...?... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

49 There!... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

50 Configuration 6... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

51 What to do...?... $ $ $ $ x... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

52 There... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

53 Configuration 7... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

54 What to do...?... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

55 There!... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

56 Configuration 8... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

57 What to do...?... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

58 There!... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

59 Configuration 9... $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

60 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

61 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

62 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

63 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

64 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3

65 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3 Halt! No Action defined for State 3!

66 Configuration $ $ $ $ $... 1 $ R 1 1 x $ 1 1 # L 2 2 $ L 2 2 # R 3 Halt! No Action defined for State 3!

67 Summing up As promised, the machine started in standard configuration, converted all x symbols to $ symbols, and halted in standard configuration. A simple machine, but one which demonstrated the basic symbol manipulation abilities of Turing machines. Study it well. But the best way to learn to write Turing machines is to study as many of them as possible. So lets look at another example.

68 Turing machines for arithmetic functions We have seen examples of the sort of devices (abacuses and so on) that people have devised during history to calculate arithmetic functions (that is, addition, multiplication, division and so on). Unsurprisingly, Turing machines can calculate such functions too. Let s look at an example: a Turing machine for addition.

69 Representing numbers The first thing to decide is how to represent numbers. Now we could use ordinary decimal notation, or we could use binary notation, but it will make things simpler if we start by using monadic notation. What is monadic notation? Simple! the numeral 1 is represented by 1, 2 is represented by 11, 3 is represented by 111, 4 by 1111, 5 by 11111, and so on (we won t bother to introduce a convention for handling 0, though it is easy to do so). This notation is makes it easy to do addition. For example, is which is just That is, we really just need to stick the two blocks of one representing the input together! How can we devise a Turing machine for doing this?

70 The basic idea We re going to present a Turing machine that will take as input two blocks of 1s (separated by a blank square). The machine will start in standard configuration. It will then stick the two blocks together (thus performing the monadic addition). Furthermore, we shall design the machine is such a way that it halts in standard configuration. Why bother starting and stopping in standard configuration? It may not seem like a very important concept. But is is. We shall see why in the following lecture.

71 The algorithm in more detail Begin by scanning the leftmost 1. Move right, and keep doing so until you get to the blank that divides the two blocks on input ones. When you find the blank separating the two blocks of input, write 1 in the blank, and switch into a new state. Keep moving right until you come to the blank at the right-hand-end of the (now continuous) block of 1s. Move one space left, switch state, and change the 1 to a blank (now the correct number of 1s are on the tape). Move left, change state and keep moving left until you come to the blank at the left-hand-end of the block of 1s. Move right one space and halt. And here s a machine that realizes this algorithm...

72 Monadic addition R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

73 Example input R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

74 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

75 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

76 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

77 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5 We stick the blocks together by changing the blank between them to a 1. This means we have one 1 too many. So we will need to remember to delete rightmost 1.

78 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

79 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

80 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

81 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

82 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

83 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5 There! We changed the rightmost 1 to a blank. So we have the right number of symbols on the tape. The only thing left to do is get the head back to the leftmost 1 symbol.

84 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

85 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

86 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

87 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

88 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

89 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

90 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5

91 Configuration R 1 1 # R 2 2 # L # 3 3 # L L 4 4 # R 5 Halt! State 5 is a final state!

92 Remarks This machine clearly does perform monadic addition. Thus we have seen our first example of a Turing machine which calculates an arithmetic function. Our use of monadic notation for number made addition easy but it makes other operations quite involved. For example, monadic multiplication takes a little more thought. Writing a Turing machine for monadic multiplication is one of the problems on Worksheet 3. It is perhaps worth emphasizing that we used monadic addition because it gave us a simple way to show how arithmetic functions could be implemented. But we weren t forced to do so! We could have defined addition in decimal or binary. And indeed, you ll find such exercises on Worksheet 3.

93 A decision problem Our first Turing machine (the x to $ converter) performed a typical computer science problem, namely converting one kind of symbol to another. Our second Turing machine, the monadic addition machine, carried out a typical mathematical task. Our third machine is going to carry out a typical logical task it is going to provide a yes/no answer to a question. That is, it is going to implement what logicians call a decision procedure. In particular, the machine is going to decide whether the input number (written in monadic) is odd or even.

94 The Algorithm We start in the initial state reading a 1. Assume that there are two other states: an even state and an odd state. Move right, go into the odd state, and then... Odd state: if you read a blank, write odd and halt. If you read a 1, move right, and go into the even state. Even state: if you read a blank, write even and halt. If you read a 1, move right, and go into the odd state. Just keep flipping between the previous two moves!

95 Odd or even? R 3 2 # even R 3 3 # odd R 2 Here s a machine for the algorithm just described. State 2 is the even state and State 3 is the odd state.

96 Example input R 3 2 # even R 3 3 # odd R 2 And here s some typical (even) input. Ready to go... blah blah blah

97 Configuration R 3 2 # even R 3 3 # odd R 2 We start in State 1. We re reading the first 1, so we have so far seen one 1, an odd number. So we go into the odd state (State 3).

98 Configuration R 3 2 # even R 3 3 # odd R 2 We re in the odd state. But we re reading a 1. So now we have seen an even number of 1s. So we go into the even state (State 2).

99 Configuration R 3 2 # even R 3 3 # odd R 2 We re in the even state. But we re reading a 1. So now we have seen an odd number of 1s. So we go into the odd state (State 3).

100 Configuration R 3 2 # even R 3 3 # odd R 2 We re in the odd state. But we re reading a 1. So now we have seen an even number of 1s. So we go into the even state (State 2).

101 Configuration R 3 2 # even R 3 3 # odd R 2 We re in the even state. But now we are seeing a blank square. So we know that the input was an even number. So we write even on the tape and go into State 4

102 Configuration even R 3 2 # even R 3 3 # odd R 2 And this is where we halt, because State 4 is a final state. blah blah blah

103 Remarks So the machine really does implement the odd/even decision problem. Study this machine carefully, and then do the exercise on Worksheet 3 which asks you to modify it in a certain way.

104 Summary of Lecture 3 Key concepts: We introduced the key concept underlying this course, namely the concept of a Turing machine. Turing machines are abstract machines, nonetheless they are based on an analysis of concrete calculations. Indeed Turing machines are essentially a simple mechanism for realizing algorithms using three fundamental concepts: the tape, some basic operations operations (moving right, moving left, and writing symbols), and internal states.

105 Summary of Lecture 3 continued Key concepts: We examined three Turing machines in detail. The first performed a symbol conversion task: it took in a string of mixed xs and $s, changed all the xs to $s, and halted leaving a string of dollars on the tape. The second performed a typical mathematical task: it calculated values of the addition function. For this task we represented numbers using monadic notation, as this greatly simplified the task. Finally, we introduced the notion of a decision problem, and illustrated this with a machine that could decide whether an input number was odd or even. Note: all three machines are calculating functions think about it.

106 Summary of Lecture 3 continued Key concepts: We also mentioned several times the concept of standard configuration for input and output, though as yet we have not seen why this idea is so important. We will see why next week when we start to talk about to build Turing machines to perform more interesting tasks.