# Notes on Complexity Theory Last updated: August, Lecture 1

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look at Sipser s book [3].) A Turing machine is defined by an integer k 1, a finite set of states Q, an alphabet Γ, and a transition function δ : Q Γ k Q Γ k 1 {L, S, R} k where: k is the number of (infinite, one-dimensional) tapes used by the machine. In the general case we have k 3 and the first tape is a read-only input tape, the last is a write-once output tape, and the remaining k 2 tapes are work tapes. For Turing machines with boolean output (which is what we will mostly be concerned with in this course), an output tape is unnecessary since the output can be encoded into the final state of the Turing machine when it halts. Q is assumed to contain a designated start state q s and a designated halt state q h. (In the case where there is no output tape, there are two halting states q h,0 and q h,1.) We assume that Γ contains {0, 1}, a blank symbol, and a start symbol. There are several possible conventions for what happens when a head on some tape tries to move left when it is already in the left-most position, and we are agnostic on this point. (Anyway, by our convention, below, that the left-most cell of each tape is marked there is really no reason for this to ever occur... ). The computation of a Turing machine M on input x {0, 1} proceeds as follows: All tapes of the Turing machine contain the start symbol followed by blank symbols, with the exception of the input tape which contains the start symbol followed by x (and then the remainder of the input tape is filled with blank symbols). The machine starts in state q = q s with its k heads at the left-most position of each tape. Then, until q is a halt state, repeat the following: 1. Let the current contents of the cells being scanned by the k heads be γ 1,..., γ k Γ. 2. Compute δ(q, γ 1,..., γ k ) = (q, γ 2,..., γ k, D 1,..., D k ) where q Q and γ 2,..., γ k Γ and D i {L, S, R}. 3. Overwrite the contents of the currently scanned cell on tape i to γ i for 2 i k; move head i to the left, to the same position, or to the right depending on whether D i = L, S, or R, respectively; and then set the current state to q = q. The output of M on input x, denoted M(x), is the binary string contained on the output tape (between the initial start symbol and the trailing blank symbols) when the machine halts. (When there is no output tape, then the output is 1 if M halts in state q h,1 and the output is 0 is M halts in state q h,0.) It is also possible that M never halts when run on some input x. We return to this point later. 1-1

2 The running time of a Turing machine M on input x is simply the number of steps M takes before it halts; that is, the number of iterations (equivalently, the number of times δ is computed) in the above loop. Machine M is said to run in time T ( ) if for every input x the running time of M(x) is at most T ( x ). The space used by M on input x is the number of cells written to by M on all its work tapes 1 (a cell that is written to multiple times is only counted once); M is said to use space T ( ) if for every input x the space used during the computation of M(x) is at most T ( x ). We remark that these time and space measures are worst-case notions; i.e., even if M runs in time T (n) for only a fraction of the inputs of length n (and uses less time for all other inputs of length n), the running time of M is still said to be T. (Average-case notions of complexity have also been considered, but are somewhat more difficult to reason about.) A Turing machine M computes a function f : {0, 1} {0, 1} if M(x) = f(x) for all x. Assuming f is a total function, and so is defined on all inputs, this in particular means that M halts on all inputs. We will focus most of our attention on boolean functions, a context in which it is more convenient to phrase computation in terms of languages. A language is simply a subset of {0, 1}. There is a natural correspondence between languages and boolean functions: for any boolean function f we may define the corresponding language L as the set L = {x f(x) = 1}. Conversely, for any language L we can define the boolean function f so that f(x) = 1 iff x L. A Turing machine M decides a language L if x L M(x) = 1 (we sometimes also say that M accepts L, though we will try to be careful); this is the same as computing the boolean function f that corresponds to L. 1.1 Comments on the Model Turing machines are not meant as a model of modern computer systems. Rather, they were introduced (before computers were even built!) as a mathematical model of what computation is. Explicitly, the axiom is that any function that can be computed in the physical world, can be computed by a Turing machine ; this is the so-called Church-Turing thesis. (The thesis cannot be proved unless one can formally define what it means to compute a function in the physical world without reference to Turing machines. In fact, several alternate notions of computation have been defined and shown to be equivalent to computation by a Turing machine; there are no serious candidates for alternate notions of computation that are not equivalent to computation by a Turing machine. See [1] for further discussion.) In fact, an even stronger axiom known as the strong Church-Turing thesis is sometimes assumed to hold: this says that any function that can be computed in the physical world, can be computed with at most a polynomial reduction in efficiency by a Turing machine. This thesis is challenged by notions of randomized computation that we will discuss later. In the past 15 years or so, however, this axiom has been called into question by results on quantum computing that show polynomial-time algorithms in a quantum model of computation for problems not known to have polynomial-time algorithms in the classical setting. There are several variant definitions of Turing machines that are often considered; none of these contradict the strong Church-Turing thesis. (That is, any function that can be computed on any of these variant Turing machines, including the variant defined earlier, can be computed on any other 1 Note that we do not count the space used on the input or output tapes; this allows us to meaningfully speak of sub-linear space machines (with linear- or superlinear-length output). 1-2

3 variant with at most a polynomial increase in time/space.) Without being exhaustive, we list some examples (see [1, 2] for more): One may fix Γ to only include {0, 1} and a blank symbol. One may restrict the tape heads to only moving left or right, not staying in place. One may fix k = 3, so that there is only one work tape. In fact, one may even consider k = 1 so that there is only a single tape that serves as input tape, work tape, and output tape. One can allow the tapes to be infinite in both directions, or two-dimensional. One can allow random access to the work tapes (so that the contents of the ith cell of some tape can be read in one step). This gives a model of computation that fairly closely matches real-world computer systems, at least at an algorithmic level. The upshot of all of this is that it does not matter much which model one uses, as long as one is ok with losing polynomial factors. On the other hand, if one is concerned about low level time/space complexities then it is important to fix the exact model of computation under discussion. For example, the problem of deciding whether an input string is a palindrome can be solved in time O(n) on a two-tape Turing machine, but requires time Ω(n 2 ) on a one-tape Turing machine. 1.2 Universal Turing Machines and Uncomputable Functions An important observation (one that is, perhaps, obvious nowadays but was revolutionary in its time) is that Turing machines can be represented by binary strings. In other words, we can view a program (i.e., a Turing machine) equally well as data, and run one Turing machine on (a description of) another. As a powerful example, a universal Turing machine is one that can be used to simulate any other Turing machine. We define this next. Fix some representation of Turing machines by binary strings, and assume for simplicity that every binary string represents some Turing machine (this is easy to achieve by mapping badly formed strings to some fixed Turing machine). Consider the function f(m, x) = M(x). Is f computable? Note: Here f is a partial function, since in this context the given Turing machine M may not halt on the given input x and we leave f undefined in that case. A partial function f is computable if there is a Turing machine U such that for all x where f is defined we have U(x) = f(x). When f(x) is undefined the behavior of U may be arbitrary. An alternative is to consider the (total) function { f (M, x, 1 t 1 if M(x) halts within t steps with output 1 ) = 0 otherwise, whose computability is closely linked to that of f. Another natural possibility is to consider the (total) function { 1 if M(x) halts with output 1 f halt (M, x) = 0 otherwise ; as we will see, however, f halt is not computable. 1-3

4 Perhaps surprisingly, f is computable. We stress that here we require there to be a fixed Turing machine U, with a fixed number of tapes and a fixed alphabet (not to mention a fixed set of states) that can simulate the behavior of an arbitrary Turing machine M that may use any number of tapes and any size alphabet. A Turing machine computing f is called a universal Turing machine. Theorem 1 There exists a Turing machine U such that (1) U(M, x) = M(x) for all x for which M(x) is defined; furthermore, (2) for every M there exists a constant c such that the following holds: for all x, if M(x) halts within T steps, then U(M, x) halts within c T log T steps. Proof We only sketch the proof here. We consider the case where M computes a boolean function, and so has no output tape; U will not have an output tape either. U will use 3 work tapes, and the alphabet Γ that only includes {0, 1}, a blank symbol, and a start symbol. At a high level, U proceeds as follows: 1. First, U applies a transformation to M that results in a description of an equivalent machine M that uses only a single work tape (in addition to its input tape). This is known to be possible, and moreover is possible in such a way that the following holds: if M(x) halts within T steps, then M (x) halts within O(T log T ) steps (see [2, Chap. 12] or [1, Sect. 17]). The description of M is stored on the second work tape of U (the remaining work tapes of U are used to perform the transformation). 2. Next, U applies a transformation to M that results in a description of an equivalent machine M that uses the binary alphabet (plus blank and start symbol). This is known to be possible with only a constant-factor loss of efficiency (see [1, 2]). Thus if M(x) halts within T steps, then M (x) halts within O(T log T ) steps. The description of M is stored on the first work tape of U (the 3rd work tape of U can be used to perform the transformation). 3. Finally, U simulates the execution of M on input x. It can do this by recording the current state of M on its second work tape (recall that the description of M itself is stored on the first work tape of U) and using its third work tape to store the contents of the work tape of M. To simulate each step of M, we have U simply scan the entire description of M until it finds a transition rule matching the current state of M, the current value being scanned in the input x, and the current value being scanned in the work tape of M. This rule is then used by U to update the recorded state of M, to move its heads on its input tape and third work tape (which is storing the work tape of M ), and to rewrite the value being scanned on the work tape of M. If M halts, then U simply needs to check whether the final state of M was an accepting state or a rejecting state, and move into the appropriate halting state of its own. It is not hard to see that U(M, x) = M(x) for any x for which M(x) halts. As for the claim about the running time, we note the following: the first and second steps of U take time that depends on M but is independent of x. In the third step of U, each step of M is simulated using some number of steps that depends on M (and hence M) but is again independent of x. We have noted already that if M(x) halts in T steps then M (x) halts in c T log T steps for some constant c that depends on M but not on x. Thus U(M, x) halts in c T log T steps for some constant c that depends on M but not on x. We have shown that (the partial function) f is computable. What about (the function) f halt? By again viewing Turing machines as data, we can show that this function is not computable. 1-4

5 Theorem 2 The function f halt is not computable. Proof Say there is some Turing machine M halt computing f halt. Then we can define the following machine M : On input (a description of) a Turing machine M, output M halt (M, M). If the result is 1, output 0; otherwise output 1. What happens when we run M on itself? Consider the possibilities for the result M (M ): Say M (M ) = 1. This implies that M halt (M, M ) = 0. But that means that M (M ) does not halt with output 1, a contradiction. Say M (M ) = 0. This implies that M halt (M, M ) = 1. But that means that M (M ) halts with output 1, a contradiction. It is not possible for M (M ) to never halt, since M halt (M, M ) is a total function (and so is supposed to halt on all inputs). We have reached a contradiction in all cases, implying that M halt as described cannot exist. Remark: The fact that f halt is not computable does not mean that the halting problem cannot be solved in practice. In fact, checking termination of programs is done all the time in industry. Of course, they are not using algorithms that are solving the halting problem this would be impossible! Rather, they use programs that may give false negative, i.e., that may claim that some other program does not halt when it actually does. The reason this tends to work in practice is that the programs that people want to reason about in practice tend to have a form that makes them amenable to analysis. References [1] S. Arora and B. Barak. Computational Complexity: A Modern Approach. Cambridge University Press, [2] J.E. Hopcroft and J.D. Ullman. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley Publishing Company, [3] M. Sipser. Introduction to the Theory of Computation (2nd edition). Course Technology,

### Notes on Complexity Theory Last updated: August, 2011. Lecture 1

Notes on Complexity Theory Last updated: August, 2011 Jonathan Katz Lecture 1 1 Turing Machines I assume that most students have encountered Turing machines before. (Students who have not may want to look

### CS 3719 (Theory of Computation and Algorithms) Lecture 4

CS 3719 (Theory of Computation and Algorithms) Lecture 4 Antonina Kolokolova January 18, 2012 1 Undecidable languages 1.1 Church-Turing thesis Let s recap how it all started. In 1990, Hilbert stated a

### Computability Theory

CSC 438F/2404F Notes (S. Cook and T. Pitassi) Fall, 2014 Computability Theory This section is partly inspired by the material in A Course in Mathematical Logic by Bell and Machover, Chap 6, sections 1-10.

### CS154. Turing Machines. Turing Machine. Turing Machines versus DFAs FINITE STATE CONTROL AI N P U T INFINITE TAPE. read write move.

CS54 Turing Machines Turing Machine q 0 AI N P U T IN TAPE read write move read write move Language = {0} q This Turing machine recognizes the language {0} Turing Machines versus DFAs TM can both write

### Turing Machines: An Introduction

CIT 596 Theory of Computation 1 We have seen several abstract models of computing devices: Deterministic Finite Automata, Nondeterministic Finite Automata, Nondeterministic Finite Automata with ɛ-transitions,

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### 1 Definition of a Turing machine

Introduction to Algorithms Notes on Turing Machines CS 4820, Spring 2012 April 2-16, 2012 1 Definition of a Turing machine Turing machines are an abstract model of computation. They provide a precise,

### Lecture 2: Universality

CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

### 3515ICT Theory of Computation Turing Machines

Griffith University 3515ICT Theory of Computation Turing Machines (Based loosely on slides by Harald Søndergaard of The University of Melbourne) 9-0 Overview Turing machines: a general model of computation

### CSE 135: Introduction to Theory of Computation Decidability and Recognizability

CSE 135: Introduction to Theory of Computation Decidability and Recognizability Sungjin Im University of California, Merced 04-28, 30-2014 High-Level Descriptions of Computation Instead of giving a Turing

### Computational Models Lecture 8, Spring 2009

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown Univ. p. 1 Computational Models Lecture 8, Spring 2009 Encoding of TMs Universal Turing Machines The Halting/Acceptance

### NP-Completeness and Cook s Theorem

NP-Completeness and Cook s Theorem Lecture notes for COM3412 Logic and Computation 15th January 2002 1 NP decision problems The decision problem D L for a formal language L Σ is the computational task:

### Theory of Computation Chapter 2: Turing Machines

Theory of Computation Chapter 2: Turing Machines Guan-Shieng Huang Feb. 24, 2003 Feb. 19, 2006 0-0 Turing Machine δ K 0111000a 01bb 1 Definition of TMs A Turing Machine is a quadruple M = (K, Σ, δ, s),

### Lecture 1: Time Complexity

Computational Complexity Theory, Fall 2010 August 25 Lecture 1: Time Complexity Lecturer: Peter Bro Miltersen Scribe: Søren Valentin Haagerup 1 Introduction to the course The field of Computational Complexity

### CAs and Turing Machines. The Basis for Universal Computation

CAs and Turing Machines The Basis for Universal Computation What We Mean By Universal When we claim universal computation we mean that the CA is capable of calculating anything that could possibly be calculated*.

### Outline. Database Theory. Perfect Query Optimization. Turing Machines. Question. Definition VU , SS Trakhtenbrot s Theorem

Database Theory Database Theory Outline Database Theory VU 181.140, SS 2016 4. Trakhtenbrot s Theorem Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 4.

### Computing basics. Ruurd Kuiper

Computing basics Ruurd Kuiper October 29, 2009 Overview (cf Schaum Chapter 1) Basic computing science is about using computers to do things for us. These things amount to processing data. The way a computer

### We give a basic overview of the mathematical background required for this course.

1 Background We give a basic overview of the mathematical background required for this course. 1.1 Set Theory We introduce some concepts from naive set theory (as opposed to axiomatic set theory). The

### Turing Machine and the Conceptual Problems of Computational Theory

Research Inventy: International Journal of Engineering And Science Vol.4, Issue 4 (April 204), PP 53-60 Issn (e): 2278-472, Issn (p):239-6483, www.researchinventy.com Turing Machine and the Conceptual

### Turing Machines, Part I

Turing Machines, Part I Languages The \$64,000 Question What is a language? What is a class of languages? Computer Science Theory 2 1 Now our picture looks like Context Free Languages Deterministic Context

### Introduction to computer science

Introduction to computer science Michael A. Nielsen University of Queensland Goals: 1. Introduce the notion of the computational complexity of a problem, and define the major computational complexity classes.

### Properties of Stabilizing Computations

Theory and Applications of Mathematics & Computer Science 5 (1) (2015) 71 93 Properties of Stabilizing Computations Mark Burgin a a University of California, Los Angeles 405 Hilgard Ave. Los Angeles, CA

### Markov Algorithm. CHEN Yuanmi December 18, 2007

Markov Algorithm CHEN Yuanmi December 18, 2007 1 Abstract Markov Algorithm can be understood as a priority string rewriting system. In this short paper we give the definition of Markov algorithm and also

### FORMAL LANGUAGES, AUTOMATA AND COMPUTATION

FORMAL LANGUAGES, AUTOMATA AND COMPUTATION REDUCIBILITY ( LECTURE 16) SLIDES FOR 15-453 SPRING 2011 1 / 20 THE LANDSCAPE OF THE CHOMSKY HIERARCHY ( LECTURE 16) SLIDES FOR 15-453 SPRING 2011 2 / 20 REDUCIBILITY

### Classical Information and Bits

p. 1/24 Classical Information and Bits The simplest variable that can carry information is the bit, which can take only two values, 0 or 1. Any ensemble, description, or set of discrete values can be quantified

### The Halting Problem is Undecidable

185 Corollary G = { M, w w L(M) } is not Turing-recognizable. Proof. = ERR, where ERR is the easy to decide language: ERR = { x { 0, 1 }* x does not have a prefix that is a valid code for a Turing machine

### Introduction to Turing Machines

Automata Theory, Languages and Computation - Mírian Halfeld-Ferrari p. 1/2 Introduction to Turing Machines SITE : http://www.sir.blois.univ-tours.fr/ mirian/ Automata Theory, Languages and Computation

### Lecture 2: The Complexity of Some Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

### 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008

MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

### Diagonalization. Ahto Buldas. Lecture 3 of Complexity Theory October 8, 2009. Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach.

Diagonalization Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Background One basic goal in complexity theory is to separate interesting complexity

### Reading 13 : Finite State Automata and Regular Expressions

CS/Math 24: Introduction to Discrete Mathematics Fall 25 Reading 3 : Finite State Automata and Regular Expressions Instructors: Beck Hasti, Gautam Prakriya In this reading we study a mathematical model

### Formal Languages and Automata Theory - Regular Expressions and Finite Automata -

Formal Languages and Automata Theory - Regular Expressions and Finite Automata - Samarjit Chakraborty Computer Engineering and Networks Laboratory Swiss Federal Institute of Technology (ETH) Zürich March

### 6.080/6.089 GITCS Feb 12, 2008. Lecture 3

6.8/6.89 GITCS Feb 2, 28 Lecturer: Scott Aaronson Lecture 3 Scribe: Adam Rogal Administrivia. Scribe notes The purpose of scribe notes is to transcribe our lectures. Although I have formal notes of my

### A Formalization of the Turing Test Evgeny Chutchev

A Formalization of the Turing Test Evgeny Chutchev 1. Introduction The Turing test was described by A. Turing in his paper [4] as follows: An interrogator questions both Turing machine and second participant

### Complexity of Algorithms

1 Complexity of Algorithms Lecture Notes, Spring 1999 Peter Gács Boston University and László Lovász Yale University 2 Contents 1 Introduction and Preliminaries 1 1.1 The subject of complexity theory.........................

### THE TURING DEGREES AND THEIR LACK OF LINEAR ORDER

THE TURING DEGREES AND THEIR LACK OF LINEAR ORDER JASPER DEANTONIO Abstract. This paper is a study of the Turing Degrees, which are levels of incomputability naturally arising from sets of natural numbers.

### (IALC, Chapters 8 and 9) Introduction to Turing s life, Turing machines, universal machines, unsolvable problems.

3130CIT: Theory of Computation Turing machines and undecidability (IALC, Chapters 8 and 9) Introduction to Turing s life, Turing machines, universal machines, unsolvable problems. An undecidable problem

### Sets and Subsets. Countable and Uncountable

Sets and Subsets Countable and Uncountable Reading Appendix A Section A.6.8 Pages 788-792 BIG IDEAS Themes 1. There exist functions that cannot be computed in Java or any other computer language. 2. There

### 24 Uses of Turing Machines

Formal Language and Automata Theory: CS2004 24 Uses of Turing Machines 24 Introduction We have previously covered the application of Turing Machine as a recognizer and decider In this lecture we will discuss

### The Classes P and NP

The Classes P and NP We now shift gears slightly and restrict our attention to the examination of two families of problems which are very important to computer scientists. These families constitute the

### Universal Turing Machine: A Model for all Computational Problems

Universal Turing Machine: A Model for all Computational Problems Edward E. Ogheneovo Lecturer I, Dept of Computer Science, University of Port Harcourt, Port Harcourt Nigeria. ABSTRACT: Turing machines

### Chapter 1. Computation theory

Chapter 1. Computation theory In this chapter we will describe computation logic for the machines. This topic is a wide interdisciplinary field, so that the students can work in an interdisciplinary context.

### Automata and Computability. Solutions to Exercises

Automata and Computability Solutions to Exercises Fall 25 Alexis Maciel Department of Computer Science Clarkson University Copyright c 25 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### Introduction to Theory of Computation

Introduction to Theory of Computation Prof. (Dr.) K.R. Chowdhary Email: kr.chowdhary@iitj.ac.in Formerly at department of Computer Science and Engineering MBM Engineering College, Jodhpur Tuesday 28 th

### Universality in the theory of algorithms and computer science

Universality in the theory of algorithms and computer science Alexander Shen Computational models The notion of computable function was introduced in 1930ies. Simplifying (a rather interesting and puzzling)

### 2. Methods of Proof Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try.

2. METHODS OF PROOF 69 2. Methods of Proof 2.1. Types of Proofs. Suppose we wish to prove an implication p q. Here are some strategies we have available to try. Trivial Proof: If we know q is true then

### Homework Five Solution CSE 355

Homework Five Solution CSE 355 Due: 17 April 01 Please note that there is more than one way to answer most of these questions. The following only represents a sample solution. Problem 1: Linz 8.1.13 and

### Fundamentals of Mathematics Lecture 6: Propositional Logic

Fundamentals of Mathematics Lecture 6: Propositional Logic Guan-Shieng Huang National Chi Nan University, Taiwan Spring, 2008 1 / 39 Connectives Propositional Connectives I 1 Negation: (not A) A A T F

### A TM. Recall the A TM language: A TM = { M, w M is a TM and M accepts w.} is undecidable. Theorem: The language. p.2/?

p.1/? Reducibility We say that a problem Q reduces to problem P if we can use P to solve Q. In the context of decidability we have the following templates : If A reduces to B and B is decidable, then so

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### (Refer Slide Time: 1:41)

Discrete Mathematical Structures Dr. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture # 10 Sets Today we shall learn about sets. You must

### Introduction to NP-Completeness Written and copyright c by Jie Wang 1

91.502 Foundations of Comuter Science 1 Introduction to Written and coyright c by Jie Wang 1 We use time-bounded (deterministic and nondeterministic) Turing machines to study comutational comlexity of

### ECS 120 Lesson 17 Church s Thesis, The Universal Turing Machine

ECS 120 Lesson 17 Church s Thesis, The Universal Turing Machine Oliver Kreylos Monday, May 7th, 2001 In the last lecture, we looked at the computation of Turing Machines, and also at some variants of Turing

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### 11 Multivariate Polynomials

CS 487: Intro. to Symbolic Computation Winter 2009: M. Giesbrecht Script 11 Page 1 (These lecture notes were prepared and presented by Dan Roche.) 11 Multivariate Polynomials References: MC: Section 16.6

### Theory of Computation Lecture Notes

Theory of Computation Lecture Notes Abhijat Vichare August 2005 Contents 1 Introduction 2 What is Computation? 3 The λ Calculus 3.1 Conversions: 3.2 The calculus in use 3.3 Few Important Theorems 3.4 Worked

### Computational complexity theory

Computational complexity theory Goal: A general theory of the resources needed to solve computational problems What types of resources? Time What types of computational problems? decision problem Decision

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### OHJ-2306 Introduction to Theoretical Computer Science, Fall 2012 8.11.2012

276 The P vs. NP problem is a major unsolved problem in computer science It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute to carry a \$ 1,000,000 prize for the

### Binary numbers. Try converting 3, 22, and 71 to binary; try converting 10101, 1100, and to decimal. Satisfaction problem

Binary numbers We are used to expressing numbers in base 10, but binary numbers are base 2. Here are some simple examples of decimal numbers and their binary equivalents. Decimal Binary 0 00 1 01 2 10

### Deterministic Finite Automata

1 Deterministic Finite Automata Definition: A deterministic finite automaton (DFA) consists of 1. a finite set of states (often denoted Q) 2. a finite set Σ of symbols (alphabet) 3. a transition function

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### Computing Functions with Turing Machines

CS 30 - Lecture 20 Combining Turing Machines and Turing s Thesis Fall 2008 Review Languages and Grammars Alphabets, strings, languages Regular Languages Deterministic Finite and Nondeterministic Automata

### Lecture 1: Elementary Number Theory

Lecture 1: Elementary Number Theory The integers are the simplest and most fundamental objects in discrete mathematics. All calculations by computers are based on the arithmetical operations with integers

### Incompleteness and Artificial Intelligence

Incompleteness and Artificial Intelligence Shane Legg Dalle Molle Institute for Artificial Intelligence Galleria 2, Manno-Lugano 6928, Switzerland shane@idsia.ch 1 Introduction The implications of Gödel

### Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24

Introduction to Algorithms Review information for Prelim 1 CS 4820, Spring 2010 Distributed Wednesday, February 24 The final exam will cover seven topics. 1. greedy algorithms 2. divide-and-conquer algorithms

### Turing Machines and Understanding Computational Complexity

Turing Machines and Understanding Computational Complexity Paul M.B.Vitányi CWI, Science Park 123, 1098XG Amsterdam The Netherlands 1. Introduction A Turing machine refers to a hypothetical machine proposed

### Lecture 36: Turing Machines [Fa 14]

Caveat lector: This is the zeroth (draft) edition of this lecture note. In particular, some topics still need to be written. Please send bug reports and suggestions to jeffe@illinois.edu. Think globally,

### Finite Automata and Formal Languages

Finite Automata and Formal Languages TMV026/DIT321 LP4 2011 Lecture 14 May 19th 2011 Overview of today s lecture: Turing Machines Push-down Automata Overview of the Course Undecidable and Intractable Problems

### NUMBER SYSTEMS CHAPTER 19-1

NUMBER SYSTEMS 19.1 The Decimal System 19. The Binary System 19.3 Converting between Binary and Decimal Integers Fractions 19.4 Hexadecimal Notation 19.5 Key Terms and Problems CHAPTER 19-1 19- CHAPTER

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### MATH 289 PROBLEM SET 1: INDUCTION. 1. The induction Principle The following property of the natural numbers is intuitively clear:

MATH 89 PROBLEM SET : INDUCTION The induction Principle The following property of the natural numbers is intuitively clear: Axiom Every nonempty subset of the set of nonnegative integers Z 0 = {0,,, 3,

### Discrete Mathematics Lecture 3 Elementary Number Theory and Methods of Proof. Harper Langston New York University

Discrete Mathematics Lecture 3 Elementary Number Theory and Methods of Proof Harper Langston New York University Proof and Counterexample Discovery and proof Even and odd numbers number n from Z is called

### Regular Languages and Finite Automata

Regular Languages and Finite Automata 1 Introduction Hing Leung Department of Computer Science New Mexico State University Sep 16, 2010 In 1943, McCulloch and Pitts [4] published a pioneering work on a

### Quantum Computation CMU BB, Fall 2015

Quantum Computation CMU 5-859BB, Fall 205 Homework 3 Due: Tuesday, Oct. 6, :59pm, email the pdf to pgarriso@andrew.cmu.edu Solve any 5 out of 6. [Swap test.] The CSWAP (controlled-swap) linear operator

### 13 Infinite Sets. 13.1 Injections, Surjections, and Bijections. mcs-ftl 2010/9/8 0:40 page 379 #385

mcs-ftl 2010/9/8 0:40 page 379 #385 13 Infinite Sets So you might be wondering how much is there to say about an infinite set other than, well, it has an infinite number of elements. Of course, an infinite

### Lecture 7: NP-Complete Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 7: NP-Complete Problems David Mix Barrington and Alexis Maciel July 25, 2000 1. Circuit

### NUMBER SYSTEMS. William Stallings

NUMBER SYSTEMS William Stallings The Decimal System... The Binary System...3 Converting between Binary and Decimal...3 Integers...4 Fractions...5 Hexadecimal Notation...6 This document available at WilliamStallings.com/StudentSupport.html

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### CHAPTER 5: MODULAR ARITHMETIC

CHAPTER 5: MODULAR ARITHMETIC LECTURE NOTES FOR MATH 378 (CSUSM, SPRING 2009). WAYNE AITKEN 1. Introduction In this chapter we will consider congruence modulo m, and explore the associated arithmetic called

University of Virginia - cs3102: Theory of Computation Spring 2010 Final Exam - Comments Problem 1: Definitions. (Average 7.2 / 10) For each term, provide a clear, concise, and precise definition from

### Math 4310 Handout - Quotient Vector Spaces

Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable

### On the Relationship between Classes P and NP

Journal of Computer Science 8 (7): 1036-1040, 2012 ISSN 1549-3636 2012 Science Publications On the Relationship between Classes P and NP Anatoly D. Plotnikov Department of Computer Systems and Networks,

### Implementation of Recursively Enumerable Languages using Universal Turing Machine in JFLAP

International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 4, Number 1 (2014), pp. 79-84 International Research Publications House http://www. irphouse.com /ijict.htm Implementation

### The Chinese Remainder Theorem

The Chinese Remainder Theorem Evan Chen evanchen@mit.edu February 3, 2015 The Chinese Remainder Theorem is a theorem only in that it is useful and requires proof. When you ask a capable 15-year-old why

### Computation. Chapter 6. 6.1 Introduction

Chapter 6 Computation 6.1 Introduction In the last two chapters we saw that both the logical and the cognitive models of scientific discovery include a condition to prefer simple or minimal explanations.

### CS_226 Computability Theory

Examples CS_226 Computability Theory http://www.cs.swan.ac.uk/ csetzer/lectures/computability/ 08/index.html Course Notes, Michaelmas Term 2008 Anton Setzer (Dept. of Computer Science, Swansea) http://www.cs.swan.ac.uk/

### GROUPS SUBGROUPS. Definition 1: An operation on a set G is a function : G G G.

Definition 1: GROUPS An operation on a set G is a function : G G G. Definition 2: A group is a set G which is equipped with an operation and a special element e G, called the identity, such that (i) the

### MAT2400 Analysis I. A brief introduction to proofs, sets, and functions

MAT2400 Analysis I A brief introduction to proofs, sets, and functions In Analysis I there is a lot of manipulations with sets and functions. It is probably also the first course where you have to take

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Automata and Formal Languages

Automata and Formal Languages Winter 2009-2010 Yacov Hel-Or 1 What this course is all about This course is about mathematical models of computation We ll study different machine models (finite automata,

### Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU

Welcome to... Problem Analysis and Complexity Theory 716.054, 3 VU Birgit Vogtenhuber Institute for Software Technology email: bvogt@ist.tugraz.at office hour: Tuesday 10:30 11:30 slides: http://www.ist.tugraz.at/pact.html

### Outline 2. 1 Turing Machines. 2 Coding and Universality. 3 The Busy Beaver Problem. 4 Wolfram Prize. 5 Church-Turing Thesis.

Outline 2 CDM Turing Machines Turing Machines 2 Coding and Universality Klaus Sutner Carnegie Mellon University 3-turing-mach 205/8/30 9:46 3 The Busy Beaver Problem 4 Wolfram Prize 5 Church-Turing Thesis

### Finite Sets. Theorem 5.1. Two non-empty finite sets have the same cardinality if and only if they are equivalent.

MATH 337 Cardinality Dr. Neal, WKU We now shall prove that the rational numbers are a countable set while R is uncountable. This result shows that there are two different magnitudes of infinity. But we

### 1. Nondeterministically guess a solution (called a certificate) 2. Check whether the solution solves the problem (called verification)

Some N P problems Computer scientists have studied many N P problems, that is, problems that can be solved nondeterministically in polynomial time. Traditionally complexity question are studied as languages:

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted