Course Manual Automata & Complexity 2015

Course Manual Automata & Complexity 2015 Course code: Course homepage: Coordinator: Teachers lectures: Teacher exercise classes: Credits: X_401049 http://www.cs.vu.nl/~tcs/ac prof. dr. W.J. Fokkink home: http://www.cs.vu.nl/~wanf/ e-mail: w.j.fokkink"at"vu.nl phone: 020-59 87735 dr. ir. J.J.A. Keiren home: http://www.jeroenkeiren.nl e-mail: j.j.a.keiren"at"vu.nl F. Geiger BSc 6 ECTS 1 Introduction This course is the last course in the track on Logic & Algorithms, and one of the last courses in the BSc programme. In this course, concepts and results from earlier courses will be put into a broader perspective, and fundamental questions regarding automata & complexity theory will be studied. We introduce important foundations of computer science. In this area, typical questions that are answered are: What is a (programming) language?, How can languages be recognised by computers (automata), What problems can be solved using a class of automata?, How much time and memory does solving a problem require?. The lecture series consists of three parts: automata & languages, complexity theory and an introduction to quantum computing. In the first part, on automata and languages, the concepts of formal language, grammar, and automaton are introduced. Two types of languages, with the corresponding grammars and automata are covered: regular and contextfree languages. Regular languages are used, e.g., in formulate search queries, in the form of regular expressions. Context-free languages are suitable to describe programming languages. The automata-theoretic counterparts here are finite automata and the more powerful pushdown automata. Of both types of automata it will be investigated whether their deterministic versions are less powerful than their non-deterministic counterpart. A central question in this 1

part of the course is how it can be determined whether or not a language is regular or context-free. Here the notion of pumping lemmas is introduced. Finally, with each type of language we associate a class of grammars. Left- and rightlinear grammars, and context-free grammars. For the latter we also describe normal forms and we show how these normal forms can be used to construct a parser. This can, e.g., be used to parse a program in a programming language. In the second part of the course, on complexity theory, the central question is What computations can be performed on a computer?. To reason about this question, Turing machines are introduced; we also introduce the closely related notions of recursive- and recursively enumerable languages and unrestrictedand context-sensitive grammars and linear bounded automata. We introduce the Church-Turing thesis, along with examples of undecidable problems such as the halting problem and the Post-correspondence problem. We show how undecidability of new problems can be shown by reduction from a known undecidable problem. Finally, we introduce the most important complexity classes from the complexity hierarchy: P, NP, PSpace, EXP and NEXP, and we discuss NP- and PSpace completeness with the corresponding reduction arguments. In the final part we introduce the basic concepts in quantum computing: qubits, entanglement and quantum-operations. We show how quantum computing can improve computing first using a parity game, and later by introducing Simon s algorithm. The latter solves a problem in quantum-polynomial time, where in the traditional setting the problem has exponential time complexity. The principles introduced in Simon s algorithm are similar to, yet simpler than, those required by Shor s algorithm for prime factorisation. We conclude with the complexity classes BQP and BPP. 2 Entry Conditions There are no strict entry conditions for this course. However, prior knowledge of data structures and algorithms is recommended. Furthermore, basic knowledge of matrices, vectors and complex numbers is helpful. 3 Course goals Upon completion of the course: 1. The student has knowledge of fundamental concepts in computer science, such as automata, languages and computability, and can explain their meaning. 2. The student can apply the algorithmic meaning of concepts on simple problems. This means, among others, that she is able to design simple grammars and automata, can derive automata from grammars and vice versa, and is able to manipulate grammars and automata. 2

3. The student can explain the restrictions of different types of automata and grammars, and the restrictions implied by the notion of computability for computer science, i.e. some problems cannot be solved algorithmically. 4. The student can apply techniques to show these restrictions (pumping lemmas and reductions) to simple examples. 5. The student knows basic quantum computing concepts and can explain their meaning. Furthermore, the student can execute simple quantum algorithms to show his understanding of these concepts. The course goals are given in more detail for the individual lectures in Section 5. 4 Literature 4.1 Required reading The material for this class is based on the following book: Peter Linz. An Introduction to Formal Languages and Automata. Jones and Bartlett. (3 rd, 4 th or 5 th edition.) 4.2 Additional reading For some of the lectures, additional background material is available that discusses topics not described by Linz, or that covers topics more thoroughly. Chapter 8 (LL parsing) from Dick Grune and Ceriel J.H. Jacobs. Parsing Techniques - A Practical Guide. Springer, 2008 (2 nd edition). This gives more background on LL parsing (lecture 4). Sections 3.4 and 7.2 from Harry R. Lewis and Christos H. Papadimitriou. Elements of the Theory of Computation. Prentice Hall, 1998 (2 nd edition). Section 3.4 give a translation from context-free grammar to pushdown automaton, section 7.2 describes the bounded tiling problem and Cook s theorem (lectures 5, 9). Sections 12.4, 17.3 and 17.4 from Thomas A. Sudkamp. Languages and Machines. Addison Wesley, 2006 (3 rd edition). Section 12.4 describes Rice s theorem, the other sections discuss PSpace-completeness (lectures 8, 9). Chapter 10 from Sanjeev Arora and Boaz Barak. Computational Complexity: A Modern Approach. Cambridge University Press, draft version from 2007, available from http://www.cs.princeton.edu/theory/ 3

index.php/compbook/draft#quantum. This provides an introduction to quantum computing (lectures 10, 11). 5 Lectures The course consists of a series of 11 lectures. The learning outcomes of each of the lectures are described below. 1. Languages as sets, deterministic- and nondeterministic automata. The student is able to describe languages in terms of sets. She is able to define both types of automata, formulate the similarities and differences, and transform a simple nfa into a dfa using the subset construction. 2. Grammars and regular expressions. The student is able to describe regular languages in terms or right- and left- linear grammars. He knows that right-linear grammars and finite automata are equi-expressive, and is able to reduce right-linear grammars into finite automata and vice versa for small examples. Finally, the student knows what regular expressions are, and is able to describe regular languages in terms of regular expressions, the student can convert examples of regular expressions into nfas and vice versa. 3. Properties of regular languages. The student can reason about regular languages using their basic properties. She can also reduce small examples of dfas to obtain the minimal language equivalent dfa. The student can explain the problem of string matching, and is able to apply Thomson s algorithm to match strings on small examples. Finally, she is able to explain the pumping lemma for regular languages, and use it to show that a language is not regular. The student can also explain how to show that a language is regular. 4. Context free languages. The student can describe context free languages in terms of context free grammars. She can determine whether a grammar is context free, and explain whether a context free grammar is ambiguous using derivation trees. She can simplify simple context free grammars by removing λ- and unit productions, removing non-productive variables, and transforming the grammar to Chomsky normal form. The student can parse simple input strings with respect to simple context free grammars using the CYKand LL parsing algorithms. 5. Pushdown automata. The student can describe simple context free languages as a nondeterministic pushdown automaton. He can explain the difference between 4

nondeterministic and deterministic pushdown automata, and their difference in expressiveness. He can also reduce simple context free grammars to nondeterministic pushdown automata and vice versa. 6. Properties of context free languages, and Turing machines. The student can reason about context free languages using their basic properties. She can show that a language is not context-free using the pumping lemma. The student also knows what deterministic Turing machines are, and can design simple Turing machines. She can define the concept of recursively enumerable language, and is able to explain the Church-Turing thesis, and its significance for compute science. 7. Variations of Turing machines. The student can describe a number of generalisations of deterministic Turing machines, and argue that they do not add expressivity. He can explain unrestricted grammars, and is able to reduce simple Turing machines to unrestricted grammars. He can also explain the restrictions imposed by context-sensitive grammars, and their relation to linear bounded automata. Finally, the student can reason about context-sensitive and recursive languages using elementary properties. He can also explain the difference between recursive and recursively enumerable languages. The student can explain the Chomsky hierarchy. 8. Undecidability. The student can explain the concepts of decidability and undecidability, and can prove undecidability of the halting problem. She is familiar with Rice s theorem, and can apply it to show undecidability of properties of recursively enumerable languages. The student can give examples of undecidable problems (MPCP, PCP), and can use reductions to prove undecidability of additional problems, e.g. undecidable problems for context free languages. 9. Computational complexity. The student can define complexity classes P and NP, and can explain NPcompleteness. He knows the bounded tiling problem is NP-complete, and can reduce language containment of a concrete nondeterministic Turing machine to the bounded Tiling problem. He can also prove other problems NP-complete, especially the satisfiability problem. The student can define the class co-np, and can prove that if there is an NP-complete language in co-np, then NP = co-np. He can explain the nature of problems in PSpace, NPSpace, EXP and NEXP, and the relation between all complexity classes discussed in this lecture. 10. Quantum computing. The student can explain qubits and entanglement. She can apply quan- 5

tum operations to qubits; in particular, she is able to compute the quantum solution to an instance of a parity game. 11. Simon s algorithm. The student can describe quantum circuits in terms of gates. He can apply Simon s algorithm to simple problem instances, and explain the advantage of the quantum algorithm compared to a brute force algoritm. The student can furthermore explain what problem is solved by Shor s algorithm, and he can explain the difference between probabilistic- and quantum algorithms and the corresponding complexity classes BQP and BPP. 6 Exercise classes In the lectures you build up your knowledge on automata & complexity theory. You are going to apply this knowledge to small examples in the exercise classes. You are expected to prepare the exercises before the exercise classes, so that in class the focus can be on discussing problems. The goal of the exercise class is thus to improve your capabilities in applying the theory of the course to small examples. The book also comes with the tool JFLAP that allows you to experiment with the concepts covered in the course. Sometimes you can check your answers using this tool as well, so use it to your advantage! 7 Hand-in exercises To test your knowledge of the topics covered in the lectures and exercise classes, every week a set of hand-in exercises is made available. In total there will be 6 sets of hand in exercises consisting of exam-level questions. If you hand in these exercises, they will be graded and provided with feedback, so that you can check your understanding. For each set of exercises you can obtain 0.1 point bonus on the exam (this means that, in total, you can get 0.6 bonus on the exam). The bonus will be (sum of the marks of the hand-in exercises/10). For some sets of excercises 1 point can be obtained by drawing the automata using JFLAP. 8 Schedule The course consists of 2 2 hours of lecture and 2 2 hours of exercise class every week. To be able to make the most of your own learning experience in the course, it is expected that you prepare the lectures and exercise classes: 6

as preparation for the lectures, read the indicated material and identify difficulties, for the exercise classes, do the exercises before the exercise class, so that during the exercise class time can be spent discussing the problems encountered during the exercises. The preparation work is indicated in the schedule below. Session Date Time Location Preparation L1 2/2 13:30 E1 3/2 13:30 L2 5/2 13:30 E2 6/2 11:00 L3 9/2 13:30 E3 10/2 13:30 L4 12/2 13:30 E4 13/2 11:00 L5 16/2 13:30 E5 17/2 13:30 L6 19/2 13:30 E6 20/2 11:00 WN-M623 Write down 3 expectations you have for this course. Read Linz 1.1, 1.2, 2.1 2.3 WN-F637 Make exercises 1.2: 2, 4, 8, 10; 2.1: 1, 2d, 3, 7b, 9b,f 11; 2.2: 12; and 2.3: 2, 3, 6, 12. WN-P647 Read Linz: 1.2, 3.1 3.3 WN-S607 Make exercises 1.2: 11a,b,c, 13, 14a,b,e,f,h; 3.1: 5, 7, 13, 16a,b; 3.2: 1, 2, 4b,d 9, 10a,c, 13a; and 3.3: 1, 3, 6, 12 WN-M623 Read Linz: 2.4, 4.1 4.3 WN-F637 Make exercises 4.1: 17, 20, 26 (note that regular should be right-linear twice); 4.2: 5, 12; 2.4: 1, 4, 6; 4.3: 1, 3, 4e,f WN-P647 Read Linz: 5.1, 5.2, 6.1 6.3 WN-S607 Make exercises 5.1: 2, 7b,e, 8b,h 21, 22; 5.2: 14, 6.1: 6, 7, 9, 15, 16; 6.2: 3, 4; 6.3: 1, 2, 3. Also do the extra exercise on LL parsing WN-M623 Read Linz: 7.1 7.3. Additionally you can read 3.4 from Lewis and Papadimitriou. WN-F637 Make exercises 7.1: 3a, 4c,g 5, 11; 7.2: 5, 6, 15; 7.3: 3, 5, 8, 16, 18 WN-P647 Read Linz: 8.1, 8.2, 9.1, 9.3 WN-S607 Make exercises 8.1: 2, 3, 5, 8c; 8.2: 1, 11, 22, 23; 9.1: 4, 5, 7d,e, 8 7

Session Date Time Location Preparation L7 23/2 13:30 E7 24/2 13:30 L8 26/2 13:30 E8 27/2 11:00 L9 2/3 13:30 E9 3/3 13:30 L10 5/3 13:30 E10 6/3 11:00 L11 9/3 13:30 E11 10/3 13:30 WN-M623 Read Linz: 10.2, 10.3, 10.5, 11.1 11.4 WN-F637 Make exercises 11.1: 3, 6, 7, 8, 10; 11.2: 6, 7; 11.3: 1b,d, 3; 10.5: 4c,d WN-P647 Read Linz: 12.1 12.3. Additionally you can read 12.4 from Sudkamp. WN-P647 Make exercises 12.1: 3, 7, 9; 12.3: 1, 3; 12.4: 3, 7, 8 WN-M623 Read Linz: 14.*. Additionally you can read 7.2 from Lewis and Papadimitriou and 17.3, 17.4 from Sudkamp. WN-F637 Make exercises 14.2: 5, 14.3: 2, 3. Also do the 7 extra questions on complexity. WN-P647 Read 10.1 10.3 from Arora and Barak WN-S607 WN-M623 WN-F637 L12 12/3 no lecture E12 13/3 no exercise class L13 16/3 no lecture E13 17/3 no exercise class L14 19/3 13:30 E14 20/3 11:00 WN-P647 WN-S607 Make 7 extra exercises on quantum computing Read 10.5, 10.7 from Arora and Barak Make 8th exercise on quantum computing (application of Simon s algorithm); do the exam from 2010 Q & A session. Prepare questions you still have on the material of the lectures. It is encouraged to submit questions by e-mail to the lecturer a day before the start of the lecture. Q & A session on the exercises of the exercise classes. Note, the numbers in the table above refer to the number in editions 4 and 5 of Linz book. If you own a copy of edition 3, please let me know and I can provide you with the alternative exercises. 8

9 Study load The study load of the course is 6 ECTS, which is 6 28 = 168 hours. Of these, about 50 are spent in lectures and exercise classes, and the exam takes almost 3 hours. This means there are 115 hours left for self-study activities. Given that the course takes 7 weeks, this means you are expected to spend about 16 hours on self-study every week! This allows for plenty of time preparing the lectures and exercise classes, as well as studying for the exam. 10 Examination Examination of the course is through a written exam. At the exam copies of the slides without handwritten notes may be used. Other material may not be used. The exam dates are as follows: (exam) 25 March 2015, 12.00 14.45h, room MF-FG2. (resit) 10 June 2015, 18.30 21.15h, room to be announced. By handing in the hand-in exercises, a maximum of 0.7 bonus on the exam grade can be obtained. You are only eligible for the bonus on the first exam, not in the resit. The final grade for the course is obtained by max(e + B, R), where E is the grade for the exam, R is the grade for the resit, and B are the bonus points. 9