Theory of Computation Class Notes 1

Transcription

1 Theory of Computation Class Notes 1 1 based on the books by Sudkamp and by Hopcroft, Motwani and Ullman

2 ii

3 Contents 1 Introduction Sets Functions and Relations Countable and uncountable sets Proof Techniques Languages and Grammars Languages Regular Expressions Grammars Classification of Grammars and Languages Normal Forms of Context-Free Grammars Chomsky Normal Form (CNF) Greibach Normal Form (GNF) Finite State Automata Deterministic Finite Automata (DFA) Nondeterministic Finite Automata (NFA) NFA with Epsilon Transitions (NFA-ε or ε-nfa)) Finite Automata and Regular Sets Removing Nondeterminism Expression Graphs Regular Languages and Sets Regular Grammars and Finite Automata Closure Properties of Regular Sets Pumping Lemma for Regular Languages Pushdown Automata and Context-Free Languages Pushdown Automata Variations on the PDA Theme Pushdown Automata and Context-Free Languages The Pumping Lemma for Context-Free Languages Closure Properties of Context-Free Languages A Two-Stack Automaton Turing Machines The Standard Turing Machine Notation for the Turing Machine Turing Machines as Language Acceptors Alternative Acceptance Criteria Multitrack Machines iii

4 iv CONTENTS 6.5 Two-Way Tape Machines Multitape Machines Nondeterministic Turing Machines Turing Machines as Language Enumerators The Chomsky Hierarchy The Chomsky Hierarchy Decidability Decision Problems The Church-Turing Thesis The Halting Problem for Turing Machines A Universal Machine The Post Correspondence Problem Undecidability Problems That Computers Cannot Solve Programs that Print Hello, World The Hypothetical Hello, World Tester Reducing One Problem to Another A Language That Is Not Recursively Enumerable Enumerating the Binary Strings Codes for Turing Machines The Diagonalization Language Proof that L d is not Recursively Enumerable Complements of Recursive and RE languages The Universal Language Undecidability of the Universal Language Undecidable Problems About Turing Machines Reductions Turing Machine That Accepts the Empty Language Rice s Theorem and Properties of RE Languages Post s Correspondence Problem The Modified PCP Other Undecidable Problems Undecidability of Ambiguity for CFG s The Complement of a List Language Intractable Problems The Classes P and N P Problems Solvable in Polynomial Time An Example: Kruskal s Algorithm An N P Example: The Travelling Salesman Problem NP-complete Problems The Satisfiability Problem NP-Completeness of 3SAT

5 List of Figures 2.1 Derivation tree Example DFA L(M 1 ) L(M 2 ) L(M 1 )L(M 2 ) L(M 1 ) Sample Union Construction Machines that accept the primitive regular sets An NFA-ε Equivalent DFA Expression Graph Expression Graph Transformation (a) w, (b) w1 w 2(w 3 w 4 (w 1 ) w 2 ) Example (a),(b) Example (a),(b) Example (c),(d) Example NFA accepts a (a b + ) Example L = {a i i 0} {a i b i i 0} PDA L(M) = ww R Pumping Lemma for CFL A Turing Machine Turing Machine COPY TM accepting (a b) aa(a b) TM accepting a i b i c i A k-tape TM for L = a i b i c i Halting Machine Turing Machine D with R(M) as input Turing Machine D with R(D) as input Universal Machine Post Correspondence System Post Correspondence Solution Example Hello-World Program Fermat s last theorem expressed as a hello-world program A hypothetical program H that is a hello-world detector v

6 vi LIST OF FIGURES 9.4 H 1 behaves like H, but it says hello, world instead of no H 2 behaves like H 1, but uses its input P as both P and I What does H 2 do when given itself as input? Reduction of P 1 to P The table that represents acceptance of strings by Turing machines Relationship between the recursive, RE, and non-re languages Construction of a TM accepting the complement of a recursive language Simulation of two TM s accepting a language and its complement Organization of a universal Turing machine Reduction of L d to L u Reductions turn positive instances into positive and negative to negative Construction of a NTM to accept L ne Plan of TM M constructed from (M, w) Construction of a M for the proof of Rice s Theorem Turing Machine that accepts after guessing 10 strings Turing Machine that simulates M on w Turing Machine for L L u A graph

7 Chapter 1 Introduction 1.1 Sets A set is a collection of elements. To indicate that x is an element of the set S, we write x S. The statement that x is not in S is written as x / S. A set is specified by enclosing some description of its elements in curly braces; for example, the set of all natural numbers 0, 1, 2, is denoted by N = {0, 1, 2, 3, }. We use ellipses (i.e.,...) when the meaning is clear, thus J n = {1, 2, 3,, n} represents the set of all natural numbers from 1 to n. When the need arises, we use more explicit notation, in which we write S = {i i 0, i is even} for the last example. We read this as S is the set of all i, such that i is greater than zero, and i is even. Considering a universal set U, the complement S of S is defined as S = {x x U x / S} The usual set operations are union ( ), intersection ( ), and difference( ), defined as S 1 S 2 = {x x S 1 x S 2 } S 1 S 2 = {x x S 1 x S 2 } S 1 S 2 = {x x S 1 x / S 2 } The set with no elements, called the empty set is denoted by. It is obvious that S = S = S S = = U S = S A set S 1 is said to be a subset of S if every element of S 1 is also an element of S. We write this as S 1 S If S 1 S, but S contains an element not in S 1, we say that S 1 is a proper subset of S; we write this as S 1 S 1

8 2 CHAPTER 1. INTRODUCTION The following identities are known as the de Morgan s laws, 1. S 1 S 2 = S 1 S 2, 2. S 1 S 2 = S 1 S 2, 1. S 1 S 2 = S 1 S 2, x S 1 S 2 x U and x / S 1 S 2 x U and (x S 1 or x S 2 ) x U and ( (x S 1 ) and (x S 2 )) x U and (x / S 1 and x / S 2 ) (x U and x / S 1 ) and (x U and x / S 2 ) (x S 1 and x S 2 ) x S 1 S 2 (def.union) (negation of disjunction) (def.complement) (def.intersection) If S 1 and S 2 have no common element, that is, S 1 S 2 =, then the sets are said to be disjoint. A set is said to be finite if it contains a finite number of elements; otherwise it is infinite. The size of a finite set is the number of elements in it; this is denoted by S (or #S). A set may have many subsets. The set of all subsets of a set S is called the power set of S and is denoted by 2 S or P (S). Observe that 2 S is a set of sets. Example If S is the set {1, 2, 3}, then its power set is 2 S = {φ, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}} Here S = 3 and 2 S = 8. This is an instance of a general result, if S is finite, then 2 S = 2 S Proof: (By induction on the number of elements in S). Basis: S = 1 2 S = {, S} 2 S = 2 1 = 2 Induction Hypothesis: Assume the property holds for all sets S with k elements. Induction Step: Show that the property holds for (all sets with) k + 1 elements. Denote S k+1 = {y 1, y 2,..., y k+1 } = S k {y k+1 }

9 1.2. FUNCTIONS AND RELATIONS 3 where S k = {y 1, y 2, y 3,..., y k } 2 S k+1 = 2 S k {y k+1 } {y 1, y k+1 } {y 2, y k+1 }... {y k, y k+1 } x,y Sk {x, y, y k+1 }... S k+1 2 S k has 2 k elements by the induction hypothesis. The number of sets in 2 S k+1 which contain y k+1 is also 2 k. Consequently 2 S k+1 = 2 2 k = 2 k+1. A set which has as its elements ordered sequences of elements from other sets is called the Cartesian product of the other sets. For the Cartesian product of two sets, which itself is a set of ordered pairs, we write Example Let S 1 = {1, 2} and S 2 = {1, 2, 3}. Then S = S 1 S 2 = {(x, y) x S 1, y S 2 } S 1 S 2 = {(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3)} Note that the order in which the elements of a pair are written matters; the pair (3, 2) is not in S 1 S 2. Example If A is the set of throws of a coin, i.e., A ={head,tail}, then A A = {(head,head),(head,tail),(tail,head),(tail,tail)} the set of all possible throws of two coins. The notation is extended in an obvious fashion to the Cartesian product of more than two sets; generally S 1 S 2 S n = {(x 1, x 2,, x n ) x i S i } 1.2 Functions and Relations A function is a rule that assigns to elements of one set (the function domain a unique element of another set (the range). We write f : S 1 S 2 to indicate that the domain of the function f is a subset of S 1 and that the range of f is a subset of S 2. If the domain of f is all of S 1, we say that f is a total function on S 1 ; otherwise f is said to be a partial function on S Domain f = {x S 1 (x, y) f, for some y S 2 } = D f 2. Range f = {y S 2 (x, y) f, for some x S 1 } = R f

10 4 CHAPTER 1. INTRODUCTION 3. The restriction of f to A S 1, f A = {(x, y) f x A} 4. The inverse f 1 : S 2 S 1 is {(y, x) (x, y) f} 5. f : S 1 S 1 is called a function on S 1 6. If x D f then f is defined at x; otherwise f is undefined at x; 7. f is a total function if D f = S f is a partial function if D f S 1 9. f is an onto function or surjection if R f = S 2. If R f S 2 then f is a function from S 1 (D f ) into S f is a one to one function or injection if (f(x) = z and f(y) = z) x = y 11. A total function f is a bijection if it is both an injection and a surjection. A function can be represented by a set of pairs {(x 1, y 1 ), (x 2, y 2 ), }, where each x i is an element in the domain of the function, and y i is the corresponding value in its range. For such a set to define a function, each x i can occur at most once as the first element of a pair. If this is not satisfied, such a set is called a relation. A specific kind of relation is an equivalence relation. A relation denoted r on X is an equivalence relation if it satisfies three rules, the reflexivity rule: the symmetry rule: and the transitivity rule: (x, x) r x X (x, y) r then (y, x) r x, y X (x, y) r, (y, z) r then (x, z) r x, y, z X An equivalence relation on X induces a partition on X into disjoint subsets called equivalence classes X j, j X j = X, such that elements from the same class belong to the relation, and any two elements taken from different classes are not in the relation. Example The relation congruence mod m (modulo m) on the set of the integers Z. i = j mod m if i j is divisible by m; Z is partitioned into m equivalence classes: {, 2m, m, 0, m, 2m, } {, 2m + 1, m + 1, 1, m + 1, 2m + 1, } {, 2m + 2, m + 2, 2, m + 2, 2m + 2, } {, m 1, 1, m 1, 2m, 3m 1, }

11 1.3. COUNTABLE AND UNCOUNTABLE SETS Countable and uncountable sets Cardinality is a measure that compares the size of sets. The cardinality of a finite set is the number of elements in it. The cardinality of a finite set can thus be obtained by counting the elements of the set.two sets X and Y have the same cardinality if there is a total one to one function from X onto Y (i.e., a bijection from X to Y ). The cardinality of a set X is less than or equal to the cardinality of a set Y if there is a total one to one function from X into Y. We denote cardinality of X by #X or X. A set that has the same cardinality as the set of natural numbers N, is said to be countably infinite or denumerable. Sets that are either finite or denumerable are referred to as countable sets. The elements of a countable set can be indexed (or enumerated) using N as the index set. Sets that are not countable are said to be uncountable. The cardinality of denumerable sets is #N = ℵ 0 ( aleph 0 ) The cardinality of the set of the real numbers, #R = ℵ 1 ( aleph 1 ) A set is infinite if it has proper subset of the same cardinality. Example The set J = N {0} is countably infinite; the function s(n) = n + 1 defines a one-to-one mapping from N onto J. The set J, obtained by removing an element from N, has the same cardinality as N. Clearly, there is no one to one mapping of a finite set onto a proper subset of itself. It is this property that differentiates finite and infinite sets. Example The set of odd natural numbers is denumerable. The function f(n) = 2n + 1 establishes the bijection between N and the set of the odd natural numbers. The one to one correspondence between the natural numbers and the set of all integers exhibits the countability of set of integers. A correspondence is defined by the function { n f(n) = if n is odd n 2 if n is even Example #Q + = #J = #N Q + is the set of the rational numbers p q > 0, where p and q are integers, q Proof Techniques We will give examples of proof by induction, proof by contradiction, and proof by Cantor diagonalization. In proof by induction, we have a sequence of statements P 1, P 2,, about which we want to make some claim. Suppose that we know that the claim holds for all statements P 1, P 2,, up to P n. We then try to argue that this implies that the claim also holds for P n+1. If we can carry out this inductive step for all positive n, and if we have some starting point for the induction, we can say that the claim holds for all statements in the sequence. The starting point for an induction is called the basis. The assumption that the claim holds for statements P 1, P 2,, P n is the induction hypothesis, and the argument connecting the induction

12 6 CHAPTER 1. INTRODUCTION hypothesis to P n+1 is the induction step. Inductive arguments become clearer if we explicitly show these three parts. Example Let us prove by mathematical induction. We establish n i=0 i2 = n(n+1)(2n+1) 6 (a) the basis by substituting 0 for n in n i=0 i2 = n(n+1)(2n+1) 6 and observing that both sides are 0. (b) For the induction hypothesis, we assume that the property holds with n = k; k i=0 i2 = k(k+1)(2k+1) 6 (c) In the induction step, we show that the property holds for n = k + 1; i.e., Since k i=0 i2 = (k)(k+1)(2k+1) 6 k+1 i=0 i2 = (k+1)(k+2)(2k+3) 6 k+1 i=0 i2 = k i=0 i2 + (k + 1) 2 and in view of the induction hypothesis, we need only show that (k)(k+1)(2k+1) 6 + (k + 1) 2 = (k+1)(k+2)(2k+3) 6 The latter equality follows from simple algebraic manipulation. In a proof by contradiction, we assume the opposite or contrary of the property to be proved; then we prove that the assumption is invalid. Example Show that 2 is not a rational number. As in all proofs by contradiction, we assume the contrary of what we want to show. Here we assume that 2 is a rational number so that it can be written as 2 = n m, where n and m are integers without a common factor. Rearranging ( 2 = n m ), we have 2m 2 = n 2 Therefore n 2 must be even. This implies that n is even, so that we can write n = 2k or and 2m 2 = 4k 2 m 2 = 2k 2

13 1.4. PROOF TECHNIQUES 7 Therefore m is even. But this contradicts our assumption that n and m have no common factor. Thus, m and n in ( 2 = n m ) cannot exist and 2 is not a rational number. This example exhibits the essence of a proof by contradiction. By making a certain assumption we are led to a contradiction of the assumption or some known fact. If all steps in our argument are logically sound, we must conclude that our initial assumption was false. To illustrate Cantor s diagonalization method, we prove that the set A = {f f a total function, f : N N }, is uncountable. This is essentially a proof by contradiction; so we assume that A is countable, i.e., we can give an enumeration f 0, f 1, f 2, of A. To come to a contradiction, we construct a new function f as f(x) = f x (x) + 1 x N The function f is constructed from the diagonal of the function values of f i A as represented in the figure below. For each x, f differs from f x on input x. Hence f does not appear in the given enumeration. However f is total and f : N N. Hence the set A is uncountable since such an f can be given for any chosen enumeration. Therefore A cannot be enumerated; hence A is uncountable. f 0 f 0 (0) f 0 (1) f 0 (2) f 1 f 1 (0) f 1 (1) f 1 (2) f 2 f 2 (0) f 2 (1) f 2 (2) f 3 f 3 (0) f 3 (1) f 3 (2) Remarks: The set of all infinite sequences of 0 s and 1 s is uncountable. With each infinite sequence of 0 s and 1 s we can associate a real number in the range [0, 1). As a consequence, the set of real numbers in the range [0, 1) is uncountable. Note that the set of all real numbers is also uncountable.

14 8 CHAPTER 1. INTRODUCTION

15 Chapter 2 Languages and Grammars 2.1 Languages We start with a finite, nonempty set Σ of symbols, called the alphabet. From the individual symbols we construct strings (over Σ or on Σ), which are finite sequences of symbols from the alphabet. The empty string ε is a string with no symbols at all. Any set of strings over/on Σ is a language over/on Σ. Example Example Σ = {c} L 1 = {cc} L 2 = {c, cc, ccc} L 3 = {w w = c k, k = 0, 1, 2,...} Σ = {a, b} L 1 = {ab, ba, aa, bb, ε} L 2 = {w w = (ab) k, k = 0, 1, 2, 3,...} = {ε, ab, abab, ababab,...} The concatenation of two strings w and v is the string obtained by appending the symbols of v to the right end of w, that is, if and w = a 1 a 2... a n v = b 1 b 2... b m, then the concatenation of w and v, denoted by wv, is which completes the induction step. wv = a 1 a 2... a n b 1 b 2... b m If w is a string, then w n is the string obtained by concatening w with itself n times. As a special case, we define w 0 = ε, 9

16 10 CHAPTER 2. LANGUAGES AND GRAMMARS for all w. Note that εw = wε = w for all w. The reverse of a string is obtained by writing the symbols in reverse order; if w is a string as shown above, then its reverse w R is If w R = a n... a 2 a 1 w = uv, then v is said to be prefix and u a suffix of w. The length of a string w, denoted by w, is the number of symbols in the string. Note that, ε = 0 If u and v are strings, then the length of their concatenation is the sum of the individual lengths, uv = u + v Let us show that uv = u + v. To prove this by induction on the length of strings, let us define the length of a string recursively, by a = 1 wa = w + 1 for all a Σ and w any string on Σ. This definition is a formal statement of our intuitive understanding of the length of a string: the length of a single symbol is one, and the length of any string is incremented by one if we add another symbol to it. Basis: uv = u + v holds for all u of any length and all v of length 1 (by definition). Induction Hypothesis: we assume that uv = u + v holds for all u of any length and all v of length 1, 2,..., n. Induction Step: Take any v of length n + 1 and write it as v = wa. Then, v = w + 1, uv = uwa = uw + 1. By the induction hypothesis (which is applicable since w is of length n). so that which completes the induction step. uw = u + w. uv = u + w + 1 = u + v. If Σ is an alphabet, then we use Σ to denote the set of strings obtained by concatenating zero or more symbols from Σ. The set Σ and Σ + are always infinite since there is no limit on the length of the strings in these sets. A language can thus be defined as a subset of Σ. A string w in a language L is also called a word or a sentence of L. Example 2.1.3

17 2.1. LANGUAGES 11 Σ = {a, b}. Then Σ = {ε, a, b, aa, ab, ba, bb, aaa, aab,...}. The set {a, aa, aab}. is a language on Σ. Because it has a finite number of words, we call it a finite language. The set L = {a n b n n 0} is also a language on Σ. The strings aabb and aaaabbbb are words in the language L, but the string abb is not in L. This language is infinite. Since languages are sets, the union, intersection, and difference of two languages are immediately defined. The complement of a language is defined with respect to Σ ; that is, the complement of L is L = Σ L The concatenation of two languages L 1 and L 2 is the set of all strings obtained by concatenating any element of L 1 with any element of L 2 ; specifically, L 1 L 2 = {xy x L 1 and y L 2 } We define L n as L concatenated with itself n times, with the special case for every language L. Example L 1 = {a, aaa} L 2 = {b, bbb} L 0 = {ε} Example L 1 L 2 = {ab, abbb, aaab, aaabbb} For then L = {a n b n n 0}, L 2 = {a n b n a m b m n 0, m 0} The string aabbaaabbb is in L 2.The star-closure or Kleene closure of a language is defined as and the positive closure as L = L 0 L 1 L 2 = L i i=0 L + = L 1 L 2 = L i i=0

18 12 CHAPTER 2. LANGUAGES AND GRAMMARS 2.2 Regular Expressions Definition Let Σ be a given alphabet. Then, 1., ε (representing {ε}), a (representing {a}) a Σ are regular expressions. They are called primitive regular expressions. 2. If r and r 1 are regular expressions so are (r), (r ), (r 1 + r 2 ), (r.r 1 ). 3. A string is a regular expression if it can be derived from the primitive regular expressions by applying a finite number of the operations +, * and concatenation. A regular expression denotes a set of strings, which is therefore referred to as a regular set or language. Regarding the notation of regular expression, texts will usually print them boldface; however, we assume that it will be understood that, in the context of regular expressions, ε is used to represent {ε} and a is used to represent {a}. Example b (ab ab ) is a regular expression. Example (c + da bb) = {c, dbb, dabb, daabb,...} = {ε, c, cc,..., dbb, dbbdbb,..., dabb, dabbdabb,..., cdbb, cdabb,...} Beyond the usual properties of + and concatenation, important equivalences involving regular expressions concern porperties of the closure (Kleene star) operation. Some are given below, where α, β, γ stand for arbitrary regular expressions: 1. (α ) = α. 2. (αα ) = α α. 3. αα + ε = α. 4. α(β + γ) = αβ + αγ. 5. α(βα) = (αβ) α. 6. (α + β) = (α + β ). 7. (α + β) = (α β ). 8. (α + β) = α (βα ). In general, the distribution law does not hold for the closure operation. For example, the statement α + β =? (α + β) is false because the right hand side denotes no string in which both α and β appear.

19 2.3. GRAMMARS Grammars Definition A grammar G is defined as a quadruple G = (V, Σ, S, P ) where V is a finite set of symbols called variables or nonterminals, Σ is a finite set of symbols called terminal symbols or terminals, S V is a special symbol called the start symbol, P is a finite set of productions or rules or production rules. We assume V and Σ are non-empty and disjoint sets. Production rules specify the transformation of one string into another. They are of the form where Given a string w of the form x y x (V Σ) + and y (V Σ). w = uxv we say that the production x y is applicable to this string, and we may use it to replace x with y, thereby obtaining a new string, w z; we say that w derives z or that z is derived from w. Successive strings are derived by applying the productions of the grammar in arbitrary order. production can be used whenever it is applicable, and it can be applied as often as desired. If A we say that w 1 derives w, and write w 1 w. w 1 w 2 w 3 w The * indicates that an unspecified number of steps (including zero) can be taken to derive w from w 1. Thus w w is always the case. If we want to indicate that atleast one production must be applied, we can write Let G = (V, Σ, S, P ) be a grammar. Then the set w + v L(G) = {w Σ s w} is the language generated by G. If w L(G), then the sequence S w 1 w 2 w is a derivation of the sentence (or word) w. The strings S, w 1, w 2,, are called sentential forms of the derivation.

20 14 CHAPTER 2. LANGUAGES AND GRAMMARS Example Consider the grammar G = ({S}, {a, b}, S, P ) with P given by, S asb S ε Then S asb aasbb aabb, so we can write S aabb. The string aabb is a sentence in the language generated by G. Example P: < sentence > < Noun phrase >< V erb phrase > < Noun phrase > < Determiner >< Noun phrase > < Adjective >< Noun > < Noun phrase > < Article >< Noun > < V erb phrase > < V erb >< Noun phrase > < Determiner > T his < Adjective > Old < Noun > Man Bus < V erb > Missed < Article > T he Example < expression > < variable > < expression >< operation >< expression > < variable > A B C Z < operation > + / Leftmost Derivation < expression > < expression >< operation >< expression > < variable >< operation >< expression > A < operation >< expression > A+ < expression > A+ < expression >< operation >< expression > A+ < variable >< operation >< variable > A + B < operation >< expression > A + B < expression > A + B < variable > A + B C

21 2.3. GRAMMARS 15 4 $& > (*)+5 / (*),+- = 0 Ä7 (*),+-. / 0!7698/,$: 9;/ 0 (),+4 "!# $&%' 1(*),+-2 < B,, Figure 2.1: Derivation tree This is a leftmost derivation of the string A + B C in the grammar (corresponding to A + (B C)). Note that another leftmost derivation can be given for the above expression. A grammar G (such as the one above) is called ambiguous if some string in L(G) has more than one leftmost derivation. An unambiguous grammar for the language is the following: < expr > < multi expr > < multi expr >< add expr >< expr > < multi expr > < variable > < variable >< multi op >< variable > < multi op > / < add op > + < variable > A B C Z Note that, for an inherently ambiguous language L, every grammar that generates L is ambiguous. Example Show that L(G) = L G : S ε asb bsa SS L = {w n a (w) = n b (w)} 1. L(G) L. (All strings derived by G, are in L.) For w L(G), all productions of G add a number of a s which is same as the number of b s added; n a (w) = n b (w) w L 2. L L(G) Let w L. By definition of L, n a (w) = n b (w). We show that w L(G) by induction (on the

22 16 CHAPTER 2. LANGUAGES AND GRAMMARS length of w). Basis: ε is in both L and L(G). w = 2. The only two strings of length 2 in L are ab and ba S asb ab S bsa ba Induction Hypothesis: w L with 2 w 2i, we assume that w L(G). Induction Step: Let w 1 L, w 1 = 2i + 2. (a) w 1 of the form w 1 = awb (or bwa) where w = 2i w L(G) (by I. H.) We derive w 1 = awb using the rule S asb. We derive w 1 = bwa using the rule S bsa. (b) w 1 = awa or w 1 = bwb Let us assign a count of +1 to a and -1 to b; Thus for w 1 L the total count = 0. Example We will now show that count goes through 0 at least once within w 1 = awa (case bwb is similar) w 1 = a (count = +1) (count goes through 0) (count = -1) a (by end, count = 0). w 1 = w (count = 0) w where w L, w L. We also have w 2 and w 2 so that w 2i and w 2i w, w L(G) (I. H.) w 1 = w w can be derived in G from w and w, using the rule S SS. L(G) = {a 2n n 0} G = (V, T, S, P ) where V = {S, [, ], A, D} T = {a} P :S [A] [ [D ε D] ] DA AAD ] ε A a

23 2.3. GRAMMARS 17 For example, let us derive a 4. S [A] [DA] [AAD] [AA] [DAA] [AADA] [AAAAD] [AAAA] εaaaaε AAAA aaaa a 4 Example L(G) = {w {a, b, c} n a (w) = n b (w) = n c (w)} V = {A, B, C, S} T = {a, b, c} P : S ε ABCS AB BA AC CA BC CB BA AB CA AC CB BC A a B b C c derive ccbaba Solution: S ABCS ABCABCS ABCABCε ABCABC ACBACB CABCAB CACBBA CCABBA CCBABA cababa Example S ε asb L(G) = {ε, ab, aabb, aaabbb,...} L = {a i b i i 0} To prove that L = L(G) 1. L(G) L 2. L L(G)

24 18 CHAPTER 2. LANGUAGES AND GRAMMARS 2. L L(G) : Let w L, w = a k b k we apply S asb (k times), thus then S ε S a k Sb k S a k b k 1. L(G) L: We need to show that, if w can be derived in G, then w L. ε is in the language, by definition. We first show that all sentential forms are of the form a i Sb i, by induction on the length of the sentential form. Basis: (i = 1) asb is a sentential form, since S asb. Induction Hypothesis: Sentential form of length 2i + 1 is of the form a i Sb i. Induction Step: Sentential form of length 2(i + 1) + 1 = 2i + 3 is derived as S asb a(a i Sb i )b = a i+1 Sb i+1. To get a sentence, we must apply the production S ε; i.e., S a i Sb i a i b i represents all possible derivations; hence G derives only strings of the form a i b i (i 0). 2.4 Classification of Grammars and Languages A classification of grammars (and the corresponding classes of languages) is given with respect to the form of the grammar rules x y, into the Type 1, Type 2 and Type 3 classes, respectively. Type 1 If all the grammar rules x y satisfy x y, then the grammar is context sensitive or Type 1. Grammar G will generate a language L(G) which is called a context-sensitive language. Note that x has to be of length at least 1 and thereby y too. Hence, it is not possible to derive the empty string in such a grammar. Type 2 If all production rules are of the form x y where x = 1, then the grammar is said to be context-free or Type 2 (i.e., the left hand side of each rule is of length 1). Type 3 If the production rules are of the following forms: A xb A x where x Σ (a string of all terminals or the empty string), and A, B V (variables), then the grammar is called right linear. Similarly, for a left linear grammar, the production rules are of the form A Bx A x

25 2.5. NORMAL FORMS OF CONTEXT-FREE GRAMMARS 19 For a regular grammar, the production rules are of the form A ab A a A ε with a Σ. A language which can be generated by a regular grammar will (later) be shown to be regular. Note that, a language that can be derived by a regular grammar iff it can be derived by a right linear grammar iff it can be derived by a left linear grammar. 2.5 Normal Forms of Context-Free Grammars Chomsky Normal Form (CNF) Definition A context-free grammar G = (V, Σ, P, S) is in Chomsky Normal Form if each rule is of the form i) A BC ii) A a iii) S ε where B, C V {S} Theorem Let G = (V, Σ, P, S) be a context-free grammar. There is an algorithm to construct a grammar G = (V, Σ, P, S) in Chomsky normal form that is equivalent to G (L(G ) = L(G)). Example Convert the given grammar G to CNF. Solution: A CNF equivalent G can be given as : G : S A T 1 a A a T 1 AT 2 T 2 BC A A A B B T 3 B C B b T 3 C B C C C c C c G :S aabc a A aa a B bcb bc C cc c Greibach Normal Form (GNF) If a grammar is in GNF, then the length of the terminals prefix of the sentential form is increased at every grammar rule application, thereby enabling the prevention of the left recursion.

26 20 CHAPTER 2. LANGUAGES AND GRAMMARS Definition A context-free grammar G = (V, Σ, P, S) is in Greibach Normal Form if each rule is of the form, i) A aa 1 A 2... A n ii) A a iii) S ε

27 Chapter 3 Finite State Automata 3.1 Deterministic Finite Automata (DFA) Definition A deterministic finite automaton (DFA) is a quintuple M = (Q, Σ, δ, q 0, F ) where Q is a finite set of states, Σ a finite set of symbols called the alphabet, q 0 Q a distinguished state called the start state, F a subset of Q consisting of the final or accepting states, and δ a total function from Q Σ to Q called the transition function. Example Figure 3.1: Example DFA Some strings accepted by the machine are: baab baaab babaabaaba aaa a All of the above strings are characterized by the presence of at least one aa substring. According to the definition of a DFA, the following are identified: Q = {q 0, q 1, q 2 } Σ = {a, b} δ : Q Σ Q : (q i, a) q j where i can be equal to j and the mapping is given by the transition table below. Transition Table: 21

28 22 CHAPTER 3. FINITE STATE AUTOMATA δ a b q 0 q 1 q 0 q 1 q 2 q 0 q 2 q 2 q 2 A sample computation, on the string abaab, is represented as [q 0, abaab] [q 1, baab] [q 0, aab] [q 1, ab] [q 2, b] [q 2, ε] Definition Let M = (Q, Σ, δ, q 0, F ) be a DFA. The language of M, denoted L(M), is the set of strings in Σ accepted by M. A DFA can be considered as a language acceptor; the language recognized by the machine is the set of strings that are accepted by its computations. Two machines that accept the same language are said to be equivalent. Definition The extended transition function ˆδ of a DFA with transition function δ is a function from Q to Q Σ defined by recursion on the length of the input string. i) Basis: length(w) = 0. Then w = ε and ˆδ(q i, ε) = q i. length(w) = 1. Then w = a, for some a Σ and ˆδ(q i, a) = δ(q i, a). ii) Recursive step: Let w be a string of length n > 1. Then w = ua and ˆδ(q i, ua) = δ(ˆδ(q i, u), a). The computation of a machine in state q i with string w halts in state ˆδ(q i, w). A string w is accepted if ˆδ(q 0, w) F. Using this notation, the language of a DFA M is the set L(M) = {w ˆδ(q 0, w) F }. 3.2 Nondeterministic Finite Automata (NFA) Definition A nondeterministic finite automaton is a quintuple M = {Q, Σ, δ, q 0, F } where Q is a finite set of states, Σ a finite set of symbols called the alphabet, q 0 Q a distinguished state known as the start state, F a subset of Q consisting of the final or accepting states, and δ a total function from Q Σ to P(Q) known as the transition function. Note that a deterministic finite automaton is considered a special case of a nondeterministic one. The transition function of a DFA specifies exactly one state that may be entered from a given state and on a given input symbol, while an NFA allows zero, one or more states to be entered. Hence, a string input to an NFA may generate several distinct computations. For the language over Σ = {a, b} where each string has at least one occurrence of a double a, an NFA can be given with the following transition table: δ a b q 0 {q 0, q 1 } {q 0 } q 1 {q 2 } q 2 {q 2 } {q 2 }

29 3.3. NFA WITH EPSILON TRANSITIONS (NFA-ε OR ε-nfa)) 23 Two computations on the string aabaa are given by: and [q 0, aabaa] [q 1, abaa] [q 2, baa] [q 2, ba] [q 2, a] [q 2, ε] [q 0, aabaa] [q 0, abaa] [q 0, baa] [q 0, ba] [q 1, a] [q 2, ε] We will further show that a language accepted by an NFA is also accepted by a DFA. As an example, the language accepted by the above NFA is also accepted by the DFA of Example Definition The language of an NFA M, denoted L(M), is the set of strings accepted by M. That is, L(M) = {w there is a computation [q 0, w] [q i, ε] with q i F }. 3.3 NFA with Epsilon Transitions (NFA-ε or ε-nfa)) So far, in the discussion of Finite State automatons, the reading head was required to move at each step of the transitions. Intuitively, an ε-transition allows the reading head of the automaton to remain at a cell during a transition. Such a transition is called an ε-transition. Definition A nondeterministic finite automaton with ε-transitions is a quintuple M = (Q, Σ, δ, q 0, F ) where Q, δ, q 0, and F are as in an NFA. The transition function is a function from Q (Σ {ε}) to 2 Q. Epsilon transitions can be used to combine existing machines to construct more complex composite machines. Let M 1 and M 2 be two finite automata which consists of a single start state and a final state where there are no arcs entering the start state, and no arcs leaving the accepting state. Composite machines that accept L(M 1 ) L(M 2 ), L(M 1 )L(M 2 ), and L(M 1 ) are constructed from M 1 and M 2 as depicted in Figures The NFA-ε of Example accepts the language over Σ = {a, b} where each string has at least one occurrence of aa or bb. The states of machines M 1 and M 2 are given distint names. A possible computation on the string bbaaa is given below. Example [q 0, bbaaa] [ε 0, bbaaa] [ε 1, baaa] [ε 2, aaa] [q 2, aa] [q 2, a] [q 2, ε]

30 24 CHAPTER 3. FINITE STATE AUTOMATA Figure 3.2: L(M 1 ) L(M 2 ) Figure 3.3: L(M 1 )L(M 2 ) Figure 3.4: L(M 1 ) 3.4 Finite Automata and Regular Sets Theorem The set of languages accepted by finite state automata consists precisely of the regular sets over Σ First we will show that every regular set is accepted by some NFA-ε. This follows from the recursive definition of regular sets. The regular sets are built from the basis elements, {ε} and the singletons containing a symbol from the alphabet. Machines that accept these sets are given in Figure 3.6. The regular sets are constructed from the primitive regular sets using union, concatenation, and Kleene star operations Removing Nondeterminism Definition The ε-closure of a state q i, denoted ε-closure(q i ), is defined recursively by, i) Basis: q i ε-closure(q i ). ii) Recursive step: Let q j be an element of ε-closure(q i ). If q k δ(q j, ε), then q k ε-closure(q i ).

31 3.4. FINITE AUTOMATA AND REGULAR SETS 25 Figure 3.5: Sample Union Construction Figure 3.6: Machines that accept the primitive regular sets iii) Closure: q j is in ε-closure(q i ) only if it can be obtained from q i by a finite number of applications of operations in ii). Algorithm 1 Construction of DM, a DFA Equivalent to NFA-ε M (see text) Example For the NFA-ε of Figure 3.7, we derive the DFA of Figure 3.8.

32 26 CHAPTER 3. FINITE STATE AUTOMATA Figure 3.7: An NFA-ε!! (Note: the diagram of the figure is missing a transition from FG to BCE on 1, and transitions on 0 and 1 at Φ.) Figure 3.8: Equivalent DFA

33 3.4. FINITE AUTOMATA AND REGULAR SETS Expression Graphs Definition An expression graph is a labeled directed graph in which the arcs are labeled by regular expressions. An expression graph, like a state diagram, contains a distinguished start node and a set of accepting nodes. Example The expression graph given in (fig 3.9) accepts the regular expressions u and u vw. Figure 3.9: Expression Graph Figure 3.10: Expression Graph Transformation The reduced graph has atmost two nodes, the start node and an accepting node. If these are the same node, the reduced graph has the form (fig 3.11(a)), accepting w. A graph with distinct start and accepting nodes reduces to (fig 3.11(b)) and accepts the expression w 1 w 2(w 3 w 4 (w 1 ) w 2 ). This expression may be simplified if any of the arcs in the graph are labeled Φ. Figure 3.11: (a) w, (b) w 1 w 2(w 3 w 4 (w 1 ) w 2 )

34 28 CHAPTER 3. FINITE STATE AUTOMATA Algorithm 2 Construction of a Regular Expression from a Finite Automaton input: state diagram G of a finite automaton and the nodes of G are numbered 1, 2,..., n 1. Make m copies of G, each of which has one accepting state. Call these graphs G 1, G 2,..., G m. Each accepting node G is the accepting node of G t, for some t = 1, 2,..., m. 2. for each G t, do 2.1. repeat choose a node i in G t, that is neither the start nor the accepting node of G t delete the node i from G t according to the procedure: for every j,k not equal to i (this includes j = k) do i) if w j,i Φ, and w i,i = Φ then add an arc from node j to node k labeled w j,i w i,k ii) if w j,i Φ, w i,k Φ, and w i,i Φ then add an arc from node j to node k labeled w j,i (w i,i ) w i,k iii) if nodes j and k have arcs labeled w 1, w 2,..., w s connecting them then replace them by a single arc labeled w 1 w 2... w s iv) remove the node i and all arcs incident to it in G t until the only nodes in G t are the start node and the single accepting node. end for 2.2. determine the expression accepted by G t. end for 3. The regular expression accepted by G is obtained by joining the expressions for each G t with. The deletion of the node i is accomplished by finding all paths j, i, k of length two that have i as the intermediate node. An arc from j to k is added by passing the node i. If there is no arc from i to itself, the new arc is labeled by the concatenation of the expressions on each of the component arcs. If w i,i Φ, then the arc w i,i can be traversed any number of times before following the arc from i to k. The label for the new arc is w j,i (w i,i ) w i,k. These graph transformations are illustrated in (fig 3.10).

35 3.4. FINITE AUTOMATA AND REGULAR SETS 29 Example Example 1: Fig 3.12(a) shows the original DFA which is reduced to an expression graph shown in fig 3.12(b). Figure 3.12: Example (a),(b) 2. Example 2: Explanation of elimination: Sequence of steps where one state is eliminated at each step. Step 1: Given: fig 3.13(a) Step 2: Eliminating i at this step, fig 3.13(b) Figure 3.13: Example (a),(b) Step 3: After eliminating all but initial and final state in G i, fig 3.14(c) Step 4: Final regular expression,fig 3.14(d) 3. Example 3: Fig shows the different steps where

36 30 CHAPTER 3. FINITE STATE AUTOMATA Figure 3.14: Example (c),(d) L = r 1 r 2(r 3 + r 3 r 4r 1 r 2r 3 ) = r 1r 2 (r 3 + r 4 r 1r 2 ) or L = r1 fig 3.14(d)

37 3.4. FINITE AUTOMATA AND REGULAR SETS 31 Figure 3.15: Example

38 32 CHAPTER 3. FINITE STATE AUTOMATA

39 Chapter 4 Regular Languages and Sets 4.1 Regular Grammars and Finite Automata This chapter corresponds to Chapter 7 of the course textbook Theorem Let G = (V, Σ, P, S) be a regular grammar. Define the NFA M = (Q, Σ, δ, S, F ) as follows: { V {Z} wherez / V, if P contains a rule A a i) Q = V otherwise ii) δ(a, a) = iii) F = { B Z whenevera ab P whenevera a P { {A A ε P } {Z} ifz Q {A A ε P } otherwise Then L(M) = L(G). Example The grammar G generates and the NFA M accepts the language a (a b + ) G :S as bb a B bb ε The derivation of a string such as aabb is explained below: In G: S a aas aabb aabbb aabbε aabb 33

40 34 CHAPTER 4. REGULAR LANGUAGES AND SETS Figure 4.1: NFA accepts a (a b + ) In M: [S, aabb] [S, abb] [S, bb] [B, b] [B, ε] Similarly, a regular grammar that accepts the L(M) is constructed from the automaton M. G :S as bb az B bb ε Z ε The transitions provide the S rules and the first B rule. The varepsilon rules are added since B and Z are accepting states. Note: Example A regular grammar is constructed from the given DFA in fig 4.2. Figure 4.2: Example 4.1.2

41 4.2. CLOSURE PROPERTIES OF REGULAR SETS 35 S bb aa A as bc B ac bs ε C ab ba 4.2 Closure Properties of Regular Sets A language over an alphabet Σ is regular if it is i) a regular set (expression) over Σ ii) accepted by DFA, NFA, or NFA-ε iii) generated by a regular grammar. Theorem Let L 1 and L 2 be two regular languages. The languages L 1, L 2, L 1 L 2, and L 1 are regular languages. Theorem Let L be a regular language over Σ. The language L is regular. L = Σ L Theorem Let L 1 and L 2 be regular languages over Σ. The language L 1 L 2 is regular. Proof: By DeMorgan s law L 1 L 2 = L 1 L 2 The right-hand side of the equality is regular since it is built from L 1 and L 2 using union and complementation. Theorem Let L 1 be a regular language and L 2 be a context-free language. The language L 1 L 2 is not necessarily regular. Proof: Let L 1 = a b and L 2 = {a i b i 0}. L 2 is context-free since it is generated by the grammar S asb ε. The intersection of L 1 and L 2 is L 2, which is not regular. 4.3 Pumping Lemma for Regular Languages Pumping a string refers to constructing new strings by repeating (pumping) substrings in the original string. Theorem Let L be a regular language that is accepted by a DFA M with n states. Let w be any string in L with length(w) n. Then w can be written as xyz with length(xy) n, length(y) > 0, and xy k z L for all k 0. Example Prove that the languge L = {a i b i i 0} is not regular using the Pumping lemma for regular languages. Proof: By contradiction: Assume L is regular; then the pumping lemma holds. Let w = a n b n. By splitting a n b n into xyz, we get x = a i, y = a j, and z = a n i j b n

42 36 CHAPTER 4. REGULAR LANGUAGES AND SETS where Pumping y to y 2 gives, Therefore, L is not regular. i + j n and j > 0 a i a j a j a n i j b n = a n a j b n / L (contradiction with the pumping lemma). Example The language L = {a i i is prime} is not regular. Assume L is regular, and that a DFA with n states accepts L. Let m be a prime greater than n. The pumping lemma implies that a m can be decomposed as xyz, y ε, such that xy k z is in L for all k 0. The length of s = xy m+1 z must be prime if s is in L. But, length(xy m+1 z) = length(xyzy m ) = length(xyz) + length(y m ) = m + m(length(y)) = m(1 + length(y)) (4.1) Since its length is not prime, xy m+1 z is not in L (contradiction with the pumping lemma). Hence, L is not regular. Corollary Let DFA M have n states. i. L(M) is not empty if, and only if, M accepts a string w with length(w) < n. ii. L(M) has an infinite number of strings if, and only if, M accepts a string w where n length(z) < 2n. Theorem Let M be a DFA. There is a decision procedure to determine whether, i. L(M) is empty; ii. L(M) is finite; iii. L(M) is infinite.

43 Chapter 5 Pushdown Automata and Context-Free Languages 5.1 Pushdown Automata Definition A pushdown automaton is a six tuple (Q, Σ, Γ, δ, q 0, F ), where Q is a finite set of states, Σ a finite set called the input alphabet, Γ a finite set called the stack alphabet, q 0 the start state, F Q a set of final states, and δ a transition function from Q (Σ {ε}) (Γ {ε}) to subsets of Q (Γ {ε}). Example The language L = {a i i 0} {a i b i i 0} contains strings consisting solely of a s or an equal number of a s and b s. The stack of the PDA M that accepts L maintains a record of the number of a s processed until a b is encountered or the input string is completely processed. Figure 5.1: L = {a i i 0} {a i b i i 0} When scanning an a in state q 0, there are two transitions that are applicable. A string of the form a i b i, i > 0, is accepted by a computation that remains in states q 0 and q 1. If a transition to state q 2 follows the processing of the final a in a string a i, the stack is emptied and the input is accepted. Reaching q 2 in any other manner results in an unsuccessful computation, since no input is processed after q 2 is entered. 37

44 38 CHAPTER 5. PUSHDOWN AUTOMATA AND CONTEXT-FREE LANGUAGES The ε-transition from q 0 allows the machine to enter q 2 after the entire input string has been read, since a symbol is not required to process an ε-transition. The transition, which is applicable whenever the machine is in state q 0, introduces nondeterministic computations of M. Example The even-length palindromes over {a, b} are accepted by the PDA That is, L(M) = {ww R w {a, b} }. Figure 5.2: PDA L(M) = ww R A successful computation remains in state q 0 while processing the string w and enters state q 1 upon reading the first symbol in w R. 5.2 Variations on the PDA Theme Pushdown automata are often defined in a manner that differs slightly from Definition In this section we examine several alterations to our definition that preserve the set of accepted languages. Along with changing the state, a transition in a PDA is accompanied by three actions: popping the stack, pushing a stack element, and processing an input symbol. A PDA is called atomic if each transition causes only one of the three actions to occur. Transitions in an atomic PDA have the form [q j, ε] δ(q i, a, ε) [q j, ε] δ(q i, ε, A) [q j, A] δ(q i, ε, ε) Theorem shows that the languages accepted by atomic PDAs are the same as those accepted by PDAs. Moreover, it outlines a method to construct an equivalent atomic PDA from an arbitrary PDA. Theorem Let M be a PDA. Then there is an atomic PDA M with L(M ) = L(M). Proof: To construct M, the nonatomic transitions of M are replaced by a sequence of atomic transitions. Let [q j, B] δ(q i, a, A) be a transition of M. The atomic equivalent requires two new states, p 1 and p 2, and the transitions [p 1, ε] δ(q i, a, ε) δ(p 1, ε, A) = {[p 2, ε]} δ(p 2, ε, ε) = {[q j, B]} In a similar manner, a transition that consists of changing the state and performing two additional actions can be replaced with a sequence of two atomic transitions. Removing all nonatomic transitions produces an equivalent atomic PDA. An extended transition is an operation on a PDA that replaces the stack top with a string of symbols, rather than just a single symbol. The transition [q j, BCD] δ(q i, u, A) replaces the stack top A with the string BCD with B becoming the new stack top. The apparent generalization does