Context Free Languages

Transcription

1 Computer Science Department Fall 2014

2 Outline Context Free Grammars (CFGs) Ambiguity Chomsky Normal Form Pushdown Automata (PDAs) Equivalence between CFGs and PDAs The Pumping Lemma

3 Formal Grammars We ve seen that regular languages can be recognized by finite automata. Other types of automata recognize other classes of languages. Formal languages can also be described by formal grammars. Intuitively a grammar is a set of variables and terminal symbols, and a set of rules which state when a variable can be replaced with another string. Example A 0A1 A B B # A, B are variables (A is the start ), and 0, 1, # terminals. The grammar generates the language {0 n #1 n n 1}. Starting with A, there is a sequence of rule applications in which variables on the left side are replaced with strings on the right. The language above is not regular. It is context free, however.

4 Formal Grammar Definition A formal grammar G = (V, Σ, R, S) is a 4-tuple consisting of 1. a finite set V of nonterminal symbols or variables; 2. a finite set Σ of terminal symbols; 3. a finite set R of substitution (production) rules u v, where u and v are both strings from (V Σ), and u contains a variable; 4. a designated start symbol S V ; Sets V, Σ, and R are disjoint. Placing restrictions on the form of rules determines whether the language of the grammar falls into a particular class (e.g., regular).

5 Context Free Grammars and Languages Definition A formal grammar G = (V, Σ, R, S) is context free if and only if for each rule u v in R, u is a single variable. If u, v and w are strings from (V Σ) + and A w a rule of R, then uav yields uwv, written uav uwv. u derives v, written u v, if and only if u v or there is a sequence u u 1 u k v, with k 0. L(G) = {w S w} is the language of the grammar. Example A 0A1 A B B # A 0A1 00A11 000A B #111 L(G) = {0 n #1 n n 1}

6 Formal Grammars Multiple rules for the same variable are often written using. Other types of automata recognize other classes of languages. Example A 0A1 A B B # is equivalent to A 0A1 B B #

7 Example Here, variables are indicated with ( and ). Example (SENTENCE) (NOUN-PHRASE) (VERB-PHRASE) (NOUN-PHRASE) (CMPLX-NOUN) (CMPLX-NOUN)(PREP-PHRASE) (VERB-PHRASE) (CMPLX-VERB) (CMPLX-VERB)(PREP-PHRASE) (PREP-PHRASE) (PREP) (CMPLX-NOUN) (CMPLX-NOUN) (ARTICLE) (NOUN) (CMPLX-VERB) (VERB) (VERB)(NOUN-PHRASE) (ARTICLE) a the (NOUN) boy girl flower (VERB) touches likes sees (PREP) with The above grammar generates strings such as. a girl with a flower likes the boy

8 Combining Grammars Grammars can be combined to form new grammars. This fact can be used to design grammars for particular languages. Example Let G 1 be S 1 0S 1 1 ε. L(G 1 ) = {0 n 1 n n 0}. Let G 2 be S 2 1S 2 0 ε. L(G 2 ) = {1 n 0 n n 0}. Let G be S S 1 S 2 S 1 0S 1 1 ε S 2 1S 2 0 ε L(G) = {1 n 0 n n 0} {0 n 1 n n 0}.

9 CFGs for Regular Languages Context free grammars can be easily constructed for regular languages. Let D = (Q, Σ, δ, q 0, F ) be a DFA. Construct a CFG G = (V, Σ, R, S) as follows: 1. V = {R i q i Q} (make a variable for each state of Q). 2. S = R 0 (S corresponds to the start state q 0). 3. For each transition δ(q i, a) = q j, add rule R i ar j to R. 4. For each q i F, add rule R i ε to R. In general, a grammar is regular if all rules are of the form: A ab A a A ε where A, B V and a Σ. A language is regular if some regular grammar generates it.

10 In-class Questions L(M) = CSCI {w 2670 w ends Context in FreeaLanguages 1}. Give the regular grammar for the language which can be accepted by the following DFA M. The above DFA M can be formally defined as follows: Q = {q 1, q 2}. Σ = {0, 1}. δ is given by the table: δ 0 1 q 1 q 1 q 2 q 2 q 1 q 2 q 0 = q 1. F = {q 2}

11 In-class Questions Give the regular grammar for the language which can be accepted by the following DFA M. The regular grammar G can be formally defined as follows: V = {R 1, R 2}. Σ = {0, 1}. the set of rules R: S = R 1. R 1 0R 1 1R 2 R 2 0R 1 1R 2 ε

12 Parse Trees The derivation of a string using a grammar can also be represented using a parse tree.

13 Ambiguity In a given CFG, it s possible for there to be multiple parse trees for the same string w. Such a grammar is said to be ambiguous. Definition A derivation of a string w in a grammar is a leftmost derivation if, at each step of the derivation, it is the leftmost variable of the string that is replaced. A string w is derived ambiguously in CFG G if it has two or more leftmost derivations. A CFG G is ambiguous if it generates some string ambiguously. Some languages are inherently ambiguous. They can only be generated by ambiguous grammars.

14 Ambiguity Example This grammar is ambiguous (but the language generated by it is not inherently ambiguous). < EXPR > < EXPR > + < EXPR > < EXPR > < EXPR > (< EXPR >) a

15 Chomsky Normal Form Definition A CFG G is in Chomsky Normal Form (CNF) if and only if every rule of the grammar is of the form: A BC A a S ε where S is the start variable, A is any variable, B and C are variables other than S, and a is any terminal. Theorem For each CFL L, there exists a grammar G in CNF such that L(G) = L.

16 Chomsky Normal Form Theorem For each CFL L, there exists a grammar G in CNF such that L(G) = L. Proof. 1. Add a new start symbol and rule S 0 S, where S is the old start. 2. For each A ε (A S 0), delete it and for each B w, where A appears in w, add rules B w, where w is w with one or more As removed. If w = ε and B ε was deleted, don t re-add it. Repeat until all ε rules are deleted. 3. Remove each rule A B: If rule B u exists, add rule A u unless it was previously removed. 4. For each rule of the form A u 1u 2... u n, where each u i V Σ and n 3, replace the rule with rules A u 1A 1, A 1 u 2A 2,... A n 2 u n 1u n, where each A i is a new variable. 5. Replace each rule of the form A ub, u Σ, with A UB and U u.

17 Chomsky Normal Form Convert the following to CNF: S ASA ab A B S B b ε Step 1: Add a new start symbol and rule. Step 2: Process ε rules. S 0 S S ASA ab A B S B b ε S 0 S S ASA ab a A B S ε B b ε

18 Chomsky Normal Form Step 2: Process ε rules. S 0 S S ASA ab a AS SA S A B S ε B b Step 3: Process unit rules (S S). Here, no new rules need to be added. S 0 S S ASA ab a AS SA S A B S B b

19 Chomsky Normal Form Step 3: Process unit rules (S 0 S). S 0 S ASA ab a AS SA S ASA ab a AS SA A B S B b Step 3: Process unit rules (A S). S 0 ASA ab a AS SA S ASA ab a AS SA A B S ASA ab a AS SA B b

20 Chomsky Normal Form Step 3: Process unit rules (A B). S 0 ASA ab a AS SA S ASA ab a AS SA A B b ASA ab a AS SA B b Step 4: Process remaining rules. S 0 AA 1 ab a AS SA S AA 1 ab a AS SA A b AA 1 ab a AS SA B b A 1 SA

21 Chomsky Normal Form Step 4: Process remaining rules. S 0 AA 1 UB a AS SA S AA 1 UB a AS SA A b AA 1 UB a AS SA B b A 1 SA U a At this point, the grammar is in Chomsky Normal Form.

22 Stack

23 Conceptual View of a Stack

24 Stack: Push Operation

25 Stack: Pop Operation

26 Pushdown Automata Context free languages are recognized by pushdown automata (PDAs). A PDA has a memory in the form of a stack. At each step of computation, the PDA occupies some state q, a symbol a can be read from the input string (optionally, nothing need be read), and a symbol b can be popped from the top of the stack (optionally, nothing need be popped). Based on the state, input symbol, and stack symbol, the PDA optionally pushes a symbol onto the stack, and changes state. A PDA accepts a string w if it occupies an accept state after reading the last symbol of the input. Note that PDAs can be nondeterministic. Nondeterministic PDAs are more powerful than deterministic PDAs.

27 Pushdown Automata Definition A pushdown automaton P is a 6-tuple (Q, Σ, Γ, δ, q o, F ), where 1. Q is a finite set of states, 2. Σ is a finite input alphabet, 3. Γ is a finite stack alphabet, 4. δ : (Q Σ ε Γ ε ) P(Q Γ ε ) is the transition function, 5. q 0 is the start state, 6. F Q is a set of accept states. δ maps triples (q, a, b) where q is a state, a an input symbol, and b a stack symbol to sets of pairs (q, b ), where q is a state and b a stack symbol. a,b, b can also be ε. δ is nondeterministic. The PDA needn t read an input or stack symbol, and it needn t write a stack symbol. In general, there might be multiple transitions for a given state, input symbol, and stack symbol.

28 Transition Tables for PDAs Example Let M 1 = (Q, Σ, Γ, δ, q 1, F ), where 1. Q = {q 1, q 2, q 3, q 4 }, 2. Σ = {0, 1}, 3. Γ = {0, $}, 4. F = {q 1, q 4 }, and 5. δ is given by the below table. Input 0 1 ε Stack 0 $ ε 0 $ ε 0 $ ε q 1 {(q 2, $)} q 2 {(q 2, 0)} {(q 3, ε)} q 3 {(q 3, ε)} {(q 4, ε)} q 4 The symbol $ is, by convention, used to mark an empty stack. As shown in the table, when started, the PDA immediately pushes it onto the stack.

29 State Diagrams for PDAs Example The PDA M 1 recognizes the language {0 n 1 n n 0}. The PDA may be represented using the following state diagram. A label a, b c means that a is read from the input, b is popped from the stack, and c is pushed onto the stack.

30 Acceptance for PDAs Definition Let P = (Q, Σ, Γ, δ, q o, F ) be a PDA and w = w 1 w 2... w m a string such that each w i Σ ɛ. P accepts w if and only if there exists a sequence r 0, r 1,..., r m of states of Q, and a sequence of strings s 0, s 1,..., s m, each s i Γ (these represent the stack contents at a given step), such that 1. r 0 = q 0 and s 0 = ε (initially, the stack is empty), 2. for each 0 i < m, (r i+1, b) δ(r i, w i, a), where s i = at and s i+1 = bt for some a, b Γ ε and t Γ, and 3. r m F.

31 Example PDAs Example The below PDA M 2 recognizes the language {a i b j c k i, j, k 0 and i = j or i = k}.

32 Example PDAs Example The below PDA M 3 recognizes the language {ww R w {0, 1} }.

33 Pushdown Automata Example Given an informal description for a PDA recognizing the below language. 1. {a i b j c k i = j or j = k where i, j, k 0}

34 Equivalence between CFGs and PDAs Theorem A language is context free if and only if it is recognized by some pushdown automaton. This can be broken down into two lemmas (which will be proven). Lemma If a language is context free, then it is recognized by a pushdown automaton. Lemma If a language is recognized by a pushdown automaton, then it is context free.

35 Equivalence between CFGs and PDAs Lemma If a language is context free, then it is recognized by a pushdown automaton. We construct a PDA to recognize context free language L. We modify the definition of the PDA slightly to allow whole strings to be pushed onto the stack in a single step (this causes no harm). The machine only has three states: q start, q loop, and q accept. The input alphabet of the PDA is Σ, the alphabet of the grammar. The stack alphabet is {$} V Σ. The transition functions for the PDA are as follows: δ(q start, ε, ε) = {(q loop, S$), where S is the start state of the grammar. If A w is a rule of G, then (q loop, w) δ(q loop, ε, A). If a Σ, then (q loop, ε) δ(q loop, a, a). (q accept, ε) δ(q loop, ε, $).

36 Equivalence between CFGs and PDAs The conversion produces a PDA of the form below. It accepts w if it enters q accept (and there is no more input). (Undefined transitions go to a trap state).

37 Equivalence between CFGs and PDAs Implementing pushing a compound string onto the stack.

38 Equivalence between CFGs and PDAs Construct a PDA for the following grammar. S atb b T Ta ε

39 Equivalence between CFGs and PDAs

40 Equivalence between CFGs and PDAs Construct a PDA for the following grammar. Try processing (()()). S SS (S) ()

41 Equivalence between CFGs and PDAs Lemma If a language is recognized by a pushdown automaton, then it is context free. For this direction, we construct a grammar to generate the language recognized by the PDA P. P must have a certain form. It has a single accept state, q accept. An existing PDA can be modified by adding a new accept state and connecting old states to it via ε-edges. It can only accept a string if its stack is empty. This can be accomplished by adding loops ε, a ε to the accept state for each a Γ. Each step pops or pushes a stack symbol but never both. This can be accomplished by adding new states and modifying transitions: 1. If a transition pops and pushes, split the transition into separate pop and push transitions going through a new state. 2. If a transition neither pops nor pushes, add transitions to push and then pop an arbitrary symbol, going through a new state.

42 Equivalence between CFGs and PDAs Given PDA P = (Q, Σ, Γ, δ, q 0, {q accept}), we construct the grammar G = (V, Σ, R, S) as follows: V = {A pq p Q and q Q} (we add a variable for each pair of states). S = A q0 q accept. R is defined as follows: For each p Q, add A pp ε. For each p, q, r Q, add rule A pq A pr A rq. For each p, q, r, s Q, t Γ, and a, b Σ ε, if (r, t) δ(p, a, ε) and (q, ε) δ(s, b, t), then add A pq aa rsb to R. In the grammar, if one starts in state A pq and applies the rules, all of the strings that take the PDA P from state p (and an empty stack) to state q (with an empty stack) will be generated. A q0 q accept generates all strings that take P from q 0 to q accept (beginning and ending with an empty stack). I.e., A q0 q accept generates L(P). The starting and ending with the empty stack implies that it is possible to begin and end with the same stack contents, regardless of what that is.

43 Equivalence between CFGs and PDAs Proposition If A pq generates x, then x can bring P from p with empty stack to q with empty stack. The proof is by induction on the length of the derivation of x from variable A pq. Basis: If the derivation takes only one step then the rule used must be A pp ε. All other rules contain variables on the RHS. And so x = ε and q = p. It is clearly possible to go from p to p reading ε (by staying put).

44 Equivalence between CFGs and PDAs Induction: Suppose the claim holds for derivations of length k or less, k 1, and suppose the derivation of x from A pq takes k + 1 steps. Since the derivation is of length greater than 1, it must be that the rule used has either the form A pq aa rsb or A pr A rq. We consider each in turn. If the rule has the form A pq aa rsb, then x = awb and A rs w. By the construction of G, (r, t) δ(p, a, ε) and (q, ε) δ(s, b, t), for some a, b Σ ε and t Γ. That is, one may go from p to r reading input a and pushing t onto the stack. By the inductive hypothesis, w can bring P from state r to state s, leaving the stack in the same state as when it started (here, with t as the top symbol). Given that (q, ε) δ(s, b, t), x thus can bring p with empty stack to q with empty stack. If the rule has the form A pq A pr A rq, then x can be split into halves y and z such that A pr y and A rq z. By the inductive hypothesis, y can bring P from p with empty stack to r with empty stack, and z can bring P from r with empty stack to q with empty stack. And so x = yz can bring P from p with empty stack to q with empty stack. One of these cases must hold, and so the claim holds.

45 Equivalence between CFGs and PDAs Proposition If x can bring P from p with empty stack to q with empty stack, then A pq generates x. The proof is by induction on the number of steps in the computation brings P from p with empty stack to q with empty stack on input x. Basis: Suppose the computation has 0 steps. Then p = q and x must be ε. Since A pp ε is rule of G, A pq generates x.

46 Equivalence between CFGs and PDAs Induction: Suppose the claim holds for all computations taking k or fewer steps, k 0, and suppose a computation of P on input x takes k + 1 steps. Suppose the stack is empty only at the beginning and end of the computation. Then an input symbol a is read and a symbol t is pushed onto the stack on the first move, and P enters state r. t must be popped from the stack in the last move. Let b be the symbol read in the last move and s the state of P before the last move. Then (r, t) δ(p, a, ε) and (q, ε) δ(s, b, t). As such, rule A pq aa rsb exists as a rule of G. Let x = ayb. y can bring P from state r to state s never popping t from the stack, and so y can bring P from r with empty stack to s with empty stack. By the ind. hyp., A rs generates y. From this, A pq generates x. If the stack is empty at some intermediary step with corresponding state r, then the computations from p to r and from r to q both have length less than k + 1. Let y be the input read going from state p to r in the first computation, and let z be the input read during the second computation. By the ind. hyp., A pr generates y and A rq generates z. As rule A pq A pr A qr is a rule of G, it follows that A pq generates x.

47 Every regular language is context free It has been proved that pushdown automata recognize the class of context-free languages. Every regular language is recognized by a finite automaton. Every finite automaton is automatically a pushdown automaton that simply ignores its stack. Hence, every regular language is a context-free language. { regular languages } { context free languages }

48 A Pumping Lemma for CFLs Like regular languages, CFLs have strings that can be pumped. For large strings s, the parse tree for s has a branch with a repeated variable R. A later occurrence of R can be replaced with the subtree under an earlier occurrence. Similarly, an earlier occurrence can be replaced with the subtree under the last occurrence. As seen below, s = uvxyz, and v and y can be pumped.

49 A Pumping Lemma for CFLs Theorem If A is a context free language, then there exists an integer p such for all s A with s p, s may be divided into five pieces s = uvxyz such that 1. uv i xy i z A for each i 0, 2. vy > 0, and 3. vxy p. Observe that v or y can be empty, but not both. (Without this, the theorem would be trivial).

50 A Pumping Lemma for CFLs Let G be a CFG and let b be the maximum number of symbols on the RHS of any rule in G. It is clear that the branching factor of any parse tree generated by G will be at most b, and so if a parse tree for a string s has height h, then s has at most b h symbols. Similarly, if s b h + 1, it must be that any parse tree for s has height at least h + 1. (Proof of the Pumping Lemma) Let A be a CFL with grammar G, and let p = b V +1, where b is the maximum number of symbols on the RHS of any rule of G. Let s be a string generated by G such that s p. Let τ be a minimal parse tree for s (minimal in the sense that no other parse tree for s has fewer nodes than τ). Assume the root of τ is T. It follows that τ has height V + 1, and so there exists a branch B of length (in edges) V + 1. Since that is so, some variable R must appear at least twice on the branch.

51 A Pumping Lemma for CFLs (Proof of the Pumping Lemma) We may chose any branch with length V + 1, and so we assume B to be the longest branch of τ. We also assume R is a variable on B in which multiple occurrence occur on the last V + 1 variables on the path, and we let R 1 and R 2 be the last two occurrences of R on the branch. It is clear that s can be divided into uvxyz, where vxy consists of the leaves of the tree rooted at R 1, x consists of the leaves of the tree rooted at R 2, and uvxyz consists of the leaves of the tree rooted at T. Since R 1 and R 2 are occurrences of R, it is possible to replace the subtree rooted at R 2 with the one rooted at R 1, yielding a larger tree. This process can be repeated infinitely, giving parse trees for uv i xy i z for each i > 1. And so the resulting strings are all in A. Similarly, is possible to replace the subtree rooted at R 1 with the one rooted at R 2, yielding the tree τ and corresponding string uv 0 xy 0 z. If both v and y are empty, then uxz would equal uvxyz. Since τ has fewer nodes than τ, this would mean that τ is not a minimal tree for s. But we have assumed τ to be minimal, and so one of v or y must be nonempty.

52 A Pumping Lemma for CFLs (Proof of the Pumping Lemma) Since R 1 and R 2 appear among the last V + 1 variables on B, it must be that the subtree rooted at R 1 has height at most V + 1. But a tree of that height can generate strings of length at most b V +1, where b is the maximum number of symbols on the RHS of any rule of G. And so vxy b V +1. However, p = b V +1, and so vxy p.

53 A Pumping Lemma for CFLs Example B = {a n b n c n n 0} is not context free. Proof. Suppose that B is context free, and let s = a p b p c p, where p is the pumping length of B. By the pumping lemma, s = uvxyz such that vy > 0 and vxy p. Given that vxy p, v and y can consist of at most two types of symbols (e.g., a and b). Neither will contain the third (e.g., c). Thus, pumping up v and y will ruin the symmetry of s, which implies that uv 2 xy 2 z / B, contradicting the pumping lemma.

54 A Pumping Lemma for CFLs Example B = {a i b j c k 0 i j k} is not context free. Proof. Suppose that B is context free, and let s = a p b p c p, where p is the pumping length of B. By the pumping lemma, s = uvxyz such that vy > 0 and vxy p. Given that vxy p, v and y can consist of at most two types of symbols (e.g., a and b). Neither will contain the third (e.g., c). We consider the possibilities. If neither contain c, then v and y can be pumped so that the number of a s and/or b s outnumber the c s. If neither contain b, then v and y must be all a s (or empty) or else all c s. In the former case, the a s can be pumped up to outnumber the b s. In the latter, v and y can be pumped down so so that the number of b s outnumber the c s. If neither contain a s, we can again pump down so that the a s outnumber the b s, or the b s outnumber the c s.

55 A Pumping Lemma for CFLs Example 1. The language A 1 = {0 n 1 n 0 n 1 n n 0} 2. The language A 2 = {w#w w {0, 1} } 3. The language A 3 = {0 n #0 2n #0 3n n 0} 4. The language A 4 = {w#t w is a substring of t, where w, t {a, b} } Proof.???