FORMAL LANGUAGES, GRAMMARS, AND AUTOMATA

Transcription

1 FORMAL LANGUAGES, GRAMMARS, AND AUTOMATA Outline of theory of computationm What is a language? May refer either to the (human) capacity for acquiring and using complex systems of communication, or to a specific c instance of such a system of complex communication. 1

2 WHAT IS AUTOMATA THEORY? Study of abstract computing devices, or machines Automaton = an abstract computing device Note: A device need not even be a physical hardware! A fundamental question in computer science: Find out what different models of machines can do and cannot do The theory of computation 3 (A pioneer of automata theory) ALAN TURING ( ) Father of Modern Computer Science English mathematician Studied abstract machines called Turing machines even before computers existed Heard of the Turing test? 4 2

3 THEORY OF COMPUTATION: A HISTORICAL PERSPECTIVE 1930s Alan Turing studies Turing machines Decidability Halting problem s Finite automata machines studied Noam Chomsky proposes the Chomsky Hierarchy for formal languages 1969 Cook introduces intractable problems or NP-Hard problems Modern computer science: compilers, computational & complexity theory evolve 5 LANGUAGES & GRAMMARS Or words Languages: A language is a collection of sentences of finite length all constructed from a finite alphabet of symbols Grammars: A grammar can be regarded as a device that enumerates the sentences of a language - nothing more, nothing less 6 Image source: Nowak et al. Nature, vol 417,

4 THE CHOMSKY HIERACHY A containment hierarchy of classes of formal languages Regular (DFA) Contextfree (PDA) Contextsensitive (LBA) Recursivelyenumerable (TM) 7 AUTOMATA Finite Automata Pushdown Automata Linear Bounded Automata Turing Machines 4

5 THE CENTRAL CONCEPTS OF AUTOMATA THEORY 9 ALPHABET An alphabet is a finite, non-empty set of symbols We use the symbol (sigma) to denote an alphabet Examples: Binary: = {0,1} All lower case letters: = {a,b,c,..z} Alphanumeric: = {a-z, A-Z, 0-9} DNA molecule letters: = {a,c,g,t} 10 5

6 STRINGS A string or word is a finite sequence of symbols chosen from Empty string is (or epsilon ) Length of a string w, denoted by w, is equal to the number of (non- ) characters in the string E.g., x = x = 6 x = x =? xy = concatentation of two strings x and y 11 STRINGS Operator Power Concatenate x = a y = bc x y = abc s = s s = s Substring x = bc Reflection x = ab y = ba Power sn = s s s.. s ( n times) s 0 = x = a x 4 = aaaa Let be an alphabet. k = the set of all strings of length k * = 0 U 1 U 2 U + = 1 U 2 U 3 U 12 6

7 LANGUAGES L is a said to be a language over alphabet, only if L * this is because * is the set of all strings (of all possible length including 0) over the given alphabet Examples: Let L be the language of all strings consisting of n 0 s followed by n 1 s: L = {,01,0011,000111, } Let L be the language of all strings of with equal number of 0 s and 1 s: L = {,01,10,0011,1100,0101,1010,1001, } 13 CARDINALITY Language cardinality: number of strings in the language Cardinality language L L Finite language -> Finite cardinality Definition: Ø denotes the Empty language Let L1 = { }; Is L2=Ø? No L1 = 1 L2 = 0 7

8 LANGUAGE OPERATION Concatenation L1 = {a, b} L2 = {c, d} L = L1 L2 = = {xy x L1, y L2} = = {ac, ad, bc, bd} Union L1 = {a, b} L2 = {c, d} L = L1 L2= = {x L1 L2} = = {a, b, c, d} KLEENE STAR Given a set (alphabet) define 0 = { } (the language consisting only of the empty word), 1 = (the language consisting only of string of length 1) and define recursively the set i+1 = * 1 If is a formal language, then i, the i-th power of the set, is a shorthand for the concatenation of set with itself i times. That is, - i can be understood to be the set of all strings that can be represented as the concatenation of i strings in. The definition of Kleene star on is n= 0 1 v2. n is Kleene plus n= 1 v2. n 8

9 THE MEMBERSHIP PROBLEM Given a string w *and a language L over, decide whether or not w L. Example: Let w = Is w the language of strings with equal number of 0s and 1s? 17 GRAMMARS A formal grammar G is any compact, precise mathematical definition of a language L. As opposed to just a raw listing of all of the language s legal sentences, or just examples of them. A grammar implies an algorithm that would generate all legal sentences of the language. Often, it takes the form of a set of recursive definitions. A popular way to specify a grammar recursively is to specify it as a phrase-structure grammar. 9

10 PHRASE-STRUCTURE GRAMMARS A phrase-structure grammar (abbr. PSG) G = (V,T,S,P) is a 4-tuple, in which: V is a set of special words called nonterminals. (Representing concepts like noun ). T is a set of words called terminals Actual words of the language. S V is a special nonterminal, the start symbol. P is a set of productions (to be defined). Rules for substituting one sentence fragment for another. A phrase-structure grammar is a special case of the more general concept of a string-rewriting system, due to Post. PRODUCTIONS A production p P is a pair p=(, ) of sentence fragments (not necessarily in L), which may generally contain a mix of both terminals and nonterminals that is,, (V T)* We often denote the production a. Read goes to (like a directed graph edge) Call the before string, the after string. It is a kind of recursive definition meaning that If l r LT, then l r LT. Language tamplate for the grammar G That is, if l r is a legal sentence template, then so is l r. That is, we can substitute in place of in any sentence template. A phrase-structure grammar imposes the constraint that each Language Template must contain a nonterminal symbol. 10

11 LANGUAGES FROM PSGS The recursive definition of the language G(L) (language generated by G), or simply L. defined by the PSG: G = (V, T, S, P): Rule 1: S LT (LT is L s template language) The start symbol is a sentence template (member of LT). Rule 2: ( ) P: l,r (V T)*: l r LT l r LT Any production, after substituting in any fragment of any sentence template, yields another sentence template. l r l r. (read, l r is derivable from l r ). Rule 3: ( σ LT: n V: n σ) σ L All sentence templates that contain no nonterminal symbols are sentences in L. Abbreviate this using lbr lar. (read, lar is directly derivable from lbr ). PSG EXAMPLE ENGLISH FRAGMENT We have G = (V, T, S, P), where: V = {(sentence), (noun phrase), (verb phrase), (article), (adjective), (noun), (verb), (adverb)} T = {a, the, large, hungry, rabbit, mathematician, eats, hops, quickly, wildly} S = (sentence) P = { (sentence) (noun phrase) (verb phrase), (noun phrase) (article) (adjective) (noun), (noun phrase) (article) (noun), (verb phrase) (verb) (adverb), (verb phrase) (verb), (article) a, (article) the, (adjective) large, (adjective) hungry, (noun) rabbit, (noun) mathematician, (verb) eats, (verb) hops, (adverb) quickly, (adverb) wildly } 11

12 BACKUS-NAUR FORM sentence ::= noun phrase verb phrase noun phrase ::= article [ adjective ] noun verb phrase ::= verb [ adverb ] article ::= a the adjective ::= large hungry noun ::= rabbit mathematician verb ::= eats hops adverb ::= quickly wildly Square brackets [] mean optional Vertical bars mean alternatives A SAMPLE SENTENCE DERIVATION (sentence) (noun phrase) (verb phrase) (article) (adj.) (noun) (verb phrase) the (adj.) (noun) (verb phrase) the (adj.) (noun) (verb) (adverb) the large (noun) (verb) (adverb) the large rabbit (verb) (adverb) the large rabbit hops (adverb) the large rabbit hops quickly 12

13 ANOTHER EXAMPLE Let G = (V,P,S,T} V= {A,B,S} T = {a,b } P = {S ABa, A BB, B ab, AB b}). One possible derivation in this grammar is: S ABa Aaba BBaba Bababa abababa. DERIVABILITY Recall that the notation w 0 w 1 means that ( ) P: l,r (V T)*: w 0 = l r w 1 = l r. The template w 1 is directly derivable from w 0. If w 2, w n-1 : w 0 w 1 w 2 w n, then we write w 0 * w n, and say that w n is derivable from w 0. The sequence of steps w i w i+1 is called a derivation of w n from w 0. Note that the relation * is just the transitive closure of the relation. 13

14 A SIMPLE DEFINITION OF L(G) The language L(G) (or just L) that is generated by a given phrase-structure grammar G=(V,T,S,P) can be defined by: L(G) = {w T* S * w} That is, L is simply the set of strings of terminals that are derivable from the start symbol. LANGUAGE GENERATED BY A GRAMMAR Example: Let G = ({S,A,a,b},{a,b}, S, {S aa, S b, A aa}). What is L(G)? Easy: We can just draw a tree of all possible derivations. We have: S aa aaa. and S b. Answer: L = {aaa, b}. aa aaa S b Example of a derivation tree or parse tree or sentence diagram. 14

15 GENERATING INFINITE LANGUAGES A simple PSG can easily generate an infinite language. Example: S 11S, S 0 (T = {0,1}). The derivations are: S 0 S 11S 110 S 11S 1111S and so on L = {(11)*0} the set of all strings consisting of some number of concatenations of 11 with itself, followed by 0. ANOTHER EXAMPLE Construct a PSG that generates the language L = {0 n 1 n n N}. 0 and 1 here represent symbols being concatenated n times, not integers being raised to the nth power. Solution strategy: Each step of the derivation should preserve the invariant that the number of 0 s = the number of 1 s in the template so far, and all 0 s come before all 1 s. Solution: S 0S1, S. 15

16 TYPES OF GRAMMARS - CHOMSKY HIERARCHY OF LANGUAGES Venn Diagram of Grammar Types: Type 0 Phrase-structure Grammars Type 1 Context-Sensitive Type 2 Context-Free Type 3 Regular DEFINING THE PSG TYPES Type 1: Context-Sensitive PSG: All after fragments are either longer than the corresponding before fragments, or empty: if, then < =. Type 2: Context-Free PSG: All before fragments have length 1: if, then = 1 ( V). Type 3: Regular PSGs: All after fragments are either single terminals, or a pair of a terminal followed by a nonterminal. if, then T TV. 16

17 CLASSIFYING GRAMMARS Given a grammar, we need to be able to find the smallest class in which it belongs. This can be determined by answering three questions: Are the left hand sides of all of the productions single non-terminals? If yes, does each of the productions create at most one non-terminal and is it on the right? Yes regular No context-free If not, can any of the rules reduce the length of a string of terminals and non-terminals? Yes unrestricted No context-sensitive Example The language { a n b n n 1} is context free. A grammar for this language is given by: S asb ab A derivation from this grammar is:- S asb aasbb (using S asb) aaabbb (using S ab) which derives a 3 b 3 17

18 Example The language { a n b n c n n 1} is contextsensitive but not context free. A grammar for this language is given by: S asbc abc CB BC ab ab bb bb bc bc cc cc A derivation from this grammar is:- S asbc aabcbc (using S abc) aabcbc (using ab ab) aabbcc (using CB BC) aabbcc (using bb bb) aabbcc (using bc bc) aabbcc (using cc cc) which derives a 2 b 2 c 2. 18