Building Finite Automata From Regular Expressions

Transcription

1 Building Automt From Regulr Expressions We mke n FA from regulr expression in two steps: Trnsform the regulr expression into n NFA. Trnsform the NFA into deterministic FA. The first step is esy. Regulr expressions re ll uilt out of the tomic regulr expressions (where is chrcter in Σ) nd y using the three opertions ABnd A B nd A *. Other opertions (like A + ) re just revitions for comintions of these. NFAs for nd re trivil: Suppose we hve NFAs for A nd B nd wnt one for A B. We construct the NFA shown elow: Automton for A A Automton for B B The sttes leled A nd B were the ccepting sttes of the utomt for A nd B; we crete new ccepting stte for the comined utomton. A pth through the top utomton ccepts strings in A, nd pth through the ottom utomtion ccepts strings in B, so the whole utomton mtches A B. The construction for AB is even esier. The ccepting stte of the comined utomton is the sme stte tht ws the ccepting stte of B. We must follow pth through A s utomton, then through B s utomton, so overll A B is mtched. We could lso just merge the ccepting stte of A with the initil stte of B. We chose not to only ecuse the picture would e more difficult to drw. Automton for A Automton for B A

2 Finlly, let s look t the NFA for A *. The strt stte reches n ccepting stte vi, so is ccepted. Alterntively, we cn follow pth through the FA for A one or more times, so zero or more strings tht elong to A re mtched. Automton for A A Creting Deterministic Automt The trnsformtion from n NFA N to n equivlent DFA D works y wht is sometimes clled the suset construction. Ech stte of D corresponds to set of sttes of N. The ide is tht D will e in stte {x, y, z} fter reding given input string if nd only if N could e in ny one of the sttes x, y, or z, depending on the trnsitions it chooses. Thus D keeps trck of ll the possile routes N might tke nd runs them simultneously. Becuse N is finite utomton, it hs only finite numer of sttes. The numer of susets of N s sttes is lso finite, which mkes trcking vrious sets of sttes fesile. An ccepting stte of D will e ny set contining n ccepting stte of N, reflecting the convention tht N ccepts if there is ny wy it could get to its ccepting stte y choosing the right trnsitions. The strt stte of D is the set of ll sttes tht N could e in without reding ny input chrcters tht is, the set of sttes rechle from the strt stte of N following only trnsitions. Algorithm close computes those sttes tht cn e reched following only trnsitions. Once the strt stte of D is uilt, we egin to crete successor sttes: We tke ech stte S of D, nd ech chrcter c, nd compute S s successor under c. S is identified with some set of N s sttes, {n 1, n 2,...}. We find ll the possile successor sttes to {n 1, n 2,...} under c, otining set {m 1, m 2,...}. Finlly, we compute T = CLOSE({ m 1, m 2,...}). T ecomes stte in D, nd trnsition from S to T leled with c is dded to D. We continue dding sttes nd trnsitions to D until ll possile successors to existing sttes re dded. Becuse ech stte corresponds to finite suset of N s sttes, the

3 process of dding new sttes to D must eventully terminte. Here is the lgorithm for - closure, clled close. It strts with set of NFA sttes, S, nd dds to S ll sttes rechle from S using only trnsitions. void close(nfaset S) { while (x in S nd x y nd y notin S) { S = S U {y} }} Using close, we cn define the construction of DFA, D, from n NFA, N: DFA MkeDeterministic(NFA N) { DFA D ; NFASet T D.StrtStte = { N.StrtStte } close(d.strtstte) D.Sttes = { D.StrtStte } while (sttes or trnsitions cn e dded to D) { Choose ny stte S in D.Sttes nd ny chrcter c in Alphet T = {y in N.Sttes such tht x c y for some x in S} close(t); if (T notin D.Sttes) { D.Sttes = D.Sttes U {T} D.Trnsitions = D.Trnsitions U {the trnsition S c T} } } D.AcceptingSttes = { S in D.Sttes such tht n ccepting stte of N in S} } Exmple To see how the suset construction opertes, consider the following NFA: Stte 1 hs itself s successor under. When stte 1 s - successor, 2, is included, {1,2} s successor is {1,2}. {3,4,5} s successors under nd re {5} nd {4,5}. {4,5} s successor under is {5}. Accepting sttes of D re those stte sets tht contin N s ccepting stte which is 5. The resulting DFA is: We strt with stte 1, the strt stte of N, nd dd stte 2 its - successor. D s strt stte is {1,2}. Under, {1,2} s successor is {3,4,5}. 1,2 3,4,5 5 4,

4 It is not too difficult to estlish tht the DFA constructed y MkeDeterministic is equivlent to the originl NFA. The ide is tht ech pth to n ccepting stte in the originl NFA hs corresponding pth in the DFA. Similrly, ll pths through the constructed DFA correspond to pths in the originl NFA. Wht is less ovious is the fct tht the DFA tht is uilt cn sometimes e much lrger thn the originl NFA. Sttes of the DFA re identified with sets of NFA sttes. If the NFA hs n sttes, there re 2 n distinct sets of NFA sttes, nd hence the DFA my hve s mny s 2 n sttes. Certin NFAs ctully exhiit this exponentil lowup in size when mde deterministic. Fortuntely, the NFAs uilt from the kind of regulr expressions used to specify progrmming lnguge tokens do not exhiit this prolem when they re mde deterministic. As rule, DFAs used for scnning re simple nd compct. If creting DFA is imprcticl (ecuse of size or speed-ofgenertion concerns), we cn scn using n NFA. Ech possile pth through n NFA is trcked, nd rechle ccepting sttes re identified. Scnning is slower using this pproch, so it is used only when construction of DFA is not prcticl Optimizing Automt We cn improve the DFA creted y MkeDeterministic. Sometimes DFA will hve more sttes thn necessry. For every DFA there is unique smllest equivlent DFA (fewest sttes possile). Some DFA s contin unrechle sttes tht cnnot e reched from the strt stte. Other DFA s my contin ded sttes tht cnnot rech ny ccepting stte. It is cler tht neither unrechle sttes nor ded sttes cn prticipte in scnning ny vlid token. We therefore eliminte ll such sttes s prt of our optimiztion process. We optimize DFA y merging together sttes we know to e equivlent. For exmple, two ccepting sttes tht hve no trnsitions t ll out of them re equivlent. Why? Becuse they ehve exctly the sme wy they ccept the string red so fr, ut will ccept no dditionl chrcters. If two sttes, s 1 nd s 2, re equivlent, then ll trnsitions to s 2 cn e replced with trnsitions to s 1. In effect, the two sttes re merged together into one common stte. How do we decide wht sttes to merge together?

5 We tke greedy pproch nd try the most optimistic merger of sttes. By definition, ccepting nd non-ccepting sttes re distinct, so we initilly try to crete only two sttes: one representing the merger of ll ccepting sttes nd the other representing the merger of ll non-ccepting sttes. This merger into only two sttes is lmost certinly too optimistic. In prticulr, ll the constituents of merged stte must gree on the sme trnsition for ech possile chrcter. Tht is, for chrcter c ll the merged sttes must hve no successor under c or they must ll go to single (possily merged) stte. If ll constituents of merged stte do not gree on the trnsition to follow for some chrcter, the merged stte is split into two or more smller sttes tht do gree. As n exmple, ssume we strt with the following utomton: c d c Initilly we hve merged nonccepting stte {1,2,3,5,6} nd merged ccepting stte {4,7}. A merger is legl if nd only if ll constituent sttes gree on the sme successor stte for ll chrcters. For exmple, sttes 3 nd 6 would go to n ccepting stte given chrcter c; sttes 1, 2, 5 would not, so split must occur We will dd n error stte s E to the originl DFA tht is the successor stte under ny illegl chrcter. (Thus reching s E ecomes equivlent to detecting n illegl token.) s E is not rel stte; rther it llows us to ssume every stte hs successor under every chrcter. s E is never merged with ny rel stte. Algorithm Split, shown elow, splits merged sttes whose constituents do not gree on common successor stte for ll chrcters. When Split termintes, we know tht the sttes tht remin merged re equivlent in tht they lwys gree on common successors. Split(FASet StteSet) { repet for(ech merged stte S in StteSet) { Let S correspond to {s 1,...,s n } for(ech chr c in Alphet){ Let t 1,...,t n e the successor sttes to s 1,...,s n under c if(t 1,...,t n do not ll elong to the sme merged stte){ Split S into two or more new sttes such tht s i nd s j remin in the sme merged stte if nd only if t i nd t j re in the sme merged stte} } until no more splits re possile }

6 Returning to our exmple, we initilly hve sttes {1,2,3,5,6} nd {4,7}. Invoking Split, we first oserve tht sttes 3 nd 6 hve common successor under c, nd sttes 1, 2, nd 5 hve no successor under c (equivlently, hve the error stte s E s successor). This forces split, yielding {1,2,5}, {3,6} nd {4,7}. Now, for chrcter, sttes 2 nd 5 would go to the merged stte {3,6}, ut stte 1 would not, so nother split occurs. We now hve: {1}, {2,5}, {3,6} nd {4,7}. At this point we re done, s ll constituents of merged sttes gree on the sme successor for ech input symol. Once Split is executed, we re essentilly done. Trnsitions etween merged sttes re the sme s the trnsitions etween sttes in the originl DFA. Thus, if there ws trnsition etween stte s i nd s j under chrcter c, there is now trnsition under c from the merged stte contining s i to the merged stte contining s j. The strt stte is tht merged stte contining the originl strt stte. Accepting sttes re those merged sttes contining ccepting sttes (recll tht ccepting nd non-ccepting sttes re never merged) Returning to our exmple, the minimum stte utomton we otin is d c 1 2,5 3,6 4,7 196