1. Introduction Texts and their processing

Transcription

1

2 Chpter /7/97 1. Introduction 1.1. Texts nd their processing One of the simplest nd nturl types of informtion representtion is y mens of written texts. Dt to e processed often does not decompose into independent records. This type of dt is chrcterized y the fct tht it cn e written down s long sequence of chrcters. Such liner sequence is clled text. The texts re centrl in "word processing" systems, which provide fcilities for the mnipultion of texts. Such systems usully process ojects which re quite lrge. For exmple, this ook contins proly more thn million chrcters. Text lgorithms occur in mny res of science nd informtion processing. Mny text editors nd progrmming lnguges hve fcilities for processing texts. In iology, text lgorithms rise in the study of moleculr sequences. The complexity of text lgorithms is lso one of the centrl nd most studied prolems in theoreticl computer science. It could e sid tht it is the domin where the prctice nd theory re very close together. The sic textul prolem is the prolem clled pttern mtching. It is used to ccess informtion nd proly mny computers re solving in this moment this prolem s frequently used opertion in some ppliction system. Pttern-mtching is comprle in this sense to sorting, or to sic rithmetic opertions. Consider the prolem of reder of the French dictionry "Grnd Lrousse", who wnts ll entries relted to the word "Mrie-Curie-Sklodowsk". This is n exmple of pttern mtching prolem, or string-mtching. In this cse, the word "Mrie-Curie-Sklodowsk" is the pttern. Generlly we my wnt to find string clled pttern of length m inside text of length n where n is greter thn m. The pttern cn e descried in more complex wy to denote set of strings nd not only single word. In mny cses m is very lrge. In genetics the pttern cn correspond to genome tht cn e very long, in imge processing the digitized imges sent serilly tke millions of chrcters ech. The string-mtching prolem is the sic question considered in the ook, together with its vritions. String-mtching is lso the sic suprolem in very other lgorithmic prolems on texts. Below there is (not exclusive) list of sic groups of prolems discussed in the ook : vritions on the string-mtching prolem, prolems relted to the structure of the segments of text, dt compression, pproximtion prolems, finding regulrities, extensions to two-dimensionl imges, extensions to trees, optiml time-spce implementtions, optiml prllel implementtions. The forml definition of the string-mtching nd mny other prolems is given in the next chpter. We introduce now some of them informlly in the context of pplictions. Efficient Algorithms on Texts 3 M. Crochemore & W. Rytter

3 1.2 Text file fcilities The Unix system uses s min feture text files for exchnging informtion. The user cn get informtion from the files nd trnsform them through different existing commnds. The tools often ehve s filters tht red their input once nd produce the output simultneously. These tools cn e esily connected with ech others, nd especilly through the pipelining fcility. This often reduces the cretion of new commnds to few lines of lredy existing commnds. One of these useful commnds is grep, cronym of "generl regulr expression print". An exmple of the formt of grep is grep Mrie-Curie-Sklodowsk Grnd-Lrousse provided "Grnd-Lrousse" is file on your computer. The output of this commnd is the list of lines from the file which contin n occurrence of the word "Mrie-Curie-Sklodowsk". This is n instnce of the string-mtching prolem. Another exmple with more complex pttern cn e grep '^Chpter [0-9]' Book to list the titles of ook ssuming titles egin with Chpter followed y numer. In this cse the pttern denotes set of strings (even potentilly infinite), nd not only one string. The nottion to specify ptterns is known s regulr expressions. This is n instnce of the regulr-expression-mtching prolem. The indispensle complement of grep is sed (strem editor). It is designed to trnsform its input. It cn replce ptterns of the input y specific strings. Regulr expressions re lso ville with sed. But the editor contins n even more powerful nottion. This llows, for instnce, the ction on line of the input text contining twice sme word. It cn e pplied to delete two consecutive occurrences of sme word in text. This is oth n instnce of the repetition-finding prolem, pttern-mtching prolem nd, more generlly, the prolem of finding regulrities in strings. The very helpful mtching device sed on regulr expressions is omnipresent in the Unix system. It cn e used inside text editors such s ed nd vi, nd generlly in lmost ll Unix commnds. The ove tools, grep nd sed, re sed on this mechnism. There is even progrmming lnguge sed on pttern mtching ctions. It is the wk lnguge, where the nme wk comes from the initils of the uthors, Aho, Weinerger nd Kernighn. A simple wk progrm is sequence of pttern-ction sttements : pttern1 { ction1 } pttern2 { ction2 } pttern3 { ction3 }

4 Chpter /7/97 The sic components of this progrm re pttern to e find inside the lines of the current file. When pttern is found, the corresponding ction is pplied to the line. So, severl ctions my e pplied sequentilly to sme line. This is n instnce of the multi-pttern mtching prolem. The lnguge wk is ment for converting dt from one form to nother form, counting things, dding up numers nd extrcting informtion for reports. It contins n implicit input loop, nd the pttern-ction prdigm often eliminte control flow. This lso frequently reduces the size of progrm to few sttements. For instnce, the following wk progrm prints the numer of lines of the input tht contin the word "rcdr" : rcdr { count++ } END { print count } The pttern "END" mtches the end of input file, so tht the result is printed fter the input hs een entirely processed. The lnguge contins nice fetures tht strengthen the simplicity of the pttern mtching mechnism, such s defult initiliztion for vriles, implicit declrtions, nd ssocitive rrys providing ritrry kinds of suscripts. All this mkes wk convenient tool for rpid prototyping. The lnguge wk cn e considered s generliztion of nother Unix tool, lex, imed t producing lexicl nlyzers. The input of lex progrm is the specifiction of lexicl nlyzer y mens of regulr expressions (nd few other possiilities). The output is the source in the C progrmming lnguge of the specified lexicl nlyzer. A specifiction in lex is minly sequence of pttern-ction sttements s in wk. Actions re pieces of C code to e inserted in the lexicl nlyzer. At run time, these pieces of code execute the ction corresponding to the ssocited pttern, when found. The following line is typicl sttement of lex progrm : [A-Z-z]+([A-Z-z0-9])* { yylvl = Instll(); return(id); } The pttern specifies identifiers, tht is, strings of chrcters strting with one letter nd contining only letters nd digits. The ction leds the generted lexicl nlyzer to store the identifier nd to return the string type "ID" to the clling prser. It is nother instnce of the regulr expression mtching prolem. The question of constructing pttern-mtching utomt is n importnt component hving prcticl ppliction in the lex softwre. Lex is often comined with ycc nd twig for the development of prsers, trnsltors, compilers, or simply prototypes of these. Ycc llows the specifiction of prser y grmmr rules. It produces the source of the specified prser in C lnguge. Twig is tree-mnipultion lnguge tht helps to design efficient code genertors. It trnsforms tree-trnsltion scheme into code genertor sed on fst top-down tree-pttern mtching lgorithm. The output genertor lso uses dynmic progrmming techniques for optimiztion purposes ut the Efficient Algorithms on Texts 5 M. Crochemore & W. Rytter

5 essentil feture is the tree-pttern mtching ction for tree rewriting. It hs een shown, y experimentl results on few significnt pplictions, tht twig-generted code genertors re often fster thn hnd-generted compilers even when the ltter incorporte some tricky optimiztions. One of the sic prolems considered for the design of such tool is the tree pttern-mtching prolem. Texts such s ooks or progrms re likely to e chnged during elortion. Even fter their chievement they often support periodic upgrdes. These questions re relted to text comprisons. Sometimes lso we wish to find string, nd we do not completely rememer it. The serch hs to e performed with non entirely specified pttern. This is n instnce of the pproximte pttern mtching. Keeping trck of ll consecutive versions of text sometimes does not help ecuse the text cn e very long nd chnges my e hrd to find. The resonle wy to control the process is to hve n esy ccess to differences etween the vrious versions. There is no universl notion on wht re the differences, nor conversely wht re the similrities etween two texts. However, it cn e greed tht the intersection of two texts is longest common sutext of oth. This is clled in our ook the longest common fctor prolem. So tht the differences etween two texts re the respective complements of the common prt. The Unix commnd diff uilds on this notion. Consider the texts A nd B s follows (Molière's joke, 1670) Text A : Belle Mrquise, vos eux yeux me font mourir d'mour Text B : D'mour mourir me font, Belle Mrquise, vos eux yeux The commnd "diff A B" produces 01 > D'mour mourir me font, 3d3 < me font mourir d'mour An option of the commnd diff produces sequence of ed instructions to trnsform one text into the other text. The similrity of texts cn e mesured s miniml numer of edit opertions to trnsform one text into the other. The computtion of such mesure is n instnce of the edit distnce prolem. 1.3 Dictionries

6 Chpter /7/97 The serch for words or ptterns in sttic texts is quite different question thn the previous pttern mtching mechnism. Usul dictionries, for instnce, re orgnized in order to speed up the ccess to entries. Another exmple of the sme question is given y indexes. Technicl ooks often contin n index of chosen terms tht gives pointers to prts of text relted to words in index. The lgorithms involved in the cretion of n index form specific group. The use of dictionries or lexicons is often relted to nturl lnguge processing. Lexicons of progrmming lnguges re smll, nd their representtion is not hrd prolem during the development of compiler. A contrrio, English contins pproximtively 100,000 words, nd even twice more if inflected forms re considered. In French inflected forms produce more thn 500,000 words. The representtion of lexicons of this size mkes the prolem it hrder. A simple use of dictionries is mde y spelling checkers. The Unix commnd spell reports the words of its input tht re not stored in the lexicon. This rough pproch does not yield pertinent checker, ut it prcticlly helps to find typing errors. The lexicon used y spell contins round 70,000 entries stored within less thn 60 kiloytes of rndom-ccess memory. Quick ccess to lexicons is necessry condition to produce good prsers. The dt structure useful for such ccesses is clled n index. In our ook indexes correspond to dt structures representing ll fctors of given (presumly long) text. We consider prolems of the construction such structures : suffix trees, directed cyclic word grphs, fctor utomt, suffix rrys. The tool PAT developed t the NOED Center (Wterloo, Cnd) is n implementtion of one of these structures tilored to work on lrge texts. There re severl pplictions tht effectively require some understnding of phrses in nturl lnguges, such s dt retrievl systems, interctive softwres nd chrcter recognition. An imge scnner is kind of photocopier. It is used to give digitlised version of n imge. When the imge is the pge of text, the nturl output of the scnner must e digitl form ville y text editor. The trnsformtion of digitlised imge of text into usul computer representtion of the text is relized y n Opticl Chrcter Reder (OCR). Scnning text with n OCR cn e 50 times fster thn retyping the text on keyord. OCR softwres re thus likely to ecome more common. But they still suffer from high degree of imprecision. The verge rte of errors in the recognition of chrcters is round one percent. Even if this my hppen s rther smll, this mens tht scnning ook produces round one error per line. This is to e compred with the usul very high qulity of texts checked y specilized persons. Technicl improvements on the hrdwre cn help to eliminte certin kinds of errors occurring on scnned texts in printed forms. But this cnnot void the prolem of recognizing texts written y hnd. In this cse, isoltion of chrcters is much hrder, nd miguous forms cn e encountered t recognition stge. Reduction on the numer of errors cn thus only e chieved y considering the context of chrcters, which ssumes some Efficient Algorithms on Texts 7 M. Crochemore & W. Rytter

7 understnding of the text. Imge processing is relted to the prolem of two-dimensionl pttern mtching. Another relted prolem is the dt structure for ll suimges, this is discussed in the ook in the context of the dictionry of sic fctors (suimges). No system is presently le to prse nturl lnguges. Indeed, it is questionle whether such n effective system cn exist. Words re certinly units of syntx in lnguges, ut they re not units of mening. This is oth ecuse words cn hve severl menings, nd ecuse they cn e prts of compound phrses tht crry mening only s whole. It ppers tht the proportion in the lexicon (of French nd severl other Europen lnguges) of idiomtic sentences, of metphoric nd technicl sentences whose mening cnnot e inferred from their elements, is very high. The nturl consequence is tht compound expressions nd the so-clled frozen phrses must e included in computerized dictionries. The dt se must lso incorporte t lest the syntctic properties of the elements. At present, no complete dictionry estlished ccording to these principles exists. This certinly represent of huge mount of dt to e collected ut this seems n indispensle work. Dt ses for the Europen lnguges, French, English, Itlin, nd Spnish re presently eing uilt. And studies on Aric, Chinese, Germn, Koren nd Mdgscn hve een performed. These question re studied, for instnce, t LADL (Pris), nd the reder cn refer to [Gr 91] for more informtion on specific spects of computtionl linguistics. The theoreticl pproch to the representtion of lexicons is y mens of trees or finite stte utomt. It ppers tht oth pproches re eqully efficient. This shows the prcticl importnce of the utomt theoretic pproch to text prolems. We hve shown t LITP (Pris) tht the use of utomt to represent lexicons is prticulrly efficient. Experiments hve een done on 500,000 word lexicon of LADL (Pris). The representtion supports direct ccess to ny word of the lexicon nd tkes only 300 kytes of rndom-ccess memory.

8 Chpter /7/ Dt compression One of the sic prolems in keeping lrge mount of textul informtion is the text compression prolem. Text compression mens reducing the representtion of text. It is ssumed tht the originl text cn e recovered from its compressed form. No loss of informtion is llowed. Text compression is relted to Huffmn coding prolem nd the fctoriztion prolem. This kind of compression contrsts with other kind of compression technics pplied to sounds or imges, where pproximtion is cceptle. Avilility of lrge mss storge does not decrese the interest for compressing dt. Indeed, users lwys use extr ville spce to store more dt or new kind of dt. Moreover the question remins importnt for storing dt on secondry storge devices. Exmples of implementtions of dictionries reported ove show tht dt compression is importnt in severl domins relted to nturl lnguge nlysis. Text compression is lso useful for telecommunictions. It ctully reduces the time to trnsmit documents vi telephone networks, for instnce. The success of Fx is perhps to e credited to compression technics. Generl compression methods often dpts themselves to the dt. This phenomenon is centrl in getting high compression rtios. But, it ppers in prctice tht methods tilored for specific dt led to the est results. We hve experimented this fct on dt sent y géosttionnires stellites. The dt hve een compressed to 7% without ny loss of informtion. The compression is very successful if there re redundncies nd regulrities in the informtion messge. The nlysis of dt is relted to the prolem of detecting regulrities in texts. Efficient lgorithms re prticulrly useful to expertize the dt. 1.5 Applictions of text lgorithms in genetics Molecules of nucleic cids crry lrge prt of informtion out the fundmentl determinnts of life, nd in prticulr out the reproduction of cells. There re two types of nucleic cids known s desoxyryonucleic cid (DNA) nd rionucleic cid (RNA). DNA is usully found s doule-strnded molecules. In vivo, the molecule is folded up like ll of string. The skeleton of DNA is sequence on the four-letter lphet of nucleotides : denine (A), gunine (G), cytosine (C), nd thymine (T). RNA molecules is usully single-strnded composed of rionucleotides : A, G, C, nd urcil (U). Processes of trnscription nd trnsltion led to the production of proteins, which hve lso primry structure s string Efficient Algorithms on Texts 9 M. Crochemore & W. Rytter

9 composed of 20 mino cids. In first pproch ll these molecules cn e viewed s texts. The discovery of powerful sequencing techniques fifteen yers go hs led to rpid ccumultion of sequence dt (more thn 10 million nucleotides of sequences). From the collect of sequences up to their nlysis lot of lgorithms on texts re implied. Moreover, only fst lgorithms re often fesile ecuse of the huge mount of dt. Collecting sequences is mde through gel udiordiogrphs. The utomtic trnscription of these gels into sequences is typicl two-dimensionl pttern mtching prolem in two dimensions. The reconstruction of whole sequence from smll segments is nother exmple of prolem tht occurs during this step. This prolem is clled the shortest common superstring prolem : construction of the shortest text contining severl given smller texts. Once new sequence is otined, the first importnt question on it is whether it resemles to ny other sequence lredy stored in dtnks. Before dding new moleculr sequence into n existing dtse one needs to know whether the sequence is lredy present or not. The comprison of severl sequences is usully relized y writing one over nother. The result is known s n lignment of the set of sequences. This is common disply in moleculr iology tht cn gives n ide on the evolution of the sequences though the muttions, insertions nd deletions of nucleotides. Alignment of two sequences is the edit distnce prolem : compute the miniml numer of edit opertions to trnsform one string into nother. It is relized y lgorithms sed on dynmic progrmming techniques similr to the one used y the Unix commnd diff. A tool, clled grep, developed t the University of Arizon is devoted to these questions, relted to pproximte string-mtching. Further questions on sequences re relted to their nlysis. The im is to discover the functions of ll prts of the sequence. For instnce, DNA sequences contin importnt regions (coding sequences) for the production of proteins. But, no good nswer is presently known to find ll coding sequences of DNA sequence. Another question on sequences is the reconstruction of their three-dimensionl structure. It seems tht prt of the informtion resides in the sequence itself. This is ecuse during the folding process of DNA, for instnce, nucleotides mtch pirwise (A with T, nd C with G). This produces pproximte plindromic symmetries (s TTAGCGGCTAA). Involved in ll these questions re pproximte serches for specific ptterns, for repetitions, for plindromes or ny other regulrities. The prolem of the longest common susequence is vrition of the lignment of sequences.

10 Chpter /7/ Structure of the ook The structure of our ook reflects oth the reltionships etween pplictions of text lgorithmic techniques nd the clssifiction of lgorithms ccording to the mesures of complexity considered. The ook cn e viewed s "prde" of lgorithms in which our min im is to discuss the foundtions of these lgorithms nd their interconnections. The ook egins with the lgorithmic nd theoreticl foundtions: mchine models, discussion on lgorithmic efficiency, review of sic prolems nd their interction, nd some comintorics on words. It is chpter 2. The topic of Chpters 3 nd 4 is STRING-MATCHING. The chpters del with two fmous lgorithms, Knuth-Morris-Prtt nd Boyer-Moore lgorithms respectively. The upmost improvements nd results on these lgorithms re incorported. We discuss here those string-mtching lgorithms which re efficient with respect to only one mesure of complexity sequentil time. Hence, the chpter cn e treted s "wrm-up" for other chpters devoted to the sme topic. We lter discuss in Chpter 13 more difficult lgorithms TIME-SPACE OPTIMAL STRING-MATCHING lgorithms tht work in liner time nd simultneously use only very smll dditionl memory. The considertion of two complexity mesures together usully leds to most complicted lgorithms. Chpters 3 nd 4 cn lso e treted s preprtion to Chpters 9 nd 14 where re presented dvnced PARALLEL STRING- MATCHING lgorithms, nd to Chpter 12 tht dels with two-dimensionl pttern mtching. Chpters 5 nd 6 re sic chpters of the ook. They cover dt structures representing succinctly ll suwords of text, nmely, SUFFIX TREES nd SUBWORD GRAPHS. The considertions re typicl from the point of view of dt structures mnipultions : trees, links, vectors, lists, etc. The prepred dt structures re used lter in the discussion on other prolems; especilly in the computtion of the longest common fctor of texts, fctoriztion prolems nd finding squres in words. Also it is used to construct in Chpter 7 certin kind of utomt relted to string-mtching. Chpter 7 gives n utomt theoreticl pproch to string-mtching tht leds to nturl solution of the MULTI-PATTERN MATCHING prolem. It covers Aho-Corsick utomt for multi-pttern mtching, utomt for the set of fctors nd suffixes of given text, nd ppliction of them. Chpter 8 considers REGULARITIES IN TEXTS: SYMMETRIES AND REPETITIONS. It covers prolems relted to symmetries of words s well s prolems Efficient Algorithms on Texts 11 M. Crochemore & W. Rytter

11 relted to repetitions. Surprisingly it turns out tht finding plindrome in word in liner time is much esier thn finding squre. Generlly, prolems relted to symmetries re esier thn tht relted to repetitions. The sic component of the chpter is new dt structure, clled the dictionry of sic fctors, tht is introduced s sic technique for the development of efficient, ut not necessrily optiml, lgorithms. Chpter 9 dels with PARALLEL ALGORITHMS ON TEXTS. Only one, very generl model of prllel computtions is considered. The prllel implementtion nd pplictions of the dictionry of sic fctors, suffix trees nd suword grphs re presented. We discuss prllel version of the Krp-Miller-Rosenerg lgorithm (Chpter 8) nd some computtions relted to trees. Chpter 10 is out dt compression. It discusses only some of the COMPRESSION TECHNIQUES FOR TEXTS. This re is exceptionlly rod s compression of dt is the vitl prt of mny rel world systems. We focus on few importnt lgorithms. Two of the sujects here re the Ziv-Lempel lgorithm, nd pplictions of dt structures from Chpters 5 nd 6 to the text fctoriztion. Questions relted to lgorithms involving the notion of APPROXIMATE PATTERNS re presented in Chpter 11. It strts with lgorithms to compute edit distnce of sequences (lignment of sequences). Two specific clssicl notions on pproximte ptterns re then considered, the first for string-mtching with errors, nd the second for ptterns with don t cre symols. TWO-DIMENSIONAL PATTERN MATCHING, in Chpter 12, is the prolem of identifying suimge inside lrger one. The notion nturlly extends tht of pttern mtching in texts. Bsic techniques on the suject show the importnce of lgorithms presented in Chpter 7. Chpter 13 presents three different TIME-SPACE OPTIMAL STRING-MATCHING lgorithms. None of them hve never ppered in textook nor even in journl, nd the third one is presented in completely new version. This is n dvnced mteril since the lgorithms rely hevily on the comintorics of words, mostly relted to periodicities. The discussion on OPTIMAL PARALLEL ALGORITHM ON TEXTS is the suject of Chpter 14. It includes more technicl considertions thn Chpter 9. Are presented there the well-known Vishkin's lgorithm nd recent optiml prllel lgorithm for string-mtching, sed on suffix versus prefix computtions, designed y Kedem, Lndu nd Plem.

12 Chpter /7/97 The lst chpter considers severl miscellneous prolems like tree-pttern mtching prolem, or the computtion shortest common superstrings, mximl suffixes, Lyndon fctoriztions One cn prtition lgorithmic prolems discussed in this ook into prcticl nd theoreticl prolems. Certinly the string-mtching nd dt compression re in the first clss, while most prolems relted to symmetries nd repetitions re in the second. However we elieve tht ll the prolems re interesting from n lgorithmic point of view nd enle the reder to pprecite the importnce of comintorics on words. In most textooks on lgorithms nd dt structures the presenttion of efficient lgorithms on words is quite short s compred to issues in grph theory, sorting, serching nd some other res. At the sme time, there re mny presenttions of interesting lgorithms on words ccessile only in journls nd in the form directed minly to specilists. We hope tht this ook will cover gp on lgorithms on words in the ook literture, nd ring together mny results presently dispersed in the msses of journl rticles. Selected references A.V. AHO, B.W. KERNIGHAN & P.J. WEINBERGER, The AWK Progrmming Lnguge, Addison-Wesley, Reding, Mss., A.V. AHO, R. SETHI & J. ULLMAN, Compilers, Addison-Wesley, Reding, Mss., 1988, Chpter 3. T.C. BELL, J.C. CLEARY & I.H. WITTEN, Text Compression, Prentice Hll, Englewood Cliffs, New Jersey, M.J. BISHOP & C.J. RAWLINGS, Nucleic cid nd protein sequence nlysis : prcticl pproch, IRL Press Limited, Oxford, Englnd, M. GROSS, Constructing lexicon-grmmrs, in : (B.T.S. Atkins & A. Zmpolli eds, Computtionl pproches to the lexicon, Oxford University Press, 1991). G. SALTON, Automtic Text Processing, Addison-Wesley, Reding, Mss., P.D. SMITH, An introduction to text processing, The MIT Press, Cmridge, Mss., M.S. WATERMAN, Mthemticl methods for DNA sequences, CRC Press, Boc Rton, Florid, Efficient Algorithms on Texts 13 M. Crochemore & W. Rytter

13 Chpter /7/97 2. Foundtions The chpter gives lgorithmic nd theoreticl foundtions required in further prts of the ook. The models of computtions re discussed, nd sic prolems on texts re exposed in precise wy. The chpter ends with comintoril properties of texts Mchine models nd presenttion of lgorithms The model of sequentil computtions is Rndom Access Mchine (RAM). Evlution of the time complexity of lgorithms is done ccording to the uniform cost criterion, i.e. one time unit per sic opertion. However, to void "uses" of the model, the sizes of ll integers occurring in lgorithms will e "resonle". The integers will e within polynomil rnge of input size nd cn e represented y logrithmic numers of its. We refer the reder to The design nd nlysis of computer lgorithms y A.V. Aho J.E. Hopcroft nd J.D. Ullmn for more detils on the RAM model. The sequentil lgorithms of this ook re minly presented in Pscl progrmming lnguge. Few more complex lgorithms re written in n informl lnguge, in style nd semntic similr to tht of Pscl. Sometimes, the initil lgorithm for given prolem is usully more understndle using kind of pseudo-lnguge. In this lnguge we cn use terminology pproprite to the nturl model for the prolem. The flow-of-control constructs cn e the sme s in Pscl, ut with the dvntge of hving informl nmes for ojects. For exmple, in lgorithms on trees we cn use notions such s "next son", or "fther of the fther", etc. In this sitution, n informl lgorithm is given to ctch the min ides of the construction. We generlly ttempt to convert such n informl lgorithm into Pscl progrm. t e x t s n r r y Figure 2.1. Arry representtion of text. We consider texts s rrys of chrcters. We stick to the convention tht indices of chrcters of text run from 1 to the length of the text. The symols of text text of length n re stored in text[1], text[2],, text[n]. We my ssume tht loction text[n+1] contins specil chrcter. Throughout the ook, we consider two glol vriles pt nd text tht re declred y vr text : text_type; pt : pttern_type; The types cn e themselves defined y Efficient Algorithms on Texts 15 M. Crochemore & W. Rytter

14 type text_type = rry[1..n] of chr ; pttern_type = rry[1..m] of chr, ssuming tht n nd m re Pscl integer constnts declred in proper wy. We ssume lzy evlution of oolen expressions. For instnce, the expression (i>0) nd (t[i]=c) is flse when i=0, even if t[0] does not exists. And the expression (i=0) or (t[i]=c) is true under the sme conditions. Replcing ll oolen expression to void the convention is strightforwrd exercise. Doing it in the presenttion of lgorithms would trnsform them into n unredle form. Pscl functions re presented in the ook with the nonstndrd (ut common) "return" sttement. Such instruction exists in lmost ll modern lnguges nd helps writing more structured progrms. The mening of the sttement "return(expression)" is tht the function returns the vlue of the expression to the clling procedure, nd then stops. The following function shows n ppliction of the return sttement. function text_inequlity(text1, text2: text_type): oolen; egin { left-to-right scn of text1 nd text2 } m := length(text1); n := length(text2); j := 0; while j <= min(m, n) do { the prefixes of size j re the sme } if text1[j+1]=text2[j+1] then j := j+1 else return(true); return(flse); end; The "return" sttement is very useful to shorten the presenttion of lgorithms. It is lso esy to implement. It cn e replced y text_inequlity := expression; goto L where L is the lel of the lst end of the function. The following description of the function text_inequlity is equivlent to the ove text.

15 Chpter /7/97 function text_inequlity(text1, text2: text_type): oolen; lel 1992; egin { left-to-right scn of text1 nd text2 } m := length(text1); n := length(text2); j := 0; while j <= min(m, n) do { the prefixes of size j re the sme } if text1[j+1]=text2[j+1] then j := j+1 else egin text_inequlity:=true; goto 1992 end; text_inequlity:= flse; 1992: end; Severl models of sequentil computtions more theoreticl thn the RAM re relevnt to text processing : Turing mchines, multihed finite utomt etc. For exmple, it is known tht string-mtching cn e done in rel time on Turing mchine or even on multihed finite utomton (without ny memory ut with constnt numer of heds). Generlly, we do not discuss lgorithms on such highly theoreticl models; in few cses we only give some rief remrks relted to the computtions on these models. Concerning prllel computtions, very generl model is ssumed, since we re minly interested in exposing the prllel nture of some prolems without going into the detils of the prllel hrdwre. The prllel rndom ccess mchine (PRAM), prllel version of the rndom ccessed mchine is used s stndrd model for presenttion of prllel lgorithms. Processors Common Rndon Access Memory Figure 2.2. PRAM mchine. The PRAM consists of numer of processors working synchronously nd communicting through common rndom ccess memory. Ech processor is rndom ccess mchine with the usul opertions. The processors re indexed y the consecutive Efficient Algorithms on Texts 17 M. Crochemore & W. Rytter

16 nturl numers, nd they synchronously execute the sme centrl progrm; however, the ction of given processor depends lso on its numer (known to the processor). In one step, processor cn ccess one memory loction. The models differ with respect to simultneous ccess of the sme memory loction y more thn one processor. For the CREW (concurrent red, exclusive write) vriety of PRAM mchine ny numer of processors cn red from the sme memory loction simultneously, ut write conflicts re not llowed : no two processors cn ttempt to write simultneously into the sme loction. Processors Common Rndon Access Memory Figure 2.3. A Concurrent Red. We denote y CRCW (concurrent red, concurrent write) the PRAM model in which, in ddition to concurrent red, write conflicts re llowed : mny processors cn ttempt to write into the sme loction simultneously ut only if they ll ttempt to write the sme vlue. Processors Common Rndon Access Memory Figure 2.4. A Concurrent Write. Prllel lgorithms re presented in similr wy s in Efficient prllel lgorithms y A. Gions nd W. Rytter. There is no generlly ccepted universl lnguge for the presenttion of prllel lgorithms. The PRAM is rther idelized model. We hve chosen this model s est suitle for the presenttion of lgorithms nd especilly for the presenttion of the inherent prllelism of some prolems. This would e difficult to nicely present these lgorithms with lnguges oriented towrds concrete existing hrdwre of prllel

17 Chpter /7/97 computer. Moreover, the PRAM model is widely ccepted in the literture on prllel computtion on texts. Prllelism will e expressed y the following type of prllel sttement : for ll i in X in prllel do ction(i). Execution of this sttement consists of : ssigning processor to ech element of X; executing in prllel y ssigned processors the opertions specified y ction(i). Usully the prt "i in X" looks like "1 i n" if X is n intervl of integers. We discuss this more deeply in Chpter Efficiency of lgorithms Efficient lgorithms cn e clssified ccording to wht we men y efficiency. There re different notions of efficiency depending on the complexity mesure involved. Severl such mesures re discussed in this ook : sequentil time, memory spce, prllel time nd numer of processors. Our ook dels with "fesile" prolems. We cn define them s prolems hving efficient lgorithms, or s solvle in time ounded y smll-degree polynomil. In the cse of sequentil computtions we re interested to lower down the degree of the polynomil corresponding to time complexity. The most efficient lgorithms usully solve prolem in liner time complexity. We re lso interested in spce complexity. Optiml spce complexity often mens constnt numer of (smll integer) registers in ddition to input dt. So, we sy tht n lgorithm is time-spce optiml iff it works simultneously in liner time nd constnt extr spce. These re the most dvnced sequentil lgorithms, nd the most interesting lso oth from prcticl nd theoreticl point of view. In the cse of prllel computtions we re generlly interested in the prllel time T(n) s well s in the numer of processors P(n) required for the execution of the prllel lgorithm. The totl numer of elementry opertions performed y the prllel lgorithm is not greter thn the product T(n)P(n). Efficient prllel lgorithms re those tht operte in no more thn polylogrithmic ( polynomil of log s of input size) time with polynomil numer of processors. The clss of prolems solvle y such lgorithms is denoted y NC nd we cll the relted lgorithms NC-lgorithms. A NC-lgorithm is optiml iff the totl numer of opertions T(n)P(n) is liner. Another possile definition is tht this numer is essentilly the sme s the time complexity of the est known sequentil lgorithm solving the given prolem. However we dopt here the first option ecuse lgorithms on strings usully hve time complexity t lest liner. Evluting precisely the complexity of n lgorithm ccording to some mesure is often difficult, nd moreover it is unlikely to e of some use. The ig O nottion mkes cler wht Efficient Algorithms on Texts 19 M. Crochemore & W. Rytter

18 re the importnt terms of complexity expression. It estimtes the symptotic order of the complexity of n lgorithm nd helps to compre lgorithms etween ech other. Recll tht if f nd g re two function from nd to integers, then we sy tht f = O(g) if f(n) C.g(n) when n>n, for some constnts C nd N. We write f = Θ(g) when the functions f nd g re of the sme order, which mens tht oth equlities f = O(g) nd g = O(f) hold. Compring functions through their symptotic orders leds to these kind of inequlities : O(n 0.7 ) < O(n) < O(n.log(n)), or O(n log(n) ) < O(log(n) n ) < O(n!). Within sequentil models of mchine one cn distinguish further types of computtions : off-line, on-line nd rel-time. These types re relted lso to efficiency. It is understood tht rel-time computtions re more efficient thn generl on-line, nd tht on-line computtions re more efficient thn off-line. Ech lgorithm is n off-line lgorithm : "off-line" conceptully mens tht the whole input dt cn e put into the memory efore the ctul computtion strts. We re not interested then in the intermedite results computed y the lgorithm, ut only the finl result is sought (though this finl result cn e sequence or vector). The time complexity is mesured y the totl time from the moment the computtion strts (with ll input dt previously memorized) up to the finl termintion. In contrst, n on-line lgorithm is like sequentil trnsducer. The portions of the input dt re "swllowed" y the lgorithm step fter step, nd fter ech step n intermedite result is expected (relted to the input dt red so fr). It then reds the next portion of the input, nd so on. In on-line lgorithms the input cn e treted s n infinite strem of dt, nd so we re not minly interested in the termintion of the lgorithm for ll such dt. The min interest for us is the totl time T(n) for which we hve to wit to get the n-th first outputs. The time T(n) is mesured strting t the eginning of the whole computtion (ctivtion of the trnsducer). Suppose tht the input dt is sequence nd tht fter reding the n-th symol we wnt to print "1" if the text red to this moment contins given pttern s suffix, otherwise we print "0". Hence we hve two strems of dt : the strem of input symols nd n output strem of nswers "1" or "0". The min feture of the on-line lgorithm is tht we hve to give n output vlue efore reding the next input symol. The rel time computtions re these on-line lgorithms which re in certin sense optiml; the elpsing time etween reding two consecutive input symols (the time spent for computing only the lst output vlue) should e ounded y constnt. Most liner on-line lgorithms re in fct rel-time. We re mostly interested in off-line computtions whose worst cse time is liner, however lso on-line nd rel-time computtions s well s verge complexities will e sometimes shortly discussed in the ook.

19 Chpter /7/ Nottion nd forml definitions of sic prolems on texts Let A e n input lphet finite set of symols. Elements of A re clled the letters, the chrcters, or the symols. Typicl exmples of lphets re : the set of ll ordinry letters, or the set of inry digits. The texts (lso clled words or strings) over A re sequences of elements of A. The length (size) of text is the numer of its elements (with repetitions). So, the length of is 3. The length of word x is denoted y x. The input dt for our prolems will e words nd the size n of the input prolem will e usully the length of the input word. In some situtions, n will denote the mximum length or the totl length of severl words if the input of the prolem consists of severl words. The i-th element of the word x is denoted y x[i] nd i is its position in x. We denote y x[i j] the suword x[i]x[i+1] x[j] of x. If i>j, y convention, the word x[i j] is the empty word (the sequence of length zero). We sy tht the word x of length n is fctor (lso clled suword) of the word y if x=y[i+1 i+n] for some integer i. We lso sy tht x occurs in y t position i or tht the position i is mtch for x in y. We define lso the notion of susequence (sometimes clled suword). The word x is susequence of y if x cn e otined from y y removing zero or more (non necessrily djcent) letters from it. Equivlently x is susequence of y if x=y[i1]y[i2] y[im], where i1, i2,, im is n incresing sequence of indices in y. Below we define sic prolems nd groups of prolems covered in the ook. We often consider two texts pt (the pttern) nd text of respective lengths m nd n. 1. String-mtching (the sic prolem) Given texts pt nd text, verify if pt occurs in text. Hence it is decision prolem : the output is oolen vlue. It is usully ssumed tht m n. So, the size of the prolem is n. A slightly dvnced version is to serch for ll occurrences of pt in text, tht is, to compute the set of positions of pt in text.. Denote y MATCH(pt, text) this set. In most cses n lgorithm computing MATCH(pt, text) is trivil modifiction of the decision lgorithm, hence we sometimes present only the decision lgorithm for string-mtching. Insted of one pttern one cn consider finite set of ptterns nd sk if given text text contins pttern from this set. The size of the prolem is now the totl length of ll ptterns plus the length of text. 2. Construction of string-mtching utomt For given pttern pt, construct the miniml deterministic finite utomton ccepting ll words contining pt s suffix. Denote such n utomton y SMA(pt). A more generl Efficient Algorithms on Texts 21 M. Crochemore & W. Rytter

20 prolem is to construct similr utomton SMA( ) ccepting the set of ll words contining s suffix one of the ptterns from the set ={pt1,, ptk }. 3. Approximte string-mtching We re given gin pttern pt nd text text. The prolem consists in finding n pproximte occurrence of pt in text, tht is, position i such tht dist(pt, text[i+1 i+m]) is miniml. Another instnce of the prolem is to find ll positions i such tht dist(pt, text[i+1 i+m]) is less thn given constnt. Here, dist is function defining distnce etween two words. There re two stndrd distnces : the Hmming distnce (numer of positions t which two strings differ) nd the edit distnce (see Prolem 7 elow). 4. String-mtching with don't cre symols We re given specil universl symol ø tht mtches ny other symol (including itself). It is clled the don't cre symol. In fct, ø cn e common nme for severl unimportnt symols of the lphet. For two symols nd, we write iff they re equl or t lest one of them is don't cre symol. For two words of the sme length we write x y if, for ech i, x[i] y[i]. The prolem consists in verifying whether pt text[i+1 i+m] for some i, or more generlly, to compute the set of ll such positions i. 5. Two-dimensionl pttern mtching For this prolem, the pttern pt nd the text text re two-dimensionl rrys (or mtrices) whose elements re symols. We re to nswer whether pt occurs in text s surry, or to find ll positions of pt in text. The pproximte nd don't cre versions of the prolem cn e defined similrly s for the (one-dimensionl) string-mtching prolem. Insted of rrys one cn lso consider some other "shpes", e.g. trees. 6. Longest common fctor Compute the mximl length of longest common fctors of two words x, y (denoted y LCF(x, y)). One cn consider lso longest common fctors for more thn two words. 7. Edit distnce Compute the edit distnce, EDIT(x, y), etween two words. It is the miniml numer of edit opertions needed to trnsform one string into the other. Edit opertions re : insertion of one chrcter, deletion of one chrcter, nd chnge of one chrcter. The prolem is closely relted to the pproximte string-mtching nd to the prolem defined elow. It is similr to the notion of lignment of DNA sequences in Moleculr Biology. 8. Longest common susequence

21 Chpter /7/97 Compute the mximl length of common susequences of two words x, y. The length is denoted y LCS(x, y). Note tht severl susequences my hve this length nd tht we my lso wnt to compute one or ll this words. This prolem is relted to the edit distnce prolem s follows : if l=lcs(x, y), then one cn trnsform x to y y first deleting m-l symols of x (ll ut those of longest common susequence) nd then inserting n-l symols to get y. So, the computtion of LCS s is prticulr cse of the computtion of edit distnces. 9. Longest repeted fctor We re to compute longest fctor x of the text text tht occurs t lest twice in text. Its length is denoted y LRF(text). Similrly we cn consider longest fctor tht occurs t lest k times in text. Its length is LRFk(text). 10. Finding squres nd powers A much hrder prolem is tht of computing the consecutive repetitions of sme fctor. A squre word (or simply squre) is nonempty word of the form xx. Let us define the predicte Squre-free(text) whose vlue is 'true' iff text does not contin ny squre word fctor. Similrly one cn define cues nd the predicte Cue-free(text). Generlly one cn define the predicte Power-free k (text) whose vlue is 'true' iff text does not contin ny fctor of the form x k (with x non-empty). These prolems hve seemingly no prcticl ppliction. However, they re relted to the most clssicl mthemticl prolems on words (whose investigtion strted t the eginning of this century). 11. Computing symmetries in texts The word x is symmetricl if x is equl to its reversed x R. A plindrome is ny symmetricl word contining t lest two letters ( one-letter word hs n uninteresting ovious symmetry!). Even plindromes re plindromes of n even length. Denote y Pl nd P the set of ll plindromes nd the set of even length plindromes, respectively. There re severl interesting prolems relted to symmetries. Compute PREFPAL(text), the length of the shortest prefix of text tht is plindrome. Another interesting prolem is to find ll fctors of the text tht re plindromes. Or, compute MAXPAL(text), the length of longest symmetricl fctor of text.. Another question is to count the numer of ll plindromes which re fctors of text. A word is clled plstr if it is composition of plindromes (it is contined in the lnguge Pl * ); it is n even plstr if it is composition of even plindromes. Two lgorithmic prolems relted to plstrs re : verify if word is plstr, or n even plstr. Efficient Algorithms on Texts 23 M. Crochemore & W. Rytter

22 Insted of sking out the composition of some numer of plindromes we cn e interested in compositions of fixed numer k of plindromes. Let Pk(text) e the predicte whose vlue is true if text is composition of k even plindromes; nlogously we cn define Plk(text) s predicte relted to compositions of ny k plindromes (odd or even) 12. Numer of ll distinct fctors The prolem is to compute the numer of ll distinct fctors of text. Oserve tht there is qudrtic numer of fctors, while liner time lgorithm is possile. Such liner time counting of qudrtic numer of ojects is possile due to succinct (liner-sized) representtions of the set of ll fctors of given text. 13. Mximl suffix Compute the lexicogrphiclly mximl suffix of the word text (it is denoted y mxsuf(text) ). Oserve tht the lexicogrphiclly mximl suffix of text is the sme s its lexicogrphiclly mximl fctor. The lexicogrphic order is the sme order s used, for exmple, commonly in encyclopedis or dictionries (if we omit the question of ccents, uppercse letters, hyphens, ). 14. Cyclic equivlence of words Check whether two words x nd y re cycliclly equivlent, tht is to sy, whether x=uv, y=vu for some words u, v. A strightforwrd liner time lgorithm for this prolem cn e constructed y pplying string-mtching procedure to the pttern pt=x nd the text text=yy. However it is possile to design much simpler efficient lgorithm. 15. Lyndon fctoriztion Compute the decomposition of the text into non-incresing sequence of Lyndon words Lyndon words re lexicogrphiclly (strictly) miniml within the set of ll their cyclic shifts. It is known tht ny word cn e uniquely fctorized into sequence of Lyndon words 16. Compression of texts For given word text we would like to hve most succinct representtion of text. The solution depends on wht we men y representtion. If y representtion we men n encoding of single symols y inry strings, then, we hve the prolem of the efficient construction of Huffmn codes nd Huffmn trees. Much more complicted prolems rise if some prts of the text text re encoded using other prts of the text. This leds to some fctoriztion nd textul sustitution prolems. We spend whole chpter on this suject. 17. Unique decipherility prolem

23 Chpter /7/97 We hve code which is function h ssigning to ech symol of the lphet K word over n lphet A. The function cn e in nturl wy extended to ll words over the lphet K using the equlity h(xy)=h(x)h(y). We sk whether the so-extended function is oneto-one (denote y UD(h) the corresponding predicte). The size n of the prolem is the totl length of ll codewords h() over ll symols in K. The next three prolems re relted to the construction of succinct (ut useful) representtion of sets of ll fctors or ll suffixes of given word text. Denote y Fc(text), Suf(text) the set of ll respectively fctors, suffixes of text. There cn e qudrtic numer of elements to e represented ut the size of the representtion is required to e liner. Moreover, we lso require tht the construction time is liner in time. 18. Suffix trees The set Fc(text) is prefix closed. This mens tht prefixes of words in Fc(text) re lso elements of the set. The nturl representtion for ech prefix closed set F of words is tree T1 whose edges re lelled with symols of the lphet. The set F equls the set of lels of ll pths from the root down to nodes of the tree. However, such tree T1 corresponding to Fc(text) cn e too lrge. It cn hve qudrtic numer of internl nodes. Fortuntely, the numer of leves is lwys liner nd hence the numer of internl nodes hving t lest two sons (out-degree igger thn one) is lso liner. Hence, we cn construct tree T whose nodes re ll "essentil" internl nodes nd leves of T1. The edges of T correspond to mximl pths (chins) in T1 consisting only of nodes of out-degree one; the lel of ech such edge is the composite lel of ll edges of the corresponding chin. Denote the constructed tree T y ST(text). It is clled the suffix (or the fctor) tree of text. The size of the suffix tree is liner nd we present lso liner time lgorithm to construct it. The suffix tree is very useful representtion of ll fctors; the rnge of pplictions is essentilly the sme s of two other dt structures considered elow : suffix nd fctor utomt. 19. Smllest suffix utomt nd directed cyclic word grphs Compute the smllest deterministic finite utomton SA(text) ccepting the set Suf(text) of ll suffixes of text. The size of the utomton turns out to e liner nd we even prove lter tht its size does not depend on the size of the lphet (if trnsitions to ded stte re not counted). The utomton SA(text) is essentilly the sme dt structure s the so-clled directed cyclic word grph (dwg, for short). Dwg s re directed grphs whose edges re lelled y symols, nd such grph represents the set of ll words "spelled" y its pths from the root to the sink. Efficient Algorithms on Texts 25 M. Crochemore & W. Rytter