ONLINE TRANSLATION SERVICES FOR THE LAO LANGUAGE

Transcription

1 ONLINE TRANSLATION SERVICES FOR THE LAO LANGUAGE Vincent BERMENT GETA-CLIPS (IMAG) BP Grenoble Cedex 9, France INALCO 2, rue de Lille Paris Cedex 7, France ABSTRACT The students learning Laotian or French language, the professional or occasional translators, all have the same few means for helping them in their translations task. Those means are the often frozen, old and tiresome to use dictionaries, like Marc Reinhorn s Lao-French dictionary or Pierre Somchine Nginn s French- Lao dictionary. This paper presents LaoTrans, a set of online translation support services between Lao and French (both directions), that has all the flexibility and evolutionarity that the new Information and Communication Technologies (ICT) can bring. RÉSUMÉ Les étudiants laotiens apprenant le français ou français apprenant le laotien, les traducteurs professionnels ou occasionnels disposent de peu de moyens pour les aider dans leur tâche. Il s agit des dictionnaires souvent anciens, figés et dont l utilisation est fastidieuse comme le dictionnaire laotien-français de Marc Reinhorn ou comme celui, français-laotien, de Pierre Somchine Nginn. Le présent article présente LaoTrans, un ensemble de services d aide à la traduction en ligne entre français et lao, qui dispose de toute la souplesse et de l évolutivité que permettent aujourd hui les nouvelles Technologies de l Information et des Communications (TIC). KEYWORDS Machine translation, translation support tool, pi-language, π-language, Lao language, language computerization, unsegmented writing system, syllabic segmentation, word segmentation, lexical database, electronic dictionary, LaoTrans, LaoWord, LaoUniKey, Lao Software INTRODUCTION Thanks to a continuous effort made over the last 30 years, the Lao system of writing has now achieved a satisfactory level of computerization: mature fonts, text input tools and word processors are now available and the Lao Unicode is used over the Internet. So a lot of people is now waiting for what may be looked as a natural next step of this computerization effort: a high quality machine translation service that would allow the Lao people to read English, Chinese, Japanese, French or whatsoever language, translated in their own one. In contrast with the pioneering age where isolated individuals could offer complete solutions, the much more complex machine translation step must rely on groups of linguists and computer scientists to get through. For example, the order of magnitude for producing a good quality Lao-English / English-Lao machine translation system is estimated to 50 men-years (Lafourcade, 1994), only for the lingware, when an add-in such as LaoWord which provides a full set of word processing functions only requires about 6 men-years. So even if the methods used are well adapted to the computerization of under-resourced languages or, as they are called now, π-languages (Berment, 2004), it may take time before a good translation software is available for the Lao language. 1 / 11

2 On the way of this highly desirable future, an interesting translation support can be brought to professional and occasional translators by simpler tools mostly based on the technology developed for word processors. In this paper, we present LaoTrans, a set of online translation services that offers, for the first time, a translation support on the (dictionary, word for word translation of texts). Doing so, we will see how the reuse of a technology previously developed dramatically eased the realization of this new software. Then, looking towards the future, we will also show how the LaoTrans experience can contribute to the design of a full machine translation service. We will also introduce LaoUniKey, a text input tool, and LaoLex, an online collaborative service for building dictionaries. I FUNCTIONAL DESCRIPTION OF LAOTRANS LaoTrans is a set of two online services which are available, among other software dedicated to the Lao language, at the following address: I.1 TRANSLATION OF WORDS: LAOTRANS-WORD The first of the two services is called LaoTrans-Word. It is a look-up dictionary that provides both Lao-French and French-Lao translations for isolated words. Figure 1: LaoTrans-Word translates ແປ into French (example) Using this service from Lao to French, the tool displays all the possible French translations as shown in the figure above for the word ແປ. When using the dictionary in the other direction, from French to Lao, the engine looks for all the translations that include the French word, for example the following translations will be given for chaussure (see next page). 2 / 11

3 Figure 2 : LaoTrans-Word translates chaussure into Lao (example) This is very convenient to find all the occurrences of a word including, for example, plural or conjugated forms. Actually, this was done so because the original lexicon used in LaoTrans is a Lao- French dictionary in which an entry like ກ ງ is translated by the periphrasis l'autre partie d'une paire and another like ເກ ບ is translated by the plural forms souliers and chaussures. So it appeared natural to search the French word not as an exact translation but only as any sub-chain. In both directions, Lao-French and French-Lao, the user can investigate further his translated words as they are displayed with hyperlinks. For example, when the user will click on the first translation given for chaussure (ເກ ບ), LaoTrans-Word will return the three following results. Figure 3 : LaoTrans-Word translates ເກ ບ into plural forms (example) 3 / 11

4 I.2 TRANSLATION OF TEXTS: LAOTRANS-TEXT The second service offered by LaoTrans is more complex as it aims at easing the reading and the translation of a full piece of Lao text ( active reading support tool). With this service, the user receives a word for word translation in response to his Lao text as shown in the figure below. As you can see in this figure, a word which has more than one translation in the dictionary will be displayed with the list of all its translations separated with / characters. Figure 4 : LaoTrans-Text translates a Lao text into a list of its words translated into French Obviously, this is not yet a complete machine translation system with production of sentences. But it is also obvious that the human user will easily and quickly produce his own translation thanks to the translations he got for the words. 4 / 11

5 I.3 TEXTS INPUT IN UNICODE: LAOUNIKEY Unicode is an initiative for standardizing the representation in computers of all the writing systems in the world. Though the Lao writing system is part of the Unicode standard since June 1992 (Unicode version 1.01, see Unicode fonts that work properly for Lao only appeared a few years ago 1. So, after a long period during which Lao Unicode remained almost unused, we now observe that more and more sites adopt Unicode so we are confident that it will become THE standard in a close future and that Lao texts will at last be readable on any computer in the world. In LaoTrans, all the Lao texts are encoded in Unicode. As Lao text input is still not natively present in Microsoft Windows (people in Laos mainly use Windows 98, Me and XP), the users need to rely on additional software to type their texts. LaoUniKey (ລາວຢ ນ ກ ) is such a software. It is available from the same site as LaoTrans: This software is very flexible (formats, hotkeys...) and its human-machine interface is trilingual: Lao, French and English. Figure 5 : LaoUniKey configuration window (Lao language selected) Alternatively, the text can also be copied from an existing document, and in particular from sites using the Unicode encoding. 1 In May 2001, we developed one of the first Unicode font which was properly working for Lao. It was presented during the Papillon seminar in Tokyo in May 2002 where it was used to show the Lao language in PapiLex, a mockup of collaborative online Unicode dictionary. See (Berment 2002) and (Berment 2004). The most popular fonts at the moment are done by John Durdin (e.g. Saysettha OT). 5 / 11

6 I.4 DICTIONARY CONTENT: THE REUSE OF LINGWARE In order to offer a translation service of good quality, the software itself has to be good but it s not enough. A good dictionary is also a key issue as it has to contain as many word-senses and as little mistakes as possible. Fortunately, the dictionaries made by teachers or by linguists with means such as word processors can often be recycled to become lexical databases. This process can be helped by tools such as Recupdic in which the textual structure of the dictionary is formally described in order to automate its transformation into a lexical base (Doan-Nguyen, 1998). In the case of LaoTrans, we could recycle a Lao-French lexicon of about 15,000 word-senses that was created by Paul Jadin 2 who originally typed it in Excel. Its structure had two great advantages: The entries and their translations were already separated in two different columns, The entries corresponding to two different word-senses were in two different rows 3. This 15,000 entries dictionary happens to provide a good coverage for the test texts. The process to transform Paul Jadin s Excel file into a lexical database followed the three steps: Save the Excel file into a.csv file, Import this.csv file into a MySQL table by using phpmyadmin 4, Transforming the proprietary encoding of the Lao entries into Unicode 5. If we now look towards the future, we may wonder what is likely to be reusable in this translation support stuff. Actually, we think that the dictionary is the most essential element to reuse. The current step will have driven to a correct and large dictionary over a period of several months. This will be even more evident with the interactive use of LaoTrans and LaoLex as demonstrated in chapter III. The improvement, by adding lexical content (examples...) to the entries, participates to the lingware s development. So this enriched and improved dictionary can be later used for itself to do the lexical transfer of a machine translation system. Moreover, it can also be used together with LaoTrans and a corpus to provide a help for analyzing and understanding the morphosyntactic structure of the Lao language. 2 Paul Jadin is an independent lexicographer who lives in Belgium. We got in contact thanks to the Internet. This last remark highlights the essential fact that Internet changed the way people are working together, and especially people involved in π-language computerization because they are very few and scattered around the world. The collaboration between Paul Jadin and the author is a typical example as we actually never physically met until now. 3 For example, the word ແປ with the meanings Traduire and Plat were in two different rows. 4 After selecting the table in the database, the Structure page is displayed. This page proposes to Insert data from a text file into table at its bottom. This can be used in particular for inserting a.csv file. For more details, see and 5 This last action is handled by a C++ program deriving from LaoWord which can transform a text from any given encoding into any other, and in particular Unicode. This C++ program is called by a PHP code. 6 / 11

7 II ARCHITECTURE AND ALGORITHMS: THE REUSE OF SOFTWARE II.1 LAOTRANS-WORD SOFTWARE ARCHITECTURE AND ALGORITHMS The LaoTrans-Word software is running on a Linux server. The languages used are PHP and C++, and the dictionary is stored in a MySQL database. The architecture is shown in the figure below. Client Form Response LaoTrans Code (PHP) Request Lexical Data Lexical Base (MySQL) Characters Chain to Process Response LaoProc Code (C++) Server Figure 6 : LaoTrans-Word: Architecture The browser ( client) sends the word to translate to the server in a form (see Figure 1). The principle of the operation in the server is summarized in the following drawing. Client Form Server Response Canonic formatting (LaoProc) Word retrieval (LaoTrans/Lexical base) Response construction (LaoTrans) Figure 7 : LaoTrans-Word: Processing sequence First, the server transforms the received word into a canonical format (for example ໜ ຫນ 6 ) which is the format of the Lao headwords in the database. Then, it simply searches the standardized word in the database and returns the response to the client. 6 For more details, see (Berment, 2004). 7 / 11

8 II.2 LAOTRANS-TEXT SOFTWARE ARCHITECTURE AND ALGORITHMS From a main bricks point of view, the structure of LaoTrans-Text is the same as the one of LaoTrans-Word. The differences are that LaoProc also has to segment the text into syllables and to compute a phonological transcription for each of them. Then the PHP code of LaoTrans associates the syllables together to form words by using a longest matching algorithm (see (Rarunrom, 1991)). Client Form Server Response Canonic formatting (LaoProc) Syllabic segmentation (LaoProc) Phonological transcription (LaoProc) Word retrieval (LaoTrans/Lexical base) Response construction (LaoTrans) Figure 8 : LaoTrans-Text: Processing sequence II.3 GAIN REACHED BY REUSING AN EXISTING SOURCE CODE Though the three linguistic algorithms (canonic formatting, syllabic segmentation and phonological transcription) are not obvious, their part in the development of LaoProc did not even take 15 hours, thanks to the reuse of the source code of LaoWord (Berment, 2003a). Total Three linguistic modules LaoWord > 4,000 h 2,500 h LaoProc 300 h < 15 h Figure 9 : Compared development efforts 7 The gain 8 reached for the three linguistic modules is greater than 99 % (the rest of the two software programs is not comparable so talking of gain would not be relevant). The global gain for the LaoProc program is then (2, ) / 2, %. The same order of magnitude was already observed for LaoUniKey which also derives from LaoWord ( 93 % for the linguistic modules) as well as for other software reuse experiences (Berment, 2004). We may observe that this software reuse was possible because the programming language (C++) can be compiled on both Windows and Linux platforms (LaoWord is running on Windows and LaoProc on a Linux server). 7 Development time, not including feasibility studies, mockups, design & test documentation, user s manuals, distribution, support and maintenance. 8 This gain is defined by the formula : (T S T A ) / T S, with T S = development time without reuse and T A = development time with reuse. 8 / 11

9 III DICTIONARY ENRICHMENT: LAOLEX As any other dictionary, the dictionary used in LaoTrans may contain mistakes and entries may also be missing. The LaoLex software is designed to add and modify lexical entries in online dictionaries. Though this existing software was first presented in 2003, it has not been upgraded yet to handle Unicode. However, as it will soon be available as a natural complement to LaoTrans, we will say a few words here about its capabilities. For more details, see (Berment and Thongvilu 2003), (Berment et al. 2003) and (Berment 2003b). Figure 10 : Creation of an entry in LaoLex An efficient manner to improve the existing dictionary is to couple LaoTrans together with LaoLex dynamically. This means that every time a recorded user asks for a translation and the result does not give full satisfaction, he may modify or add entries in the dictionary 9. The main dictionary will not be modified directly by LaoLex. Only the personal dictionaries of the recorded users will. However, when the content of a personal dictionary becomes significant, it can be checked by the linguist in charge of the main dictionary and then be added. This collaborating way of building the main dictionary can be very efficient as the users themselves do guarantee its lexical quality and completeness. 9 This coupled mechanism is foreseen to be included in the future implementation of LaoLex. 9 / 11

10 As you can see in Figure 10, LaoLex can not only enrich the headwords and their translations (currently the only available data in the dictionary) but also the lexical content of the existing entries. The main additional lexical items are: Old spellings, Alternative spellings, Part of speech, Language level, Specifier, Definition (with its translation), Examples (with their translations), Idioms (with their translations), Comments. Every Lao word in the dictionary can be associated with a category (ປະເພດ). This category, or part of speech, is attributed according to syntactical criteria (semantic and morphological criteria also participate to categorization). In LaoLex, we use a description in three levels: A group level, A category level, A subcategory level. The first level, called group because its elements group together several categories, coincides with the seven parts of speech of the traditional grammar (Comité Littéraire, 1962): Nouns (ຄຳນາມ), Pronouns (ຄຳສ ບພະນາມ / ຄຳສ ພນາມ), Verbs (ຄຳກ ຣ ຍາ), Predicatives 10 (ຄຳວ ເສດ), Prepositions (ຄຳບ ບພະບ ດ / ຄຳບ ພບ ດ), Conjunctions (ຄຳສ ນທານ), Interjections (ຄຳອ ທານ). The second and third levels mostly derive from Marc Reinhorn's works completed by Lamvieng Inthamone's. See (Berment et al. 2003). The levels of language are also important in the Lao language. We are using the following ones: General use Respectful Colloquial Slangy Specialty Refined Monk Royal Literary Spoken Archaic 10 Term borrowed from Marc Reinhorn s grammar (Reinhorn, 1975). 10 / 11

11 REFERENCES LIST Berment Vincent Several Technical Issues for Building New Lexical Bases. Papillon Seminar, July 15 18, 2002, NII, Hitotsubashi, Chiyoda-ku, Tokyo, Japan. Berment Vincent, Thongvilu Houmphanh Cooperative Lao ICT framework. Case study: construction of Lao lexical resources. Regional Conference on Digital GMS, February 26 28, 2003, Asian Institute of Technology, Bangkok, Thailand. Berment Vincent, Jacqmin Thakkhinh, Dechanet Blandine Parts of Speech for the LaoLex Dictionary. Pan Asia Networking All Partners 2003, March 3 10, 2003, Vientiane Novotel, Vientiane, Laos. Berment Vincent. 2003a. LaoWord's Word Processing Functions. Pan Asia Networking All Partners 2003, March 3 10, 2003, Vientiane Novotel, Vientiane, Laos. Berment Vincent. 2003b. Current status of the Papillon-Lao database: Tools (LaoLex), dictionary (LaoDict), XML schema and export towards Papillon. Papillon Seminar, July 3 5, 2003, Sapporo University, Sapporo, Japan. Berment Vincent Méthodes pour informatiser des langues et des groupes de langues «peu dotées». Ph.D. thesis, Grenoble University, France. Comité Littéraire (ກ ມວ ນນະຄະດ ) Lao grammar (ໄວຍາກອນລາວ). Doan-Nguyen Hai Techniques génériques d accumulation d ensembles lexicaux structurés à partir de ressources dictionnairiques informatisées multilingues hétérogènes. Ph.D. thesis, Grenoble University, France. Jadin Paul Dictionnaire laotien-français. Non publié. Lafourcade Mathieu Génie logiciel pour le génie linguiciel. Ph.D. thesis, Grenoble University, France. Nginn Pierre Somchine Dictionnaire français-lao. Idase, Paris. Rarunrom Sampan Dictionary-based Thai word separation,. Senior project report, Thailand. Reinhorn Marc Dictionnaire laotien-français. CNRS. Reinhorn Marc Grammaire de la langue lao. INALCO. 11 / 11