MACHINE TRANSLATION. I.1 Introduction to Statistical Machine Translation

Transcription

1 Universitat Politècnica de València Master en Inteligencia Artificial, Renoconocimiento de Formas e Imagen Digital Master in Artificial Intelligence, Pattern Recognition and Digital Imaging MACHINE TRANSLATION STATISTICAL MACHINE TRANSLATION I.1 Introduction to Statistical Francisco Casacuberta [email protected] September 22, 2015 MIARFID-UPV Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 1

2 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 2 MT objectives [ M. Turchi. Introduction to machine translation. MT Marathon 2014] Information societ and production of multilingua content: 7 billion people, 193 countries, over 150 official languages Globalization and demand for translation services: 1,000 global companies operating in at least 160 countries Size of worldwide translation market: 12.5 billion $ per ear 34 million $ per da. Size of translation industr: 3,000 translation companies and > 250,000 translators MT can improve productivit of human translators: integration of MT with human translation (post editing) MT can suppl cheap gist translation: competitive qualit cost speed trade off MIARFID-UPV September 22, 2015 SMT-1: 3

3 MT objectives: Erroneous conceptions MT is a waste of time because a machine never will translate Shakespeare. Generall, the qualit of translation ou can get from an MT sstem is ver low. MT threatens the jobs of translators There is an MT sstem that translates what ou sa into Japanese and translates the other speaker s replies in English. There is an amazing South American Indian language with a structure of such logical perfection that it solves the problem of design MT sstems. MT sstems are machines, and buing an MT sstem should be ver much like buing a car. MIARFID-UPV September 22, 2015 SMT-1: 4 MT objectives: Facts MT is useful. There are man situations that MT sstems produce reliable, if less than perfect, translations at high speed. In some circunstances, MT sstems can produce good qualit outputs. MT does not threaten translators jobs: High demand of translations and too repetitive translation jobs. Speech-to-speech MT is still a research topic. There are man open research problems in MT. Building a traditional MT sstem is a time consuming job. A user will tpicall have to invest a considerable amount of effort in customizing an MT sstem. MIARFID-UPV September 22, 2015 SMT-1: 5

4 Need of pre/post-editing While the number of errors and bad constructions is high, post-editing can make the result useful. Man problems could have been avoided b making the source tet simpler. source target PRE-EDITING TRANSLATION POST-EDITING MIARFID-UPV September 22, 2015 SMT-1: 6 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 7

5 Approaches to MT: Analsis detail SOURCE INTERLINGUA Depth of analsis SEMANTIC TRANSFER SYNTACTIC TRANSFER DIRECT Generation TARGET MIARFID-UPV September 22, 2015 SMT-1: 8 Approaches to MT: Technologies (Linguistic) knowledge-based methods (Memorized) eample-based methods Translation memories Statistical models Alignment models Finite-state models Neural networks Hbrid models MIARFID-UPV September 22, 2015 SMT-1: 9

6 Assessment Test sentences Subjective evaluation based on the number of words that need to be corrected Test sentences with reference translation Automatic assessment Editing Distances: Translation Word Error Rate (TWER) Translation Error Rate (TER) Multireference TWER N-gram based: BLEU and NIST scores... MIARFID-UPV September 22, 2015 SMT-1: 10 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 11

7 Brief histor of MT 1949 Weaver: Information-theor based approach 1957 Chomsk: Natural language is not governed b statistics 1960 ALPAC (Automatic Language Processing Advisor Committee) report: No useful MT results are foreseen 1960-nowadas SYSTRAN sstem: based on dictionaries Several (linguistic) knowledge-based approaches Empiricists methods are introduced: corpus-based and statistical approaches (IBM, 1989) 1995-nowadas Empiricists methods are thriving. Speechto-speech MT in limited domains MIARFID-UPV September 22, 2015 SMT-1: 12 Recent histor of MT: Empiricists methods Statistical approach to MT b IBM Yorktown Heights researchers Hansards Parallel English/French transcriptions of parliamentar discussions DARPA competitive assessment (1994): Results comparable to those achieved b traditional approaches Development of statistical techniques and other empiricists methods Great progress on the statistical approach: from word-based to phrasebased models and hierarchical phrase-based models; log-linear combination of models, etc. Other empiricist techniques: Memor-based, finite-state, etc. Progress in grammars and sntactic analsis Computer assisted translation Practical use of statistical machine tranlation: Great progress of the statistical approach: Industrial use of Moses Other techniques: Continuous space representation Advanced computer assisted translation: interaction, adaptative,.., Adaptation, online learning MIARFID-UPV September 22, 2015 SMT-1: 13

8 Machine translation sstems Tet translation sstems SYSTRAN: SisHiTra: Moses: Google: SALT: WebSphere: Apertium: MIARFID-UPV September 22, 2015 SMT-1: 14 Machine translation sstems Computer assisted (aided) translation sstems TRANSTYPE: Déjàvu: TRADOS: CasMaCat: MIPRCV-CAT: CAITRA: MIARFID-UPV September 22, 2015 SMT-1: 15

9 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 16 Notation and basic concepts and will generall denote source and target tets, respectivel CONDITIONAL AND UNCONDITIONAL PROBABILITIES: Pr(Y = X = ) Pr( ), Pr(Y = ) Pr() BAYES RULE: Pr( ) Pr() = Pr( ) Pr() JOINT PROBABILITY: Pr(, ) = Pr() Pr( ) Pr( 1, 2,..., I ) = Pr( 1 ) Pr( 2 1 ) Pr( I 1,..., I 1 ) (Pr( I 1) = Pr( 1 ) Pr( 2 1 ) Pr( I I 1 1 )) Pr() = Pr(, ) ma A Pr() = Pr(argma Pr()) A A Pr() ma A Pr() MIARFID-UPV September 22, 2015 SMT-1: 17

10 General Framework Ever sentence from a target language is considered as a possible translation of a given sentence from a source language. For each possible pair of sentences,, there is a probabilit Pr( ). Pr( ) should be low for pairs (, ) such as: ( una habitación con vistas al mar, are all epenses included in the bill? ) Pr( ) should be high for pairs such as: ( ha alguna habitación tranquila libre?, is there a quiet room available? ) Goal of statistical machine translation: given a source sentence search for a target sentence ŷ such that (Risk minimization): ŷ = argma Pr( ) MIARFID-UPV September 22, 2015 SMT-1: 18 An inverse (Baesian) approach Decompose Pr( ) using Baes rule: Pr( ) Pr() ŷ = argma Pr( ) = argma Pr() = argma Pr( ) Pr() A distorted (nois) channel model P r() P r( ) Need: a target-language model + alignment and leicon models MIARFID-UPV September 22, 2015 SMT-1: 19

11 A generative (finite-state) approach The direct probabilit can be decomposed in a different wa: Pr(, ) ŷ = argma Pr( ) = argma Pr() = argma Pr(, ) A joint model P r(, ) A stochastic finite-state transducer can model the joint distribution MIARFID-UPV September 22, 2015 SMT-1: 20 A direct (discriminative) approach Search for a target sentence with maimum posterior probabilit: ŷ = argma Pr( ) A direct model P r( ) Log-linear combination of models MIARFID-UPV September 22, 2015 SMT-1: 21

12 Log-linear combination of models Search for a target sentence with maimum posterior probabilit: ŷ = argma ŷ = argma Pr( ) ( K ) ep k=1 λ kh k (, ) ( ep K k=1 λ kh k (, ) ) = argma K λ k h k (, ) k=1 h 1 (, ) = log P r(), a language model h 2 (, ) = log P r( ), a translation model model h 3 (, ) = log P r( ), an inverse translation model model... MIARFID-UPV September 22, 2015 SMT-1: 22 Training and search Search argma P r( ) ŷ MIARFID-UPV September 22, 2015 SMT-1: 23

13 Training and search ( 1, 1 )... ( n, n ) Training models P r( ) Search argma P r( ) ŷ MIARFID-UPV September 22, 2015 SMT-1: 24 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 25

14 Speech-input translation Given an input acoustic sequence v, search for a target sentence with maimum posterior probabilit: ŷ = argma Pr( v) But this can be seen as a two-step process : v where the hidden variable accounts for all possible input decodings of v: ŷ = argma Pr(, v) = argma Pr(, ) Pr(v ) (with the assumption: Pr(v, ) does not depend on the target sentence ) MIARFID-UPV September 22, 2015 SMT-1: 26 Speech-input translation: An integrated approach argma Pr( v) argma ma (Pr(, ) Pr(v )) Pr(v ) ACOUSTIC MODELS Pr(, ) STOCHASTIC FINITE-STATE TRANSDUCERS MIARFID-UPV September 22, 2015 SMT-1: 27

15 Speech-input translation: A serial approach argma ma (Pr(, ) Pr(v )) = argma ma (Pr( ) Pr() Pr(v )) 1. argma 2. ŷ argma Pr( ) Pr() Pr(v ) = argma Pr( v) Pr(v ) ACOUSTIC MODELS Pr( ) TRANSLATION MODELS Pr() SOURCE LANGUAGE MODELS MIARFID-UPV September 22, 2015 SMT-1: 28 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 29

16 Tet prediction for Interactive in CAT Given a source tet and a correct prefi p of the target tet, search for a suffi ŷ s, that maimizes the posterior probabilit over all possible sufies: ŷ s = argma s Pr( s, p ) Taking into account that Pr(, p ) does not depend on s, we can write: Pr( p, s ) ŷ s = argma s Pr( p ) = argma s Pr( p s ) Main difference with tet-input machine translation: search over the set of suffies. MIARFID-UPV September 22, 2015 SMT-1: 30 Target language dictation in CAT A human translator dictates the translation of a source tet,, producing a target language acoustic sequence v. Given v and, the sstem should search for a most likel decoding of v: ŷ = argma Pr(, v) B the assumption that Pr(v, ) does not depend on, ŷ = argma Pr(v ) Pr( ) Pr() Pr(v ) (TARGET LANGUAGE) ACOUSTIC MODELS Pr( ) TRANSLATION MODEL Pr() TARGET LANGUAGE MODEL Similar to plain speech decoding, where: ŷ = argma Pr(v ) Pr() MIARFID-UPV September 22, 2015 SMT-1: 31

17 Inde 1 Objectives of machine translation (MT) 2 2 Approaches to MT 7 3 Brief histor of MT and MT sstems 11 4 Tet-input machine translation 16 5 Speech-input machine translation 25 6 Computer-assisted translation 29 7 Bibliograph 32 MIARFID-UPV September 22, 2015 SMT-1: 32 Bibliograph 1. P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, J. Laffert, R. Mercer, P. Roosin: A statistical approach to machine translation. Computational Linguistics, 16:79 85, P. Brown, S. Chen, S. Della Pietra, V. Della Pietra, A. Kehler, R. Mercer: Automatic speech recognition in machine aided translation. Computer Speech and Language, 8(3): F. Casacuberta, H. Ne, F. J. Och, E. Vidal, J. M. Vilar, S. Barrachina, I. García-Varea, D. Llorens, C. Martínez, S. Molau, F. Nevado, M. Pastor, D. Picó, A. Sanchis, Ch. Tillmann: Statistical and finitestate approaches to speech-to-speech translation. Computer Speech and Language, 18:25-47, P. Koehn. Statistical. Cambridge Universit Press P. Langlais, G. Foster, G. Lapalme; Unit Completion for a Computer-aided Translation Tping Sstem., 15(4): , A. Lopez. Statistical. ACM Computing Surves, 40(3):1 49, E. Vidal, F. Casacuberta, L. Rodríguez, J. Civera and C. Martínez Computer-assisted translation using speech recognition. IEEE Transactions on Speech and Audio Processing, 14(3): , S. Barrachina, O. Bender, F. Casacuberta, J. Civera, E. Cubel, S. Khadivi, A. Lagarda H. Ne, J. Tomás, and E. Vidal. Statistical approaches to computer-assisted translation. Computational Linguistics, 35(1):3-28, Francisco Casacuberta, Marcello Federico, Hermann Ne, Enrique Vidal. Recent Efforts in Spoken Language. Translation. IEEE Signal Processing Magazine, Vol. 25 (3), pp MIARFID-UPV September 22, 2015 SMT-1: 33