TS3: an Improved Version of the Bilingual Concordancer TransSearch

Size: px

Start display at page:

Download "TS3: an Improved Version of the Bilingual Concordancer TransSearch"

Roy Shaw
10 years ago
Views:

1 TS3: an Improved Version of the Bilingual Concordancer TransSearch Stéphane HUET, Julien BOURDAILLET and Philippe LANGLAIS EAMT Barcelona June 14, 2009

2 Computer assisted translation Preferred by professional translators Exploits a translation memory One of these tools: bilingual concordancer Retrieves from a bitext parts associated with a query Currently operates at the sentence level TransSearch: a web-based concordancer with 177,000 queries/month 2

Retrieves from a bitext parts associated with a query Currently operates at

3 Current version: highlighted query sentence alignment 3

4 Prototype version: TS3 highlighted query sentence alignment translation of the query 4

5 Prototype version: TS3 several translations of the query context of use 5

6 Outline Spotting of the query translation Refinement of translation spotting Translation variants merging Corpora Experimental results 6

7 Translation spotting (or Transpotting) Identification in a sentence of the translation of a query This is in keeping with that strategy. La présente mesure est conforme à cette stratégie. Query: in keeping with 7

8 Translation spotting (or Transpotting) Identification in a sentence of the translation of a query This is in keeping with that strategy. La présente mesure est conforme à cette stratégie. Query: in keeping with Transpot: conforme à 8

9 Word alignment This is in keeping with that strategy. La présente mesure est conforme à cette stratégie. Use of an IBM2 model Discontinuous transpots Not the best method to transpot 9

10 Transpotting algorithm This is in keeping with that strategy. La présente mesure est conforme à cette stratégie. Algorithm of [Simard 03] Contiguous transpots Best performance among several tested methods 10

11 Transpotting algorithm This is in keeping with that strategy. La présente mesure est conforme à cette stratégie. Algorithm of [Simard 03] Contiguous transpots Best performance among several tested methods 11

12 The need for post-processing Query: in keeping with Proposed transpots conforme à (45) conformément à (29) à (21) dans (20) conforme aux (18) de (14) conforme (13) conformément aux (13) conforme au (12) conformes à (11) d actualité (1) gestes en (1) correspond à (1) respectent (1) 12

de (14) conforme (13) conformément aux (13) conforme au (12) conformes

13 The need for post-processing Query: in keeping with Proposed transpots after filtering conforme à (45) conformément à (29) à (21) dans (20) conforme aux (18) de (14) conforme (13) conformément aux (13) conforme au (12) conformes à (11) d actualité (1) gestes en (1) correspond à (1) respectent (1) 13

conforme aux (18) de (14) conforme (13) conformément aux (13) conforme au

14 The need for post-processing Query: in keeping with Proposed transpots after filtering and merging conforme à (45) conformément à (29) conforme aux (18) conforme (13) conformément aux (13) conforme au (12) conformes à (11) correspond à (1) respectent (1) 14

conformément à (29) conforme aux (18) conforme (13) conformément

15 Filtering bad transpots At the level of a pair of sentences Computation of 3 sets of features Size of the transpot, size of the query Statistical word alignment features: min and max likelihood, Viterbi scores... Linguistic features: grammatical word ratio, article counts, preposition counts... Training of various classifiers Voted-perceptron, SVM, decision tree, voting 15

16 Merging translation variants At the level of the transpot list found for a query High complexity when building all possible clusters Neighbor-joining method of [Saiou and Nei 87] Builds a distance matrix Q between all pairs Is a greedy algorithm that at each step Merges the two closest transpots Updates Q Uses a word-based distance Minimal cost between 2 inflected forms of a lemma Edition costs smaller for grammatical words 16

between all pairs Is a greedy algorithm that at each step Merges the two closest transpots Updates Q Uses a

17 Example for the merging process conforme au conforme aux correspondant au dans le sens de l dans les sens des 17

18 Example for the merging process conforme au conforme aux correspondant au dans le sens de l dans les sens des 18

19 Example for the merging process conforme au conforme aux correspondant au dans le sens des dans le sens de l 19

20 Example for the merging process correspondant au conforme au conforme aux dans le sens des dans le sens de l 20

21 Example for the merging process dans le sens des dans le sens de l correspondant au conforme au conforme aux 21

22 Detection of similar variants dans le sens des dans le sens de l correspondant au conforme au conforme aux 22

23 Corpus used in the experiments 5,000 most frequent queries Canadian Hansard 8.3 M pairs of sentences Retrieved Retrieved pairs of pairs sentences of sentences 23

24 Reference corpus for filtering Annotation of 530 queries (23 translations per query) 24

25 Results for classification of transpots Trained on the annotated queries Tested by 10-fold cross-validation Correct classification F-measure for bad transpots All good 62 0 Grammatical ratio > Best classifier Similar results for the 4 tested classifiers: voted-perceptron, SVM, decision stump, AdaBoost Most informative features: grammatical and statistical word alignment 25

26 Reference corpus for transpotting Retrieved Retrieved pairs of pairs sentences of sentences Bilingual lexicon Transpotted pairs of sentences Reference = 1.4 M pairs of sentences 26

27 Metrics for transpotting Precison 2/4 suggested transpot Je crois qu il est tout à fait conforme à l esprit du projet de loi. reference 27

28 Metrics for transpotting Precison 2/4 suggested transpot Je crois qu il est tout à fait conforme à l esprit du projet de loi. Recall 1/2 reference suggested transpot Cela n est pas conforme aux normes des Nations Unies. reference Averaged for each query, then averaged on the overall corpus 28

29 Results for transpotting and filtering precision recall F-measure Transpotting Transpotting + filtering Filtering of 7.9% of pairs of sentences Improvement of F-measure, in particular of recall 29

30 Evaluation of variant merging Significant reduction of the number of translations proposed for a query: Higher diversity among the top translations Example: query as described rank before décrits décrite décrit tel que décrit comme l a after décrits prévu comme l a tel que prescrit comme le propose 30

31 Evaluation of variant merging Task: retrieving the 5 best transpots of a query Example {décrits, décrite, décrit, tel que décrit, comme l a} 31

32 Evaluation of variant merging Task: retrieving the 5 best transpots of a query Example {décrits, décrite, décrit, tel que décrit, comme l a} bag of unigrams {décrits, décrite, décrit, tel, que, comme, l, a} 32

33 Evaluation of variant merging Task: retrieving the 5 best transpots of a query Example {décrits, décrite, décrit, tel que décrit, comme l a} bag of unigrams grammatical words removed {décrits, décrite, décrit} 33

34 Evaluation of variant merging Task: retrieving the 5 best transpots of a query Example {décrits, décrite, décrit, tel que décrit, comme l a} bag of unigrams grammatical words removed lemmatization {décrit} 34

35 Results for variant merging Task: retrieving the 5 best transpots of a query Experiments done on the manually annotated corpus precision recall F-measure Before merging After merging Slight decrease of precision and significant improvement of recall => higher diversity 35

36 Conclusion Use of word alignment in a bilingual concordancer Quantitative evaluation of a transpotting algorithm Two new issues Filtering erroneous transpots Merging similar variants of translations 36

37 Future work Improvement of word alignment Higher level IBM models Phrase-based models Use of pseudo-relevance feedback to improve transpotting Evaluation with end users 37

38 Thank you for your attention 38

Statistical NLP Spring 2008. Machine Translation: Examples

Statistical NLP Spring 2008. Machine Translation: Examples Statistical NLP Spring 2008 Lecture 11: Word Alignment Dan Klein UC Berkeley Machine Translation: Examples 1 Machine Translation Madame la présidente, votre présidence de cette institution a été marquante.