Agenda. Introduction Predictions

Transcription

1

2 Introduction Predictions Agenda MT usage & the market impact Translators profession Quality Productivity Compensation TM/MT integration Small-scale experiment at KU Leuven

3 About the presenter Postgraduate EMT in Specialised Translation Research Assistant for the SCATE project at KU Leuven Experience with training translators on CAT tools No special expertise in MT

4 Smart computer-aided translation environment (SCATE) in a nutshell Time-frame: 01/03/ /02/2018 Consortium: KU Leuven, 6 university teams, 10 companies Funding: Flemish government, industry, IWT Primary goal: Develop techniques to improve translation efficiency and consistency! How? Better exploitation of resources, e.g. translation memories, terminology Syntax-based fuzzy matching Automatic bilingual term extraction from comparable corpora Stronger integration of speech recognition models with machine translation Personalised user interfaces

5 Project Consortium SCATE Participants

6 SCATE Participants Industrial Advisory Committee

7 13 years ago. About Machine Translation A decent MT machine is just around the corner A computer will never understand, but it can translate, at least to some extent. And, since translation without understanding is meaningless, the future of the human translator is proofsensing what a machine has pre-translated. Champollion (2001) In 6 years from now We can expect that by 2020 automated translation tools will be embedded into every device, application and website. TAUS

8 Present Is the human translator proof-reading what a machine has pre-translated? YES

9 MT for Big Data Many languages Increasing content volumes Big Language

10 What about the LSP s? in the professional market, human translation still rules, though a quiet but pronounced shift to machine translation with post editing is unquestionably underway in the agencies, some of whom are also actively developing their own engines using open source software. LT-Innovate Report % of the LSPs use MT Source: CSA

11 MT for Translation Rule-based Statistical Hybrid Linguistic rules Specialised dictionaries per domain, customer, product line! Suitable for software documentation, technical manuals, E-learning courseware, UI Consistent and predictable quality Out-of-domain translation quality Lack of fluency Hard to handle exceptions to rules It does not cover all languages! Expensive to develop and customize Large bilingual corpora Terminology choices hard to manage! Suitable for user-generated content Unpredictable translation quality Poor out-of domain quality Does not know grammar High CPU and disk space required Good fluency Good for catching expeptions to rules It covers all languages! Rapid and cost-effective development Rules +Statistics Better output Solves ambiguity More expensive to manage two systems! Systran, Asia Online, Prompt Systran, Apertium, Babelfish, Prompt Google, Moses, Language Weaver, Asia Online

12 What is the market impact? In a nutshell: Impact on the translation profession o o o Quality o o Deprofessionalization of the translator? New jobs, e.g. Post-Editor, MT quality evaluator How to maintain creativity? MT Process vs. quality of the MT output What is the final expected quality? Productivity o Raw MT output, Light post-editing vs full post-editing Compensation o Per word, per hour or both?

13 Benefits of MT for o o o o End-clients Reduce costs Reduce time to market Increase productivity Translate new content Translators New opportunities Increase productivity Higher income More consistency

14 What do translators say? While we see that MT technology adoption continues to grow, hopefully because of clearly demonstrated benefits and measured production efficiencies, we still see that the dialogue between the technology developers / business sponsors and translators/posteditors is often strained, and communications can often be dysfunctional and sometimes even hostile. Kirty Vashee, empty Pages

15 What do translators say? how to protect the language skills and mental function of those routinely exposed to toxic machine language. (Translator s blog) I fear it might be the post-mt crash (of our industry, in terms of rates and quality). (Translator, Proz ) post-processing of machine translations, is likely to be much more deadly to the translating community, because this tool was designed by our beloved translation industry leaders to simply kill off the translating profession and replace it by a new type of workers called post-processors of machine translations. (Translator s blog)

16 What do translators say? My proposal is that you start investing time and money to update the skills of your human base, current freelance translators and interpreters, so that you may flip it around...i urge industry leaders to invite us, everyday translators and interpreters, to become an integral part (not a side story) of this equation. We have been severed from the most important conversations about our own future. Many of us are afraid of the new technologies because there is yet no clear answer to the question what s in it for me?. The industry as a whole may well benefit from inviting us to become part of the equation going forward. If translators and interpreters do not learn quickly and swiftly to use 21st century technologies, we may not survive as a viable profession. Unless all the parties are on the same page, the industry as a whole may not evolve the way industry leaders might have anticipated. We need each other. Claudia Brauer (2014), blog post Will you help me flip it?

17 What do translators say? it is safe to say that the number of professional translators who are using MT has risen in the past couple of years. I suspect that you would find large differences of usage among different language groups, type of projects, experience levels, and even in the way it is used. Significantly, MT seems to have transitioned into one of many productivity tools that are useful to some but not all, and I think the stigma it used to have among translators is gradually going away. ATA Chronicle (Feb. 2013) Conversation: Charlotte Brasler and Jost Zetzsche

18 What do you say?

19 Post-editing is just a step in the MT process Pre-production Consultation Content Customization Production Piloting Processing Post-editing Post-production Metrics Maintenance Based on the MT process described by Lori Thicke in The industrial process for quality machine translation in JoSTrans, Issue 19

20 Translation vs post-editing Translation process of a translator (TAUS 2010) Translation process of a post-editor (TAUS 2010)

21 Translators vs Post-editors Skills? Translators Fluent at least in two languages o o Excellent knowledge of SL Perfect command of TL Excellent written and verbal communication skills Understand the cultural differences Specialised domain knowledge Word-processing skills Knowledge of language technologies Post-editors Excellent word-processing and editing skills Ability to work and make corrections directly on-screen Strong knowledge of general MT and its shortcomings Specific knowledge of the approach of the MT systems in use, especially their weaknesses and the reason for producing the encountered errors Knowledge of the SL, TL and the audience Ability to make quick decisions and reduce errors as needed Ability to balance post-editing speed and effort with respect to the required quality level; Knowledge of CAT environments and of the differences between post-editing MT output as opposed to editing the past human translations present in fuzzy matches. Source: Doherty & Gaspari 2014, GALA blog

22 Quality Source: language-translations-can-be-funny-right/

23 Quality Quality is relative The evaluation of the translation qualitywhether human or computerized has to take into account the following intra-linguistic and inter-linguistic factors: morphology, syntax, content, terminology, style, conformity. (Bruderer 1978) Typically, there are many perfect translations of a given source sentence. These translations may vary in word choice or in word order even when they use the same words. And yet humans can clearly distinguish a good translations from a bad one. (Papineni et al., 2002)

24 Quality EN-15038: simple to control! Lucija Biel, 2006

25 Quality more difficult to control! Source: Asia Online

26 Quality: MT Evaluation methods Human No standardised method Word Error Rate (WER) Subjective Sentence Error Rate (SSER) Metrics: o Fluency and Adequacy (TAUS DQF, COSTA MT- Evaluation, Appraise etc.) Post-editing effort Automatic Metrics: o o o o o BLEU NIST WER METEOR LEPOR

27 Productivity Post-editing MT output can significantly increase the productivity of human translators! Guerberof 2009; Plitt and Masselot 2010; Federico et al. 2012; Green et al. 2013; Läubli et al. 2013

28 Productivity TEP Translation: 250 words per hour Editing: 1000 words per hour Proofing: 4000 words per hour MT Post-editing words per hour

29 Productivity With Customised MT Source: Asia Online, Custom MT engines with Language Studio TM

30 Productivity Autodesk 2010

31 PE Compensation Various experiments, no standard Hourly rates or price per word? o 65%-85% of new word rate (Lionbridge) o 40%-85% of new word rate (CSA report) o $45/hr (Asia Online)

32 PE Compensation It depends on certain factors: o Quality of the MT output o Translator s productivity (speed) o Language pair o Content type

33 PE Compensation Less PE effort More words per hour More earnings? Good MT output

34 TM/MT integration MT becomes even better when integrated in the translators TM environment! (O Brien & Moorkens 2014) A number of CAT tools connect to general and customized MT engines via API:

35 SDL Trados Studio 2014 memoq 2014

36 OmegaT In the OmegaT user manual it is stated the following: If you do not wish to send your source segments to Google to get translated, untick the Google Translate menu entry in Options. Note that nothing but your source segment is sent to the MT service. The online version of Google Translate allows the user to correct the suggestion and send the corrected segment in. This feature, however, is not implemented in OmegaT.

37 Integrated PE analysis features Edit distance memoq 2014

38 Post-edit Modifications Analysis in Trados Studio 2014

39 MateCat

40 MemSource Post-editing Analysis report

41 Post-editing analysis & Compensation Signs and Simptoms of Translation- Blog by Emma Goldsmith

42 Small-scale experiment at University of Leuven Goal: o To test the TM/MT integration o To investigate the impact of MT on productivity and the final translation quality Participants and language combinations: o 21 MA/Postgraduate students with EN > NL (Productivity test) o Translation Technology class Source text: o 5 short English texts, technical documentation, same level of difficulty Tools: o Trados Studio 2014 Resources: o TM with output from Google

43 What did we measure? Productivity (time): human translation vs post-editing MT Quality: human-translations vs post-edited translations The usefulness of MT Translators expectations

44 How? 1. Translation from scratch, no TM/MT provided. 2. Translation with TM. 3. Translation with TM/MT. 4. Track the translation time of the whole project. 5. Answer few questions about the usefulness of MT

45 Language combination used for the assignment:

46 Did you study English in the previous years?

47 Source text EN NL

48 Results Productivity Total average time per text for EN-NL Nr of words per text varied between

49 Quality Evaluation: Results Rate the translations on a scale from 1-4 in terms of: o Clarity: How easily can you understand the translation? o Accuracy: How much of the meaning in the source text is also expressed in the translation? o Style: Is the language used in the translation appropriate for this content type? Fiederer & O Brien (2009)

50 CLARITY: How easily can you understand the translation? In general

51 CLARITY: How easily can you understand the translation? Human Translation vs. Post-editing

52 ACCURACY: How much of the meaning in the source text is also expressed in the translation? In general

53 ACCURACY: How much of the meaning in the source text is also expressed in the translation? Human Translation vs. Post-editing

54 STYLE: Is the language used in the translation appropriate for a software product user manual? In general

55 STYLE: Is the language used in the translation appropriate for a software product user manual? Human Translation vs. Post-editing

56 General Usefulness of MT Final quality Terms Colloc ations

57 Did you find the machine translation output useful? Nr of students

58 Terminology? Collocations? Nr of students Very useless Useless Useful Very useful 0 Useful for terminology Useful for collocations

59 Do you think the machine translation output had an impact on the quality of your final translation?

60 Is MT a disruptive technology? or is it simply.

61 What did students think of MT?

62 What did students think of MT?

63 Limitations Students had to record the time themselves Not all of them had a good knowledge of English Knowledge of the technical domain Starter-knowledge of Trados Studio 2014

64 The debate that puts machine translation and professional translation on the opposite sides is misleading. The question isn t about choosing eitheror because those two solve totally different problems. Professional translation is the best choice when your goal is to get someone to understand your message. MT is a risky choice in those kinds of situations but is a more suitable tool when you want to understand someone else s message. Multilizer, Translation Blog

65 Thank you!

66 References Thicke, Lori. The industrial process for quality machine translation. Retrieved from on 5 November Guerberof Arenas, Ana (2009). Productivity and Quality in the Post editing of Outputs from Translation Memories and Machine Translation. Localisation Focus 7(1): 11-21http://isg.urv.es/library/papers 2009_Ana_Guerberof_Vol_7-11.pdf Guerberof Arenas, Ana (2013) What do professional translators think about postediting?. The Journal of Specialised Translation 19: issue19/art_guerberof.pdf O Brien, Sharon (2002) Teaching Post-editing: A Proposal for Course Content. Proceedings of the 6th EAMT Workshop on Teaching Machine Translation. EAMT/ BCS, UMIST, Manchester, UK Language Technology at Autodesk. Translation and Post-editing Productivity (2011). Doherty Stephen & Gaspari Federico, QT LaunchPad Project (2014) Effective Post- Editing in Human and Machine Translation Workflows /effective-post-editing-in-human-and-machine-translation-workflows/

67 References O'Brien, Sharon and Moorkens, Joss (2014). Towards intelligent post-editing interfaces. In: FIT XXth World Congress 2014, 4-6 Aug 2014, Berlin, Germany. Retrieved from on 5 November MateCat: SDL: Kilgray Technologies: MemSource: empty Pages Blog: Improving the MT Technology through Translator Dialogue. Patenttranslator blog: There Are Many Ways to Commit Suicide, But Post-Processing of Machine Translations Would Be a Horrible Way To Go (2014) : patenttranslator.wordpress.com/2014/10/22/there-are-many-ways-to-commit-suicide-butpost-processing-of-machine-translations-would-be-a-horrible-way-to-go Translationtribulations.com

68 References Plitt.M, Masselot F. A productivity test of statistical machine translation post-editing in a typical localisation context. Retrieved on October 21 from De Palma, Don. Post-editing in practice: Asia Online, Fair-compensation of Post-editing MT: FairCompensationForMTPostEditing.aspx. Retrieved on 27 October 2014 Smets, Martine. Microsoft and the need for agile translations: default.aspx?cdtid=266 Deliverables D5.1 Baseline Definition Results on KPI 1: Post-editing Effort: deliverables/d5-1-baseline-definition/results-on-kpi-1-post-editing-effort/ Apertium: ATA Chronicle (February 2013): Example of a Translation Integrated Workflow: default.aspx?qid=21