Service model for semi-automatic generation of multilingual terminology resources

Transcription

1 Service model for semi-automatic generation of multilingual terminology resources Mārcis Pinnis, Andrejs Vasiļjevs, Tatiana Gornostay Tilde TKE 2014 / Berlin/

2

3

4 Complexity of terminology work Term identification in the source text Consulting online databases and local files for translation equivalents Creating and maintaining terminology glossaries Sharing term glossaries and involving others in their polishing Structuring data in the industry standard formats Integrating term glossaries in CAT and other productivity tools Keeping terminology up to date etc.

5 SaaS Software as a Service

6 PaaS Platform as a Service

7 TaaS Terminology as a Service

8 Cloud-based service model The automation of individual tasks in terminology work is rendered as a set of interoperable cloud-based services integrated into workflows Collaborative involvement of users in the refinement and enrichment of raw terminological data Automated sharing and integration of the terminology in various use cases by language workers and language processing applications: computer-assisted translation tools machine translation systems terminology management and terminology lookup platforms etc.

9 Terminology as a Service cloud-based platform for acquiring, cleaning up, sharing, and reusing multilingual terminological data

10 TaaS Partners Tilde TAUS Kilgray University of Cologne University of Sheffield Latvia (Coordinator) Netherlands Hungary Germany UK

11 TaaS Industry & Research Collaboration project Supported by EU 7 th R&I Framework Programme Resulted in TaaS cloud-based services Accessible for free at the online portal termunity.com

12 TaaS User Needs Survey Results Professional profile Potential TaaS user groups: What professional groups we address? Technical writers Translators Interpreters Software localizers Domain experts Language teachers/learners Terminologists 7.6% 6.4% 5.1% 4.8% 3.3% 2.8% 1.0% 31.0% 38.0% Technical writer Translator Other Interpreter Terminologist Domain expert Language teacher Software localizer Language learner The prevailing working status of the potential users of TaaS are Employed with an industrial company with 51.1% (mainly technical editors/writers and terminologists) and Freelance with 34.6% (mainly translators and interpreters).

13 TaaS User Needs Survey Results: Importance of terminology work 14.8% 1.8% 43.5% Very important Quite important Less important Not important 39.9%

14 TaaS User Needs Survey Results Addressing the actual needs of potential users: Problems faced when doing online terminology search and optimization potential in this area. Problematic issues % Lack of resources/insufficient terminology management 24.5 Poor quality/up-to-dateness 14.5 Lack of information (contexts, definitions, local and companyspecific use etc.) 13.6 Lack of convincing verification/misleading information online 12.3 Inconsistent use of terminology 10.9 Technical issues 8.9 Lack of covered specific subject fields 6.6 Lack of freely accessible standardized data/ Unwillingness to share information 5.4 Ambiguity/Synonymy 4.5 Lack of covered languages and language pairs 4.1 Lack of equivalents 2.4 Other 11.2

15 TaaS User Needs Survey Results Technical specification of the TaaS platform: Mostly used programs and formats when doing terminology work. Mostly used formats DOC XML TXT RTF TMX CSV XLIFF TBX Other 9.6% 12.2% 17.6% 24.4% 26.8% 33.0% 45.1% 52.4% 84.0%

16 TaaS User Needs Survey: willingness to share 60.5% 39.5% Yes, provided that No, because 16,7% 24,9% 8.3% 4.6% 6,0% 16.5% 48.6% 7,6% 11,4% 19,2% 14,2% Joint contribution to the DB Access control Legal aspects External quality control Little effort Anonymity Other 22.0% Legal restrictions Poor quality/lack of time Own asset Risk of misunderstanding

17 TaaS Goals Simplify the process for language workers to prepare, store and share of task-specific multilingual term glossaries Provide instant access to term translation equivalents and translation candidates for professional translators through CAT tools Domain adaptation of statistical machine translation systems by dynamic integration with TaaS provided terminology data

18 Key services of TaaS Automatic extraction of monolingual term candidates from user uploaded documents Automatic retrieval of translation equivalents from different public and industry terminology databases Translation candidate acquisition from multilingual web data Facilities for cleaning-up by users automatically acquired terminological data; Data sharing and integration facilities through APIs and export tools

19 TaaS Services

20 Simple Search Search for terms in various sources

21 Term candidate identification Tagging the document using part-of-speech or morphosyntactic taggers Term candidate identification using: linguistic filtering with morpho-syntactic term phrase regular expressions statistical ranking methods with co-occurrence measures (e.g., log likelihood, modified mutual information etc.) for terms of two or more tokens and the TF*IDF measure for unigram terms Marking terms in the document

22 Evaluation of service for term candidate extraction Language Information Technology Mechanical Engineering Correct Total Precision Correct Total Precision English % % German % % Hungarian % % Latvian % % Comment: The average agreement rate of the Latvian annotators was only 63.3%.

23 Automatic retrieval of translation equivalents from terminology databases Normalization of terms transforming terms from their surface forms into corresponding canonical forms as used in dictionaries and term banks Lemmatization for single word terms Rule-based methods for multiword terms Automatic search and retrieval from terminology databases: EuroTermBank databank of federated multilingual terminology IATE inter-institutional termbank of European Union TAUS Data repository of multilingual translation memories

24 Service for Translation Equivalent Retrieval from the Web Extracting bilingual terminology from the parallel corpora using bilingual phrase alignment techniques Terminology extraction from focused comparable corpora Focussed Monolingual Crawler FMC (Mastropavlos and Papavassiliou, 2011) Bilingual terminology extraction from comparable RSS news corpora Bilingual terminology extraction from Wikipedia Application of the ACCURAT Toolkit Context independent term mapping - (Pinnis, 2013) Supervised term mapping - Aker et al. (2013)

25 Workflows for multilingual terminology extraction from comparable Web corpora

26 Visualised example of terminological data from the Statistical Data Base

27 Automated Lookup

28

29 Term identification and annotation

30 Identifying and marking terms New W3C standard for Internationalization Tag Set ITS 2.0 ITS 2.0 enriched content Showcase Web Page ITS 2.0 enriched content Terminology Annotation Web Service API Plaintext TaaS Terminology Services Human users (e.g., translators, terminologists) ITS2.0 term-annotated content export / visualisation ITS2.0 term-annotated content ITS 2.0 enriched content Machine users Term-annotated content ITS2.0 term-annotated content CAT Tools MT Systems

31 Term Annotation

32 HTML Term Annotation Term entries for terms identified in EuroTermBank are stored in TBX format in a <script> element that is placed in the HTML5 document.

33 Refine & Approve

34 Share & Collaborate

35 Integration Support for industry standard formats Integration into CAT and productivity tools API to integrate TaaS services into various software applications

36 http/https html TaaS Architecture Web Browsers CAT tools MT External TDBs Presentation Layer https REST https REST https REST Web Page UI Public API included included Application Logic Layer Terminology collection management User management Terminology collection search Terminology collection creation Data Storage Layer (Shared Term Repository) Term extraction workflows Full collection creation workflow Monolingual collection creation Translation candidate extraction Text tagging with terms... Shared Term Repository DB File Store Statistical DB High-performance Computing (HPC) Cluster HPC frontend CPU CPU CPU CPU CPU CPU CPU CPU SGE CPU CPU CPU CPU Modules Term extraction TXT extractor TWSC Kilgray Term Extractor Term normalizer Collection creator Statistical DB acquisition Parameter retriever Bilingual Term Extraction System Statistical DB feeding Translation lookup ETB & STR IATE TAUS API Statistical DB Collection merger Result processing Collection Importer Marked Text enrichment

37 koks timber How to instruct SMT to use the right terms?

38 Put TaaS in the service for MT

39

40 Boost in the quality of machine translation Narrow Domain Automotive MT English Latvian DATA 2 M unique parallel sentences 1.9 M monolingual sentences 0.2 M in-domain monolingual QUALITY 16% improvement from terminology integration

41 Try it yourself! termunity.com

42 Towards a Public EU Infrastructure for Automated Translation Deliver the strategic vision, functional, technical and operational specifications of the infrastructure for EU public service for automated translation and other multilingual services and resources Design a sustainable governance model for the multilingual infrastructure Foster multi-stakeholders alliances to ensure its commitment and support for the deployment of the MLi and its usage

43 Thank you! The research within the project TaaS leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ), Grant Agreements n and n