From monolithic XML for print/web to lean XML for data: realising linked data for dictionaries

Size: px

Start display at page:

Download "From monolithic XML for print/web to lean XML for data: realising linked data for dictionaries"

Louisa Berry
10 years ago
Views:

1 From monolithic XML for print/web to lean XML for data: realising linked data for dictionaries Matt Kohl & Sandro Cirulli Language Technologists Oxford University Press (OUP) 7 June 2014

2 Introduction Oxford University Press 2/18 World-renowned dictionary publisher Licensing partner for lexical data

3 Introduction Shifts in Publishing 3/18 New trends & demands Emerging technologies & markets Importance of well-structured, semantically-rich data Speed!

4 Data Modelling Our Current Dictionary Data Models 4/18 Print-oriented: designed to capture dictionary layout Monolithic: one enormous document Permissive: continually loosened to accommodate new texts Can t give us the flexibility we need

5 Data Modelling Requirements 5/18 A new approach should: Represent language concepts, not layouts Enable data reusability for different products & services Allow only one, clear way to model any given lexical item

6 Data Modelling The New Lexical Schema

7 Data Conversion Moving Data into the Lexical Schema 7/18 Conversion Framework Requirements Scalability: convert 40+ data-sets Standardization: harmonize variation inside the data-sets Modularity: enable customization, slotting in & out of QA, etc.

8 Data Conversion Tools 8/18 XProc XSpec Schematron & XML Schema Jenkins CI Agile methodology

9 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es"

10 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es" XSL transformations

11 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es" XSL transformations Schematron validation

12 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es" XSL transformations Schematron validation enhanced XML

13 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es" XSL transformations Schematron validation XSL transformations enhanced XML XML Schema validation Schematron validation

14 Data Conversion Simplified XProc pipeline 9/18 print-focused XML +xml:lang = "es" XSL transformations Schematron validation XSL transformations enhanced XML XML Schema validation Schematron validation Lexical Data

15 Data Conversion Build Workflow Check code in SVN Jenkins build - Ant script - XSpec unit tests - XProc pipeline

16 Data Conversion Build Workflow Jenkins build Check code in SVN - Ant script Jenkins build - XSpec unit tests Linguistic QA - XProc pipeline

17 Data Conversion Build Workflow Jenkins build Check code in SVN - Ant script Jenkins build - XSpec unit tests Linguistic QA - XProc pipeline Update code No Passes?

18 Data Conversion Build Workflow Jenkins build Check code in SVN - Ant script Jenkins build - XSpec unit tests Linguistic QA - XProc pipeline Update code No Passes? Yes Archive artefacts Tag release in SVN via Jenkins

19 Results & Discussion Source data 11/18 A sense of malaúva from a monolingual Spanish dictionary <ACEPCIO ACEP="2"> <AREA-GEO>Esp</AREA-GEO> <NIVELL>coloquial</NIVELL> <SIGNIFICAT>Persona que tiene mal carácter o mala intención.</significat> <SINONIM>malaleche.</SINONIM> </ACEPCIO>

20 Results & Discussion OUP XML Print-focused DTD <se2 num="2"> <lg> <ge>esp</ge> <reg>coloquial</reg> </lg> <msdict type="core"> <df>persona que tiene mal carácter o mala intención.</df> <syn>malaleche</syn> </msdict> </se2> 12/18

21 Results & Discussion OUP XML Print-focused DTD <se2 num="2"> <lg> <ge>esp</ge> <reg>coloquial</reg> </lg> <msdict type="core"> <df>persona que tiene mal carácter o mala intención.</df> <syn>malaleche</syn> </msdict> </se2> New Lexical XSD <sense register="informal" region="es"> <definitions> <definition> <text>persona que tiene mal carácter o mala intención</text> </definition> </definitions> <synonyms> <synonym>malaleche</ synonym> </synonyms> </sense> 12/18

22 Results & Discussion OUP XML Print-focused DTD <se2 num="2"> <lg> <ge>esp</ge> <reg>coloquial</ reg> </lg> <msdict type="core"> <df>persona que tiene mal carácter o mala intención.</df> <syn>malaleche</syn> </msdict> </se2> New Lexical XSD <sense register="informal" region="es"> <definitions> <definition> <text>persona que tiene mal carácter o mala intención</text> </definition> </definitions> <synonyms> <synonym>malaleche</ synonym> </synonyms> </sense> 12/18

23 Next steps Scale It Up 13/18 Consolidate data in an XML database Build an RDF layer on top of the XML database Leverage Semantic Web to enhance our data

24 Next Steps Prototype RDF/XML 14/18 <Sense rdf:about="sense:es_noun_malauva_se_2"> <isdescribedby rdf:resource=" definition:es_noun_malauva_se_2_def_1"/> <hasregister rdf:resource="register:informal" /> <hasregion rdf:resource="region:es"/> <hassynonym rdf:resource="lemma:a5e644"/> </Sense> <StandardDefinition rdf:about="definition:es_noun_malauva_se_2_def_1"> <rdfs:label xml:lang="es">persona que tiene mal carácter o mala intención</rdfs:label> </StandardDefinition>

25 RDF Data extraction Musical terms in English & Spanish 15/18 hook: hook tune: melodía chorale: coral song: canción aria: aria choir: coro chorus: estribillo strain: tono chorus: choral chorus: coro air: aire ensemble: conjunto chant: salmodia

26 Inference mechanism 16/18 hasantonym hassynonym word sense X word sense Y word sense Z hasantonym

27 Summary 17/18 Overall project requirements Moving from products to platforms and services Supporting current business needs while innovating Adapting in nimble ways to fast changing market requirements Focusing on time and cost efficiency Data model Content driven Machine interpretable Modular Evolvable/adaptable Conversion process Highly automated Modular Scalable

28 Thank you for your attention! Any questions? Matt Kohl: Sandro Cirulli:

Continuous Integration for XML and RDF Data

Continuous Integration for XML and RDF Data Sandro Cirulli Language Technologist Oxford University Press (OUP) 6 June 2015 Table of contents 1. Context 2. Continuous Integration with Jenkins 3. Automatic