!! National Aeronautics and Recent Advances in Eclipse QVTO! Nicolas Rouquette Principal Computer Scientist, Systems and Software Division 2012. Government sponsorship acknowledged.
Outline! A Condensed History of QVT @ OMG QVTO @ Eclipse JPL s improvements to Eclipse QVTO (Helios) Expose the QVTO compiler separately from the QVTO interpreter Performance improvements via selective Google Guava caching optimization Continuous integration support via JUnit result files OASIS XML Catalog-aware EMF ResourceSet QVTO: Compilation & Execution Overview & Internals 2
A Condensed History of QVT @ OMG! OMG Specification Development & Revision Cycles: RFP ad/2002-02-04 MOF 2.0 Query/View/Transformation Formal formal/2008-04-03 MOF QVT version 1.0 Formal formal/2011-01-01 MOF QVT version 1.1 The QVT specification defines: 3 languages 2 conformance criteria Next Revision, QVT 1.2 Due Sept. 2012 What is Eclipse QVTO conformance????? 3
A Condensed History of QVTO @ Eclipse! Eclipse QVTO originates from a Borland contribution Sergey Boyko (formerly at Borland) is the project lead/committer Eclipse QVTO @ OMG Used extensively for automating the production of Revision Task Force artifacts UML 2.4.1 SysML 1.2 SysML 1.3 SysML/Modelica 1.0 Use of OMG QVT in other OMG specifications Diagram Definition & Interchange UML profile for BPMN SysML/Modelica 1.0 4
JPLʼs improvements to Eclipse QVTO (Helios)![ Available pending completion of legal! review by JPL & Caltech lawyers ]! 1. Expose the QVTO compiler separately from the QVTO interpreter Invoke the compiler from the QVTO Editor Cancellable QVTO compilation from the QVTO Builder Separate compilation vs. execution in QVTO Ant tasks 2. Performance improvements via selective Google Guava caching optimization QVT Operational provides powerful mechanisms for extending types & operations Eclipse QVTO 3.1 uses Eclipse OCL 3.0 for type resolution, lookup & reflection Eclipse OCL & EMF APIs are inefficient for extensive type lookup & reflection 3. Continuous integration support via JUnit result files QVTO Compilation JUnit result includes all compilation errors (if any) QVTO Execution JUnit result includes all exceptions & assertion violations (if any) 4. OASIS XML Catalog-aware EMF ResourceSet Protégé/OWL already uses OASIS XML Catalogs for redirecting ontology URIs QVTO s Catalog-aware ResourceSet does the same for redirecting Resource URIs 5. Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) 5
QVTO: Compilation & Execution (overview)! *.qvto Compiler *.qvtotrace (in) <configuration Parameters> *.qvtox Eclipse QVTO Interpreter (inout) (out) 6
QVTO: Compilation & Execution (internals)! *.qvto Compiler *.qvtotrace (in) *.qvtox Interpreter (out) <configuration Parameters> (inout) 7
Eclipse QVTO Compliance! 8
Eclipse QVTO Compliance: SyntaxExecutable! *.qvto Compiler *.qvtotrace (in) *.qvtox Interpreter (out) <configuration Parameters> (inout) 9
Eclipse QVTO Compliance: XMIExecutable! *.qvto Compiler *.qvtotrace *.qvtox (in) *.qvtox Interpreter (out) <configuration Parameters> (inout) 10
Eclipse QVTO Compliance: SyntaxExportable! *.qvto Compiler *.qvtotrace (in) *.qvtox Interpreter (out) <configuration Parameters> (inout) *.qvto (out) 11
Eclipse QVTO Compliance: XMIExportable! *.qvto Compiler *.qvtotrace (in) *.qvtox Interpreter (out) <configuration Parameters> (inout) *.qvtox (out) 12
Eclipse QVTO = A missed conformance opportunity?! With JPL s patch, Eclipse QVTO could claim XMI Executable interoperability compliance 13
JPL Patch adds XMIExecutable compliance, OASIS XML Catalog Resource URI mapping & JUnit result instrumentation! *.qvto OASIS XML Catalog *.qvtox (in) <configuration Parameters> Compiler Interpreter (inout) *.qvtox Compilation JUnit result *.qvtotrace (out) Execution JUnit result 14
JPLʼs improvements to Eclipse QVTO (Helios)! 1. Expose the QVTO compiler separately from the QVTO interpreter Invoke the compiler from the QVTO Editor Cancellable QVTO compilation from the QVTO Builder Separate compilation vs. execution in QVTO Ant tasks 2. Performance improvements via selective Google Guava caching optimization QVT Operational provides powerful mechanisms for extending types & operations Eclipse QVTO 3.1 uses Eclipse OCL 3.0 for type resolution, lookup & reflection Eclipse OCL & EMF APIs are inefficient for extensive type lookup & reflection 3. Continuous integration support via JUnit result files QVTO Compilation JUnit result includes all compilation errors (if any) QVTO Execution JUnit result includes all exceptions & assertion violations (if any) 4. OASIS XML Catalog-aware EMF ResourceSet Protégé/OWL already uses OASIS XML Catalogs for redirecting ontology URIs QVTO s Catalog-aware ResourceSet does the same for redirecting Resource URIs 5. Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) 15
Guava-accelerated Eclipse QVTO! Eclipse QVTO 3.1 is architecturally an extension of Eclipse OCL 3.0 Sensible: QVT Operational defines Imperative OCL as an extension of OCL But: OCL architecture was designed for expressiveness, not efficiency Mismatch: QVT Operational compilation & execution is slow Why? QVT Operational allows extending any metamodel QVT Operational allows defining new types QVT Operational allows defining new operations for existing types How does this flexibility work? Eclipse QVTO specializes OCL s architecture: AbstractTypeResolver EcoreEnvironment UMLReflection 16
Eclipse QVTO Anti-Patterns! Properties with complex initialization expressions Issue: No error handling; vulnerable to oclisinvalid() & oclisundefined() Remedy: Create an initialization helper Accessing type properties using dot notation Issue: Feature lookup is expensive (due to OCL s UMLReflection API) Remedy 1: Use an EMF code generator with efficient feature dispatch Remedy 2: Replace dot access with calls to an access helper Redundant imports Issue: Unnecessary overhead Remedy: Avoid them! 17
Eclipse project practices made optimizing Eclipse QVTO possible!! Eclipse QVTO has 2 test suites: org.eclipse.m2m.tests.qvt.oml 837 unit tests org.eclipse.m2m.tests.qvt.oml.ui 184 unit tests After patching: org.eclipse.m2m.tests.qvt.oml 841 unit tests (+4) 827 passed, 4 errors & 10 failures org.eclipse.m2m.tests.qvt.oml.ui 184 unit tests 183 passed, 1 failure Optimization: added Google Guava caches to optimize the lookup of expensive OCL operations in QVTO s implementaiton of: AbstractTypeResolver +1 cache EcoreEnvironment +1 cache UMLReflection +12 caches 18
Guava-optimized Eclipse QVTO implementation org.eclipse.m2m.internal.qvt.oml.stdlib.qvtumlreflection! The new method delegates to the cache The old method became the cache s load method 19
An Example of Guava cache in Eclipse QVTO! Creating Google Guava caches is easy. The challenge is to make sure the cache is invalidated when necessary => Systematic regression testing & profiling is important to judge the effectiveness of cache optimizations 20
Managing Guava caches with EMF Adapters! Most of OCL s APIs used in QVTO compute derived features from base features (e.g., Class::attributes) 1. Cache the computation of derived features (reflection makes this particularly expensive) 2. Invalidate the derived feature cache whenever the base feature changes 21
JPLʼs improvements to Eclipse QVTO (Helios)! 1. Expose the QVTO compiler separately from the QVTO interpreter Invoke the compiler from the QVTO Editor Cancellable QVTO compilation from the QVTO Builder Separate compilation vs. execution in QVTO Ant tasks 2. Performance improvements via selective Google Guava caching optimization QVT Operational provides powerful mechanisms for extending types & operations Eclipse QVTO 3.1 uses Eclipse OCL 3.0 for type resolution, lookup & reflection Eclipse OCL & EMF APIs are inefficient for extensive type lookup & reflection 3. Continuous integration support via JUnit result files QVTO Compilation JUnit result includes all compilation errors (if any) QVTO Execution JUnit result includes all exceptions & assertion violations (if any) 4. OASIS XML Catalog-aware EMF ResourceSet Protégé/OWL already uses OASIS XML Catalogs for redirecting ontology URIs QVTO s Catalog-aware ResourceSet does the same for redirecting Resource URIs 5. Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) 22
Eclipse QVTO & Eclipse JUnit result format! New suite: org.eclipse.m2m.tests.qvt.oml.ant.qvtantcompilertests 23
Example of Eclipse QVTO JUnit results! For complex model transformation workflows deployed in a continuous integration environment, the new QVTO Junit result files are very useful Caveat: Junit.org does not define the XML Schema for Junit results Workaround: Defined an Ecore model of Eclipse JDT JUnit result format 24
JPLʼs improvements to Eclipse QVTO (Helios)! 1. Expose the QVTO compiler separately from the QVTO interpreter Invoke the compiler from the QVTO Editor Cancellable QVTO compilation from the QVTO Builder Separate compilation vs. execution in QVTO Ant tasks 2. Performance improvements via selective Google Guava caching optimization QVT Operational provides powerful mechanisms for extending types & operations Eclipse QVTO 3.1 uses Eclipse OCL 3.0 for type resolution, lookup & reflection Eclipse OCL & EMF APIs are inefficient for extensive type lookup & reflection 3. Continuous integration support via JUnit result files QVTO Compilation JUnit result includes all compilation errors (if any) QVTO Execution JUnit result includes all exceptions & assertion violations (if any) 4. OASIS XML Catalog-aware EMF ResourceSet Protégé/OWL already uses OASIS XML Catalogs for redirecting ontology URIs QVTO s Catalog-aware ResourceSet does the same for redirecting Resource URIs 5. Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) 25
Finding Models & Transformations! /org.eclipse.m2m.tests.qvt.oml/buildertestdata/simpleuml_to_rdb.qvto Eclipse provides several mechanisms, including: Eclipse QVTO defines Eclipse extension points for registering metamodels Eclipse EMF defines several extension points for URI mapping, URI handling, etc At JPL, we have integrated OWL2 ontologies with MOF2 models Ontologies are models too! Protégé OWL uses OASIS XML Catalogs for mapping ontology URIs Key idea: use the same mechanism (OASIS XML Catalog) for ontologies & models 26
OASIS XML Catalogs vs. EMF URI Converters! ExtensibleURIConverterImpl is an implementation What if a ResourceSet has a different URIConverter??? 27
JPLʼs improvements to Eclipse QVTO (Helios)! 1. Expose the QVTO compiler separately from the QVTO interpreter Invoke the compiler from the QVTO Editor Cancellable QVTO compilation from the QVTO Builder Separate compilation vs. execution in QVTO Ant tasks 2. Performance improvements via selective Google Guava caching optimization QVT Operational provides powerful mechanisms for extending types & operations Eclipse QVTO 3.1 uses Eclipse OCL 3.0 for type resolution, lookup & reflection Eclipse OCL & EMF APIs are inefficient for extensive type lookup & reflection 3. Continuous integration support via JUnit result files QVTO Compilation JUnit result includes all compilation errors (if any) QVTO Execution JUnit result includes all exceptions & assertion violations (if any) 4. OASIS XML Catalog-aware EMF ResourceSet Protégé/OWL already uses OASIS XML Catalogs for redirecting ontology URIs QVTO s Catalog-aware ResourceSet does the same for redirecting Resource URIs 5. Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) 28
Conversion of YAML hashes to QVTO Dict(String,? extends OclAny) QVTO Blackbox library for: Loading a YAML file from a URL as a QVTO Dict(String, OclAny) OCL-like conversions: oclasyamldictkind() oclasyamldicttype() convert values according to a Dict() type signature oclasyamlsequence() convert values according to a List() type signature 29
Questions?! Contact: Nicolas.F.Rouquette@jpl.nasa.gov 30