Data integration for metagenomics: current status and future plans

Transcription

1 integration for metagenomics: current status and future plans Neil Wipat Computing Science University of Newcastle NERC Microbial Metagenomics

2 Overview metamicrobase Current method of data integration Problems in data and database integration Future possibilities: Semantic data integration using the comparagrid approach

3 Metagenomics Informatics pipeline overview Enviromental Sample Sample Metadata EMBL Submission Clone Library Barcode DB Meta Microbase Clone 1 Clone 2 etc. DB & Cache Sequencing Facility First Pass YAMAP QC

4 Metagenomics Informatics pipeline overview Enviromental Sample Sample Metadata EMBL Submission Clone Library Barcode DB Meta Microbase Clone 1 Clone 2 etc. DB & Cache Sequencing Facility First Pass YAMAP QC

5 MetaMicrobase overview Comparison BlastN BlastP Ssearch Collector MetaGenome Cache Mummer Warehouse Notification server Primary Promer Secondary Metabolic reconstruction Sharkhunt Orfans Secondary analysis Orthologues User defined

6 integration - current approach BlastN Comparison BlastP Ssearch BlastN BlastP results Seqeunce Features Locus tag Accession no. Mummer SSearch Mummer Metabolic reconstruction Promer Sharkhunt Promer Orfans EC numbers Secondary analysis Orfans Orthologues Orthologues User defined Secondary

7 integration - current approach BlastN Comparison BlastP Ssearch BlastN BlastP results Seqeunce Features Locus tag Accession no. Mummer SSearch Mummer Metabolic reconstruction Promer Sharkhunt Promer Orfans EC numbers Secondary analysis Orfans Orthologues Orthologues User defined Secondary

8 integration - user defined tasks cause problems User defined Queries User defined Warehouse mygrid Workflows Secondary

9 integration - user defined tasks cause problems User defined Queries User defined Warehouse Secondary mygrid

10 mygrid workflows don t solve data integration They call Web services, carry out analysis and gather more data, and produce their own data

11 mygrid workflows don t solve data integration We need a way to join it all together...

12 Problems in data and database integration Variation in computational access to data sources Attribute and table names vary between databases (often not descriptive) The use of non-standard terminology or different standard terminologies to describe date entries Mismatch between attribute data types and their formats (domain types). Not all data describing the contents of tables is readily computationally accessible. Some metadata and data is assumed by context.

13 Approaches to data and database integration Variation in computational access to data sources SQL based computational access to databases can be standardised using interfaces such as JDBC, ODBC or OGSA-DAI. Approaches to distributed query processing using such standards have been developed e.g. OGSA-DQP and BioMart These tackle the technical issues about querying over data stored in multiple locations but don t ensure queries are semantically meaningful (you need to know the db schema)

14 Semantic data integration using ComparaGrid Developing Grid-based systems for integrating comparative genomics data EBI, Institute for Food Research, John Innes Centre, Manchester, Newcastle University (Maths & Stats & Computing Science), SCRI A generic architecture for semantic data and database integration Approach:

15 Approach Problems relating to differences in table and attribute names can be addressed by semantically defining a database by mapping tables and attributes to a formal ontology. (attribute semantics, attribute value semantics and table semantics) Ontology base Table org Species organism_tab enz thing spec organism enzyme Bacteria bact_spec_tab ec animal plant bacterium vertebrate invertebrate Vertebrates verteb_tab name ename Difference relating to differences in terminology describing data entries can be overcome by reference to controlled vocabularies/ontologies or by reference to an equivalence table.

16 Approach To resolve issues where table data is assumed by context, tables can be further defined by reference to a term in an ontology (in a similar fashion to attribute names) The reasoning system interprets data from the tables, takes the extra information into account, effectively populating the missing properties. e.g. thing Bacteria bact_spec_tab ec organism enzyme animal plant bacterium vertebrate invertebrate

17 Approach Mismatches between domain types CANNOT (in general) be solved using a semantic integration approach. Mapping functions can be used to identify the mapping between data representations, using pairs of mapping functions e.g. Ensembl InterPro enspep IPEntry InterproID IP: ID IPEntry.ID= TRIM("IP:" enspep.interproid)

18 ComparaGrid Architecture DB 'X' SQL RS Integrator Transformer Wrapper OWL RAW OWL RAW OWL SHARED ComparaGrid ontology JDBC WS/ HTTP all DB'A' DB'Z' DB'Y' DB'X' DB 'Y' SQL RS Transformer Wrapper OWL RAW OWL RAW OWL SHARED ComparaGrid ontology JDBC WS/ HTTP SYNTAX SEMANTICS LOCATION GRAPHICAL USER QUERY AND VISUALISATION CLIENT (PUSSYCAT)

19 MetaMicrobase overview Comparison BlastN BlastP Ssearch Collector MetaGenome Cache Mummer Warehouse Notification server Primary Promer Secondary Metabolic reconstruction Sharkhunt Orfans Secondary analysis Orthologues User defined

20 Very long term aim BlastN BlastP Ssearch Comparison int DB1 ext DB1 ext DB1 ext DB1 MetaGenome Cache Primary Collector Notification server Metabolic reconstruction Mummer Promer Sharkhunt Compara Grid int DB1 int DB1 ext DB1 Orfans int DB1 Secondary analysis Orthologues int DB1 User defined

21 Acknowledgements NERC Metagenomics project ComparaGrid team Microbase team Matt Pocock and James Worthington