Data Integration of Bioinformatics and Web-Based Software Development

Transcription

1 Integration of Biological XML data Ph. D. Lecture Bioinformatics & Software Systems Lab. Woo-Hyuk Jang Information and Communications Univ. Where are we? Client-Side Info. Management Business related Issues WWW Concepts & Webbased Info. Management HTML, JavaScript, Plugin, Applet XML & XML Processing Internationalization and Privacy Web Services Semantic Web App. Of Webbased tech. S.S. Info. Management Concept CGI, Java Servlets JDBC, MySQL This Lecture? Server-Side Info. Management ICE Web-Based Software Development, Summer

2 Can you remember? Problems in Integrating Heterogeneous Information - Heterogeneity of formats, data types, units, or semantics. Information Mediation Fig 1. Mediator in Lecture 7. ICE Web-Based Software Development, Summer This Lecture Contains Information Integration in Bioinformatics - Bioinformatics Overview Is there any relationship between Web and Bioinformatics? - Difficulties to handle Biological XML data - What is it? Why? Cultures : Schema-driven, Data-driven Models : Federation, Warehousing, Mediation Integration of XML format data - Problems - Issues Summary ICE Web-Based Software Development, Summer

3 Bioinformatics A narrow sense - The application of information technology to life science research Modeling (abstraction) Analysis and collection Data integration and information retrieval - Enables the discovery and analysis of biomolecules and their properties (Structure, function, interactions) A wide sense - The use of computers to collect, analyze, and interpret biological information at the molecular level ICE Web-Based Software Development, Summer Web and Bioinformatics Use Biological Data Bio Applications Make, Publish Use Experiment, Publish Biologist Computer Scientist ICE Web-Based Software Development, Summer

4 Difficulties to Handle Biological XML data Lack of standard - Different data model and schemas - Different handling methods are needed - Different formats Monstrous volume of data - It is growing exponentially - Data are updated very frequently Newly introduced data, error fixed data ICE Web-Based Software Development, Summer Why Integration? In the post-human genome sequencing era, many analyses on the genome scale are possible Majority of human diseases are the product of multi-step pathophysiological processes The biggest challenge in interpreting the results of these analyses lies in the data integration problem ICE Web-Based Software Development, Summer

5 Two Cultures of Integration Database Integration - Schema level view - Focus on outside of data Schema 3 Schema 4 Schema 1 Schema 2 Data Integration - Data level view - Focus on inside of data Data 1 Data 2 Data 3 Data 4 ICE Web-Based Software Development, Summer Two Cultures of Integration Schema-driven (computer scientists) - Much smaller than data, (hopefully) well-defined elements - Resolve redundancy and heterogeneity at the schema level - High degree of automation once system is set-up - Focus on methods - you rarely publish a data paper Data-driven (biologists) - Value is in the data, abstraction is a result of analysis - Don t bother with schemas Abstraction is volatile and depends on experimental technique - Manual integration at data level, constant high effort - You rarely publish a (database) method paper ICE Web-Based Software Development, Summer

6 Models of Integration Federation (Multi-database) Warehousing (Materialized in house) Mediation (Virtual integration) ICE Web-Based Software Development, Summer Models of Integration Federation (Multi-database) - K2/BioKleisli, Entrez FDBMS FDBMS Component DB 1 Component DB 2 Component DB n Component DB 1 Component DB 2 Component DB n (Centralised DBMS) (Centralised DBMS) (Distributed DBMS) (Distributed DBMS) (another FDBMS) (another FDBMS) Comp DB 1 Comp DB 2.1 Comp DB 2.2 Comp DB 1 Comp DB 2.1 Comp DB 2.2 ICE Web-Based Software Development, Summer

7 Models of Integration Warehousing (Materialized in house) - GUS (Genome Unified Schema), SRS (Sequence Retrieval System) Network Internet Local Operational Integration & Storage R2 R3 Warehouse Decision Support & Mining ICE Web-Based Software Development, Summer Models of Integration Mediation (Virtual integration) - TAMBIS (Transparent Access to Multiple Bioinformatics Information Source) Network Internet Query Translation Mediator ICE Web-Based Software Development, Summer

8 Models of Integration Federation represents a more static approach using agreed couplings to allow view creation. Warehousing and Mediation addresses integration in a more dynamic way using extraction, transformation and integration processes. ICE Web-Based Software Development, Summer Warehousing vs. Mediation Warehouse - Update-driven: i.e. in warehouse repository - Heterogeneous data is integrated in advance and stored inhouse for direct query and analysis. Mediation - Wrapper and Mediator layer on top of source DBs. - Query-driven: Query to mediated schema then translated into queries appropriate to sources. - Results integrated into a global answer set. ICE Web-Based Software Development, Summer

9 Now let s study the Information Integration in Bioinformatics - Bioinformatics Overview Is there any relationship between Web and Bioinformatics? - Difficulties to handle Biological XML data - Why Integration? Cultures : Schema-driven, Data-driven Models : Federation, Warehousing, Mediation Integration of XML format data - Problems - Issues Discussion about Reading Question #6 ICE Web-Based Software Development, Summer Integration of XML format data Why XML? - Biology is a complex discipline - Wide variety of data resources and repositories No standard protocol exists to interrogate biological data stores No standard data format exists to exchange biological data. No standard data model exists. - Difficulties in using and exchanging data There exist various tools that can support XML handling ICE Web-Based Software Development, Summer

10 Integration of XML format data Problems - We focus on schema-driven integration - Warehousing model is efficient Have to analyze data Performance To implement perfect mediation model is extremely difficult - XML data should be converted into RDB - We want to make our own DB schema accommodating the data from XML files - We need to make the DB schema regarding efficiency and our own purpose - Heterogeneity and Large scale ICE Web-Based Software Development, Summer Integration of XML format data PreSPI (Prediction System for Protein Interaction) Warehouse Integration Rule Local DB1 Local DB2 Local DB3 Local General XML Wrapper (SAX) Web XML XML XML XML XML XML XML XML Sequence Structure Function ٠ ٠ ٠ Domain ICE Web-Based Software Development, Summer

11 Issues of Using XML Biological data Structure - Semi-structured: Can be expressed as trees, graphs - Theoretically, it is ideal to map them into DB regarding structural feature Method for storing XML - File system Has overhead for query Text file, invert list, compression file - Specific storing method Use XML s own structure - DB system Especially, mapping into RDB has been researched a lot Has overhead for converting into the appropriate model ICE Web-Based Software Development, Summer Issues of Using XML Biological data Object view of the XML use DOM A Class can be mapped into a Table, PCDATA or ATTRIBUTE can be column XML Objects Tables ============= ============ ============== Table A <A> object A { <B>bbb</B> B = "bbb" B C D <C>ccc</C> <=> C = "ccc" <=> <D>ddd</D> D = "ddd" </A> } bbb ccc ddd XML-view CREATE XMLVIEW xview_1( id char(20), char (30) ) AS ( select p.personnel.person@id, p.personnel.person@ from file:/home/user1/personal.xml, p; ); A generic load/extract utility for data transfer between XML documents and relational databases Bourret, R.; Bornhovd, C.; Buchmann, A.;Advanced Issues of E-Commerce and Web-Based Information Systems, ICE Web-Based Software Development, Summer

12 Issues of Using XML Biological data Direct method XML Document input XML Saver Output & execute Insert Statement input Mapping Rule A direct method of data exchange between XML and relational database Bei Jia; Cai Fei; Tao Lie-Jun; Pan Jin-Gui; Information Technology Interfaces, th International Conference on 2004 Page(s): Vol.1 ICE Web-Based Software Development, Summer Issues of Using XML Biological data Direct Method (cont d) ICE Web-Based Software Development, Summer

13 Issues of Using XML Biological data Current methods force DB to follow XML schema ID ID_A ID_A ID_A ID_A DB B C D E ID ID_B ID_C ID_D ID_E Rather than ID ID_A NAME PROTEIN_A B ID_B C ID_C D ID_D E ID_E Complex structured XML - Share the same element name even thought they should be different columns in DB (DIP, InterPro ) <protein id= ID_A" name= PROTEIN_A > <ref db= B" id= ID_B" /> <ref db= C" id= ID_C" />..... B ID_B C ID_B... Large size of file; we cannot use DOM XML updated frequently; the process should be easy ICE Web-Based Software Development, Summer Issues of Using XML Biological data Direct Method cannot cover following XML type <node id="g:1" uid="dip:232n" name="baxa_human" class="protein"> <xref db="dip" id="232n" type="src"/> <feature name="swp_ref" class="cref"> <src>swissprot</src> <val>swp:q07812</val> <xref db="swp" id="q07812" type="src"/> We want </feature> <feature name="pir_ref" class="cref"> <src>pir</src> <val>pir:a47538</val> <xref db="pir" id="a47538" type="src"/> </feature> <feature name="gi_ref" class="cref"> <src>ncbi</src> <val>gi:539664</val> <xref db="gi" id="539664" type="src"/> </feature> <att name="descr"> ID G:1 NAME BAXA_HUMAN But, ID G:1 <val>bcl-2-associated protein x, alpha splice form</val> </att> G:1 <att name="organism"> <val>homo sapiens</val> G:1 <xref db="txid" id="9606" type="ont"/> </att> G:1 </node> DIP_ID DIP:232N DB DIP SWP PIR gi SWP_ID PIR_ID Q07812 A47538 Ref_ID DIP:232N Q07812 A GI_ID Cannot integrate two more files ; Needs constraint ICE Web-Based Software Development, Summer

14 Issues of Using XML Biological data Make a data set for a tuple, which ignore sub document tree nodes Define SQL like syntax - Where condition of each column for constraints - Multiple files can be populated into one table by manipulation CREATE TABLE PROTEIN_IDs(ID_A CHAR(20), NAME CHAR(20), B CHAR(20), C CHAR(20), D CHAR(20), E CHAR(20) ) AS ( SELECT ( FILE.protein@id, FILE.protein@name, [FILE.protein.ref]@id = B, [FILE.protein.ref]@id = C, [FILE.protein.ref]@id = D, [FILE.protein.ref]@id = E, [FILE_2.ELEMENT]@value ) FROM file/protein.xml AS FILE, file/file.xml AS FILE_2); ICE Web-Based Software Development, Summer Summary Integration of biological data is a kind of Web based information management Integration in bioinformatics is a very important work because we can find out more valuable biological information via comprehensive view Biological XML data have some properties that disturb integration, so schema-driven and warehousing model are usually used for integration Thank you~~~ ICE Web-Based Software Development, Summer