Schema Advisor for Hybrid Relational/XML DBMS

Transcription

1 IBM Research Schema Advisor for Hybrid Relational/XML DBMS Mirella Moro, Universidade Federal do Rio Grande do Sul Lipyeow Lim, IBM T J Watson Yuan-chi Chang, IBM T J Watson

2 MOTIVATION - DB2 Pure XML DB2 stores XML in parsed hierarchical format CREATE TABLE dept (deptid char(8),, deptdoc xml); deptid deptdoc Relational columns are stored in relational format (tables) PR27 <dept> <emp></emp> </dept> DB2 Storage XML is stored natively as type-annotated trees 2 Hybrid Relational/XML DB Design - Mirella M Moro

3 Financial Company M <Message xmlns=" xmlns:fpml=" xmlns:aur=" xmlns:xsi=" version="0-1" xsi:schemalocation=" <header> <dtcc> <activity>new</activity> <status>submit</status> <transactiontype>trade</transactiontype> </dtcc> </header> <fpmlextensions id="trade_ "> <trade> <tradeid tradeidscheme=" </tradeId>... </trade> </fpmlextensions> <FpML version="4-1" xsi:type="requesttradeconfirmation" xmlns=" <header> <messageid messageidscheme=" MLGCD148718</messageId>... </header> <trade>... </trade> <party id="party_cpty_7837">... </party> <party id="party_book_18589">... </party> </FpML> </Message> Trades financial derivatives Derivatives contract represented in FpML (Financial products Markup Language) for inter-company messaging For persistance, the FpML is augmented with proprietary schema FpML schema changes every few weeks Internal proprietary extension Stores coarse-grain state info Has updates Internal proprietary extension Replicates part of FpML message Stores operational metadata eg. Who updated the trade etc. Has updates Inter-company FpML message versioned, no updates Certain fields heavily queried by GUI apps 3 Hybrid Relational/XML DB Design - Mirella M Moro

4 The Question Is How do we design the database schema for company M that leverages both relational & XML capabilities of modern DBMS? Logical Data Model Relational-XML Schema Advisor XML data Relational data Relational & XML data??? R-X Hybrid Database Schema 4 Hybrid Relational/XML DB Design - Mirella M Moro

5 What affects the schema design? Granularity of access/reuse Business artifacts grouping of data elements that are accessed as a single unit. Eg. PO, contracts, forms XML messages grouping of data elements into messages that are sent/received Schema variability A lot of flexibility in structure of the data. Eg. Sparse data, Optional attributes, composite fields Schema evolution Structure of data changes over time Eg. FpML format changes every 6 months Data versioning Content changes over time, but changes needs to be tracked. If no explicit DBMS support, then versioning IDs need to be included in schema Performance criteria Depends on performance characteristics of DBMS Relational columns more high performing than XML CLOBs, BLOBs, & native. Usability How easy it is to write the queries on the resultant schema? Storage redundancy normalization 5 Hybrid Relational/XML DB Design - Mirella M Moro

6 Schema Variability E-catalog: product (eg. department stores sell from tshirts to TVs) Table + columns designs PROD (id, price, size, color, fabric, weight, screensize, stereo ) PROD (id, price) TSHIRT(size, color, fabric, FK to PROD) TV(weight, screensize, stereo, FK to PROD) PROD (id, attribname, attribvalue) PROD (id, price, XMLdescription) required sparse, optional Flat model Simple, NULL Categories Complex, No NULL Join Vertical Simple, joins XML Simple, flexible, no null, no joins 6 Hybrid Relational/XML DB Design - Mirella M Moro

7 Schema Evolution Consider financial company persisting FpML messages. Current solutions shred the XML data to relational tables When FpML.org releases a new schema version A new set of relational tables for the shredded XML Re-shred the existing FpML in the DBMS FpML v4.1 Schema evolves FpML v4.1+v4.2 Evolving relational schemas is expensive! 7 Hybrid Relational/XML DB Design - Mirella M Moro

8 ReXSA : Relational lational-xml Schema Advisor Input : a logical data model annotated with information on granularity, schema variability and evolution, versioning, performance criteria. Outputs: candidate relational-xml schema designs Overview: 2 phases Phase 1 analyses the annotated LDM to partition entities into relational or XML types Phase 2 transforms the partitioned LDM into table definitions and/or XML schemas. DDL LDM analysis R-X Partitioning transform XSD annotated 8 Hybrid Relational/XML DB Design - Mirella M Moro

9 Logical Data Model Use extended entity-relationship model (UML will also work) Entities, relationships, attributes, hierarchies. require d attribut optional e attribut e KEY multivalue d attribute Person composite attribute relationship R1 entity Dept recursive relationship multi R5 relationship R3 R4 Classes weak-entity identifying relationship Dependents R2 Faculty hierarchy Lecturer Professor Undergra d Student Graduate Applicant RefLetter From: Q1: Q2: Q3: Q4: business object or document Master PhD 9 Hybrid Relational/XML DB Design - Mirella M Moro

10 Relational-XML Partitioning Which is relational? Which is XML? LDM analysis R-X Partitioning Entities that benefit from XML: Optional attributes Multi-valued attributes. Eg. Phone number (h/c/o) Composite attributes. Eg. Name : first, mi, last Weak entities Business artifacts. Eg. reference letters Frequently evolving schema Entities that benefit from relational Rigid & stable schema Performance critical elements FOR EACH ENTITY 1. Compute a score of the entity based on flexibility 2. Label entity as Relational or XML based on user-specified threshold. 10 Hybrid Relational/XML DB Design - Mirella M Moro

11 Score example DOB phone SSN ID Name Resume Name Contact Education PhD Masters Bachelor Publications Professional Actvs marital status Person first mi last R2 R1 Dependents Dept offers R3 R4 location requisite R5 Classes FOR EACH ENTITY: Person Required: SSN, DOB, ID Optional: marital status Multi-valued: phone Composed: name (first, mi, last) Document(s): resume initial suggestion Score (Person): (1 op + 1 mlt + 3 cmp + 1 doc) = 55% 9 total 11 Hybrid Relational/XML DB Design - Mirella M Moro

12 Transforming Partitioned LDM to Schema R-X Partitioning transform DDL XSD Entities in LDM have been labeled as relational or XML Transform LDM to database schema Examine entities, relationships Hierarchies can be tricky Read the paper for transformation rules A few examples presented next foreach entity 1. transform entity to table definition and/or XML schema foreach relationship 1. transform relationship to table definition or modify table for entity 2. add key constraints foreach hierarchy 1. transform entity to table definition and/or XML schema 2. add key constraints 12 Hybrid Relational/XML DB Design - Mirella M Moro

13 Transforming Entities Resume Name Contact Education PhD Masters Bachelor Publications Professional Actvs SSN marital status DOB ID Person phone Name first mi last Pure Relational : PERSON ID int PK NOT NULL SSN varchar NOT NULL DOB date NOT NULL marst char firstn varchar mi char lastn varchar phone FK to phonetable resume FK to resumetable Hybrid 1 : PERSON ID int PK NOT NULL SSN varchar NOT NULL DOB date NOT NULL marst char name XML TYPE phone XML TYPE resume XML TYPE Hybrid 2 : PERSON ID int PK NOT NULL SSN varchar NOT NULL DOB date NOT NULL Info XML TYPE Hybrid 3 : PERSON ID int PK NOT NULL Info XML TYPE 13 Hybrid Relational/XML DB Design - Mirella M Moro

14 Transforming Relationships 14 Hybrid Relational/XML DB Design - Mirella M Moro

15 Transforming Relationships: : Example exml (0..1) 1 R1 erel Table specification erel (atrib 0 datatype 0, atrib 1 datatype 1,, exml XMLtype) exml N 1 R1 erel Read only erel (atrib 0 datatype 0, atrib 1 datatype 1,, exml XMLtype) -- concatenate erel (atrib 0 datatype 0,, atrib n datatype n ) Updates exml(atrib 0 datatype 0, atrib 1 datatype 1, xmldata XMLtype) Query workload 15 Hybrid Relational/XML DB Design - Mirella M Moro

16 Transforming Hierarchies and Inheritance Resume Name Contact Education PhD Masters Bachelor Publications Professional Actvs SSN marital status DOB ID Person phone Name first mi last R1 Dept Dependents R2 Faculty Student GradDate 1 st Quarter Defense Date Lecturer Professor Undergrad Graduate Master PhD R3 Thesis Text Title Area Keywords Abstract Chapters 16 Hybrid Relational/XML DB Design - Mirella M Moro

17 Hierarchies : Relational Schemas Faculty Person Student (A) PERSON (id, SSN, DOB, marstatus, firstn, mi, lastn, phone FK) FACULTY (id, personid FK, resume FK) STUDENT (id, personid FK, graddate, firstquarter) (B) (total/disjoint inheritance: each person must be either faculty or student) FACULTY (id, SSN, DOB, marstatus, firstn, mi, lastn, phone FK, resume FK) STUDENT (id, SSN, DOB, marstatus, firstn, mi, lastn, phone FK, graddate, firstquarter) (disjoint inheritance: each person is either faculty or student + not many specialized attributes) (C) PERSON (id, SSN, DOB, marstatus, firstn, mi, lastn, phone FK, type, resume FK, graddate, firstquarter) (D) (overlapping inheritance: each person may be faculty, student, or both) PERSON (id, SSN, DOB, marstatus, firstn, mi, lastn, phone FK, Fflag, resume FK, Sflag, graddate, firstquarter) 17 Hybrid Relational/XML DB Design - Mirella M Moro

18 Hierarchies: Example 1 SSN 1 st Quarter GradDate marital status DOB ID Person Student Graduate PhD phone Name first mi last R1 R3 Dept Thesis Defense Date Text Title Area Keywords Abstract Chapters <Person id= 3d01 dept= dept001 > <SSN></SSN> <DOB></DOB> <maritalst></maritalst> <name> <first></first> <last></last> </name> <phones> <phone></phone> <phone></phone> <phone></phone> </phones> <Student> <firstquarter></firstquarter> <graddate></graddate> <Graduate> <Thesis> <DefenseDate> </DefenseDate> <Text> <Title></Title> </Text> </Thesis> <PHD> </PHD> </Graduate> </Student> </Person> 18 Hybrid Relational/XML DB Design - Mirella M Moro

19 Hierarchies : Example 2 Superclass relationship SSN marital status DOB ID phone Name first mi last <Person id= 3d01 dept= dept001 > PERSON (id, info XML, dept FK) DEPT (did, ) Person R1 Dept Subclass relationship 1 st Quarter GradDate Student Graduate PhD R3 Thesis Defense Date Text Title Area Keywords Abstract Chapters <Person > <Graduate> <Thesis> PERSON (id, info XML) THESIS (tid,, personid FK) PERSON (id, info XML) THESIS (tid,, personid FK, path) 19 Hybrid Relational/XML DB Design - Mirella M Moro

20 Artificial Usecase Artificial academic LDM Set R-X partitioning threshold to 70% DDLs look very reasonable 20 Hybrid Relational/XML DB Design - Mirella M Moro

21 HL7 Reference Information Model Health Level 7 (HL7) is an XML messaging format for healthcare industry. HL7 provides a conceptual model called the Reference Information Model (RIM) ReXSA suggests one table with an XML column Reasonable because all entities in RIM inherits from a single super-entity. 21 Hybrid Relational/XML DB Design - Mirella M Moro

22 Conclusion DBMS users don t really know what to put in XML columns and what to put in relational columns. Designing hybrid relational-xml schemas is a problem that has not been addressed before. We presented a schema design advisor that takes an annotated logical data model as input and outputs candidate schema design(s) Future work Design evaluation. Is the resultant design a good design? Incorporate data samples in the analysis Integration with performance (index, MQT) advisors 22 Hybrid Relational/XML DB Design - Mirella M Moro

23 QUESTIONS? Contacts 23 Hybrid Relational/XML DB Design - Mirella M Moro