Databases and Information Systems 1 7b Motivation of XML databases and XML compression

Transcription

1 Databases and Information Systems 1 7b Motivation of XML databases and XML compression Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

2 XML extensions of relational databases data are already stored in relational database (e.g. Oracle) extra datatype generates XML: select <kunde name =.name > <auftrag>.auftrag </auftrag> <adresse>.adresse </adresse> </kunde> from where name = kunde doc kunde auftrag adresse auftrag adresse name auftrag adresse Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression /

3 XML extensions of relational databases load XML data into relational database requires mapping (defining how to store data in RDB) mapping uses annotations in DTD or XML Schema requires DTD or XML Schema only for strongly structured XML doc name = kunde kunde auftrag adresse auftrag adresse name auftrag adresse Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression /

4 XMLSpalten in relationalen Datenbanken extra columns of type XMLDocument (e.g. Microsoft) + arbitrarily structured XML documente storable + search in imported XML documents possible + XPath queries internally mapped to ordinary queries date orderid XML document regarding the order doc name = kunde kunde auftrag adresse auftrag adresse Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression /

5 XML Speicherung in Datenbanken hybrid XMLdatabase system (e.g. DB Universal Server): DB client application SQL XQuery relational interface XML interface DB Query Engine DB Memory relational XML + relational data in relational database efficient query evaluation in relational part of database + XML data stored separately imported XML documents can be stored + both datenbase parts can be joined in queries efficient search in imported XML documents Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression /

6 XML data compression data source destination XML compress * CXML transport * CXML decompress XML query cost factors: data volume transported or stored response time: data transport time + data processing time goals / benefit: 1. less data transport cheaper, faster. less data storage cheaper. queries faster Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 6

7 Example: transport 10 MByte with Mbit/s data source destination transport XML min 10MB 10MB XML 10MB transport transport min XML 10MB min 10MB 10MB XML 10MB XML 10MB compress * transport CXML CXML min <1 min <1 min * decompress <0MB <0MB <0MB query XML 10MB Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 7

8 Benefit of XML compression Smaller data sets do more likely fit into main memory more XML can be processed fast in main memory + read operation in main memory is times faster than a disk operation applications can be processed much more efficiently run time uncompressed XML compressed XML XML data volume Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 8

9 XML compression techniques overview (1) < knr = 1 > <K 1> drop elements < anr = 7 > <A 7> predefined by <Teil nr = > <T > the structure </Teil> </T> </> </A> element names </> </K> symbols K 1 K K share multiple subtrees (DAGs) A 7 A 8 T T A 9 Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 9

10 XML compression techniques overview () compress texts separatly using String compression efficient XML structure bitstream (succint representation, e.g. Oracle): 1 < knr = 1 > 1., 1 < anr = 7 >., 1 <Teil nr = > Teil., 0 </Teil> Elementnamen nur 1x 0 </> speichern 0 </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 10

11 Leistung von XMLKompressionsverfahren Methode: benötigt DTD Strukturkompression um den Faktor Kompressionsfaktor einschließlich Texten Kompressionszeiten mit gzip für Texte Kompressionszeiten mit bzip für Texte Dekompressionszeiten mit gzip für Texte Dekompressionszeiten mit bzip für Texte AnfragePerformance BSBC no bis 1 Mbit/s 0.8 bis Mbit/s 8 bis 6. Mbit/s. bis 11. Mbit/s schneller als auf unkomprimiertem XML DTD Subtraktion yes bis 8.8 Mbit/s 0.8 bis Mbit/s.8 bis 0 Mbit/s. bis 8.8 Mbit/s gleich schnell wie auf unkomprimiertem XML Zum Vergleich: ADSL bis zu 8 Mbit/s ( ADSL+ bis zu Mbit/s ) Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 11

12 Scalability Test data: publicly available XML data, e.g. DBLP and synthetically generated XML data, e.g. XMark compression times (using a 768MB RAM 100 MHz computer): 10 MB XML min 1 GB XML 18 min GB XML 7 min 10 GB XML 180 min 16% read SAX stream, generate compressed structure 8% compress text and attribute values and compressed data output Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

13 Storage models for XML trees (): a single table for binary trees based on label, firstchild (fc) and nextsibling(ns) <> <></> </> 1 ID 1 Label(ID) fc(id) ns(id) Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

14 Converting binary XML trees into binary DAGs 1 <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

15 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

16 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 16

17 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 17

18 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 18

19 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 19

20 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 0

21 Converting binary XML trees into binary DAGs <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 1

22 Converting binary XML trees into binary DAGs 1 to do: DAG / element list ellist of Lab 6 7 <> <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / ID Label(ID) fc(id) 6 ns(id) 7

23 1. fc. fc Converting SAX into binary DAGs the goal / to do: DAG / element list ellist of Lab. fc. p. ns 9. ns 8. p 6. fc 7. p 7 1. fc <>. fc <></>. ns 8. p </> 9. ns <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / ID 1 6 Label(ID) fc(id) 6 ns(id) 7

24 1. fc. fc. fc Converting SAX into binary DAGs how to do it DAG / element list ellist of Lab 1. fc. fc. fc 1. fc <>. fc. fc <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / ID 1 Label(ID) fc(id) ns(id)

25 1. fc. fc Converting SAX into binary DAGs to do: DAG / element list ellist of Lab. fc. p. p 1. fc <>. fc <></>. ns </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / ID 1 Label(ID) fc(id) ns(id)

26 1. fc. fc Converting SAX into binary DAGs to do: DAG / element list ellist of Lab. fc. p. ns. ns 1. fc <>. fc <></>. ns </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 6 ID 1 Label(ID) fc(id) ns(id)

27 1. fc. fc Converting SAX into binary DAGs to do: DAG / element list ellist of Lab. fc. p. ns 8. p 6. fc 7. p 8. p 7 1. fc <>. fc. ns 8. p <></> </> <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 7 ID 1 6 Label(ID) fc(id) 6 ns(id) 7

28 1. fc Converting SAX into binary DAGs to do: DAG / element list ellist of Lab 9. ns. fc. fc. p. ns 9. ns 8. p 6. fc 7. p 7 1. fc <>. fc <></>. ns 8. p </> 9. ns <> <></> </> Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 8 ID 1 6 Label(ID) fc(id) 6 ns(id) 7

29 A stack for converting SAX into binary DAGs Stack operations fc : push ns : push p: pop everything including the next fc and move it to the DAG 1. fc. fc. ns 8. p. fc. p 6. fc 7. p content of the stack before step 8. p fc fc ns pop this in step 8. p popped this in steps. p and 7. p Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 9

30 Summary: converting SAX into binary DAGs 1. Use a stack to convert SAX events into a binary tree fc: push ns: push p: pop until and including last fc store popped elements in a element list containing the binary tree. look for common subtrees to generate a DAG from element list Databases and Information Systems 1 WS 008 / 09 Prof. Dr. Stefan Böttcher XML compression / 0