High Performance XML Data Retrieval

Transcription

1 High Performance XML Data Retrieval Mark V. Scardina Jinyu Wang Group Product Manager & XML Evangelist Oracle Corporation Senior Product Manager Oracle Corporation

2 Agenda Why XPath for Data Retrieval? Current XML Data Retrieval Strategies and Issues High Performance XPath Requirements Design of Extractor for XPath Extractor Use Cases

3 Why XPath for Data Retrieval? W3C Standard for XML Document Navigation since 2001 Support for XML Schema Data Types in 2.0 Support for Functions and Operators in 2.0 Underlies XSLT, XQuery, DOM, XForms, XPointer

4 Current Standards-based Data Retrieval Strategies Document Object Model (DOM) Parsing Simple API for XML Parsing (SAX) Java API for XML Parsing (JAXP) Streaming API for XML Parsing (StAX)

5 Data Retrieval Using DOM Parsing Advantages Dynamic random access to entire document Supports XPath 1.0 Disadvantages DOM In-memory footprint up to 10x doc size No planned support for XPath 2.0 Redundant node traversals for multiple XPaths

6 DOM-based XPath Data Retrieval A /A/B/C 2 /A/B/C/D B F B 2 1 C 2 C E F D

7 Data Retrieval using SAX/StAX Parsing Advantages Stream-based processing for managed memory Broadcast events for multicasting (SAX) Pull parsing model for ease of programming and control (StAX) Disadvantages No maintenance of hierarchical structure No XPath Support either 1.0 or 2.0

8 High Performance Requirements Retrieve XML data with managed memory resources Support for documents of all sizes Handle multiple XPaths with minimum node traversals Support DTD and Schema-based XML documents

9 Extractor for XPath Stream-based processing utilizing SAX Support for DTDs and XML Schemas Implements Publish/Subscribe model for scalability Handles multiple XPaths simultaneously Supports XPath 1.0; extended to 2.0

10 Extractor s Publish/Subscribe Processing Model XPath Expressions &Content Handlers SAX Extractor Outputs

11 Extractor s Function Blocks Initialization: registration of XPaths/Handlers XPath Compilation: compiles and builds index graphs XPath Tracking: maintains XPath state and matches doc XPaths with the indexed XPaths Output: sends matching XPath start/stop events along with the XML data

12 Extractor s Function Blocks XPath Expressions & Content Handlers Initialization Register XPath expressions and the instances of content handlers Output XMLSequenceBuilder XML Sequences SAX DTD/XSD Compile XPath expressions and build Index graph XMLSAXSerializer Files XML SAX Track XPath and perform XPath matching when receiving SAX events SAX Registered Handler A Registered Handler Z Extractor Runtime

13 Initialization Registration of absolute XPaths Support for XML Namespaces for differentiation Registration of execution handlers using XContentHandler() Built-in Handlers for ease of use XMLSequenceBuilder() XMLSAXSerializer()

14 XPath Compilation XPath streamability evaluation Streamable isall=true/false option True: Process only streamable XPaths False: Buffer data as needed Build XPath Predicate Table Build Index Tree XPath Dependency Tree (w/o DTD/XSD) Data Model Tree (w/ DTD/XSD) using existing Validation engine

15 Compilation of Each XPath Predicate Predicate XPath Evaluation Attributes Status Related to Predicate XPath -1 Value TRUE Location XPath -1 Predicate XPath -2 Predicate XPath-3 FALSE Predicate XPath-4 Value Pending Predicate Table (not-matched, matched, yet-to-be-matched) Location XPath -n Related to Predicate XPath Evaluation Attributes Status Predicate XPath -1 Value TRUE Predicate XPath -2 Predicate XPath-3 FALSE Predicate XPath-4 Value Pending Predicate Table (not-matched, matched, yet-to-be-matched) Also Used for isall = True condition

16 Runtime XPath Matching State Machine tokenizes and tracks In-scope Namespaces Current Element Name Current Element Attributes Node Position Relative to Siblings Number of Child Elements Implemented as a Stack

17 XPath Index Tree (w/o DTD/XSD) / X Fake Node for /* X Fake Node for // A X 1 /A/B/C B P /A/B/C/D /A/B//P C 1 X X 4 //P 5 //P/*/Q D 2 P 3 Q 5 XPath Expressions XPath Dependency Tree

18 Dependency Tree Traversal 1 /A/B/C 2 /A/B/C/D 3 4 /A/B//P //P 5 //P/*/Q X Fake Node for /* X Fake Node for // / P 3 4 P X'' X' A X'' D C B 2 D X'' X' 1 C B X'' X' X'' C 1 B X' P 4 X''' A XPath Stack A X'' Matched Node D 2 3 P XPath Dependency Tree Q 5

19 Synchronous Data Model Traversal A A B F B 2 B F B C 3 C E C /A/B/C 3 C E F D F D /A/B/C/D XML Document Tree XML Data Model

20 Extractor Output XContentHandler() Execution of Registered Content Handlers XMLSequenceBuilder() Built-in Handler Presents Result Set as XMLSequence Object Contains a Linked List of XMLItems XMLSAXSerializer Built-in Handler Serializes output to Printwriter or OutputStream

21 Extractor Use Cases Content Management Web-Services XSLT/XQuery Implementation

22 Extractor Content Management Use Case Retrieve XPath Expressions XML Pre-Process Initialize system/public ID or XML Schema Location URL XPath Expressions: /a/b JDBC DTD/XML Schema: XPaths Table XML SAX Parser Extractor Registered Handler A... Registered Handler Y JDBC MetaData Tables Registered Handler Z Connection Pool JDBC Doc Table Process Data using Content Handlers Oracle Database

23 Extractor Web Service Use Case Web Services Web Services "invoke" WS SOAP WS response SOAP Service Broker Web Service SOAP Client Register XPath/ Content Handlers XML Register Data Subscription using XPath Initialize Client 1... SAX Parser Extractor Client 2 Client 3 Clients Subscribing and Receiving the Data

24 Extractor XSLT/XQuery Use Case XSLT XQuery Resolving XPath XPath Bridge XMLSequnceObject Register XPath XML SAX Parser Extractor XML XMLSequenceBuilder

25 Oracle XML Resources Oracle Technology Network Downloads, Demos, Samples, Papers XML Support Forum Oracle Database 10g XML & SQL Design, Build, & Manage XML Applications in Java, C, C++, & PL/SQL Covers all of Oracle XML technology BetaBook Forum on OTN Available in May from Bookstores

26