Wee Keong Ng. Web Data Management. A Warehouse Approach. With 106 Illustrations. Springer

Transcription

1 Sourav S. Bhowmick Wee Keong Ng Sanjay K. Madria Web Data Management A Warehouse Approach With 106 Illustrations Springer

2 Preface vii 1 Introduction Motivation Problems with Web Data Limitations of Search Engines Limitations of Traditional Data Warehouse Warehousing the Web Architecture and Functionalities Scope of This Book Research Issues Contributions of the Book 15 2 A Survey of Web Data Management Systems Web Query Systems Search Engines Metasearch Engines W3QS WebSQL WebLog NetQL FLORID RAW Web Information Integration Systems Information Manifold TSIMMIS Ariadne WHIRL " Web Data Restructuring STRUDEL WebOQL ' ARANEUS 45

3 2.4 Semistructured Data Lore UnQL XML Query Languages Lorel XML-QL Summary 61 Node and Link Objects Introduction Motivation Our Approach - An Overview of WareHouse Object Model (WHOM) Representing Metadata of Web Documents and Hyperlinks Metadata Associated with HTML and XML Documents Node Metadata Attributes Link Metadata Attributes Representing Structure and Content of Web Documents Issues for Modeling Structure and Content Node Structural Attributes Location Attributes Representing Structure and Content of Hyperlinks Issues for Modeling Hyperlinks Link Structural Attributes Reference Identifier Node and Link Objects Node and Link Structure Trees Recent Approaches in Modeling Web Data Semistructured Data Modeling Web Data Modeling XML Data Modeling Open Hypermedia System Summary 91 Predicates on Node and Link Objects Introduction Features of Predicate Overview of Predicates Components of Comparison-Free Predicates Attribute Path Expressions Predicate Qualifier Value of a Comparison-Free Predicate Predicate Operators Comparison Predicates Components of a Comparison Predicate Types of Comparison Predicates 117

4 xvii 4.4 Summary 125 Imposing Constraints on Hyperlink Structures Introduction Overview Difficulties in Modeling Connectivities Features of Connectivities Components of Connectivities Source and Target Identifiers Link Path Expressions Types of Connectivities Simple Connectivities Complex Connectivities Transformation of Complex Connectivities Transformation of Case Transformation of Case Transformation of Case Transformation of Case Steps for Transformation Graphical Visualization of a Connectivity Conformity Conditions Simple Connectivities Complex Connectivities Summary 142 Query Mechanism for the Web Introduction Motivation Our Approach Coupling Query The Information Space Components Definition of Coupling Query Types of Coupling Query Valid Canonical Coupling Query Examples of Coupling Queries Noncanonical Coupling Query Canonical Coupling Query Valid Canonical Query Generation Outline Phase 1: Coupling Query Reduction Phase 2: Validity Checking Coupling Query Formulation Definition of Coupling Graph Types of Coupling Graph Limitations of Coupling Graphs 194

5 xviii Contents Hybrid Graph Coupling Query Results Computability of Valid Coupling Queries Browser and Browse/Search Coupling Queries Recent Approaches for Querying the Web Summary Schemas for Warehouse Data Preliminaries Recent Approaches for Modeling Schema for Web Data Features of Our Web Schema Summary of Our Methodology Importance of Web Schema in a Web Warehouse Web Schema Definition Types of Web Schema Schema Conformity Web Table Generation of Simple Web Schema Set from Coupling Query Phase 1: Valid Canonical Coupling Query to Schema Transformation Schema from Query Containing Schema-Independent Predicates Schema from Query Containing Schema-Influencing Predicates Phase 2: Complex Schema Decomposition Motivation Discussion Limitations Phase 3: Schema Pruning Motivation Classifications of Simple Schemas Schema Pruning Process Phase 1: Preprocessing Phase Phase 2: Matching Phase Phase 3: Nonoverlapping Partitioning Phase Algorithm Schema Generator Pruning Ratio Algorithm of GenerateSchemaFromQuery Algorithm for the Construct Partition Web Schema Generation in Local Operations Schema Generation Phase Schema Pruning Phase Summary 249

6 xix WHOM-Algebra Types of Manipulation Global Web Coupling Definition Global Web Coupling Operation Web Tuples Generation Phase Limitations Web Select Selection Criteria Web Select Operator Simple Web Schema Set Selection Schema Selection Condition Conformity Select Table Generation Web Project Definition Projection Attributes Algorithm for Web Project Web Distinct Web Cartesian Product Web Join Motivation and Overview Concept of Web Join Join Existence Phase Join Construction Phase When X pj ^ Joined Partition Pruning Join Construction Phase When Xj = Derivatives of Web Join a-web Join Outer Web Join Web Union Summary 351 Web Data Visualization Web Data Visualization Operators Web Nest Web Unnest Web Coalesce Web Expand Web Pack Web Unpack Web Sort Summary 365

7 xx Contents 10 Detecting and Representing Relevant Web Deltas Introduction Overview Related Work Change Detection Problem Problem Definition Types of Changes Representing Changes Decomposition of Change Detection Problem Generating Delta Web Tables Storage of Web Objects Outline of the Algorithm Algorithm Delta Conclusions and Future Work Knowledge Discovery Using Web Bags Introduction Motivation Overview Related Work PageRank Mutual Reinforcement Approach Rafiei and Mendelzon's Approach SALSA Approach of Borodin et al Concept of Web Bag Knowledge Discovery Using Web Bags Terminology Visibility of Web Documents and Intersite Connectivity Luminosity of Web Documents Luminous Paths Query Language Design Considerations Query Language for Knowledge Discovery Conclusions and Future Work The Road Ahead Summary of the Book Contributions of the Book Extending Coupling Queries and Global Web Coupling Operation Optimizing Size of Simple Schema Set Extension of the Web Algebra Schema Operators Web Correlate Web Ranking Operator Operators for Manipulation at Subpage Level Maintenance of the Web Warehouse 425

8 12.7 Retrieving and Manipulating Data from the Hidden Web Data Mining in the Web Warehouse Conclusions 427 A Table of Symbols 429 B Regular Expressions in Comparison-Free Predicate Values 431 C Examples of Comparison-Free Predicates 436 D Examples of Comparison Operators 443 E Nodes and Links 445 References 449 Index 459 xxi