My Favorite Issues in Data Warehouse Modeling
|
|
|
- Stuart Goodwin
- 8 years ago
- Views:
Transcription
1 University of Münster My Favorite Issues in Data Warehouse Modeling Jens Lechtenbörger University of Münster & ERCIS, Germany
2 Context Data Warehouse (DW) modeling ETL design DW schema design Database design Methodical process in several phases Focus here: Conceptual schema design DOLAP 2005, November 5 Jens Lechtenbörger 1
3 Outline Context Conceptual Modeling Meaning of Features Multidimensional Normal Forms Schema Versioning Conclusions
4 Conceptual Modeling (1/5) Conceptual representation of multidimensional scenario System- and implementation-independent No standard data model in sight Ad hoc E/R variants Object-oriented, based upon UML Specification of facts structure, i.e., Relevant dimensions and their inner structure ( dimension schema), Measures within their multidimensional contexts ( fact schema) DOLAP 2005, November 5 Jens Lechtenbörger 2
5 Conceptual Modeling (2/5) Fact Schema Transactions Branch BranchID City Region #Transactions Time Day Month Quarter Year CustType: Company Branch Account AccountID CustID CustType Job CustType: Person DOLAP 2005, November 5 Jens Lechtenbörger 3
6 Conceptual Modeling (3/5) Meaning of Fact Schema Universal relation Universal relation schema assumption (URSA): Semantics of attribute tied to its name Defining dimension levels form key Each arc represents functional dependency (FD) DOLAP 2005, November 5 Jens Lechtenbörger 4
7 Conceptual Modeling (4/5) Some Features (Incomplete list) Standard Features Fact schema represents M:N relationship among dimensions Arc in dimension schema represents M:1 relationship, i.e., FD Typical Features (some with challenges for summarizability) M:N relationships among dimension levels (non-strict hierarchies) Alternative and parallel paths, possibly including joining levels Optional levels allowing NULL values (heterogeneous, unbalanced, non-onto hierarchies) DOLAP 2005, November 5 Jens Lechtenbörger 5
8 Conceptual Modeling (5/5) Guidelines A rich set of features is good A set of guidelines for their proper use is even better Let s consider above typical features in turn DOLAP 2005, November 5 Jens Lechtenbörger 6
9 Outline Context Conceptual Modeling Meaning of Features Multidimensional Normal Forms Schema Versioning Conclusions
10 Meaning of Features M:N relationships (1/4) M:N relationships are generally implicitly understood Consider levels Day and City Many cities exist at a given day A city exists for many days There is no need to model this M:N relationship (if we don t do history) DOLAP 2005, November 5 Jens Lechtenbörger 7
11 Meaning of Features M:N relationships (2/4) Consider geographical levels City, Region, State, Country One Region per City, i.e., City Region M:N between Region and State, i.e., Region State One Country per State, i.e., State Country Legal instance City Region State Country ci 1 r 1 s 1 co 1 ci 1 r 1 s 2 co 2 City and State are in M:N relationship. Probably not intended. Different dimension schema needed. All Country State Region City Location DOLAP 2005, November 5 Jens Lechtenbörger 8
12 Meaning of Features M:N relationships (3/4) All Country Implicit M:N relationship No problems with summarizability Region City State Guideline Avoid M:N arcs within dimensions Joint work with Bodo Hüsemann and Gottfried Vossen, DMDW 2000 Synthesize fact schemata Follow FDs to build dimension schemata Side remark: Bridge tables of Kimball et al. arise automatically as fact schemata Location DOLAP 2005, November 5 Jens Lechtenbörger 9
13 Meaning of Features M:N relationships (4/4) All However Country Maybe there was a reason to place State above Region Roll-Up like change in granularity In general, regions fit into state boundaries But not always Then, add a new type of M:N navigational arc Region State This is not Roll-Up! City Location DOLAP 2005, November 5 Jens Lechtenbörger 10
14 Meaning of Features Joining Levels (1/5) All Country RCountry All SCountry All 1 1..* Country * 1..* Region State Region State Region State * 1..* City City City Location Location Location DOLAP 2005, November 5 Jens Lechtenbörger 11
15 Meaning of Features Joining Levels (2/5) Semantics of schema definable via admissible instances. Consider City c in Region r and State s. With universal relations, admissible instances are tables that satisfy FDs For left schema, by transitivity of FDs Country of r must be equal to Country of s With objects, associations are implemented via references Object c has references to r and s Objects r and s each have exactly one reference to a country object That object for r may be distinct from the one of s Thus, left schema on previous slide has different meaning than other two, whose meaning is the same DOLAP 2005, November 5 Jens Lechtenbörger 12
16 Meaning of Features Joining Levels (3/5) It s even worse... Consider a 3NF implementation of left schema Tables for City, Region, State, Country Table for City has foreign keys to tables for Region, State Tables for Region and State each have a foreign key to table for Country Those foreign keys need not be in sync Thus, again a city may wind up in two countries Star and snowflake schemata have different semantics! What does your favorite OLAP tool do? Gap in relational theory. Research in progress. Guideline: Use handwritten code to maintain consistency. Be careful! DOLAP 2005, November 5 Jens Lechtenbörger 13
17 Meaning of Features Joining Levels (4/5) Reuse of levels is different from joining... Sales Customer CustID Region City Country Amount Supplier SuppID... State Product ProdID Here, customer and supplier must be in the same city DOLAP 2005, November 5 Jens Lechtenbörger 14
18 Meaning of Features Joining Levels (5/5) Reuse of levels is different from joining Sales Customer CustID... CCity [City] Amount Supplier Product SuppID... ProdID... SCity [City] Notice: New notation... DOLAP 2005, November 5 Jens Lechtenbörger 15
![.. CCity [City] Amount Supplier Product SuppID... ProdID.](/docs-images/53/11650950/images/page_18.jpg ".. SCity [City] Notice: New notation.")
19 Meaning of Features Parallel vs Alternative Paths (1/5) Parallel paths allow levels from different paths in single Group-By clause, e.g.: All All Country Year Region State Quarter Month Week City Location Day Time DOLAP 2005, November 5 Jens Lechtenbörger 16
20 Meaning of Features Parallel vs Alternative Paths (2/5) Observations on parallel paths Including levels from more than one path increases level of detail E.g., grouping by Week and Month is OK Guideline: There are less problems than you might have thought DOLAP 2005, November 5 Jens Lechtenbörger 17
21 Meaning of Features Parallel vs Alternative Paths (3/5) Alternative paths require exclusive choice, e.g.: All all Context dependency CustType Person Company CustType: Person Job Branch CustType: Company Artist... Zoo director null null Airline... CustID P1 P2... P42042 C1... Customer Grouping by Job and Branch is inconsistent DOLAP 2005, November 5 Jens Lechtenbörger 18
22 Meaning of Features Parallel vs Alternative Paths (4/5) Observations on alternative paths Alternative paths usually arise from optional levels Use context dependencies to explain presence of structural NULLs Or more complex dimension constraints Hurtado and Mendelzon, PODS 2002 Guideline: Avoid/explain optional levels. Notice: Subclassing in object-oriented models expresses context dependencies DOLAP 2005, November 5 Jens Lechtenbörger 19
23 Meaning of Features Parallel vs Alternative Paths (5/5) Customer CustID CustType All Person CustID Job CustType Company CustID Branch Legal Form CustType Business P. Capital C. CustID Subs. Capital Legal Form DOLAP 2005, November 5 Jens Lechtenbörger 20
24 Outline Context Conceptual Modeling Meaning of Features Multidimensional Normal Forms Schema Versioning Conclusions
25 Multidimensional Normal Forms (1/4) Joint work with Gottfried Vossen: Multidimensional Normal Forms for Data Warehouse Design, Information Systems, 2003 Three multidimensional normal forms (MNFs) 1MNF based on analysis of FDs 2MNF requires context dependencies for optional levels 3MNF places restrictions upon context dependencies DOLAP 2005, November 5 Jens Lechtenbörger 21
26 Multidimensional Normal Forms (2/4) Implications of 1MNF Faithful representation of the application domain Completeness w.r.t. the application domain Avoidance of redundancies Avoidance of M:N relationships DOLAP 2005, November 5 Jens Lechtenbörger 22
27 Multidimensional Normal Forms (3/4) Implications of 2MNF and 3MNF Explanation for structural NULLs allows context-sensitive summarizability avoidance of contradictory queries Relational implementation of class hierarchies within dimensions without structural NULLs possible Avoidance of alternative paths DOLAP 2005, November 5 Jens Lechtenbörger 23
28 Multidimensional Normal Forms (4/4) Final remarks concerning 2MNF and 3MNF Both rely on purely relational techniques For object-oriented models considerable simplifications possible Disallow optional levels Construction (see paper in Information Systems mentioned above) As long as optional level l exists, introduce further sub-classes One with l, now mandatory The other without l DOLAP 2005, November 5 Jens Lechtenbörger 24
29 Outline Context Conceptual Modeling Meaning of Features Multidimensional Normal Forms Schema Versioning Conclusions
30 Schema Versioning (1/14) Joint work with Matteo Golfarelli, Stefano Rizzi, Gottfried Vossen. Schema Versioning in Data Warehouses: Enabling Cross-Version Querying via Schema Augmentation. To appear in Data & Knowledge Engineering. Challenges Storage of historical data under changing business requirements Non-volatility, in particular consistent re-execution of old queries Our proposal Maintenance of history of schema versions Simple graph model representing core of multidimensional models Schema augmentation to represent new schema information on old data Schema intersection to answer cross-version queries DOLAP 2005, November 5 Jens Lechtenbörger 25
31 Schema Versioning (2/14) Region Year Container Category Nation Incentive Month Size Brand Type City SaleDistrict Allowance Terms Type Carrier Date Part Customer Deal ShipMode Shipment Qty Shipped Shipping CostsDM DOLAP 2005, November 5 Jens Lechtenbörger 26
32 Schema Versioning (3/14) At t 1 = 1/1/2003, the schema undergoes a major revision. 1. The temporal granularity changes from Date to Month. 2. A classification into Subcategories is added to part hierarchy. 3. A new constraint in customer hierarchy states that SaleDistricts belong to Nations. 4. The Incentive is independent of shipment Terms. At t 2 = 1/1/2004, another version is created. 1. New measures ShippingCostsEU and ShippingCostsLIT are added. 2. The ShipMode dimension is deleted. 3. A ShipFrom dimension is added. 4. A descriptive attribute PartDescr is added to Part. DOLAP 2005, November 5 Jens Lechtenbörger 27
33 Schema Versioning (4/14) Category Region Container Subcategory Nation Incentive Year Size Brand Type City SaleDistrict Allowance Terms Month Part PartDescr Customer Deal ShipFrom Shipment Shipping CostsLIT Qty Shipped Shipping CostsDM Shipping CostsEU Resulting schema graph DOLAP 2005, November 5 Jens Lechtenbörger 28
34 Schema Versioning (5/14) Category Region Container Subcategory Nation Incentive Year Size Brand Type City SaleDistrict Allowance Terms Month Part PartDescr Customer Deal ShipFrom Shipment Shipping CostsLIT Three sample query challenges: Qty Shipped Shipping CostsDM Shipping CostsEU Compute the total quantity of each part category Shipped From each warehouse to each customer nation since July Drill down from Category to Subcategory Drill down from Nation to SaleDistrict DOLAP 2005, November 5 Jens Lechtenbörger 29
35 Schema Versioning (6/14) Schema Modification (1/4) Four schema modification operations on schema graph Add A () to add a new attribute Del A () to delete an existing attribute Add F () to add an arc involving existing attribute Del F () to remove an existing arc DOLAP 2005, November 5 Jens Lechtenbörger 30
36 Schema Versioning (7/14) Schema Modification (2/4) Consider again Region Year Container Category Nation Incentive Month Size Brand Type City SaleDistrict Allowance Terms Type Carrier Date Part Customer Deal ShipMode Shipment Qty Shipped Shipping CostsDM First goal: Delete Date DOLAP 2005, November 5 Jens Lechtenbörger 31
37 Schema Versioning (8/14) Schema Modification (3/4) Result of Del A (Date) Region Container Category Nation Incentive Year Size Brand Type City SaleDistrict Allowance Terms Type Carrier Month Part Customer Deal ShipMode Shipment Qty Shipped Shipping CostsDM Next goal: Insert Subcategory below Category DOLAP 2005, November 5 Jens Lechtenbörger 32
38 Schema Versioning (9/14) Schema Modification (4/4) Result of Add A (Subcategory) Container Category Size Brand Type... Part Shipment... Subcategory... DOLAP 2005, November 5 Jens Lechtenbörger 33
39 Schema Versioning (9/14) Schema Modification (4/4) Result of Add A (Subcategory), Add F (Type Subcategory) Container Category Subcategory Size Brand Type... Part Shipment DOLAP 2005, November 5 Jens Lechtenbörger 33
40 Schema Versioning (9/14) Schema Modification (4/4) Result of Add A (Subcategory), Add F (Type Subcategory), Add F (Subcategory Category) Container Category Subcategory Size Brand Type... Part Shipment DOLAP 2005, November 5 Jens Lechtenbörger 33
41 Schema Versioning (10/14) Schema Augmentation (1/2) Previous schema versions associated with augmented schemata Previous schema computable via projection from augmented one Designer chooses to add information to augmented schemata based on current schema modification, e.g., old data enriched with new attributes, e.g., Subcategory more constraints expressed on old data, e.g., SaleDistrict Nation Augmented schemata used by querying subsystem DOLAP 2005, November 5 Jens Lechtenbörger 34
42 Schema Versioning (11/14) Schema Augmentation (2/2) Element Condition Augm. action A is measure estimate values for A (E A) F A Diff + A is dimension disaggregate measure values A (S,S ) A is derived measure compute values for A (E A) F A is property consistently add values for A f Diff + F (S,S ) - check if f holds DOLAP 2005, November 5 Jens Lechtenbörger 35
43 Schema Versioning (12/14) Cross-version Querying (1/3) General idea: Formulation context for OLAP query is a schema graph Intersection of schema versions is the largest schema for uniform querying Query can be answered if formulation context is sub-graph of intersection More precisely, augmented schemata instead of real versions DOLAP 2005, November 5 Jens Lechtenbörger 36
44 Schema Versioning (13/14) Cross-version Querying (2/3) Category Region Container Subcategory Nation Incentive Year Size Brand Type City SaleDistrict Allowance Terms Month Part Customer Deal ShipFrom Shipment QtyShipped ShippingCostsDM Compute the total quantity of each part category shipped from each warehouse to each customer nation since July DOLAP 2005, November 5 Jens Lechtenbörger 37
45 Schema Versioning (14/14) Cross-version Querying (3/3) Observations Query well-formulated only if ShipFrom augmented Drilling down from Category to Subcategory only if subcategories established also for 2002 data Drilling down from Nation to SaleDistrict only if FD from sale districts to nations also satisfied before DOLAP 2005, November 5 Jens Lechtenbörger 38
46 Outline Context Conceptual Modeling Meaning of Features Multidimensional Normal Forms Schema Versioning Conclusions
47 Conclusions (1/3) Summary FDs help in data warehouse design Meaning and potential of multidimensional features sometimes underspecified Sub-classing helps to structure multidimensional schemata Versioning with cross-version querying is feasible DOLAP 2005, November 5 Jens Lechtenbörger 39
48 Conclusions (2/3) Schema versioning offers further potential What-if analysis Horizontal benchmarking Open issue: Generalization to hyper-graphs (cross-dimensional attributes, derived measures) DOLAP 2005, November 5 Jens Lechtenbörger 40
49 Conclusions (3/3) There s more... Taking full advantage of rich models Transformations of conceptual to logical models for ETL Alkis Simitsis: Mapping Conceptual to Logical Models for ETL Processes. DOLAP 2005 More generally, model-driven design Jose-Norberto Mazón et al.: Applying MDA to the Development of Data Warehouses. DOLAP 2005 Where do the requirements come from? Paolo Giorgini et al.: Goal-oriented requirement analysis for data warehouse design. DOLAP 2005 DOLAP 2005, November 5 Jens Lechtenbörger 41
50 Thank you for your attention! DOLAP 2005, November 5 Jens Lechtenbörger 42
Open Problems in Data Warehousing: 8 Years Later... Stefano Rizzi DEIS - University of Bologna srizzi@deis.unibo.it Summary Archeology The early 90 s Back to 1995 Into 2k At present Achievements Hot issues
Data Warehouse Schema Design
Data Warehouse Schema Design Jens Lechtenbörger Dept. of Information Systems University of Münster Leonardo-Campus 3 D-48149 Münster, Germany lechten@wi.uni-muenster.de 1 Introduction A data warehouse
Requirements are elicited from users and represented either informally by means of proper glossaries or formally (e.g., by means of goal-oriented
A Comphrehensive Approach to Data Warehouse Testing Matteo Golfarelli & Stefano Rizzi DEIS University of Bologna Agenda: 1. DW testing specificities 2. The methodological framework 3. What & How should
Data warehouse life-cycle and design
SYNONYMS Data Warehouse design methodology Data warehouse life-cycle and design Matteo Golfarelli DEIS University of Bologna Via Sacchi, 3 Cesena Italy matteo.golfarelli@unibo.it DEFINITION The term data
Conceptual Data Warehouse Design
Conceptual Data Warehouse Design Bodo Hüsemann, Jens Lechtenbörger, Gottfried Vossen Institut für Wirtschaftsinformatik Universität Münster, Steinfurter Straße 109 D-48149 Münster, Germany bodo.huesemann@uni-muenster.de
Data Warehouse Design
Data Warehouse Design Modern Principles and Methodologies Matteo Golfarelli Stefano Rizzi Translated by Claudio Pagliarani Mc Grauu Hill New York Chicago San Francisco Lisbon London Madrid Mexico City
A Hybrid Model Driven Development Framework for the Multidimensional Modeling of Data Warehouses
A Hybrid Model Driven Development Framework for the Multidimensional Modeling of Data Warehouses ABSTRACT Jose-Norberto Mazón Lucentia Research Group Dept. of Software and Computing Systems University
Data Warehouse Requirements Analysis Framework: Business-Object Based Approach
Data Warehouse Requirements Analysis Framework: Business-Object Based Approach Anirban Sarkar Department of Computer Applications National Institute of Technology, Durgapur West Bengal, India Abstract
Data Warehouse Snowflake Design and Performance Considerations in Business Analytics
Journal of Advances in Information Technology Vol. 6, No. 4, November 2015 Data Warehouse Snowflake Design and Performance Considerations in Business Analytics Jiangping Wang and Janet L. Kourik Walker
Data warehouses. Data Mining. Abraham Otero. Data Mining. Agenda
Data warehouses 1/36 Agenda Why do I need a data warehouse? ETL systems Real-Time Data Warehousing Open problems 2/36 1 Why do I need a data warehouse? Why do I need a data warehouse? Maybe you do not
Alejandro Vaisman Esteban Zimanyi. Data. Warehouse. Systems. Design and Implementation. ^ Springer
Alejandro Vaisman Esteban Zimanyi Data Warehouse Systems Design and Implementation ^ Springer Contents Part I Fundamental Concepts 1 Introduction 3 1.1 A Historical Overview of Data Warehousing 4 1.2 Spatial
Data W a Ware r house house and and OLAP Week 5 1
Data Warehouse and OLAP Week 5 1 Midterm I Friday, March 4 Scope Homework assignments 1 4 Open book Team Homework Assignment #7 Read pp. 121 139, 146 150 of the text book. Do Examples 3.8, 3.10 and Exercise
DIMENSION HIERARCHIES UPDATES IN DATA WAREHOUSES A User-driven Approach
DIMENSION HIERARCHIES UPDATES IN DATA WAREHOUSES A User-driven Approach Cécile Favre, Fadila Bentayeb, Omar Boussaid ERIC Laboratory, University of Lyon, 5 av. Pierre Mendès-France, 69676 Bron Cedex, France
A Comprehensive Approach to Data Warehouse Testing
A Comprehensive Approach to Data Warehouse Testing Matteo Golfarelli DEIS - University of Bologna Via Sacchi, 3 Cesena, Italy matteo.golfarelli@unibo.it Stefano Rizzi DEIS - University of Bologna VIale
TOWARDS A FRAMEWORK INCORPORATING FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTS FOR DATAWAREHOUSE CONCEPTUAL DESIGN
IADIS International Journal on Computer Science and Information Systems Vol. 9, No. 1, pp. 43-54 ISSN: 1646-3692 TOWARDS A FRAMEWORK INCORPORATING FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTS FOR DATAWAREHOUSE
INTEROPERABILITY IN DATA WAREHOUSES
INTEROPERABILITY IN DATA WAREHOUSES Riccardo Torlone Roma Tre University http://torlone.dia.uniroma3.it/ SYNONYMS Data warehouse integration DEFINITION The term refers to the ability of combining the content
Lecture Data Warehouse Systems
Lecture Data Warehouse Systems Eva Zangerle SS 2013 PART A: Architecture Chapter 1: Motivation and Definitions Motivation Goal: to build an operational general view on a company to support decisions in
Data Warehouse: Introduction
Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,
Basics of Dimensional Modeling
Basics of Dimensional Modeling Data warehouse and OLAP tools are based on a dimensional data model. A dimensional model is based on dimensions, facts, cubes, and schemas such as star and snowflake. Dimensional
Data warehouse design
DataBase and Data Mining Group of DataBase and Data Mining Group of DataBase and Data Mining Group of Database and data mining group, Data warehouse design DATA WAREHOUSE: DESIGN - 1 Risk factors Database
THE DIMENSIONAL FACT MODEL: A CONCEPTUAL MODEL FOR DATA WAREHOUSES 1
THE DIMENSIONAL FACT MODEL: A CONCEPTUAL MODEL FOR DATA WAREHOUSES 1 MATTEO GOLFARELLI, DARIO MAIO and STEFANO RIZZI DEIS - Università di Bologna, Viale Risorgimento 2, 40136 Bologna, Italy {mgolfarelli,dmaio,srizzi}@deis.unibo.it
Index Selection Techniques in Data Warehouse Systems
Index Selection Techniques in Data Warehouse Systems Aliaksei Holubeu as a part of a Seminar Databases and Data Warehouses. Implementation and usage. Konstanz, June 3, 2005 2 Contents 1 DATA WAREHOUSES
Sizing Logical Data in a Data Warehouse A Consistent and Auditable Approach
2006 ISMA Conference 1 Sizing Logical Data in a Data Warehouse A Consistent and Auditable Approach Priya Lobo CFPS Satyam Computer Services Ltd. 69, Railway Parallel Road, Kumarapark West, Bangalore 560020,
Multidimensional Modeling with UML Package Diagrams
Multidimensional Modeling with UML Package Diagrams Sergio Luján-Mora 1, Juan Trujillo 1, and Il-Yeol Song 2 1 Dept. de Lenguajes y Sistemas Informáticos Universidad de Alicante (Spain) {slujan,jtrujillo}@dlsi.ua.es
Mario Guarracino. Data warehousing
Data warehousing Introduction Since the mid-nineties, it became clear that the databases for analysis and business intelligence need to be separate from operational. In this lecture we will review the
Data Warehouse Logical Design. Letizia Tanca Politecnico di Milano (with the kind support of Rosalba Rossato)
Data Warehouse Logical Design Letizia Tanca Politecnico di Milano (with the kind support of Rosalba Rossato) Data Mart logical models MOLAP (Multidimensional On-Line Analytical Processing) stores data
Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1
Slide 29-1 Chapter 29 Overview of Data Warehousing and OLAP Chapter 29 Outline Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics
DATA WAREHOUSE DESIGN
DATA WAREHOUSE DESIGN ICDE 2001 Tutorial Stefano Rizzi, Matteo Golfarelli DEIS - University of Bologna, Italy 1 Motivation Building a data warehouse for an enterprise is a huge and complex task, which
Advanced Data Warehouse Design
Data-Centric Systems and Applications Advanced Data Warehouse Design From Conventional to Spatial and Temporal Applications Bearbeitet von Elzbieta Malinowski, Esteban Zimányi 1st ed. 2008. Corr. 2nd printing
6NF CONCEPTUAL MODELS AND DATA WAREHOUSING 2.0
ABSTRACT 6NF CONCEPTUAL MODELS AND DATA WAREHOUSING 2.0 Curtis Knowles Georgia Southern University ck01693@georgiasouthern.edu Sixth Normal Form (6NF) is a term used in relational database theory by Christopher
Encyclopedia of Data Warehousing and Mining
Encyclopedia of Data Warehousing and Mining Second Edition John Wang Montclair State University, USA Volume IV Pro-Z Information Science reference Hershey New York Director of Editorial Content: Director
Outline. Data Warehousing. What is a Warehouse? What is a Warehouse?
Outline Data Warehousing What is a data warehouse? Why a warehouse? Models & operations Implementing a warehouse 2 What is a Warehouse? Collection of diverse data subject oriented aimed at executive, decision
Data warehouse Architectures and processes
Database and data mining group, Data warehouse Architectures and processes DATA WAREHOUSE: ARCHITECTURES AND PROCESSES - 1 Database and data mining group, Data warehouse architectures Separation between
Transformation of OWL Ontology Sources into Data Warehouse
Transformation of OWL Ontology Sources into Data Warehouse M. Gulić Faculty of Maritime Studies, Rijeka, Croatia marko.gulic@pfri.hr Abstract - The Semantic Web, as the extension of the traditional Web,
Optimization of ETL Work Flow in Data Warehouse
Optimization of ETL Work Flow in Data Warehouse Kommineni Sivaganesh M.Tech Student, CSE Department, Anil Neerukonda Institute of Technology & Science Visakhapatnam, India. Sivaganesh07@gmail.com P Srinivasu
A Survey on Bitemporal Data Warehousing System and Data Warehouse Design Techniques.
A Survey on Bitemporal Data Warehousing System and Data Warehouse Design Techniques. P.N.V.S.Pavan Kumar, K.Bala Chowdappa, S.Subba Lakshmi CSE Department, G.Pulla Reddy Engineering College(Autonomous)
Conceptual Design Using the Entity-Relationship (ER) Model
Conceptual Design Using the Entity-Relationship (ER) Model Module 5, Lectures 1 and 2 Database Management Systems, R. Ramakrishnan 1 Overview of Database Design Conceptual design: (ER Model is used at
Determining Preferences from Semantic Metadata in OLAP Reporting Tool
Determining Preferences from Semantic Metadata in OLAP Reporting Tool Darja Solodovnikova, Natalija Kozmina Faculty of Computing, University of Latvia, Riga LV-586, Latvia {darja.solodovnikova, natalija.kozmina}@lu.lv
1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing
1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing 2. What is a Data warehouse a. A database application
Data Warehousing. Read chapter 13 of Riguzzi et al Sistemi Informativi. Slides derived from those by Hector Garcia-Molina
Data Warehousing Read chapter 13 of Riguzzi et al Sistemi Informativi Slides derived from those by Hector Garcia-Molina What is a Warehouse? Collection of diverse data subject oriented aimed at executive,
DATA WAREHOUSING AND OLAP TECHNOLOGY
DATA WAREHOUSING AND OLAP TECHNOLOGY Manya Sethi MCA Final Year Amity University, Uttar Pradesh Under Guidance of Ms. Shruti Nagpal Abstract DATA WAREHOUSING and Online Analytical Processing (OLAP) are
Bringing Business Objects into ETL Technology
Bringing Business Objects into ETL Technology Jing Shan Ryan Wisnesky Phay Lau Eugene Kawamoto Huong Morris Sriram Srinivasn Hui Liao 1. Northeastern University, jshan@ccs.neu.edu 2. Stanford University,
How To Write A Diagram
Data Model ing Essentials Third Edition Graeme C. Simsion and Graham C. Witt MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF ELSEVIER AMSTERDAM BOSTON LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE
Data Warehousing and OLAP
1 Data Warehousing and OLAP Hector Garcia-Molina Stanford University Warehousing Growing industry: $8 billion in 1998 Range from desktop to huge: Walmart: 900-CPU, 2,700 disk, 23TB Teradata system Lots
ETL evolution from data sources to data warehouse using mediator data storage
ETL evolution from data sources to data warehouse using mediator data storage Alexander Dolnik (alexander.dolnik@gmail.com) Saint-Petersburg State University Abstract. The problem of evolution in the ETL
The process of database development. Logical model: relational DBMS. Relation
The process of database development Reality (Universe of Discourse) Relational Databases and SQL Basic Concepts The 3rd normal form Structured Query Language (SQL) Conceptual model (e.g. Entity-Relationship
Business Terms as a Critical Success Factor for Data Warehousing
Business Terms as a Critical Success Factor for Data Warehousing Peter Lehmann Lawson Mardon Singen Alusingen-Platz 1 D-78221 Singen peter.lehmann@lawsonmardon.com Johann Jaszewski Otto-von-Guericke-Universität
CHAPTER-24 Mining Spatial Databases
CHAPTER-24 Mining Spatial Databases 24.1 Introduction 24.2 Spatial Data Cube Construction and Spatial OLAP 24.3 Spatial Association Analysis 24.4 Spatial Clustering Methods 24.5 Spatial Classification
Database Design Patterns. Winter 2006-2007 Lecture 24
Database Design Patterns Winter 2006-2007 Lecture 24 Trees and Hierarchies Many schemas need to represent trees or hierarchies of some sort Common way of representing trees: An adjacency list model Each
Logical Design of Data Warehouses from XML
Logical Design of Data Warehouses from XML Marko Banek, Zoran Skočir and Boris Vrdoljak FER University of Zagreb, Zagreb, Croatia {marko.banek, zoran.skocir, boris.vrdoljak}@fer.hr Abstract Data warehouse
Investigating the Effects of Spatial Data Redundancy in Query Performance over Geographical Data Warehouses
Investigating the Effects of Spatial Data Redundancy in Query Performance over Geographical Data Warehouses Thiago Luís Lopes Siqueira Ricardo Rodrigues Ciferri Valéria Cesário Times Cristina Dutra de
OLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA
OLAP and OLTP AMIT KUMAR BINDAL Associate Professor Databases Databases are developed on the IDEA that DATA is one of the critical materials of the Information Age Information, which is created by data,
Introduction to Data Warehousing. Ms Swapnil Shrivastava swapnil@konark.ncst.ernet.in
Introduction to Data Warehousing Ms Swapnil Shrivastava swapnil@konark.ncst.ernet.in Necessity is the mother of invention Why Data Warehouse? Scenario 1 ABC Pvt Ltd is a company with branches at Mumbai,
Two approaches to the integration of heterogeneous data warehouses
Distrib Parallel Databases (2008) 23: 69 97 DOI 10.1007/s10619-007-7022-z Two approaches to the integration of heterogeneous data warehouses Riccardo Torlone Published online: 23 December 2007 Springer
Data Warehousing and Mining: Concepts, Methodologies, Tools, and Applications
Data Warehousing and Mining: Concepts, Methodologies, Tools, and Applications John Wang Montclair State University, USA Information Science reference Hershey New York Acquisitions Editor: Development Editor:
Chapter 15 Basics of Functional Dependencies and Normalization for Relational Databases
Chapter 15 Basics of Functional Dependencies and Normalization for Relational Databases Copyright 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 15 Outline Informal Design Guidelines
The Entity-Relationship Model
The Entity-Relationship Model Chapter 2 Slides modified by Rasmus Pagh for Database Systems, Fall 2006 IT University of Copenhagen Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Today
Overview of Data Warehousing and OLAP
Overview of Data Warehousing and OLAP Chapter 28 March 24, 2008 ADBS: DW 1 Chapter Outline What is a data warehouse (DW) Conceptual structure of DW Why separate DW Data modeling for DW Online Analytical
Foundations of Business Intelligence: Databases and Information Management
Foundations of Business Intelligence: Databases and Information Management Content Problems of managing data resources in a traditional file environment Capabilities and value of a database management
Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and
Copyright is owned by the Author of the thesis. Permission is given for a copy to be downloaded by an individual for the purpose of research and private study only. The thesis may not be reproduced elsewhere
Designing a Dimensional Model
Designing a Dimensional Model Erik Veerman Atlanta MDF member SQL Server MVP, Microsoft MCT Mentor, Solid Quality Learning Definitions Data Warehousing A subject-oriented, integrated, time-variant, and
Concepts of Database Management Seventh Edition. Chapter 9 Database Management Approaches
Concepts of Database Management Seventh Edition Chapter 9 Database Management Approaches Objectives Describe distributed database management systems (DDBMSs) Discuss client/server systems Examine the ways
Semantic Enrichment of OLAP Cubes Multidimensional Ontologies and their Representation in SQL and OWL
Semantic Enrichment of OLAP Cubes Multidimensional Ontologies and their Representation in SQL and OWL Bernd Neumayr, Christoph Schütz, Michael Schrefl This work was supported by the FIT-IT research program
A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT
A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT Xiufeng Liu 1 & Xiaofeng Luo 2 1 Department of Computer Science Aalborg University, Selma Lagerlofs Vej 300, DK-9220 Aalborg, Denmark 2 Telecommunication Engineering
Database Applications. Advanced Querying. Transaction Processing. Transaction Processing. Data Warehouse. Decision Support. Transaction processing
Database Applications Advanced Querying Transaction processing Online setting Supports day-to-day operation of business OLAP Data Warehousing Decision support Offline setting Strategic planning (statistics)
Sterling Business Intelligence
Sterling Business Intelligence Concepts Guide Release 9.0 March 2010 Copyright 2009 Sterling Commerce, Inc. All rights reserved. Additional copyright information is located on the documentation library:
Understanding Data Warehousing. [by Alex Kriegel]
![Understanding Data Warehousing. [by Alex Kriegel]](/thumbs/27/10228953.jpg "Understanding Data Warehousing. [by Alex Kriegel]")
Understanding Data Warehousing 2008 [by Alex Kriegel] Things to Discuss Who Needs a Data Warehouse? OLTP vs. Data Warehouse Business Intelligence Industrial Landscape Which Data Warehouse: Bill Inmon vs.
Integrations of Data Warehousing, Data Mining and Database Technologies:
Integrations of Data Warehousing, Data Mining and Database Technologies: Innovative Approaches David Taniar Monash University, Australia Li Chen LaTrobe University, Australia Senior Editorial Director:
PowerDesigner WarehouseArchitect The Model for Data Warehousing Solutions. A Technical Whitepaper from Sybase, Inc.
PowerDesigner WarehouseArchitect The Model for Data Warehousing Solutions A Technical Whitepaper from Sybase, Inc. Table of Contents Section I: The Need for Data Warehouse Modeling.....................................4
investment is a key issue for the success of BI+EDW system. The typical BI application deployment process is not flexible enough to deal with a fast
EIAW: Towards a Business-friendly Data Warehouse Using Semantic Web Technologies Guotong Xie 1, Yang Yang 1, Shengping Liu 1, Zhaoming Qiu 1, Yue Pan 1, and Xiongzhi Zhou 2 1 IBM China Research Laboratory
Dimensional Modeling and E-R Modeling In. Joseph M. Firestone, Ph.D. White Paper No. Eight. June 22, 1998
1 of 9 5/24/02 3:47 PM Dimensional Modeling and E-R Modeling In The Data Warehouse By Joseph M. Firestone, Ph.D. White Paper No. Eight June 22, 1998 Introduction Dimensional Modeling (DM) is a favorite
USING ONTOLOGIES FOR THE DESIGN OF DATA WAREHOUSES
USING ONTOLOGIES FOR THE DESIGN OF DATA WAREHOUSES Jesús Pardillo 1 and Jose-Norberto Mazón 2 University of Alicante DLSI/Lucentia, Spain 1 research@jesuspardillo.com 2 jnmazon@dlsi.ua.es ABSTRACT Obtaining
Database Design Overview. Conceptual Design ER Model. Entities and Entity Sets. Entity Set Representation. Keys
Database Design Overview Conceptual Design. The Entity-Relationship (ER) Model CS430/630 Lecture 12 Conceptual design The Entity-Relationship (ER) Model, UML High-level, close to human thinking Semantic
IV. The (Extended) Entity-Relationship Model
IV. The (Extended) Entity-Relationship Model The Extended Entity-Relationship (EER) Model Entities, Relationships and Attributes Cardinalities, Identifiers and Generalization Documentation of EER Diagrams
Business Intelligence, Data warehousing Concept and artifacts
Business Intelligence, Data warehousing Concept and artifacts Data Warehousing is the process of constructing and using the data warehouse. The data warehouse is constructed by integrating the data from
Encyclopedia of Database Technologies and Applications. Data Warehousing: Multi-dimensional Data Models and OLAP. Jose Hernandez-Orallo
Encyclopedia of Database Technologies and Applications Data Warehousing: Multi-dimensional Data Models and OLAP Jose Hernandez-Orallo Dep. of Information Systems and Computation Technical University of
Goal-Oriented Requirement Analysis for Data Warehouse Design
Goal-Oriented Requirement Analysis for Data Warehouse Design Paolo Giorgini University of Trento, Italy Stefano Rizzi University of Bologna, Italy Maddalena Garzetti University of Trento, Italy Abstract
The Importance of Multidimensional Modeling in Data warehouses
COVER FEATURE Designing Data Warehouses with OO Conceptual Models Based on a subset of the Unified Modeling Language, the authors objectoriented approach to building data warehouses frees conceptual design
Module 1: Introduction to Data Warehousing and OLAP
Raw Data vs. Business Information Module 1: Introduction to Data Warehousing and OLAP Capturing Raw Data Gathering data recorded in everyday operations Deriving Business Information Deriving meaningful
DEVELOPING REQUIREMENTS FOR DATA WAREHOUSE SYSTEMS WITH USE CASES
DEVELOPING REQUIREMENTS FOR DATA WAREHOUSE SYSTEMS WITH USE CASES Robert M. Bruckner Vienna University of Technology bruckner@ifs.tuwien.ac.at Beate List Vienna University of Technology list@ifs.tuwien.ac.at
Chapter 5: Logical Database Design and the Relational Model Part 2: Normalization. Introduction to Normalization. Normal Forms.
Chapter 5: Logical Database Design and the Relational Model Part 2: Normalization Modern Database Management 6 th Edition Jeffrey A. Hoffer, Mary B. Prescott, Fred R. McFadden Robert C. Nickerson ISYS
Deductive Data Warehouses and Aggregate (Derived) Tables
Deductive Data Warehouses and Aggregate (Derived) Tables Kornelije Rabuzin, Mirko Malekovic, Mirko Cubrilo Faculty of Organization and Informatics University of Zagreb Varazdin, Croatia {kornelije.rabuzin,
IMPLEMENTING SPATIAL DATA WAREHOUSE HIERARCHIES IN OBJECT-RELATIONAL DBMSs
IMPLEMENTING SPATIAL DATA WAREHOUSE HIERARCHIES IN OBJECT-RELATIONAL DBMSs Elzbieta Malinowski and Esteban Zimányi Computer & Decision Engineering Department, Université Libre de Bruxelles 50 av.f.d.roosevelt,
Week 13: Data Warehousing. Warehousing
1 Week 13: Data Warehousing Warehousing Growing industry: $8 billion in 1998 Range from desktop to huge: Walmart: 900-CPU, 2,700 disk, 23TB Teradata system Lots of buzzwords, hype slice & dice, rollup,
Relational model. Relational model - practice. Relational Database Definitions 9/27/11. Relational model. Relational Database: Terminology
COS 597A: Principles of Database and Information Systems elational model elational model A formal (mathematical) model to represent objects (data/information), relationships between objects Constraints
How To Improve Performance In A Database
Some issues on Conceptual Modeling and NoSQL/Big Data Tok Wang Ling National University of Singapore 1 Database Models File system - field, record, fixed length record Hierarchical Model (IMS) - fixed
Building a Cube in Analysis Services Step by Step and Best Practices
Building a Cube in Analysis Services Step by Step and Best Practices Rhode Island Business Intelligence Group November 15, 2012 Sunil Kadimdiwan InfoTrove Business Intelligence Agenda Data Warehousing
Overview. DW Source Integration, Tools, and Architecture. End User Applications (EUA) EUA Concepts. DW Front End Tools. Source Integration
DW Source Integration, Tools, and Architecture Overview DW Front End Tools Source Integration DW architecture Original slides were written by Torben Bach Pedersen Aalborg University 2007 - DWML course
Lection 3-4 WAREHOUSING
Lection 3-4 DATA WAREHOUSING Learning Objectives Understand d the basic definitions iti and concepts of data warehouses Understand data warehousing architectures Describe the processes used in developing
Mastering Data Warehouse Aggregates. Solutions for Star Schema Performance
Brochure More information from http://www.researchandmarkets.com/reports/2248199/ Mastering Data Warehouse Aggregates. Solutions for Star Schema Performance Description: - This is the first book to provide
GRAnD: A Goal-Oriented Approach. to Requirement Analysis in Data Warehouses
GRAnD: A Goal-Oriented Approach to Requirement Analysis in Data Warehouses Paolo Giorgini Stefano Rizzi Maddalena Garzetti Abstract Several surveys indicate that a significant percentage of data warehouses
A Design and implementation of a data warehouse for research administration universities
A Design and implementation of a data warehouse for research administration universities André Flory 1, Pierre Soupirot 2, and Anne Tchounikine 3 1 CRI : Centre de Ressources Informatiques INSA de Lyon
A Methodology for the Conceptual Modeling of ETL Processes
A Methodology for the Conceptual Modeling of ETL Processes Alkis Simitsis 1, Panos Vassiliadis 2 1 National Technical University of Athens, Dept. of Electrical and Computer Eng., Computer Science Division,
DATA WAREHOUSING - OLAP
http://www.tutorialspoint.com/dwh/dwh_olap.htm DATA WAREHOUSING - OLAP Copyright tutorialspoint.com Online Analytical Processing Server OLAP is based on the multidimensional data model. It allows managers,
Optimizing Your Data Warehouse Design for Superior Performance
Optimizing Your Data Warehouse Design for Superior Performance Lester Knutsen, President and Principal Database Consultant Advanced DataTools Corporation Session 2100A The Problem The database is too complex
CSC 742 Database Management Systems
CSC 742 Database Management Systems Topic #4: Data Modeling Spring 2002 CSC 742: DBMS by Dr. Peng Ning 1 Phases of Database Design Requirement Collection/Analysis Functional Requirements Functional Analysis
BUILDING BLOCKS OF DATAWAREHOUSE. G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT
BUILDING BLOCKS OF DATAWAREHOUSE G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT 1 Data Warehouse Subject Oriented Organized around major subjects, such as customer, product, sales. Focusing on
Triple-Driven Data Modeling Methodology in Data Warehousing: A Case Study
Triple-Driven Data Modeling Methodology in Data Warehousing: A Case Study Authors: Yuhong Guo, Shiwei Tang, Yunhai Tong, and Dongqing Yang (Peking Univ., China) Friday, Nov. 10, 2006 Triple-driven: why
Dimensional Data Modeling for the Data Warehouse
Lincoln Land Community College Capital City Training Center 130 West Mason Springfield, IL 62702 217-782-7436 www.llcc.edu/cctc Dimensional Data Modeling for the Data Warehouse Prerequisites Students should
Business Intelligence
8 Business Intelligence Business intelligence has become a buzzword in recent years. The database tools found under the heading of business intelligence include data warehousing, online analytical processing