Semantic Data Modeling: The Key to Re-usable Data

Transcription

1 Semantic Data Modeling: The Key to Re-usable Data Stephen Brobst Chief Technology Officer Teradata Corporation

2 Enterprise Information Management Data Modeling Not just a collection of subjects......but also their relationships Single, Integrated System Party Account Product Activity Party Product Account Activity Don t model subjects individually! Model your entire business! 2 Copyright Stephen Brobst. Do not duplicate without written permission

3 Functional Views Sales Marketing Finance Rates/ Regulatory Customer Service Risk Demographics Contracts Pricing General Ledger Promotions HR Production Engineering Safety Products Works OK for OLTP, but causes data chaos for BI applications. 3 Copyright Stephen Brobst. Do not duplicate without written permission

4 Business Intelligence Requires Data Integration Product Data Customer Data Account Data Transaction Data G/L Data Market Data External Data 4 Copyright Stephen Brobst. Do not duplicate without written permission

5 Data Modeling Techniques Key observation: Practitioners in the data warehousing industry frequently confuse construction of the semantic data model, logical data model, and physical data model. A semantic data model (SDM) captures the business view of information for a specific knowledge worker community or analytic application. A logical data model (LDM) captures the business relationships in the enterprise information independent of a specific analytic application or departmental view. A physical data model (PDM) captures the implementation design of tables in the data warehouse. 5 Copyright 2005, Stephen Copyright A. Brobst All rights Stephen reserved. Brobst. Do not duplicate without written permission

6 Data Model Deployment Conceptual Data Model Enterprise Logical Data Model(3NF) Subject Area A Subject Area B xxxxx xx Subject Area A xxx xxxxx xxx xxxxx xxx xxxxx Subject Area B xxx xxxxx xxx xxxxx Subject Area C Physical Model Realization Design Meta Data Enterprise Data Standards Single Physical Data Model Project A Project B Project C Semantic Model Views 6 Copyright Stephen Brobst. Do not duplicate without written permission

7 Semantic Data Modeling Semantic data modeling is a logical data modeling technique; the semantic view of information does not necessarily need to be physicalized in the database. There may be a different semantic data model for each department/applications that uses the data warehouse. Dimensional modeling is a common technique for constructing the semantic data model for an analytic application, but is not the only viable approach. 7 Copyright 2005, Stephen Copyright A. Brobst All rights Stephen reserved. Brobst. Do not duplicate without written permission

8 Different Semantic Model Designs are Appropriate for Different Types of Knowledge Workers Dimensional Relational ADS Application Physical Data Extensions Index choices & selective table denormalizations Normalized Generic Structures 8 Copyright Stephen Brobst. Do not duplicate without written permission

9 Physical Data Model Physical data model represents the tables constructed in the database. Recommendations: Use the (3NF) LDM as the starting point for the PDM with selective denormalization when appropriate for (primarily) performance reasons. Overlay (dimensional) SDM on top of PDM using views and/or semantic metadata in your BI tool. Design LDM first, then use application-specific business requirements to derive the SDMs and performance considerations to map into the PDM. 9 Copyright 2005, Stephen Copyright A. Brobst All rights Stephen reserved. Brobst. Do not duplicate without written permission

10 Semantic Models Should be BI Tool Agnostic MicroStrategy Teradata OLAP Connector Tableau Tier 3 Tier 2 Tier 1 Access Integrated Acquisition 10 Copyright Stephen Brobst. Do not duplicate without written permission

11 What is a Semantic Modeling Building Block (SMBB) Portfolio? A collection of data modeling assets that help make database design and development faster and easier for the access layer: > Access layer provides path for data from the integrated data model to end user consumption. > When this layer not well-designed, it can impact speed, security, and simplicity in developing and delivering reports, BI applications. Re-usable building blocks provide flexibility and consistency to the development process: > SMBBs include pre-built semantic models. Focuses on a specific analytic need in a specific industry: > For example, Communications Mobile Revenue Analytics. SMBBs are to the semantic layer as ildms are to the integrated layer of a data warehouse implementation. 11 Copyright Stephen Brobst. Do not duplicate without written permission

12 Dimension Building Blocks Support a Range of Analytical Needs Dimensional Model Dimension Building Blocks Fixed, Flattened Hierarchy Variable Depth Hierarchy ( Recursive ) Fixed, Normalized Hierarchy 12 Copyright Stephen Brobst. Do not duplicate without written permission

13 What are SMBBs? How are they related to an LDM? Building from the Foundation for your Data Warehouse: An LDM is like a blueprint for a house that you are building. It serves as the foundation for your integrated data warehouse. The SMBBs are like room designs that meet specific homeowner needs. Different rooms need different designs based on their purpose. Similarly, for each new business application, new semantic models are needed. SMBBs provide different designs (building blocks) for the modeler to choose from in building the semantic models. These flexible, reusable building blocks can be used for other analytic needs. 13 Copyright Stephen Brobst. Do not duplicate without written permission

14 Relationships between the Three Types of Data Models Q: Where does it all start? A: Business requirements drive the process! Logical Data Model Physical Data Model Semantic Data Models Support data re-use Data access patterns The Logical Model is used to drive generalization and support source data leverage and reuse. The Physical Model provides core support for data integration within the information architecture. The Semantic Model captures data access patterns that must be supported by the core physical model. 14 Copyright Stephen Brobst. Do not duplicate without written permission

15 Semantic Layer Benefits Efficient table joins can be encouraged inside the SDM views. Views are low maintenance objects. Views do not consume database space. Join indexes (JIs) and aggregate join indexes (AJIs) can be created based on the access paths embedded in the SDMs. PDM is not compromised with new application requirements. Protection of code assets. 15 Copyright Stephen Brobst. Do not duplicate without written permission

16 Conclusions Critical to distinguish between logical data modeling, semantic data modeling, and physical data modeling. Separate the implementation of the semantic model from the physical data model (PDM) deployment for maximum flexibility. Selective use of PDM extensions to optimize performance. Either ANSI standard views of the semantic metadata within your BI tool of choice can be used for creating a semantic data layer without sacrificing flexibility of the PDM. 16 Copyright 2005, Stephen Copyright A. Brobst All rights Stephen reserved. Brobst. Do not duplicate without written permission