25568 Genesee Trail Rd Golden, Colorado 80401 (303) 526-0340 Data Vault Modeling and Approach DW2.0 and Unstructured Data Master Data Management and Metadata BIG DATA & the Data Warehouse 2012 Genesee Academy, LLC 25568 Genesee Trail Rd Golden, Colorado 80401 Hans Hultgren 2012 Genesee Academy, LLC
BIG DATA and the Data Warehouse WHAT TO DO WHEN THE DATA WAREHOUSE MEETS HUGE VOLUMES OF RAPIDLY ARRIVING & SHAPE- SHIFTING DATA Asser-on What it Means
About BIG DATA Typical Data Big Data Typical Data Huge Data Volumes v v A v v v v B n- Structured & Very Complex v v C Streaming & Shape- ShiBing
Big Data and architecture Big Data solu-ons are separate from the EDW solu-on
IBM View IBM note on Architecture:
Oracle View Oracle note on Architecture:
Teradata View Teradata note on Architecture:
MicrosoB View MicrosoB note on Architecture:
Big Data and the EDW today Big Data solu-ons are separate from the EDW solu-on Architectures see Big Data components as Separate layers for other forms of analy-cs Ini-al landing areas (persisted and shared) Pre- processing layers becoming sources to EDW Data pools for integrated or hybrid downstream Marts (repor-ng) The main factors defining the differences for the two layers include Schema- on- Write versus Schema- on- Read Model- driven versus Data- driven analy-cs Model- based seman-cs versus Metadata- based seman-cs All- Data versus Selected Data- on- Demand
BIG DATA and the Data Warehouse Asser-on All EDW Data is n- structured What it Means Dealing with n- structured data is not op-onal for enterprise data warehouse programs.
BIG DATA and the Data Warehouse Asser-on All EDW Data is n- structured Full EDW data integra-on is impossible What it Means Dealing with n- structured data is not op-onal for enterprise data warehouse programs. Seman-c Integra-on is the only meaningful integra-on. The EDW can address integra-on to a point, then alignment and reconcilia-on.
BIG DATA and the Data Warehouse Asser-on All EDW Data is n- structured Full EDW data integra-on is impossible All EDW BI is Fuzzy BI What it Means Dealing with n- structured data is not op-onal for enterprise data warehouse programs. Seman-c Integra-on is the only meaningful integra-on. The EDW can address integra-on to a point, then alignment and reconcilia-on. With parsing, business rules- based logic, and interpre-ve (subjec-ve) transforms, all downstream EDW BI is fuzzy BI.
BIG DATA and the Data Warehouse Asser-on All EDW Data is n- structured Full EDW data integra-on is impossible All EDW BI is Fuzzy BI For the EDW, Big Data equals Data What it Means Dealing with n- structured data is not op-onal for enterprise data warehouse programs. Seman-c Integra-on is the only meaningful integra-on. The EDW can address integra-on to a point, then alignment and reconcilia-on. With parsing, business rules- based logic, and interpre-ve (subjec-ve) transforms, all downstream EDW BI is fuzzy BI. The true EDW architecture sees Big Data in much the same way as all Data. So Big Data tools and techniques are applicable to the en-re EDW.
BIG DATA and the Data Warehouse Asser-on All EDW Data is n- structured Full EDW data integra-on is impossible All EDW BI is Fuzzy BI For the EDW, Big Data equals Data The EDW and Big Data can live together What it Means Dealing with n- structured data is not op-onal for enterprise data warehouse programs. Seman-c Integra-on is the only meaningful integra-on. The EDW can address integra-on to a point, then alignment and reconcilia-on. With parsing, business rules- based logic, and interpre-ve (subjec-ve) transforms, all downstream EDW BI is fuzzy BI. The true EDW architecture sees Big Data in much the same way as all Data. So Big Data tools and techniques are applicable to the en-re EDW. Future Big Data solu-ons and EDW programs can be deployed on a common architecture. Historized metadata layers will enable solu-ons.
EDW & Big Data: Integrated Architecture Historized Semantic Integration Metadata Source Stage Integrated Architecture Marts Pool EDW FAS BB BNYM Manual Kurre TCM Other
EDW & Big Data: Integrated Architecture Historized Semantic Integration Metadata Source Stage Integrated Architecture Marts Pool EDW FAS BB BNYM Manual Kurre TCM Other
EDW & Big Data: Integrated Architecture Historized Semantic Integration Metadata Source Stage Integrated Architecture Marts Pool EDW FAS BB BNYM Manual Kurre TCM Other
Integrated Architecture Modeling PaYern Ensemble Modeling Unified Decomposi-on Data Vault Modeling
Data Vault Model
Links and Informa-on Data Vault Cer-fica-on Course December 3-5 2012 Sydney Register Today Book Launch Modeling the Agile Data Warehouse with Data Vault Hans Hultgren Hans@GeneseeAcademy.com Twitter: gohansgo Hanshultgren.wordpress.com YouTube: DataVaultAcademy Online, on-demand training DataVaultAcademy.com