Overview. DW Source Integration, Tools, and Architecture. End User Applications (EUA) EUA Concepts. DW Front End Tools. Source Integration

Size: px
Start display at page:

Download "Overview. DW Source Integration, Tools, and Architecture. End User Applications (EUA) EUA Concepts. DW Front End Tools. Source Integration"

Transcription

1 DW Source Integration, Tools, and Architecture Overview DW Front End Tools Source Integration DW architecture Original slides were written by Torben Bach Pedersen Aalborg University DWML course 2 End User Applications (EUA) EUA Concepts The business impact of the DW! Canned reports End user application templates Provide answers to common questions Can be used as (quality-assured) building blocks for other reports Two extremes Ad hoc strategic analysis, power users, DIY query tools Fixed operational analysis, report consumers, operational reporting EUA fills the gap Tactical analysis, push-button knowledge workers Templates Layout/structure + parameters Compare sales per product in <area> for <period1> and <period2> Parameters - chosen at run-time Come from any level of the given dimension drill-down Time (All time, 2002, Q, 2002 Dec, 2002 Dec 1) possible Many different Identify report candidates Produce a list of candidates Consolidate candidate list Categorize candidates by data elements Aalborg University DWML course 3 Aalborg University DWML course 4

2 What Templates to Choose? Overview Analytical Cycle Steps (repeats) 1) How s business? current performance 2) What are the trends? performance over time 3) What s unusual? quick identification of exceptions (+/-) 4) What is driving the exceptions? find causes for exceptions 5) What if? play around with parameters and see effect 6) Make a business decision small as well as big decisions 7) Implement the decision feed analysis results into op. systems Prioritize template list Rank or group templates implement 15 most important at first DW Front End Tools Source Integration DW architecture Aalborg University DWML course 5 Aalborg University DWML course 6 Data Integration Research Projects Focus on source integration and update propagation Wrapper: convert source data into a standard format Information Manifold Sources: databases, SGML docs, unstructured files, Relational integration data model TSIMMIS Wrapper/mediator Semi-structured OEM integration data model Squirrel Powerful integration mediator WHIPS Wrapper/mediator Relational integration data model Views on DW Metadata Most DW projects: DW architecture as a stepwise flow of information from source to analyst No conceptual domain model used for integration Some questions cannot be answered DWQ project: extended metamodel to capture all relevant aspects Aalborg University DWML course 7 Aalborg University DWML course 8

3 Using DW Metadata in the Enterprise Analyst: Why can t I answer question X? Analyst wants to analyze data Gather data from operational departments through OLTP Question travels through (1)-(5) Traditional DW (previous slide) only describes step (3)-(4) Cannot solve problems like why can t I answer quest. X? Conceptual relationships between enterprise model, operational models + DW must be captured Everything is a view on the enterprise model! ( local as view ) unlike previous slide (1) (2) (5) (4) (3) Possible reasons Certain measures not included in fact table Granularity of facts too coarse Particular dimensions not in DW Descriptive attributes missing from dimensions Meaning of attributes/measures deviate from the analyst s expectation Aalborg University DWML course 9 Aalborg University DWML course 10 DWQ Metadata Three metadata perspectives must be captured Conceptual (enterprise) Logical (data model) Physical (data flow) Framework instantiated by conceptual, logical, and physical information models DW quality heavily depends on DW processes rather than schemas A process meta model is needed to capture process definitions, and the relationships to DW quality Source integration practice Focus on information integration in databases (schema and data) Two main approaches Constructing integrated enterprise model Focus on mappings between sources and DW Tools for DW management Schema integration Metadata management Based on modeling tools Tools for data integration Mapping specification ETL tools like last lecture Aalborg University DWML course 11 Aalborg University DWML course 12

4 Schema Integration Producing one global schema (one-shot or incremental) Pre-integration Analyzing and annotating source schemata Semantic enrichment of schema, often in richer data model Schema comparison Determine correlations/conflicts among schema concepts Heterogeneity conflicts different source data models Naming conflicts homonyms and synonyms Semantic conflicts different abstraction levels Structural different constructs Schema conforming Conform/align schemas to make them compatible Typically semi-automatic process Schema merging and restructuring Superimpose conformed schemas Quality: completeness, correctness, minimality, understandability Virtual Data Integration Only data definition is integrated Data only in sources, queries on views, queries shipped to sources Not suited for DW? Carnot Individual schemata mapped onto rich GCL ontology (1. order logic) Articulation axioms specify mappings, queries mapped to GCL SIMS Creates common class-based domain model to describe sources Sources are dynamically chosen and integrated at query time Query reformulation, access planning, optimization, execution Information Manifold Relational world view + information source description + correspondences Metamodel enriched using description logic/datalog rules Datalog queries, optimized by choosing minimal sources TSIMMIS Wrappers wrap sources using semi-structured OEM model Mediator performs its own integration no global integration (global as view) Aalborg University DWML course 13 Aalborg University DWML course 14 Materialized Data Integration Views on source data are materialized in integrated Squirrel Integration mediators incrementally maintain materialized views Cooperation of sources required WHIPS Relational SPJ + aggregation views specified in view tree View manager computes view and handles updates Integrator ensures view maintainability Global query processor queries sources using wrappers/mediators In combination with virtual integration? DWQ Source Integration Current DW tools cannot fully support DW quality No support for validation of interschema assertions and other specified relationships, i.e., the DW design process Conceptual perspective Domain model = enterprise model + source models Consolidated and reconciled description of important concepts Not all enterprise data captured (at first, incremental approach) Logic-based formalism allows reasoning over metadata Intermodel assertions capture interdependencies Logical perspective Source schemata + DW schema in logical data model (relational) Defined as queries over the corresponding conceptual component Physical perspective The actual data stores Aalborg University DWML course 15 Aalborg University DWML course 16

5 DWQ Source Integration Architecture DWQ Source Integration Methodology Note explicit mappings! Aalborg University DWML course 17 Source-driven integration Enterprise and source model construction Source model integration (into the domain model) Source and DW schema specification (+ mappings) Data integration and reconciliation Quality analysis steps in all phases above Client-driven integration New client query considered Reasoning determines whether query can be answered by materialized views already in DW Query containment reasoning If DW not sufficient, materialize new concepts in domain model? Otherwise, new sources must be added using source-driven integr. Aalborg University DWML course 18 Overview Lifecycle Overview DW Front End Tools Source Integration Technical Architecture Design Product Selection& Installation DW architecture Project Planning Business Requirements Definition Dimensional Modeling Physical Design Data Staging Design & Development Deployment Maintenance and Growth End-User Application Specification End-User Application Development Project Management Aalborg University DWML course 19 Aalborg University DWML course 20

6 Aalborg Copenhagen Aalborg Copenhagen Aalborg Bread Bread Copenhagen Bread Milk Milk Milk Technical DW Architecture Central DW Existing databases and systems (OLTP) ETL Data Warehouse New databases and systems (OLAP) DWHow to organize DW and s? Data Marts Clients OLAP Data mining Visualization All data in one, central DW All client queries directly on the central DW Pros Simplicity Easy to manage Cons Bad performance due to no redundancy/ workload distribution Source Central DW Source Clients Aalborg University DWML course 21 Aalborg University DWML course 22 Federated DW Tiered Architecture Data stored in separate data marts, aimed at special departments Logical DW (i.e., virtual) Data marts contain detail data Pros Performance due to distribution Cons More complex Finance mart Source Mrktng mart Logical DW Source Clients Distr. mart Central DW is materialized Data is distributed to data marts in one or more tiers Only aggregated data in cube tiers Data is aggregated/reduced as it moves through tiers Pros Best performance due to redundancy+distribution Cons Most complex Hard to manage Central DW Milk Bread Aalborg Copenhagen Milk Bread Aalborg Copenhagen Aalborg University DWML course 23 Aalborg University DWML course 24

7 Coordination w. Development Strategy Operational Data Store (ODS) Different development strategies pose different demands to the architecture elements Example: Kimball Dimensional Modeling Centralized design of (conforming) dimensions First, design of a single-source data mart Later, design of multi-source data marts Integration of existing data marts into new data marts The DW is just the union of the marts it is composed of Entails top-down ( Bus Architecture ) and bottom-up elements Consequences No initial design of DW, from which data marts are extracted Data is extracted directly from sources to data marts Allows distribution of data marts and computation on them Existing databases and systems (OLTP) New ODS ETL DW OLAP Data mining Visualization Aalborg University DWML course 25 Aalborg University DWML course 26 Operational Data Store I a subject oriented, integrated, volatile, current valued data store containing only corporate detailed data (Inmon et al.) A database which integrates and accumulates operational data in a subject-oriented structure Not dimensional, but ordinary relational An extra level between operational systems and dimensional structures Two benefits sought Integration of operational systems Basis for data warehouse Operational Data Store II ODS - pros More modeling choices The dimensional straightjacket can force sub-optimal design decisions hiding the true semantics of data No need to choose a granularity, and no need to exclude data In summary, no need to make design decisions that cannot be changed subsequently This means extra flexibility ODS cons Not feasible to do analysis directly on ODS extra complexity Separate ODS unnecessary, DW = ODS (Kimball et al.) Aalborg University DWML course 27 Aalborg University DWML course 28

8 MS Analysis Services IBM 2 OLAP Server Cheap Easy to use (R/M/H)OLAP technology Data placement as desired Intelligent pre-aggregation Server and client parts Reporting Services a separate tool Built-in data mining Decision trees Clustering MS OLE for OLAP interface Light version of Hyperion Essbase (OLAP market leader) Extra product on top of 2 (R/M/H)OLAP Data in 2 or in multidimensional structures Interfaces Hyperion Essbase API OLE for OLAP (promised) 2 can also handle aggregates Automatic summary tables Used by 2 optimizer Automatic maintenance by 2 Aalborg University DWML course 29 Aalborg University DWML course 30 Oracle 10g BI Based on Express OLAP product On the market since 1970! (R/M/H)OLAP Flexible data placement Integrates ROLAP strategy and Express OLAP Total integration with Oracle 10g RMS Storage, security, management, Best integration compared to MS and IBM Add-on data mining (10g Data Mining) Associations, classification, prediction, clustering Architecture Alternatives Cubes are smart Intuitive model Better overview Better suited for data analysis But logical cubes suffice Implementation hidden from user Architecture alternatives MS, IBM, Oracle Virtual cubes, physical cubes ROLAP, MOLAP Separate relational DW, cubes directly on source data Client tools 3*2 3 = 24 different possibilities (without clients) less in reality Aalborg University DWML course 31 Aalborg University DWML course 32

9 MS vs. IBM vs. Oracle All Good scalability Good analysis facilities Flexible storage (MOLAP, ROLAP, HOLAP) Incremental update Many client tools MS Analysis Server Built-in mining + good integration with MS SQL Server 2 OLAP Server Good integration with 2 Oracle Best RMS/MOLAP integration of the three All three products are good Dependent on the other choices + existing technical architecture Virtual vs. Physical Cubes Virtual cubes Logical cube specification directly on source data ROLAP implementation without aggregates + flexible, design can be changed quickly - performance, constant load on source Physical cubes Data for cube extracted and stored on OLAP server Several implementation choices possible + good performance, only source load at creation/update - harder to change design Aalborg University DWML course 33 Aalborg University DWML course 34 MOLAP vs. ROLAP MOLAP Data in specialized data structure, optimized for OLAP + best performance, least space consumption - changing design requires rebuilding, scalability at detail level?, detail data stored several times ROLAP Data in RMS + more flexible change of design, scalable for detail data - not as good performance, larger space consumption HOLAP Detail data in RMS (can be source ) Aggregates in multidimensional structure + good performance for higher-level queries, detail data only stored once - handling design changes, operational complexity Separate Data Warehouse? Separate DW Integration of source data in DW Cubes built from DW Sometimes the only solution + better integration and cleansing, less load on existing servers - larger complexity, design changes, updating DW Cubes directly on source data Cubes built directly from source data Cannot handle all cases + less complexity, easier to change design, no update of DW - cannot handle all forms of integration and cleansing, more load on operational servers Aalborg University DWML course 35 Aalborg University DWML course 36

10 Choosing Client Tools Many OLAP clients on the market, e.g., Hyperion, Targit, Oracle MS Reporting Services Client and server communicates via an API MS OLE for OLAP De facto standard Supported by almost all client tools Hyperion Essbase API Supported by many client tools Some criteria Functionality (web distribution, analysis, reporting, ) Support Price Architecture Alternatives - Conclusion Architecture alternatives, their pros and cons No simple general choices Choices dependent on the concrete situation Look at books Look at requirements specs Look at the latest products Think about prototyping Aalborg University DWML course 37 Aalborg University DWML course 38 Summary Mini Project DW Front End Tools Source Integration DW architecture New subtask Build a few reports in Reporting Services to answer important business questions you proposed in part (1a) Discuss the architecture of your DW system Discuss source integration in your system MS Reporting Services Tutorial Aalborg University DWML course 39 Aalborg University DWML course 40

Data Warehousing and Data Mining

Data Warehousing and Data Mining Data Warehousing and Data Mining Part I: Data Warehousing Gao Cong gaocong@cs.aau.dk Slides adapted from Man Lung Yiu and Torben Bach Pedersen Course Structure Business intelligence: Extract knowledge

More information

Data Warehousing Systems: Foundations and Architectures

Data Warehousing Systems: Foundations and Architectures Data Warehousing Systems: Foundations and Architectures Il-Yeol Song Drexel University, http://www.ischool.drexel.edu/faculty/song/ SYNONYMS None DEFINITION A data warehouse (DW) is an integrated repository

More information

IST722 Data Warehousing

IST722 Data Warehousing IST722 Data Warehousing Components of the Data Warehouse Michael A. Fudge, Jr. Recall: Inmon s CIF The CIF is a reference architecture Understanding the Diagram The CIF is a reference architecture CIF

More information

Data Warehouse: Introduction

Data Warehouse: Introduction Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,

More information

Part 22. Data Warehousing

Part 22. Data Warehousing Part 22 Data Warehousing The Decision Support System (DSS) Tools to assist decision-making Used at all levels in the organization Sometimes focused on a single area Sometimes focused on a single problem

More information

Advanced Data Management Technologies

Advanced Data Management Technologies ADMT 2015/16 Unit 2 J. Gamper 1/44 Advanced Data Management Technologies Unit 2 Basic Concepts of BI and Data Warehousing J. Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE Acknowledgements:

More information

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Appliances and DW Architectures John O Brien President and Executive Architect Zukeran Technologies 1 TDWI 1 Agenda What

More information

<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server

<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server Extending Hyperion BI with the Oracle BI Server Mark Ostroff Sr. BI Solutions Consultant Agenda Hyperion BI versus Hyperion BI with OBI Server Benefits of using Hyperion BI with the

More information

Oracle Business Intelligence 11g Business Dashboard Management

Oracle Business Intelligence 11g Business Dashboard Management Oracle Business Intelligence 11g Business Dashboard Management Thomas Oestreich Chief EPM STrategist Tool Proliferation is Inefficient and Costly Disconnected Systems; Competing Analytic

More information

Bussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University

Bussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University Bussiness Intelligence and Data Warehouse Schedule Bussiness Intelligence (BI) BI tools Oracle vs. Microsoft Data warehouse History Tools Oracle vs. Others Discussion Business Intelligence (BI) Products

More information

SAS BI Course Content; Introduction to DWH / BI Concepts

SAS BI Course Content; Introduction to DWH / BI Concepts SAS BI Course Content; Introduction to DWH / BI Concepts SAS Web Report Studio 4.2 SAS EG 4.2 SAS Information Delivery Portal 4.2 SAS Data Integration Studio 4.2 SAS BI Dashboard 4.2 SAS Management Console

More information

DATA WAREHOUSING AND OLAP TECHNOLOGY

DATA WAREHOUSING AND OLAP TECHNOLOGY DATA WAREHOUSING AND OLAP TECHNOLOGY Manya Sethi MCA Final Year Amity University, Uttar Pradesh Under Guidance of Ms. Shruti Nagpal Abstract DATA WAREHOUSING and Online Analytical Processing (OLAP) are

More information

Data Warehouse Design

Data Warehouse Design Data Warehouse Design Modern Principles and Methodologies Matteo Golfarelli Stefano Rizzi Translated by Claudio Pagliarani Mc Grauu Hill New York Chicago San Francisco Lisbon London Madrid Mexico City

More information

14. Data Warehousing & Data Mining

14. Data Warehousing & Data Mining 14. Data Warehousing & Data Mining Data Warehousing Concepts Decision support is key for companies wanting to turn their organizational data into an information asset Data Warehouse "A subject-oriented,

More information

Lection 3-4 WAREHOUSING

Lection 3-4 WAREHOUSING Lection 3-4 DATA WAREHOUSING Learning Objectives Understand d the basic definitions iti and concepts of data warehouses Understand data warehousing architectures Describe the processes used in developing

More information

Data Warehousing. Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de. Winter 2015/16. Jens Teubner Data Warehousing Winter 2015/16 1

Data Warehousing. Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de. Winter 2015/16. Jens Teubner Data Warehousing Winter 2015/16 1 Jens Teubner Data Warehousing Winter 2015/16 1 Data Warehousing Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de Winter 2015/16 Jens Teubner Data Warehousing Winter 2015/16 13 Part II Overview

More information

(Week 10) A04. Information System for CRM. Electronic Commerce Marketing

(Week 10) A04. Information System for CRM. Electronic Commerce Marketing (Week 10) A04. Information System for CRM Electronic Commerce Marketing Course Code: 166186-01 Course Name: Electronic Commerce Marketing Period: Autumn 2015 Lecturer: Prof. Dr. Sync Sangwon Lee Department:

More information

Data Warehouse Overview. Srini Rengarajan

Data Warehouse Overview. Srini Rengarajan Data Warehouse Overview Srini Rengarajan Please mute Your cell! Agenda Data Warehouse Architecture Approaches to build a Data Warehouse Top Down Approach Bottom Up Approach Best Practices Case Example

More information

A Critical Review of Data Warehouse

A Critical Review of Data Warehouse Global Journal of Business Management and Information Technology. Volume 1, Number 2 (2011), pp. 95-103 Research India Publications http://www.ripublication.com A Critical Review of Data Warehouse Sachin

More information

An Introduction to Data Warehousing. An organization manages information in two dominant forms: operational systems of

An Introduction to Data Warehousing. An organization manages information in two dominant forms: operational systems of An Introduction to Data Warehousing An organization manages information in two dominant forms: operational systems of record and data warehouses. Operational systems are designed to support online transaction

More information

Chapter 6 Basics of Data Integration. Fundamentals of Business Analytics RN Prasad and Seema Acharya

Chapter 6 Basics of Data Integration. Fundamentals of Business Analytics RN Prasad and Seema Acharya Chapter 6 Basics of Data Integration Fundamentals of Business Analytics Learning Objectives and Learning Outcomes Learning Objectives 1. Concepts of data integration 2. Needs and advantages of using data

More information

www.ijreat.org Published by: PIONEER RESEARCH & DEVELOPMENT GROUP (www.prdg.org) 28

www.ijreat.org Published by: PIONEER RESEARCH & DEVELOPMENT GROUP (www.prdg.org) 28 Data Warehousing - Essential Element To Support Decision- Making Process In Industries Ashima Bhasin 1, Mr Manoj Kumar 2 1 Computer Science Engineering Department, 2 Associate Professor, CSE Abstract SGT

More information

BUILDING BLOCKS OF DATAWAREHOUSE. G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT

BUILDING BLOCKS OF DATAWAREHOUSE. G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT BUILDING BLOCKS OF DATAWAREHOUSE G.Lakshmi Priya & Razia Sultana.A Assistant Professor/IT 1 Data Warehouse Subject Oriented Organized around major subjects, such as customer, product, sales. Focusing on

More information

Enterprise Data Warehouse (EDW) UC Berkeley Peter Cava Manager Data Warehouse Services October 5, 2006

Enterprise Data Warehouse (EDW) UC Berkeley Peter Cava Manager Data Warehouse Services October 5, 2006 Enterprise Data Warehouse (EDW) UC Berkeley Peter Cava Manager Data Warehouse Services October 5, 2006 What is a Data Warehouse? A data warehouse is a subject-oriented, integrated, time-varying, non-volatile

More information

Data Warehousing and Data Mining in Business Applications

Data Warehousing and Data Mining in Business Applications 133 Data Warehousing and Data Mining in Business Applications Eesha Goel CSE Deptt. GZS-PTU Campus, Bathinda. Abstract Information technology is now required in all aspect of our lives that helps in business

More information

1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing

1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing 1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing 2. What is a Data warehouse a. A database application

More information

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing

More information

OLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA

OLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA OLAP and OLTP AMIT KUMAR BINDAL Associate Professor Databases Databases are developed on the IDEA that DATA is one of the critical materials of the Information Age Information, which is created by data,

More information

Understanding Data Warehousing. [by Alex Kriegel]

Understanding Data Warehousing. [by Alex Kriegel] Understanding Data Warehousing 2008 [by Alex Kriegel] Things to Discuss Who Needs a Data Warehouse? OLTP vs. Data Warehouse Business Intelligence Industrial Landscape Which Data Warehouse: Bill Inmon vs.

More information

Week 3 lecture slides

Week 3 lecture slides Week 3 lecture slides Topics Data Warehouses Online Analytical Processing Introduction to Data Cubes Textbook reference: Chapter 3 Data Warehouses A data warehouse is a collection of data specifically

More information

Enterprise Solutions. Data Warehouse & Business Intelligence Chapter-8

Enterprise Solutions. Data Warehouse & Business Intelligence Chapter-8 Enterprise Solutions Data Warehouse & Business Intelligence Chapter-8 Learning Objectives Concepts of Data Warehouse Business Intelligence, Analytics & Big Data Tools for DWH & BI Concepts of Data Warehouse

More information

Introducing Oracle Exalytics In-Memory Machine

Introducing Oracle Exalytics In-Memory Machine Introducing Oracle Exalytics In-Memory Machine Jon Ainsworth Director of Business Development Oracle EMEA Business Analytics 1 Copyright 2011, Oracle and/or its affiliates. All rights Agenda Topics Oracle

More information

An Overview of Data Warehousing, Data mining, OLAP and OLTP Technologies

An Overview of Data Warehousing, Data mining, OLAP and OLTP Technologies An Overview of Data Warehousing, Data mining, OLAP and OLTP Technologies Ashish Gahlot, Manoj Yadav Dronacharya college of engineering Farrukhnagar, Gurgaon,Haryana Abstract- Data warehousing, Data Mining,

More information

Data Warehousing and Data Mining Introduction

Data Warehousing and Data Mining Introduction Data Warehousing and Data Mining Introduction General introduction to DWDM Business intelligence OLTP vs. OLAP Data integration Methodological framework DW definition Acknowledgements: I am indebted to

More information

Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

More information

Master Data Management and Data Warehousing. Zahra Mansoori

Master Data Management and Data Warehousing. Zahra Mansoori Master Data Management and Data Warehousing Zahra Mansoori 1 1. Preference 2 IT landscape growth IT landscapes have grown into complex arrays of different systems, applications, and technologies over the

More information

CHAPTER 4 Data Warehouse Architecture

CHAPTER 4 Data Warehouse Architecture CHAPTER 4 Data Warehouse Architecture 4.1 Data Warehouse Architecture 4.2 Three-tier data warehouse architecture 4.3 Types of OLAP servers: ROLAP versus MOLAP versus HOLAP 4.4 Further development of Data

More information

OLAP & DATA MINING CS561-SPRING 2012 WPI, MOHAMED ELTABAKH

OLAP & DATA MINING CS561-SPRING 2012 WPI, MOHAMED ELTABAKH OLAP & DATA MINING CS561-SPRING 2012 WPI, MOHAMED ELTABAKH 1 Online Analytic Processing OLAP 2 OLAP OLAP: Online Analytic Processing OLAP queries are complex queries that Touch large amounts of data Discover

More information

Oracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc.

Oracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc. Oracle9i Data Warehouse Review Robert F. Edwards Dulcian, Inc. Agenda Oracle9i Server OLAP Server Analytical SQL Data Mining ETL Warehouse Builder 3i Oracle 9i Server Overview 9i Server = Data Warehouse

More information

Turkish Journal of Engineering, Science and Technology

Turkish Journal of Engineering, Science and Technology Turkish Journal of Engineering, Science and Technology 03 (2014) 106-110 Turkish Journal of Engineering, Science and Technology journal homepage: www.tujest.com Integrating Data Warehouse with OLAP Server

More information

META DATA QUALITY CONTROL ARCHITECTURE IN DATA WAREHOUSING

META DATA QUALITY CONTROL ARCHITECTURE IN DATA WAREHOUSING META DATA QUALITY CONTROL ARCHITECTURE IN DATA WAREHOUSING Ramesh Babu Palepu 1, Dr K V Sambasiva Rao 2 Dept of IT, Amrita Sai Institute of Science & Technology 1 MVR College of Engineering 2 asistithod@gmail.com

More information

Data Warehousing. Overview, Terminology, and Research Issues. Joachim Hammer. Joachim Hammer

Data Warehousing. Overview, Terminology, and Research Issues. Joachim Hammer. Joachim Hammer Data Warehousing Overview, Terminology, and Research Issues 1 Heterogeneous Database Integration Integration System World Wide Web Digital Libraries Scientific Databases Personal Databases Collects and

More information

Establish and maintain Center of Excellence (CoE) around Data Architecture

Establish and maintain Center of Excellence (CoE) around Data Architecture Senior BI Data Architect - Bensenville, IL The Company s Information Management Team is comprised of highly technical resources with diverse backgrounds in data warehouse development & support, business

More information

A Survey on Data Warehouse Architecture

A Survey on Data Warehouse Architecture A Survey on Data Warehouse Architecture Rajiv Senapati 1, D.Anil Kumar 2 1 Assistant Professor, Department of IT, G.I.E.T, Gunupur, India 2 Associate Professor, Department of CSE, G.I.E.T, Gunupur, India

More information

Hybrid Support Systems: a Business Intelligence Approach

Hybrid Support Systems: a Business Intelligence Approach Journal of Applied Business Information Systems, 2(2), 2011 57 Journal of Applied Business Information Systems http://www.jabis.ro Hybrid Support Systems: a Business Intelligence Approach Claudiu Brandas

More information

IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH

IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH IMPROVING DATA INTEGRATION FOR DATA WAREHOUSE: A DATA MINING APPROACH Kalinka Mihaylova Kaloyanova St. Kliment Ohridski University of Sofia, Faculty of Mathematics and Informatics Sofia 1164, Bulgaria

More information

Data W a Ware r house house and and OLAP II Week 6 1

Data W a Ware r house house and and OLAP II Week 6 1 Data Warehouse and OLAP II Week 6 1 Team Homework Assignment #8 Using a data warehousing tool and a data set, play four OLAP operations (Roll up (drill up), Drill down (roll down), Slice and dice, Pivot

More information

Data Warehousing and OLAP Technology for Knowledge Discovery

Data Warehousing and OLAP Technology for Knowledge Discovery 542 Data Warehousing and OLAP Technology for Knowledge Discovery Aparajita Suman Abstract Since time immemorial, libraries have been generating services using the knowledge stored in various repositories

More information

Fluency With Information Technology CSE100/IMT100

Fluency With Information Technology CSE100/IMT100 Fluency With Information Technology CSE100/IMT100 ),7 Larry Snyder & Mel Oyler, Instructors Ariel Kemp, Isaac Kunen, Gerome Miklau & Sean Squires, Teaching Assistants University of Washington, Autumn 1999

More information

Data warehouse Architectures and processes

Data warehouse Architectures and processes Database and data mining group, Data warehouse Architectures and processes DATA WAREHOUSE: ARCHITECTURES AND PROCESSES - 1 Database and data mining group, Data warehouse architectures Separation between

More information

SQL SERVER BUSINESS INTELLIGENCE (BI) - INTRODUCTION

SQL SERVER BUSINESS INTELLIGENCE (BI) - INTRODUCTION 1 SQL SERVER BUSINESS INTELLIGENCE (BI) - INTRODUCTION What is BI? Microsoft SQL Server 2008 provides a scalable Business Intelligence platform optimized for data integration, reporting, and analysis,

More information

LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES

LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES MUHAMMAD KHALEEL (0912125) SZABIST KARACHI CAMPUS Abstract. Data warehouse and online analytical processing (OLAP) both are core component for decision

More information

Chapter 5. Warehousing, Data Acquisition, Data. Visualization

Chapter 5. Warehousing, Data Acquisition, Data. Visualization Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization 5-1 Learning Objectives

More information

2074 : Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000

2074 : Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000 2074 : Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000 Introduction This course provides students with the knowledge and skills necessary to design, implement, and deploy OLAP

More information

Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence. Module Curriculum

Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence. Module Curriculum Republic Polytechnic School of Information and Communications Technology C355 Business Intelligence Module Curriculum This document addresses the content related abilities, with reference to the module.

More information

Migrating a Discoverer System to Oracle Business Intelligence Enterprise Edition

Migrating a Discoverer System to Oracle Business Intelligence Enterprise Edition Migrating a Discoverer System to Oracle Business Intelligence Enterprise Edition Milena Gerova President Bulgarian Oracle User Group mgerova@technologica.com Who am I Project Manager in TechnoLogica Ltd

More information

Building Cubes and Analyzing Data using Oracle OLAP 11g

Building Cubes and Analyzing Data using Oracle OLAP 11g Building Cubes and Analyzing Data using Oracle OLAP 11g Collaborate '08 Session 219 Chris Claterbos claterbos@vlamis.com Vlamis Software Solutions, Inc. 816-729-1034 http://www.vlamis.com Copyright 2007,

More information

Data Warehouse (DW) Maturity Assessment Questionnaire

Data Warehouse (DW) Maturity Assessment Questionnaire Data Warehouse (DW) Maturity Assessment Questionnaire Catalina Sacu - csacu@students.cs.uu.nl Marco Spruit m.r.spruit@cs.uu.nl Frank Habers fhabers@inergy.nl September, 2010 Technical Report UU-CS-2010-021

More information

SENG 520, Experience with a high-level programming language. (304) 579-7726, Jeff.Edgell@comcast.net

SENG 520, Experience with a high-level programming language. (304) 579-7726, Jeff.Edgell@comcast.net Course : Semester : Course Format And Credit hours : Prerequisites : Data Warehousing and Business Intelligence Summer (Odd Years) online 3 hr Credit SENG 520, Experience with a high-level programming

More information

MDM and Data Warehousing Complement Each Other

MDM and Data Warehousing Complement Each Other Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There

More information

Data Integration and ETL Process

Data Integration and ETL Process Data Integration and ETL Process Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, second

More information

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP

OLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP Data Warehousing and End-User Access Tools OLAP and Data Mining Accompanying growth in data warehouses is increasing demands for more powerful access tools providing advanced analytical capabilities. Key

More information

70-467: Designing Business Intelligence Solutions with Microsoft SQL Server

70-467: Designing Business Intelligence Solutions with Microsoft SQL Server 70-467: Designing Business Intelligence Solutions with Microsoft SQL Server The following tables show where changes to exam 70-467 have been made to include updates that relate to SQL Server 2014 tasks.

More information

The 7 th Balkan Conference on Operational Research BACOR 05 Constanta, May 2005, Romania DATA WAREHOUSE MANAGEMENT SYSTEM A CASE STUDY

The 7 th Balkan Conference on Operational Research BACOR 05 Constanta, May 2005, Romania DATA WAREHOUSE MANAGEMENT SYSTEM A CASE STUDY The 7 th Balkan Conference on Operational Research BACOR 05 Constanta, May 2005, Romania DATA WAREHOUSE MANAGEMENT SYSTEM A CASE STUDY DARKO KRULJ Trizon Group, Belgrade, Serbia and Montenegro. MILUTIN

More information

Meta-data and Data Mart solutions for better understanding for data and information in E-government Monitoring

Meta-data and Data Mart solutions for better understanding for data and information in E-government Monitoring www.ijcsi.org 78 Meta-data and Data Mart solutions for better understanding for data and information in E-government Monitoring Mohammed Mohammed 1 Mohammed Anad 2 Anwar Mzher 3 Ahmed Hasson 4 2 faculty

More information

Data Warehousing, OLAP, and Data Mining

Data Warehousing, OLAP, and Data Mining Data Warehousing, OLAP, and Marek Rychly mrychly@strathmore.edu Strathmore University, @ilabafrica & Brno University of Technology, Faculty of Information Technology Advanced Databases and Enterprise Systems

More information

Mario Guarracino. Data warehousing

Mario Guarracino. Data warehousing Data warehousing Introduction Since the mid-nineties, it became clear that the databases for analysis and business intelligence need to be separate from operational. In this lecture we will review the

More information

Data Warehousing: Data Models and OLAP operations. By Kishore Jaladi kishorejaladi@yahoo.com

Data Warehousing: Data Models and OLAP operations. By Kishore Jaladi kishorejaladi@yahoo.com Data Warehousing: Data Models and OLAP operations By Kishore Jaladi kishorejaladi@yahoo.com Topics Covered 1. Understanding the term Data Warehousing 2. Three-tier Decision Support Systems 3. Approaches

More information

Data warehouse and Business Intelligence Collateral

Data warehouse and Business Intelligence Collateral Data warehouse and Business Intelligence Collateral Page 1 of 12 DATA WAREHOUSE AND BUSINESS INTELLIGENCE COLLATERAL Brains for the corporate brawn: In the current scenario of the business world, the competition

More information

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS PRODUCT FACTS & FEATURES KEY FEATURES Comprehensive, best-of-breed capabilities 100 percent thin client interface Intelligence across multiple

More information

COURSE OUTLINE. Track 1 Advanced Data Modeling, Analysis and Design

COURSE OUTLINE. Track 1 Advanced Data Modeling, Analysis and Design COURSE OUTLINE Track 1 Advanced Data Modeling, Analysis and Design TDWI Advanced Data Modeling Techniques Module One Data Modeling Concepts Data Models in Context Zachman Framework Overview Levels of Data

More information

B. 3 essay questions. Samples of potential questions are available in part IV. This list is not exhaustive it is just a sample.

B. 3 essay questions. Samples of potential questions are available in part IV. This list is not exhaustive it is just a sample. IS482/682 Information for First Test I. What is the structure of the test? A. 20-25 multiple-choice questions. B. 3 essay questions. Samples of potential questions are available in part IV. This list is

More information

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS

ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS Oracle Fusion editions of Oracle's Hyperion performance management products are currently available only on Microsoft Windows server platforms. The following is intended to outline our general product

More information

Moving Large Data at a Blinding Speed for Critical Business Intelligence. A competitive advantage

Moving Large Data at a Blinding Speed for Critical Business Intelligence. A competitive advantage Moving Large Data at a Blinding Speed for Critical Business Intelligence A competitive advantage Intelligent Data In Real Time How do you detect and stop a Money Laundering transaction just about to take

More information

Class News. Basic Elements of the Data Warehouse" 1/22/13. CSPP 53017: Data Warehousing Winter 2013" Lecture 2" Svetlozar Nestorov" "

Class News. Basic Elements of the Data Warehouse 1/22/13. CSPP 53017: Data Warehousing Winter 2013 Lecture 2 Svetlozar Nestorov CSPP 53017: Data Warehousing Winter 2013 Lecture 2 Svetlozar Nestorov Class News Class web page: http://bit.ly/wtwxv9 Subscribe to the mailing list Homework 1 is out now; due by 1:59am on Tue, Jan 29.

More information

How to Enhance Traditional BI Architecture to Leverage Big Data

How to Enhance Traditional BI Architecture to Leverage Big Data B I G D ATA How to Enhance Traditional BI Architecture to Leverage Big Data Contents Executive Summary... 1 Traditional BI - DataStack 2.0 Architecture... 2 Benefits of Traditional BI - DataStack 2.0...

More information

An Architectural Review Of Integrating MicroStrategy With SAP BW

An Architectural Review Of Integrating MicroStrategy With SAP BW An Architectural Review Of Integrating MicroStrategy With SAP BW Manish Jindal MicroStrategy Principal HCL Objectives To understand how MicroStrategy integrates with SAP BW Discuss various Design Options

More information

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1 Slide 29-1 Chapter 29 Overview of Data Warehousing and OLAP Chapter 29 Outline Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics

More information

A Service-oriented Architecture for Business Intelligence

A Service-oriented Architecture for Business Intelligence A Service-oriented Architecture for Business Intelligence Liya Wu 1, Gilad Barash 1, Claudio Bartolini 2 1 HP Software 2 HP Laboratories {name.surname@hp.com} Abstract Business intelligence is a business

More information

Anwendersoftware Anwendungssoftwares a. Data-Warehouse-, Data-Mining- and OLAP-Technologies. Online Analytic Processing

Anwendersoftware Anwendungssoftwares a. Data-Warehouse-, Data-Mining- and OLAP-Technologies. Online Analytic Processing Anwendungssoftwares a Data-Warehouse-, Data-Mining- and OLAP-Technologies Online Analytic Processing Online Analytic Processing OLAP Online Analytic Processing Technologies and tools that support (ad-hoc)

More information

When to consider OLAP?

When to consider OLAP? When to consider OLAP? Author: Prakash Kewalramani Organization: Evaltech, Inc. Evaltech Research Group, Data Warehousing Practice. Date: 03/10/08 Email: erg@evaltech.com Abstract: Do you need an OLAP

More information

ETL-EXTRACT, TRANSFORM & LOAD TESTING

ETL-EXTRACT, TRANSFORM & LOAD TESTING ETL-EXTRACT, TRANSFORM & LOAD TESTING Rajesh Popli Manager (Quality), Nagarro Software Pvt. Ltd., Gurgaon, INDIA rajesh.popli@nagarro.com ABSTRACT Data is most important part in any organization. Data

More information

Week 13: Data Warehousing. Warehousing

Week 13: Data Warehousing. Warehousing 1 Week 13: Data Warehousing Warehousing Growing industry: $8 billion in 1998 Range from desktop to huge: Walmart: 900-CPU, 2,700 disk, 23TB Teradata system Lots of buzzwords, hype slice & dice, rollup,

More information

DATA WAREHOUSING APPLICATIONS: AN ANALYTICAL TOOL FOR DECISION SUPPORT SYSTEM

DATA WAREHOUSING APPLICATIONS: AN ANALYTICAL TOOL FOR DECISION SUPPORT SYSTEM DATA WAREHOUSING APPLICATIONS: AN ANALYTICAL TOOL FOR DECISION SUPPORT SYSTEM MOHAMMED SHAFEEQ AHMED Guest Lecturer, Department of Computer Science, Gulbarga University, Gulbarga, Karnataka, India (e-mail:

More information

INTEROPERABILITY IN DATA WAREHOUSES

INTEROPERABILITY IN DATA WAREHOUSES INTEROPERABILITY IN DATA WAREHOUSES Riccardo Torlone Roma Tre University http://torlone.dia.uniroma3.it/ SYNONYMS Data warehouse integration DEFINITION The term refers to the ability of combining the content

More information

Breadboard BI. Unlocking ERP Data Using Open Source Tools By Christopher Lavigne

Breadboard BI. Unlocking ERP Data Using Open Source Tools By Christopher Lavigne Breadboard BI Unlocking ERP Data Using Open Source Tools By Christopher Lavigne Introduction Organizations have made enormous investments in ERP applications like JD Edwards, PeopleSoft and SAP. These

More information

Super-Charged Oracle Business Intelligence with Essbase and SmartView

Super-Charged Oracle Business Intelligence with Essbase and SmartView Specialized. Recognized. Preferred. The right partner makes all the difference. Super-Charged Oracle Business Intelligence with Essbase and SmartView By: Gautham Sampath Pinellas County & Patrick Callahan

More information

Safe Harbor Statement

Safe Harbor Statement Safe Harbor Statement "Safe Harbor" Statement: Statements in this presentation relating to Oracle's future plans, expectations, beliefs, intentions and prospects are "forward-looking statements" and are

More information

BENEFITS OF AUTOMATING DATA WAREHOUSING

BENEFITS OF AUTOMATING DATA WAREHOUSING BENEFITS OF AUTOMATING DATA WAREHOUSING Introduction...2 The Process...2 The Problem...2 The Solution...2 Benefits...2 Background...3 Automating the Data Warehouse with UC4 Workload Automation Suite...3

More information

Data Warehousing. Read chapter 13 of Riguzzi et al Sistemi Informativi. Slides derived from those by Hector Garcia-Molina

Data Warehousing. Read chapter 13 of Riguzzi et al Sistemi Informativi. Slides derived from those by Hector Garcia-Molina Data Warehousing Read chapter 13 of Riguzzi et al Sistemi Informativi Slides derived from those by Hector Garcia-Molina What is a Warehouse? Collection of diverse data subject oriented aimed at executive,

More information

Data Warehousing & Business Intelligence

Data Warehousing & Business Intelligence Data Warehousing & Business Intelligence Multicom d.o.o. Vladimira Preloga 11 10000 Zagreb multicom@multicom.hr www.multicom.hr Page 1 / 15 Table of Contents 1 General... 3 2 Solution Description... 6

More information

Microsoft Data Warehouse in Depth

Microsoft Data Warehouse in Depth Microsoft Data Warehouse in Depth 1 P a g e Duration What s new Why attend Who should attend Course format and prerequisites 4 days The course materials have been refreshed to align with the second edition

More information

<Insert Picture Here> Enhancing the Performance and Analytic Content of the Data Warehouse Using Oracle OLAP Option

<Insert Picture Here> Enhancing the Performance and Analytic Content of the Data Warehouse Using Oracle OLAP Option Enhancing the Performance and Analytic Content of the Data Warehouse Using Oracle OLAP Option The following is intended to outline our general product direction. It is intended for

More information

A Comparison of Business Intelligence Strategies and Platforms

A Comparison of Business Intelligence Strategies and Platforms Green Hill Analysis A Comparison of Business Intelligence Strategies and Platforms Comparing Microsoft, Oracle, IBM, and Hyperion By Mitch Kramer, Green Hill Analysis Prepared for Microsoft Corporation

More information

Chapter 5. Learning Objectives. DW Development and ETL

Chapter 5. Learning Objectives. DW Development and ETL Chapter 5 DW Development and ETL Learning Objectives Explain data integration and the extraction, transformation, and load (ETL) processes Basic DW development methodologies Describe real-time (active)

More information

THE Web is nowadays the world s largest source of

THE Web is nowadays the world s largest source of 940 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 20, NO. 7, JULY 2008 Integrating Data Warehouses with Web Data: ASurvey Juan Manuel Pérez, Rafael Berlanga, María JoséAramburu, and Torben

More information

Introduction to Data Warehousing. Ms Swapnil Shrivastava swapnil@konark.ncst.ernet.in

Introduction to Data Warehousing. Ms Swapnil Shrivastava swapnil@konark.ncst.ernet.in Introduction to Data Warehousing Ms Swapnil Shrivastava swapnil@konark.ncst.ernet.in Necessity is the mother of invention Why Data Warehouse? Scenario 1 ABC Pvt Ltd is a company with branches at Mumbai,

More information

A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT

A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT Xiufeng Liu 1 & Xiaofeng Luo 2 1 Department of Computer Science Aalborg University, Selma Lagerlofs Vej 300, DK-9220 Aalborg, Denmark 2 Telecommunication Engineering

More information

PREFACE INTRODUCTION MULTI-DIMENSIONAL MODEL. Chris Claterbos, Vlamis Software Solutions, Inc. dvlamis@vlamis.com

PREFACE INTRODUCTION MULTI-DIMENSIONAL MODEL. Chris Claterbos, Vlamis Software Solutions, Inc. dvlamis@vlamis.com BUILDING CUBES AND ANALYZING DATA USING ORACLE OLAP 11G Chris Claterbos, Vlamis Software Solutions, Inc. dvlamis@vlamis.com PREFACE As of this writing, Oracle Business Intelligence and Oracle OLAP are

More information