Evolving Data Warehouse Architectures

Size: px
Start display at page:

Download "Evolving Data Warehouse Architectures"

Transcription

1 Evolving Data Warehouse Architectures In the Age of Big Data Philip Russom April 15, 2014

2 TDWI would like to thank the following companies for sponsoring the 2014 TDWI Best Practices research report: Evolving Data Warehouse Architectures This presentation is based on the findings of that report. STAY TUNED At the end of this webinar, learn how to download a free copy of the report.

3 Agenda Definitions of Data Warehouse Architectures Drivers of Change Benefits & Barriers From EDWs to DWEs Role of Hadoop Analytics versus Reporting Trends among Architectural Components and Practices Top Ten Priorities PLEASE #TDWI, #EDW, #DataWarehouse, #DataArchitecture, #Analytics, #Hadoop

4 Upcoming Points There isn t one, single architecture for all data warehouses (DWs) Each org is different Expect multiple architectures A well-designed DW has multiple architectural layers Architectural approaches get mixed together into hybrids A DW architecture interacts with architectures for data integration, reporting, analytics, operational applications, etc. The warehouse is still vital, even central But it s evolving into a multiple platform environment Architecture is more important than ever, but now as a logical design that s deployed over multiple physical platforms Please don t ask me to draw a Reference Architecture for DWs Given the current diversity, there isn t just one. But I ll describe many.

5 What do you think data warehouse architecture is? Select all that apply. Source: TDWI survey run in late Based on 1197 responses from 538 respondents. 2.2 responses per respondent, on average.

6 Logical versus Physical DW Architectures And Other Architectural Components that Coexist Today s Focus Logical architecture mostly about data models and their relationships, with a focus on how these represent organizational entities and processes Data standards including standards for data modeling, data quality metrics, interfaces for data integration, programming style, format standards, etc. Physical architecture mostly a plan for deploying data and data structures based on the workload and platform requirements of each System architecture a topology of hardware servers and software servers, plus the interfaces and networks that tie them together

7 Drivers of Change Does your primary enterprise data warehouse have an architectural design? Yes 79% No 18% Don t know 3% Is the architecture of your data warehouse environment evolving? Yes moderately 54% Yes dramatically 22% No except with DW updates 22% Don t know 2% What technical issues or practices are driving change in your DW architecture? Advanced analytics 57% Increasing data volumes 56% Real-time operations 41% Business performance mgt 38% OLAP 30% Non-relational data 25% Virtualization of data 23% Cloud adoption 21% Streaming data 15% What business issues or practices are driving change in your DW architecture? Competitiveness 45% Fast-paced business processes 43% Compliance 29% Funding 29% Sponsorship 26% Reorganizations 25% Centralizing business control 30% Departmental power struggles 19% Mergers and acquisitions 18% Source: TDWI survey run in late Based on 538 respondents.

8 Benefits of Multi-Platform Architecture In priority order, based on survey responses All data analytics, in general (61%) Many new platforms are built for analytics: DW appliances, columnar databases, NoSQL databases, Hadoop. With a multi-platform portfolio, users can match an analytic workload to best platform. A diverse platform portfolio can handle a diverse range of data types. This is key to embracing the unstructured and schema-free data types found in most big data. Enables broad data exploration and discovery (43%) A more diverse platform portfolio can aid a business Additional platforms are key to addressing new business requirements (36%), especially data-oriented ones like analytics (61%), more numerous business insights (34%), business optimization (30%) Handling data in real time usually requires an additional purpose-built system. Traditional relational databases and batch-oriented Hadoop systems were not built for real-time operations (33%), though many organizations need faster business processes (26%). Adding low-cost platforms to a DW environ makes big data more affordable. DW appliances, columnar RDBMSs, Hadoop & NoSQL all lower cost for data staging for data warehousing (20%) and data archiving (16%). Source: TDWI survey run in late Based on 538 respondents.

9 Barriers to Multi-Platform Architecture In priority order, based on survey responses Inadequate staffing or skills (47%) is the most prominent barrier. Immaturity with new data types and sources (23%) plus new technologies for Hadoop, event processing, and so on make them unprepared for the complexity of multi-platform designs (25%). As usual, organizational and business issues should be settled first. Data ownership and other politics (43%), a lack of business sponsorship (38%), a lack of a compelling business case (25%) A number of data management issues should be addressed. Data integration complexity (36%), poor data quality (34%), lack of data architecture (29%), and data security, privacy, and governance issues (25%) As with any new IT initiative, proper funding is key. Account for the cost of acquiring multiple platforms (25%) and the cost of administering multiple platforms (27%) Source: TDWI survey run in late Based on 538 respondents.

10 WHY CAN T A DATA WAREHOUSE DO EVERYTHING? Square Peg Workloads may not fit Round Hole DW Architectures Most data warehouses were designed and optimized for common deliverables and methods: Standard reports, dashboards, performance mgt, online analytic processing (OLAP) This is a design and architectural decision made by users, not a failing of vendor platforms Can/should all DW & analytic workloads run on your EDW? If your EDW can handle multiple mixed concurrent workloads with performance and without impeding other workloads, then run all workloads (including analytics) on the EDW, for simplicity s sake If not, you may need additional data platforms for some workloads

11 Multi-Platform Data Warehouse Environments Many enterprise data warehouses (EDWs) are evolving into multi-platform data warehouse environments (DWEs). Users continue to add additional standalone data platforms to their warehouse tool and platform portfolio. The new platforms don t replace the core warehouse, because it is still the best platform for the data that goes into standards reports, dashboards, performance management, and OLAP. Instead, the new platforms complement the warehouse, because they are optimized for workloads that manage, process, and analyze new forms of big data, non-structured data, and real-time data.

12 Ramifications of a Multi-Platform DW Environ Workload-centric DW architecture Assumes that some workloads and their data are best offloaded from the core DW and taken to a platform more suited to them Workloads and data for advanced analytics (not OLAP), SQL-based analytics, unstructured data, massive big data, real time Distributed DW architecture This simply means that data and data structures (as defined in a logical architectural layer) are distributed across multiple physical data platforms Again, the logical layer is the big picture needed with many platforms A distributed DW architecture is both good and bad Good if it serves the unique requirements of multiple workloads and the users that depend on them Bad if platforms proliferate like the dreaded data marts of yore

13 Growing Complexity in DW System Architectures The technology stack for DW, BI, analytics, and data integration has always been a multi-platform environment. What s new? The trend toward a portfolio of many data platforms has accelerated. Over The Passage of Time Federated Data Federated Marts Data Federated Marts Data Marts Customer Mart Customer or ODS Mart or ODS Real Time ODS DW from a Merger Columnar DBMS Columnar DBMS Map Reduce Complex, Event Processing Data Warehouse Star or Multi- Snowflake dimensional Scheme Data Models Data Staging Data Areas Staging Data Areas Staging Areas Metrics for Performance Mgt OLAP Cubes OLAP DBMSs Detailed Source Detailed Data Source Detailed Data Source Data Analytic Sand Box Data Federation & Virtualization DW Appliance DW Appliances Hadoop Distributed Hadoop File Distributed Sys File Sys No-SQL Database No-SQL Database Streaming Data Tools

14 EDW Which of the following best describes your extended data warehouse environment today? Pure, central, monolithic EDWs are relatively rare (15%, far left) Likewise, environments without a DW are equally rare (15%, far right) EDWs mix well in hybrid environments (68%, middle three) Central monolithic EDW with no other data platforms Central EDW with many additional data platforms No true EDW, but many workloadspecific data platforms instead 15% 37% 16% 15% 15% DWE Central EDW with a few additional data platforms Many workload-specific data platforms; EDW is present but not the center Other (2%) Source: TDWI survey run in late Based on 538 respondents.

15 Which of the following best describes your organization s strategy for evolving your DW environment and its architecture, relative to big data? Most survey respondents plan to extend an existing DW (41%, far left) Few will deploy new data platforms (25%) 29% have no strategy for DW evolution or addressing big data Extend existing core DW to accommodate big data and other new requirements No strategy for DW architecture, though we need one Other (5%) 41% 25% 23% 6% Deploy new data management systems specifically for big data, analytics, real time, etc. No strategy for DW architecture, because we don't need one Source: TDWI survey run in late Based on 538 respondents.

16 Hadoop is a Useful Addition to DW Architectures IT COMPLEMENTS AND EXTENDS DATA WAREHOUSES HDFS extends DW Architectures Managing multi-structured data Repository for detailed source data Processing big data for analytics Advanced forms of algorithmic analytics Data staging on steroids ELT push-down processing Inexpensive compared to average DW Hadoop also contributes outside DWs Imagine HDFS as shared infrastructure, similar to SAN & NAS Imagine a huge, live archive Imagine content mgt on steroids

17 Reporting and Analytics have Different Requirements for Data and DW Architecture Reporting is mostly about entities and facts you know well, represented by highly polished data that you know well. Carefully modeled and cleansed data with rich metadata and master data that s managed in a data warehouse. Most users designed their DWs first and foremost as a repository for reporting and similar practices such as OLAP, performance management, dashboards, and operational BI. Advanced analytics enables the discovery of new facts you didn t know, based on the exploration and analysis of data that s probably new to you. Unlike the pristine data that reports operate on, advanced analytics works best with detailed source data in its original (even messy) form, using discovery oriented technologies, such as ad hoc queries, search, mining, statistics, predictive algorithms, and natural language processing.

18 Commitment & Growth Components relative to DW Architecture Some components are poised for aggressive adoption by users. Analytics is driving most adoption of new platforms & features. In-memory analytics (36%), analytic sandboxes (29%) Managing non-relational big data is also a pressing need for many organizations. HDFS (34%), open-source MapReduce (32%), vendor-built MapReduce (25%), NoSQL databases (24%) Real-time is just as important as analytics and big data. In-memory database (34%), in-database analytics (29%), solid-state drives (25%), real-time data (24%) Relational technology is more relevant than ever, but in updated forms. Columnar DBMSs (27%), DW appliances (23%)

19 Top Ten Priorities for DW Architecture These are recommendations, requirements, or rules that can guide you. 1. Recognize that successful data warehouse architectures have integrated logical and physical layers, plus other components. 2. Determine the business and technical drivers in your organization, and let those determine the evolution of your DW architecture. 3. Beware that the leading barrier to successful DW architecture is inadequate staffing and skills. 4. Address other barriers for sponsorship, funding, and improvements to data management infrastructure. 5. Turn on unused features in existing platforms. 6. Establish DW architectures and standards, but be open to exceptions. 7. Be open to hybrids and alternate standards. 8. Consider Hadoop as a DW complement. 9. Remember that analytics and reporting have different data and DW architectural requirements. 10. Don t expect the new stuff to replace the old stuff.

20 Download a free copy of the report that this Webinar is based on EVOLVING DATA WAREHOUSE ARCHITECTURES IN THE AGE OF BIG DATA Download the report in a PDF file at: tdwi.org/bpreports Feel free to distribute the PDF file of any TDWI Best Practices Report

21 Q & A Philip Russom Research Director for Data Mgt TDWI on Twitter linkedin.com/in/philiprussom

Integrating Hadoop. Into Business Intelligence & Data Warehousing. Philip Russom TDWI Research Director for Data Management, April 9 2013

Integrating Hadoop. Into Business Intelligence & Data Warehousing. Philip Russom TDWI Research Director for Data Management, April 9 2013 Integrating Hadoop Into Business Intelligence & Data Warehousing Philip Russom TDWI Research Director for Data Management, April 9 2013 TDWI would like to thank the following companies for sponsoring the

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,

More information

Evolving Data Warehouse Architectures

Evolving Data Warehouse Architectures TDWI RESEARCh Second Quarter 2014 BEST PRACTICES REPORT Evolving Data Warehouse Architectures In the Age of Big Data By Philip Russom Co-sponsored by: tdwi.org TDWI research BEST PRACTICES REPORT Second

More information

Evolving Data Warehouse Architectures

Evolving Data Warehouse Architectures TDWI research Second Quarter 2014 BEST PRACTICES REPORT Evolving Data Warehouse Architectures In the Age of Big Data By Philip Russom tdwi.org Research Sponsors Research Sponsors Actian Cloudera Datawatch

More information

Achieving Business Value through Big Data Analytics Philip Russom

Achieving Business Value through Big Data Analytics Philip Russom Achieving Business Value through Big Data Analytics Philip Russom TDWI Research Director for Data Management October 3, 2012 Sponsor 2 Speakers Philip Russom Research Director, Data Management, TDWI Brian

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management May 7, 2013 Sponsor Speakers Philip Russom TDWI Research Director, Data Management Chris Twogood VP, Product and

More information

A Next-Generation Analytics Ecosystem for Big Data. Colin White, BI Research September 2012 Sponsored by ParAccel

A Next-Generation Analytics Ecosystem for Big Data. Colin White, BI Research September 2012 Sponsored by ParAccel A Next-Generation Analytics Ecosystem for Big Data Colin White, BI Research September 2012 Sponsored by ParAccel BIG DATA IS BIG NEWS The value of big data lies in the business analytics that can be generated

More information

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS!

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS! The Bloor Group IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS VENDOR PROFILE The IBM Big Data Landscape IBM can legitimately claim to have been involved in Big Data and to have a much broader

More information

HADOOP BEST PRACTICES

HADOOP BEST PRACTICES TDWI RESEARCH TDWI CHECKLIST REPORT HADOOP BEST PRACTICES For Data Warehousing, Data Integration, and Analytics By Philip Russom Sponsored by tdwi.org OCTOBER 2013 TDWI CHECKLIST REPORT HADOOP BEST PRACTICES

More information

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Wayne W. Eckerson Director of Research, TechTarget Founder, BI Leadership Forum Business Analytics

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

Introducing Oracle Exalytics In-Memory Machine

Introducing Oracle Exalytics In-Memory Machine Introducing Oracle Exalytics In-Memory Machine Jon Ainsworth Director of Business Development Oracle EMEA Business Analytics 1 Copyright 2011, Oracle and/or its affiliates. All rights Agenda Topics Oracle

More information

E-Guide BRINGING BIG DATA INTO A DATA WAREHOUSE ENVIRONMENT

E-Guide BRINGING BIG DATA INTO A DATA WAREHOUSE ENVIRONMENT E-Guide BRINGING BIG DATA INTO A DATA WAREHOUSE ENVIRONMENT I n many organizations, the growing volume and increasing complexity of data are straining performance and highlighting the limits of the traditional

More information

Five Technology Trends for Improved Business Intelligence Performance

Five Technology Trends for Improved Business Intelligence Performance TechTarget Enterprise Applications Media E-Book Five Technology Trends for Improved Business Intelligence Performance The demand for business intelligence data only continues to increase, putting BI vendors

More information

IBM Data Warehousing and Analytics Portfolio Summary

IBM Data Warehousing and Analytics Portfolio Summary IBM Information Management IBM Data Warehousing and Analytics Portfolio Summary Information Management Mike McCarthy IBM Corporation mmccart1@us.ibm.com IBM Information Management Portfolio Current Data

More information

An Integrated Analytics & Big Data Infrastructure September 21, 2012 Robert Stackowiak, Vice President Data Systems Architecture Oracle Enterprise

An Integrated Analytics & Big Data Infrastructure September 21, 2012 Robert Stackowiak, Vice President Data Systems Architecture Oracle Enterprise An Integrated Analytics & Big Data Infrastructure September 21, 2012 Robert Stackowiak, Vice President Data Systems Architecture Oracle Enterprise Solutions Group The following is intended to outline our

More information

DAMA NY DAMA Day October 17, 2013 IBM 590 Madison Avenue 12th floor New York, NY

DAMA NY DAMA Day October 17, 2013 IBM 590 Madison Avenue 12th floor New York, NY Big Data Analytics DAMA NY DAMA Day October 17, 2013 IBM 590 Madison Avenue 12th floor New York, NY Tom Haughey InfoModel, LLC 868 Woodfield Road Franklin Lakes, NJ 07417 201 755 3350 tom.haughey@infomodelusa.com

More information

Data Warehouse design

Data Warehouse design Data Warehouse design Design of Enterprise Systems University of Pavia 07/01/2014-1- Data Warehouse design NEW TRENDS - 2- Big Data Big data is first and foremost about data volume, namely large data sets

More information

Using Big Data for Smarter Decision Making. Colin White, BI Research July 2011 Sponsored by IBM

Using Big Data for Smarter Decision Making. Colin White, BI Research July 2011 Sponsored by IBM Using Big Data for Smarter Decision Making Colin White, BI Research July 2011 Sponsored by IBM USING BIG DATA FOR SMARTER DECISION MAKING To increase competitiveness, 83% of CIOs have visionary plans that

More information

BIG DATA APPLIANCES. July 23, TDWI. R Sathyanarayana. Enterprise Information Management & Analytics Practice EMC Consulting

BIG DATA APPLIANCES. July 23, TDWI. R Sathyanarayana. Enterprise Information Management & Analytics Practice EMC Consulting BIG DATA APPLIANCES July 23, TDWI R Sathyanarayana Enterprise Information Management & Analytics Practice EMC Consulting 1 Big data are datasets that grow so large that they become awkward to work with

More information

Agile Business Intelligence Data Lake Architecture

Agile Business Intelligence Data Lake Architecture Agile Business Intelligence Data Lake Architecture TABLE OF CONTENTS Introduction... 2 Data Lake Architecture... 2 Step 1 Extract From Source Data... 5 Step 2 Register And Catalogue Data Sets... 5 Step

More information

E-Guide HADOOP MYTHS BUSTED

E-Guide HADOOP MYTHS BUSTED E-Guide HADOOP MYTHS BUSTED I n many organizations, the growing volume and increasing complexity of data are straining performance and highlighting the limits of the traditional data warehouse. Today,

More information

Traditional BI vs. Business Data Lake A comparison

Traditional BI vs. Business Data Lake A comparison Traditional BI vs. Business Data Lake A comparison The need for new thinking around data storage and analysis Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses

More information

INTEGRATING HADOOP INTO BUSINESS INTELLIGENCE AND DATA WAREHOUSING

INTEGRATING HADOOP INTO BUSINESS INTELLIGENCE AND DATA WAREHOUSING TDWI research TDWI BEST PRACTICES REPORT SECOND QUARTER 2013 INTEGRATING HADOOP INTO BUSINESS INTELLIGENCE AND DATA WAREHOUSING By Philip Russom tdwi.org Research Sponsors Research Sponsors Cloudera EMC

More information

Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data

Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data IBM Software Group Important Disclaimer THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL

More information

White Paper. Unified Data Integration Across Big Data Platforms

White Paper. Unified Data Integration Across Big Data Platforms White Paper Unified Data Integration Across Big Data Platforms Contents Business Problem... 2 Unified Big Data Integration... 3 Diyotta Solution Overview... 4 Data Warehouse Project Implementation using

More information

Unified Data Integration Across Big Data Platforms

Unified Data Integration Across Big Data Platforms Unified Data Integration Across Big Data Platforms Contents Business Problem... 2 Unified Big Data Integration... 3 Diyotta Solution Overview... 4 Data Warehouse Project Implementation using ELT... 6 Diyotta

More information

Bringing the Power of SAS to Hadoop. White Paper

Bringing the Power of SAS to Hadoop. White Paper White Paper Bringing the Power of SAS to Hadoop Combine SAS World-Class Analytic Strength with Hadoop s Low-Cost, Distributed Data Storage to Uncover Hidden Opportunities Contents Introduction... 1 What

More information

Luncheon Webinar Series May 13, 2013

Luncheon Webinar Series May 13, 2013 Luncheon Webinar Series May 13, 2013 InfoSphere DataStage is Big Data Integration Sponsored By: Presented by : Tony Curcio, InfoSphere Product Management 0 InfoSphere DataStage is Big Data Integration

More information

Evolution to Revolution: Big Data 2.0

Evolution to Revolution: Big Data 2.0 Evolution to Revolution: Big Data 2.0 An ENTERPRISE MANAGEMENT ASSOCIATES (EMA ) White Paper Prepared for Actian March 2014 IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING Table of Contents

More information

Big Data Defined Introducing DataStack 3.0

Big Data Defined Introducing DataStack 3.0 Big Data Big Data Defined Introducing DataStack 3.0 Inside: Executive Summary... 1 Introduction... 2 Emergence of DataStack 3.0... 3 DataStack 1.0 to 2.0... 4 DataStack 2.0 Refined for Large Data & Analytics...

More information

The Growing Practice of Operational Data Integration. Philip Russom Senior Manager, TDWI Research April 14, 2010

The Growing Practice of Operational Data Integration. Philip Russom Senior Manager, TDWI Research April 14, 2010 The Growing Practice of Operational Data Integration Philip Russom Senior Manager, TDWI Research April 14, 2010 Sponsor: 2 Speakers: Philip Russom Senior Manager, TDWI Research Gavin Day VP of Operations

More information

ISSN: Page 120

ISSN: Page 120 Satellite Conference ICSTSD 2016 International Conference on Science and Technology for Sustainable Development, Kuala Lumpur, Modernization of Data Warehousing through Open Source Softwares Preeti P.

More information

Hadoop for the Enterprise:

Hadoop for the Enterprise: TDWI RESEARCH SECOND QUARTER 2015 TDWI BEST PRACTICES REPORT Hadoop for the Enterprise: Making Data Management Massively Scalable, Agile, Feature-Rich, and Cost-Effective By Philip Russom Co-sponsored

More information

Artur Borycki. Director International Solutions Marketing

Artur Borycki. Director International Solutions Marketing Artur Borycki Director International Solutions Agenda! Evolution of Teradata s Unified Architecture Analytical and Workloads! Teradata s Reference Information Architecture Evolution of Teradata s" Unified

More information

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform...

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform... Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure Requirements... 5 Solution Spectrum... 6 Oracle s Big Data

More information

Driving Peak Performance. 2013 IBM Corporation

Driving Peak Performance. 2013 IBM Corporation Driving Peak Performance 1 Session 2: Driving Peak Performance Abstract We know you want the fastest performance possible for your deployments, and yet that relies on many choices across data storage,

More information

Active Data Archiving

Active Data Archiving TDWI RESEARCH TDWI CHECKLIST REPORT Active Data Archiving For Big Data, Compliance, and Analytics By Philip Russom Sponsored by: tdwi.org MAY 2014 TDWI CHECKLIST REPORT ACTIVE DATA ARCHIVING For Big Data,

More information

The Intersection of Big Data and Analytics. Philip Russom TDWI Research Director for Data Management May 5, 2011

The Intersection of Big Data and Analytics. Philip Russom TDWI Research Director for Data Management May 5, 2011 The Intersection of Big Data and Analytics Philip Russom TDWI Research Director for Data Management May 5, 2011 Sponsor 2 Speakers Philip Russom TDWI Research Director, Data Management Francois Ajenstat

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Appliances and DW Architectures John O Brien President and Executive Architect Zukeran Technologies 1 TDWI 1 Agenda What

More information

of DATA FUTURE The WAREHOUSING Best Practices Series IBM Syncsort PAGE 4 PAGE 6 WHY CLOUD IS THE FUTURE OF DATA WAREHOUSING

of DATA FUTURE The WAREHOUSING Best Practices Series IBM Syncsort PAGE 4 PAGE 6 WHY CLOUD IS THE FUTURE OF DATA WAREHOUSING IBM PAGE 4 WHY CLOUD IS THE FUTURE OF DATA WAREHOUSING Syncsort PAGE 6 A THOUGHTFUL APPROACH TO OPTIMIZING THE DATA WAREHOUSE WITH HADOOP The FUTURE of DATA WAREHOUSING Best Practices Series 2 APRIL/MAY

More information

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE Current technology for Big Data allows organizations to dramatically improve return on investment (ROI) from their existing data warehouse environment.

More information

Navigating the Big Data infrastructure layer Helena Schwenk

Navigating the Big Data infrastructure layer Helena Schwenk mwd a d v i s o r s Navigating the Big Data infrastructure layer Helena Schwenk A special report prepared for Actuate May 2013 This report is the second in a series of four and focuses principally on explaining

More information

Data Warehousing in the Cloud

Data Warehousing in the Cloud TDWI RESEARCH TDWI CHECKLIST REPORT Data Warehousing in the Cloud By David Loshin Sponsored by: tdwi.org JULY 2015 TDWI CHECKLIST REPORT Data Warehousing in the Cloud By David Loshin TABLE OF CONTENTS

More information

PRIME DIMENSIONS. Revealing insights. Shaping the future.

PRIME DIMENSIONS. Revealing insights. Shaping the future. PRIME DIMENSIONS Revealing insights. Shaping the future. Service Offering Prime Dimensions offers expertise in the processes, tools, and techniques associated with: Data Management Business Intelligence

More information

INTELLIGENT BUSINESS STRATEGIES WHITE PAPER

INTELLIGENT BUSINESS STRATEGIES WHITE PAPER INTELLIGENT BUSINESS STRATEGIES WHITE PAPER Improving Access to Data for Successful Business Intelligence Part 2: Supporting Multiple Analytical Workloads in a Changing Analytical Landscape By Mike Ferguson

More information

Practical Approaches to Big Data & Analytics: From Infrastructure to

Practical Approaches to Big Data & Analytics: From Infrastructure to 2014 Cisco and/or its affiliates. All rights reserved. Practical Approaches to Big Data & Analytics: From Infrastructure to Applications Kapil Bakshi Distinguished Architect, Cisco System Digital Government

More information

An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP Oracle ESG Data Systems Architecture

An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP Oracle ESG Data Systems Architecture An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP ESG Data Systems Architecture Big Data & Analytics as a Service Components Unstructured Data / Sparse Data of Value

More information

Ne x t gener ation Data Warehouse Pl atforms

Ne x t gener ation Data Warehouse Pl atforms four t h qua r t er 2009 TDWI best practices Report Ne x t gener ation Data Warehouse Pl atforms By Philip Russom Co-sponsored by www.tdwi.org fourth QUARTER 2009 TDWI best practices Report Ne x t gener

More information

Applied Business Intelligence. Iakovos Motakis, Ph.D. Director, DW & Decision Support Systems Intrasoft SA

Applied Business Intelligence. Iakovos Motakis, Ph.D. Director, DW & Decision Support Systems Intrasoft SA Applied Business Intelligence Iakovos Motakis, Ph.D. Director, DW & Decision Support Systems Intrasoft SA Agenda Business Drivers and Perspectives Technology & Analytical Applications Trends Challenges

More information

INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY & MANAGEMENT INFORMATION SYSTEM (IJITMIS)

INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY & MANAGEMENT INFORMATION SYSTEM (IJITMIS) INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY & MANAGEMENT INFORMATION SYSTEM (IJITMIS) International Journal of Information Technology & Management Information System (IJITMIS), ISSN 0976 ISSN 0976

More information

2015 Ironside Group, Inc. 2

2015 Ironside Group, Inc. 2 2015 Ironside Group, Inc. 2 Introduction to Ironside What is Cloud, Really? Why Cloud for Data Warehousing? Intro to IBM PureData for Analytics (IPDA) IBM PureData for Analytics on Cloud Intro to IBM dashdb

More information

IST722 Data Warehousing

IST722 Data Warehousing IST722 Data Warehousing Components of the Data Warehouse Michael A. Fudge, Jr. Recall: Inmon s CIF The CIF is a reference architecture Understanding the Diagram The CIF is a reference architecture CIF

More information

Next Generation Data Warehousing Appliances 23.10.2014

Next Generation Data Warehousing Appliances 23.10.2014 Next Generation Data Warehousing Appliances 23.10.2014 Presentert av: Espen Jorde, Executive Advisor Bjørn Runar Nes, CTO/Chief Architect Bjørn Runar Nes Espen Jorde 2 3.12.2014 Agenda Affecto s new Data

More information

High-Performance Analytics

High-Performance Analytics High-Performance Analytics David Pope January 2012 Principal Solutions Architect High Performance Analytics Practice Saturday, April 21, 2012 Agenda Who Is SAS / SAS Technology Evolution Current Trends

More information

So Many Tools, So Much Data, and So Much Meta Data

So Many Tools, So Much Data, and So Much Meta Data So Many Tools, So Much Data, and So Much Meta Data Copyright 1991-2012 R20/Consultancy B.V., The Hague, The Netherlands. All rights reserved. No part of this material may be reproduced, stored in a retrieval

More information

Table of Contents. Research Methodology and Demographics 3 Executive Summary 4 Introduction to Big Data Management 5

Table of Contents. Research Methodology and Demographics 3 Executive Summary 4 Introduction to Big Data Management 5 T DW I r e s e a r c h T DW I be s t p r ac tice s Re p o r t Managing Big Data By Philip Russom Co-SponsOred By tdwi.org Fourth Quarter 2013 Fourth QUARTER 2013 TDWI best practices Report Managing Big

More information

Ten Things You Need to Know About Data Virtualization

Ten Things You Need to Know About Data Virtualization White Paper Ten Things You Need to Know About Data Virtualization What is Data Virtualization? Data virtualization is an agile data integration method that simplifies information access. Data virtualization

More information

Getting Started Practical Input For Your Roadmap

Getting Started Practical Input For Your Roadmap Getting Started Practical Input For Your Roadmap Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson

More information

Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE

Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE INTRODUCTION Over the past several years a new category of Business Intelligence

More information

Hadoop and Relational Database The Best of Both Worlds for Analytics Greg Battas Hewlett Packard

Hadoop and Relational Database The Best of Both Worlds for Analytics Greg Battas Hewlett Packard Hadoop and Relational base The Best of Both Worlds for Analytics Greg Battas Hewlett Packard The Evolution of Analytics Mainframe EDW Proprietary MPP Unix SMP MPP Appliance Hadoop? Questions Is Hadoop

More information

Turning Big Data into Big Insights

Turning Big Data into Big Insights mwd a d v i s o r s Turning Big Data into Big Insights Helena Schwenk A special report prepared for Actuate May 2013 This report is the fourth in a series and focuses principally on explaining what s needed

More information

Data Virtualization. Paul Moxon Denodo Technologies. Alberta Data Architecture Community January 22 nd, 2014. 2014 Denodo Technologies

Data Virtualization. Paul Moxon Denodo Technologies. Alberta Data Architecture Community January 22 nd, 2014. 2014 Denodo Technologies Data Virtualization Paul Moxon Denodo Technologies Alberta Data Architecture Community January 22 nd, 2014 The Changing Speed of Business 100 25 35 45 55 65 75 85 95 Gartner The Nexus of Forces Today s

More information

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata BIG DATA: FROM HYPE TO REALITY Leandro Ruiz Presales Partner for C&LA Teradata Evolution in The Use of Information Action s ACTIVATING MAKE it happen! Insights OPERATIONALIZING WHAT IS happening now? PREDICTING

More information

Getting Started with Data Governance. Philip Russom TDWI Research Director, Data Management June 14, 2012

Getting Started with Data Governance. Philip Russom TDWI Research Director, Data Management June 14, 2012 Getting Started with Data Governance Philip Russom TDWI Research Director, Data Management June 14, 2012 Speakers Philip Russom Director, TDWI Research Daniel Teachey Senior Director of Marketing, DataFlux

More information

Oracle s Big Data solutions. Roger Wullschleger.

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here> s Big Data solutions Roger Wullschleger DBTA Workshop on Big Data, Cloud Data Management and NoSQL 10. October 2012, Stade de Suisse, Berne 1 The following is intended to outline

More information

Microsoft Analytics Platform System. Solution Brief

Microsoft Analytics Platform System. Solution Brief Microsoft Analytics Platform System Solution Brief Contents 4 Introduction 4 Microsoft Analytics Platform System 5 Enterprise-ready Big Data 7 Next-generation performance at scale 10 Engineered for optimal

More information

Big Data Analytics Platform @ Nokia

Big Data Analytics Platform @ Nokia Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform

More information

BIG DATA What it is and how to use?

BIG DATA What it is and how to use? BIG DATA What it is and how to use? Lauri Ilison, PhD Data Scientist 21.11.2014 Big Data definition? There is no clear definition for BIG DATA BIG DATA is more of a concept than precise term 1 21.11.14

More information

Whitepaper: Solution Overview - Breakthrough Insight. Published: March 7, 2012. Applies to: Microsoft SQL Server 2012. Summary:

Whitepaper: Solution Overview - Breakthrough Insight. Published: March 7, 2012. Applies to: Microsoft SQL Server 2012. Summary: Whitepaper: Solution Overview - Breakthrough Insight Published: March 7, 2012 Applies to: Microsoft SQL Server 2012 Summary: Today s Business Intelligence (BI) platform must adapt to a whole new scope,

More information

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap 3 key strategic advantages, and a realistic roadmap for what you really need, and when 2012, Cognizant Topics to be discussed

More information

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers Modern IT Operations Management Why a New Approach is Required, and How Boundary Delivers TABLE OF CONTENTS EXECUTIVE SUMMARY 3 INTRODUCTION: CHANGING NATURE OF IT 3 WHY TRADITIONAL APPROACHES ARE FAILING

More information

TDWI research TDWI BEST PRACTICES REPORT FOURTH QUARTER 2013 MANAGING BIG DATA. By Philip Russom. tdwi.org

TDWI research TDWI BEST PRACTICES REPORT FOURTH QUARTER 2013 MANAGING BIG DATA. By Philip Russom. tdwi.org TDWI research TDWI BEST PRACTICES REPORT FOURTH QUARTER 2013 MANAGING BIG DATA By Philip Russom tdwi.org Research Sponsors Research Sponsors Cloudera Dell Software Oracle Pentaho SAP SAS Fourth QUARTER

More information

Cloud Integration and the Big Data Journey - Common Use-Case Patterns

Cloud Integration and the Big Data Journey - Common Use-Case Patterns Cloud Integration and the Big Data Journey - Common Use-Case Patterns A White Paper August, 2014 Corporate Technologies Business Intelligence Group OVERVIEW The advent of cloud and hybrid architectures

More information

Modern Data Warehousing

Modern Data Warehousing Modern Data Warehousing Cem Kubilay Microsoft CEE, Turkey & Israel Time is FY15 Gartner Survey April 2014 Piloting on premise 15% 10% 4% 14% 57% 2014 5% think Hadoop will replace existing DW solution (2013:

More information

Integrating Cloudera and SAP HANA

Integrating Cloudera and SAP HANA Integrating Cloudera and SAP HANA Version: 103 Table of Contents Introduction/Executive Summary 4 Overview of Cloudera Enterprise 4 Data Access 5 Apache Hive 5 Data Processing 5 Data Integration 5 Partner

More information

The Role of the BI Competency Center in Maximizing Organizational Performance

The Role of the BI Competency Center in Maximizing Organizational Performance The Role of the BI Competency Center in Maximizing Organizational Performance Gloria J. Miller Dr. Andreas Eckert MaxMetrics GmbH October 16, 2008 Topics The Role of the BI Competency Center Responsibilites

More information

A Tour of the Zoo the Hadoop Ecosystem Prafulla Wani

A Tour of the Zoo the Hadoop Ecosystem Prafulla Wani A Tour of the Zoo the Hadoop Ecosystem Prafulla Wani Technical Architect - Big Data Syntel Agenda Welcome to the Zoo! Evolution Timeline Traditional BI/DW Architecture Where Hadoop Fits In 2 Welcome to

More information

BIG DATA CAN DRIVE THE BUSINESS AND IT TO EVOLVE AND ADAPT RALPH KIMBALL BUSSUM 2014

BIG DATA CAN DRIVE THE BUSINESS AND IT TO EVOLVE AND ADAPT RALPH KIMBALL BUSSUM 2014 BIG DATA CAN DRIVE THE BUSINESS AND IT TO EVOLVE AND ADAPT RALPH KIMBALL BUSSUM 2014 Ralph Kimball Associates 2014 The Data Warehouse Mission Identify all possible enterprise data assets Select those assets

More information

The Secrets of Analytical Leaders: Insights from Information Insiders

The Secrets of Analytical Leaders: Insights from Information Insiders The Secrets of Analytical Leaders: Insights from Information Insiders Wayne W. Eckerson Director of Research and Founder Founder, BI Leadership Forum Secrets Dan Ingle, Kelley Blue Book 1. Incremental

More information

NEWLY EMERGING BEST PRACTICES FOR BIG DATA

NEWLY EMERGING BEST PRACTICES FOR BIG DATA 2000-2012 Kimball Group. All rights reserved. Page 1 NEWLY EMERGING BEST PRACTICES FOR BIG DATA Ralph Kimball Informatica October 2012 Ralph Kimball Big is Being Monetized Big data is the second era of

More information

Using and Choosing a Cloud Solution for Data Warehousing

Using and Choosing a Cloud Solution for Data Warehousing TDWI RESEARCH TDWI CHECKLIST REPORT Using and Choosing a Cloud Solution for Data Warehousing By Colin White Sponsored by: tdwi.org JULY 2015 TDWI CHECKLIST REPORT Using and Choosing a Cloud Solution for

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

MDM for the Enterprise: Complementing and extending your Active Data Warehousing strategy. Satish Krishnaswamy VP MDM Solutions - Teradata

MDM for the Enterprise: Complementing and extending your Active Data Warehousing strategy. Satish Krishnaswamy VP MDM Solutions - Teradata MDM for the Enterprise: Complementing and extending your Active Data Warehousing strategy Satish Krishnaswamy VP MDM Solutions - Teradata 2 Agenda MDM and its importance Linking to the Active Data Warehousing

More information

Knowledge-Based Systems IS430. Mostafa Z. Ali

Knowledge-Based Systems IS430. Mostafa Z. Ali Winter 2009 Knowledge-Based Systems IS430 Data Warehousing Lesson 6 Mostafa Z. Ali mzali@just.edu.jo Lecture 2: Slide 1 Learning Objectives Understand the basic definitions and concepts of data warehouses

More information

Establish and maintain Center of Excellence (CoE) around Data Architecture

Establish and maintain Center of Excellence (CoE) around Data Architecture Senior BI Data Architect - Bensenville, IL The Company s Information Management Team is comprised of highly technical resources with diverse backgrounds in data warehouse development & support, business

More information

Detecting Anomalous Behavior with the Business Data Lake. Reference Architecture and Enterprise Approaches.

Detecting Anomalous Behavior with the Business Data Lake. Reference Architecture and Enterprise Approaches. Detecting Anomalous Behavior with the Business Data Lake Reference Architecture and Enterprise Approaches. 2 Detecting Anomalous Behavior with the Business Data Lake Pivotal the way we see it Reference

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Business Intelligence for Big Data

Business Intelligence for Big Data Business Intelligence for Big Data Will Gorman, Vice President, Engineering May, 2011 2010, Pentaho. All Rights Reserved. www.pentaho.com. What is BI? Business Intelligence = reports, dashboards, analysis,

More information

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum Big Data Analytics with EMC Greenplum and Hadoop Big Data Analytics with EMC Greenplum and Hadoop Ofir Manor Pre Sales Technical Architect EMC Greenplum 1 Big Data and the Data Warehouse Potential All

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

High Performance Data Management Use of Standards in Commercial Product Development

High Performance Data Management Use of Standards in Commercial Product Development v2 High Performance Data Management Use of Standards in Commercial Product Development Jay Hollingsworth: Director Oil & Gas Business Unit Standards Leadership Council Forum 28 June 2012 1 The following

More information

Extending Hyperion BI with the Oracle BI Server

<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server Extending Hyperion BI with the Oracle BI Server Mark Ostroff Sr. BI Solutions Consultant Agenda Hyperion BI versus Hyperion BI with OBI Server Benefits of using Hyperion BI with the

More information

MDM and Data Warehousing Complement Each Other

MDM and Data Warehousing Complement Each Other Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There

More information

BI Market Dynamics and Future Directions

BI Market Dynamics and Future Directions Inaugural Keynote Address Business Intelligence Conference Nov 19, 2011, New Delhi BI Market Dynamics and Future Directions Shashikant Brahmankar Head Business Intelligence & Analytics, HCL Content Evolution

More information

Big Data Maturity - The Photo and The Movie

Big Data Maturity - The Photo and The Movie Big Data Maturity - The Photo and The Movie Mike Ferguson Managing Director, Intelligent Business Strategies BA4ALL Big Data & Analytics Insight Conference Stockholm, May 2015 About Mike Ferguson Mike

More information

CREATING PACKAGED IP FOR BUSINESS ANALYTICS PROJECTS

CREATING PACKAGED IP FOR BUSINESS ANALYTICS PROJECTS CREATING PACKAGED IP FOR BUSINESS ANALYTICS PROJECTS A PERSPECTIVE FOR SYSTEMS INTEGRATORS Sponsored by Microsoft Corporation 1/ What is Packaged IP? Categorizing the Options 2/ Why Offer Packaged IP?

More information

Integrating Netezza into your existing IT landscape

Integrating Netezza into your existing IT landscape Marco Lehmann Technical Sales Professional Integrating Netezza into your existing IT landscape 2011 IBM Corporation Agenda How to integrate your existing data into Netezza appliance? 4 Steps for creating

More information