An Enterprise Data Hub, the Next Gen Operational Data Store

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "An Enterprise Data Hub, the Next Gen Operational Data Store"

Transcription

1 An Enterprise Data Hub, the Next Gen Operational Data Store Version: 101

2 Table of Contents Summary 3 The ODS in Practice 4 Drawbacks of the ODS Today 5 The Case for ODS on an EDH 5 Conclusion 6 About the Author 6 2

3 Summary Enterprise computing platforms are being redesigned to make use of the economics, scalability and performance of distributed computing systems such as the Apache Hadoop. Most systems in place today rely on relational databases that are brittle, expensive and limited in scale and performance (though within expectations are considered useful and important). As organizations begin the process of migrating or augmenting with Hadoop, there are many unanswered questions about the nature of applications and data management going forward. This is especially true when examining systems that evolved from an era of scarcity, such as data warehouses and Operational Data Stores (ODS). These systems were developed to provide reporting and analysis separate from the primary operational systems. The economics of enterprise computing have substantially changed. One such popular arrangement was the Corporate Information Factory 1, a complex design that separated data, data flows, governance and metadata in order to provide acceptable performance. In a typical CIF, there may be as many as 20 databases and more than 20 schema for each purpose. Today, most of these separate components can be collapsed into an enterprise data hub (EDH). Systems designed for reporting and business intelligence have historically been constrained by the cost and complexity of resources and methods. This managing from scarcity provided for some ingenious workaround approaches, but today, with scarcity mostly a thing of the past, many of these approaches are in need of review. One in particular is the ODS. A popular approach for achieving rational performance for reporting and analysis was an architecture called the Corporate Information Factory, of which the ODS was an integral part. As you can see from the diagram below the CIS was a complicated set of structures and processes with a great deal of physical movement between the components. If designed well, it delivered adequate load processing and acceptable query processing, but the relentless rate of change in organizations added a large burden to efforts to keep it running. Library & Toolbox Information Workshop Workbench Information Feedback External ERP ERP Data Acquisition Data Warehouse CIF Data Data Delivery Exploration Warehouse Data Mining Warehouse Internal Operational Data Store OLAP Data Mart Other Operational Systems Systems Data Acquisition Figure 1: The Corporate Information Factory Inmon & Imhof Tri Meta Data Operation & Administration Service Oper Mart Change 3 1 Mastering data warehouse design, Imhoff, Claudia Galemmo, Nicholas Geiger, Jonathan G.ISBN:

4 The ODS in Practice The ODS was developed 2 as a means to provide access to data from live operational systems without disturbing the processing of the operations themselves, and to overcome limitations of data warehouses, particularly the slow batch loading of data warehouses and limited scaling potential. At first glance it is reasonable to expect that the ODS is a good candidate to reside in an enterprise data hub, but there are some subtleties that need to be addressed. There are conflicting definitions for the ODS, but the original specification called for a subject oriented, current valued, integrated and detailed design. One common misconception about the ODS was that it was merely a staging area for further refinement of data for a data warehouse. Another was that it is often mistaken as part of a data warehouse, or the data warehouse itself, but it is not and it serves a different purpose. For example: Subject oriented: An ODS is not a dumping ground for all sorts of data. Each is designed for major sets of data such as CUSTOMER or PRODUCT, but not a business process such as Sales, Replenishment or Yield. Current Valued: Distinguishes an ODS from a data warehouse it only contains the current period, however that is designed (day, week, etc.) They do not retain history like a data warehouse. Integrated: Even if the ODS contains data about a single subject, that data may reside in multiple source systems and the separate feeds must be integrated to give a coherent view. Keep in mind, without the historical component; this is a much easier task than a data warehouse. Detailed: As a result of managing from scarcity, the most detailed data was often brought into the ODS, but more summarized data flowed to the data warehouse. The data warehouse typically had multiple, integrated subject areas, a much longer historical perspective and a multi-level physical schema to support various activities requiring indexing, aggregation and duplication of data. The ODS can be quite large, but can be considered more lightweight than a data warehouse. The ODS is a useful solution when certain situations prevail: There is a need to have access to detailed internal data, such as transactions from operational systems, at a level of detail finer than the data warehouse (because data warehouse size is usually limited by both cost and performance) There is a requirement for timely reporting of operations, especially if it requires integration of data from more than one system The volume of data is large, a measure that is dependent on the economics of the exiting technology Update of the data was near-real-time. This means that data can be used almost immediately, though timing differences from data trickling in have to be dealt with in the reporting application. There is no provision for versioning of data. A common application for the ODS was to support live access to integrated customer service data when the operational systems lacked the functionality to do so, but the breadth of information in the data warehouse was not necessary and its performance was not adequate. 4 2 The earliest book on the subject is Building the Operational Data Store, October 27, 1995 by W. H. Inmon, Claudia Imhoff, Greg Battas

5 Drawbacks of the ODS Today Since ODS s were typically built using the predominant relational database technologies and platforms, the ODS was an expensive proposition in terms of hardware profiles, proprietary software complexities, and labor. In addition, the ODS was integrated, meaning the data from various sources had to be blended and cleansed; so despite its proposed role, overcoming latency was still a difficult and ongoing challenge. Reporting, Data Discovery, and Analytics performance of the ODS was dependent on the physical design and the workload. There were always a number of schemas in the ODS, such as transactional 3NF design for ingesting data quickly, a dimensional schema for providing reasonably good performance for queries and all structures in the ODS were highly designed and configured based on assumption of usage patterns. Modifications to the schema generated effort to modify the upstream and downstream processes. Some relational database technologies were able to scale to meet the demand of ODS and data warehouse using massively parallel processing, but many popular offerings could not. A major limitation of all relational database technologies is that the query parser, optimizer and compute layers cannot be separated, which limited their scalability. As a result, all data structures were carefully designed using parsimonious techniques to limit the scale and usage as much as possible. All were quite costly. This is not the case with Hadoop. The Case for ODS on an EDH As organizations become aware of the value of Big Data, data flowing from a myriad of internal and external sources, and a variety of formats that a relational database cannot process, the relational-based ODS becomes untenable. Eliminating the managing from scarcity element from ODS design (reporting, business intelligence and analytics), many of the limitations of relational ODS no longer exist. Scale and latency go away, so there is no need to physically separate subject areas into different ODS s. Nor is there a need to flush history. However the ODS and the EDH remain separate concepts. While ODS data may reside in the EDH, ODS processing is only a part of the portfolio of the EDH. The EDH, built on Hadoop technology (and economics) becomes the obvious choice for ODS because: Economics: The Cloudera platform is cost-efficient versus existing relational-based architectures. Scale: Hadoop can scale to handle enormous volumes of data and concurrent work. Unlimited choices: XXX Performance: The EDH data, with its associated tools for performance, security and scalability, allow for far fewer data structures and far less maintenance of physical optimizations such as aggregation and indexing. In other words, the same physical copy of the data can support many virtual operations. Avoidance of design (schema): The whole concept of schema on read reduces the need for design and maintenance of structure s for performance and latency, as the process of parsing queries, optimizing them and presenting results can be separated into their logical locations and fit. 5

6 Conclusion One other thing to consider: the EDH can t operate as a useful enterprise tool without metadata and governance. Hadoop entered the scene as an unruly tool for singular use by data scientists. Now that it is maturing into a platform for enterprise computing, that unruliness is no longer acceptable. Cloudera provides many tools to facilitate an enterprise solution architecture with Cloudera Navigator (for governance), Impala for HDFS-based relational capabilities and a growing collection of other tools for security, development and performance. With an adequate metadata management system, the ODS can be a purely virtual structure. As an EDH contains all of the data needed for the ODS (and more of course), the ODS structures and schema can be strictly virtual. Hadoop has the processing power to present the data on request without layers of integration and physical data movement. All of the physical structures that support existing ODS s take time to maintain and with an EDH can be replaced as virtual structures with many applications using the same data. About the Author Neil Raden, based in Santa Fe, NM, is an industry analyst and active consultant, widely published author and speaker and the founder of Hired Brains Research LLC, Hired Brains provides research, advisory and consulting services in Analytics, Big Data, and Decision for clients worldwide. Neil is also the co-author of the Dresner Advisory Services Wisdom of BI series on Advanced and Predictive Analytics. Neil was a contributing author to one of the first (1995) books on designing data warehouses and he is more recently the co-author of Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions, Prentice-Hall. He is a contributor to publications such as Wall Street Week, Forbes, Information Week and ComputerWorld. He welcomes your comments at or at his blog at 6

7 About Cloudera Cloudera is revolutionizing enterprise data management by offering the first unified Platform for big data, an enterprise data hub built on Apache Hadoop. Cloudera offers enterprises one place to store, access, process, secure, and analyze all their data, empowering them to extend the value of existing investments while enabling fundamental new ways to derive value from their data. Cloudera s open source big data platform is the most widely adopted in the world, and Cloudera is the most prolific contributor to the open source Hadoop ecosystem. As the leading educator of Hadoop professionals, Cloudera has trained over 22,000 individuals worldwide. Over 1,400 partners and a seasoned professional services team help deliver greater time to value. Finally, only Cloudera provides proactive and predictive support to run an enterprise data hub with confidence. Leading organizations in every industry plus top public sector organizations globally run Cloudera in production. For additional information, please visit us at: cloudera.com or Cloudera, Inc Page Mill Road, Palo Alto, CA 94304, USA 2015 Cloudera, Inc. All rights reserved. Cloudera and the Cloudera logo are trademarks or registered trademarks of Cloudera Inc. in the USA and other countries. All other trademarks are the property of their respective companies. Information is subject to change without notice.

Operational Analytics

Operational Analytics Operational Analytics Version: 101 Table of Contents Operational Analytics 3 From the Enterprise Data Hub to the Enterprise Application Hub 3 Operational Intelligence in Action: Some Examples 4 Requirements

More information

Data Discovery, Analytics, and the Enterprise Data Hub

Data Discovery, Analytics, and the Enterprise Data Hub Data Discovery, Analytics, and the Enterprise Data Hub Version: 101 Table of Contents Summary 3 Used Data and Limitations of Legacy Analytic Architecture 3 The Meaning of Data Discovery & Analytics 4 Machine

More information

Deploying an Operational Data Store Designed for Big Data

Deploying an Operational Data Store Designed for Big Data Deploying an Operational Data Store Designed for Big Data A fast, secure, and scalable data staging environment with no data volume or variety constraints Sponsored by: Version: 102 Table of Contents Introduction

More information

INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES

INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES Data Consolidation and Multi-Tenancy in Financial Services CLOUDERA INDUSTRY BRIEF 2 Table of Contents Introduction 3 Security

More information

Integrating Cloudera and SAP HANA

Integrating Cloudera and SAP HANA Integrating Cloudera and SAP HANA Version: 103 Table of Contents Introduction/Executive Summary 4 Overview of Cloudera Enterprise 4 Data Access 5 Apache Hive 5 Data Processing 5 Data Integration 5 Partner

More information

Cloudera Enterprise Data Hub in Telecom:

Cloudera Enterprise Data Hub in Telecom: Cloudera Enterprise Data Hub in Telecom: Three Customer Case Studies Version: 103 Table of Contents Introduction 3 Cloudera Enterprise Data Hub for Telcos 4 Cloudera Enterprise Data Hub in Telecom: Customer

More information

More Data in Less Time

More Data in Less Time More Data in Less Time Leveraging Cloudera CDH as an Operational Data Store Daniel Tydecks, Systems Engineering DACH & CE Goals of an Operational Data Store Load Data Sources Traditional Architecture Operational

More information

Driving Growth in Insurance With a Big Data Architecture

Driving Growth in Insurance With a Big Data Architecture Driving Growth in Insurance With a Big Data Architecture The SAS and Cloudera Advantage Version: 103 Table of Contents Overview 3 Current Data Challenges for Insurers 3 Unlocking the Power of Big Data

More information

Cloudera in the Public Cloud

Cloudera in the Public Cloud Cloudera in the Public Cloud Deployment Options for the Enterprise Data Hub Version: Q414-102 Table of Contents Executive Summary 3 The Case for Public Cloud 5 Public Cloud vs On-Premise 6 Public Cloud

More information

Apache Hadoop in the Enterprise. Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com

Apache Hadoop in the Enterprise. Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com Apache Hadoop in the Enterprise Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com Cloudera The Leader in Big Data Management Powered by Apache Hadoop The Leading Open Source Distribution of Apache

More information

ROCANA WHITEPAPER. Improving Event Data Management and Legacy Systems

ROCANA WHITEPAPER. Improving Event Data Management and Legacy Systems ROCANA WHITEPAPER Improving Event Data Management and Legacy Systems INTRODUCTION STATE OF AFFAIRS WHAT IS EVENT DATA? There are a myriad of terms and definitions related to data that is the by-product

More information

Data Virtualization A Potential Antidote for Big Data Growing Pains

Data Virtualization A Potential Antidote for Big Data Growing Pains perspective Data Virtualization A Potential Antidote for Big Data Growing Pains Atul Shrivastava Abstract Enterprises are already facing challenges around data consolidation, heterogeneity, quality, and

More information

IST722 Data Warehousing

IST722 Data Warehousing IST722 Data Warehousing Components of the Data Warehouse Michael A. Fudge, Jr. Recall: Inmon s CIF The CIF is a reference architecture Understanding the Diagram The CIF is a reference architecture CIF

More information

VIEWPOINT. High Performance Analytics. Industry Context and Trends

VIEWPOINT. High Performance Analytics. Industry Context and Trends VIEWPOINT High Performance Analytics Industry Context and Trends In the digital age of social media and connected devices, enterprises have a plethora of data that they can mine, to discover hidden correlations

More information

An Architecture for Integrated Operational Business Intelligence

An Architecture for Integrated Operational Business Intelligence An Architecture for Integrated Operational Business Intelligence Dr. Ulrich Christ SAP AG Dietmar-Hopp-Allee 16 69190 Walldorf ulrich.christ@sap.com Abstract: In recent years, Operational Business Intelligence

More information

Symantec Global Intelligence Network 2.0 Architecture: Staying Ahead of the Evolving Threat Landscape

Symantec Global Intelligence Network 2.0 Architecture: Staying Ahead of the Evolving Threat Landscape WHITE PAPER: SYMANTEC GLOBAL INTELLIGENCE NETWORK 2.0.... ARCHITECTURE.................................... Symantec Global Intelligence Network 2.0 Architecture: Staying Ahead of the Evolving Threat Who

More information

Accelerate your Big Data Strategy. Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator

Accelerate your Big Data Strategy. Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator Accelerate your Big Data Strategy Execute faster with Capgemini and Cloudera s Enterprise Data Hub Accelerator Enterprise Data Hub Accelerator enables you to get started rapidly and cost-effectively with

More information

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Datenverwaltung im Wandel - Building an Enterprise Data Hub with Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees

More information

The Enterprise Data Hub and The Modern Information Architecture

The Enterprise Data Hub and The Modern Information Architecture The Enterprise Data Hub and The Modern Information Architecture Dr. Amr Awadallah CTO & Co-Founder, Cloudera Twitter: @awadallah 1 2013 Cloudera, Inc. All rights reserved. Cloudera Overview The Leader

More information

Unifying the Enterprise Data Hub and the Integrated Data Warehouse

Unifying the Enterprise Data Hub and the Integrated Data Warehouse Unifying the Enterprise Data Hub and the Integrated Data Warehouse CONTENTS Encompassing All of the Big Data Universe 1 The Ideal Structure 2 The Enterprise Data Hub: Refining Raw Data 3 The Integrated

More information

Traditional BI vs. Business Data Lake A comparison

Traditional BI vs. Business Data Lake A comparison Traditional BI vs. Business Data Lake A comparison The need for new thinking around data storage and analysis Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses

More information

Dell* In-Memory Appliance for Cloudera* Enterprise

Dell* In-Memory Appliance for Cloudera* Enterprise Built with Intel Dell* In-Memory Appliance for Cloudera* Enterprise Find out what faster big data analytics can do for your business The need for speed in all things related to big data is an enormous

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

How to Enhance Traditional BI Architecture to Leverage Big Data

How to Enhance Traditional BI Architecture to Leverage Big Data B I G D ATA How to Enhance Traditional BI Architecture to Leverage Big Data Contents Executive Summary... 1 Traditional BI - DataStack 2.0 Architecture... 2 Benefits of Traditional BI - DataStack 2.0...

More information

Sharing The Wealth. The Progression to the Complete CIF Environment

Sharing The Wealth. The Progression to the Complete CIF Environment Sharing The Wealth Putting it All Together in the Corporate Information Factory By Claudia Imhoff T he need for enhancements and integration within your decision support environment is growing exponentially.

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

Hadoop Data Hubs and BI. Supporting the migration from siloed reporting and BI to centralized services with Hadoop

Hadoop Data Hubs and BI. Supporting the migration from siloed reporting and BI to centralized services with Hadoop Hadoop Data Hubs and BI Supporting the migration from siloed reporting and BI to centralized services with Hadoop John Allen October 2014 Introduction John Allen; computer scientist Background in data

More information

Agile Business Intelligence Data Lake Architecture

Agile Business Intelligence Data Lake Architecture Agile Business Intelligence Data Lake Architecture TABLE OF CONTENTS Introduction... 2 Data Lake Architecture... 2 Step 1 Extract From Source Data... 5 Step 2 Register And Catalogue Data Sets... 5 Step

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

Why Big Data in the Cloud?

Why Big Data in the Cloud? Have 40 Why Big Data in the Cloud? Colin White, BI Research January 2014 Sponsored by Treasure Data TABLE OF CONTENTS Introduction The Importance of Big Data The Role of Cloud Computing Using Big Data

More information

Informatica and the Vibe Virtual Data Machine

Informatica and the Vibe Virtual Data Machine White Paper Informatica and the Vibe Virtual Data Machine Preparing for the Integrated Information Age This document contains Confidential, Proprietary and Trade Secret Information ( Confidential Information

More information

White Paper: Evaluating Big Data Analytical Capabilities For Government Use

White Paper: Evaluating Big Data Analytical Capabilities For Government Use CTOlabs.com White Paper: Evaluating Big Data Analytical Capabilities For Government Use March 2012 A White Paper providing context and guidance you can use Inside: The Big Data Tool Landscape Big Data

More information

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1

Copyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1 Slide 29-1 Chapter 29 Overview of Data Warehousing and OLAP Chapter 29 Outline Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics

More information

IBM Analytics. Just the facts: Four critical concepts for planning the logical data warehouse

IBM Analytics. Just the facts: Four critical concepts for planning the logical data warehouse IBM Analytics Just the facts: Four critical concepts for planning the logical data warehouse 1 2 3 4 5 6 Introduction Complexity Speed is businessfriendly Cost reduction is crucial Analytics: The key to

More information

Big Data for the Rest of Us Technical White Paper

Big Data for the Rest of Us Technical White Paper Big Data for the Rest of Us Technical White Paper Treasure Data - Big Data for the Rest of Us 1 Introduction The importance of data warehousing and analytics has increased as companies seek to gain competitive

More information

Build an effective data integration strategy to drive innovation

Build an effective data integration strategy to drive innovation IBM Software Thought Leadership White Paper September 2010 Build an effective data integration strategy to drive innovation Five questions business leaders must ask 2 Build an effective data integration

More information

Analytics With Hadoop. SAS and Cloudera Starter Services: Visual Analytics and Visual Statistics

Analytics With Hadoop. SAS and Cloudera Starter Services: Visual Analytics and Visual Statistics Analytics With Hadoop SAS and Cloudera Starter Services: Visual Analytics and Visual Statistics Everything You Need to Get Started on Your First Hadoop Project SAS and Cloudera have identified the essential

More information

January 2010. Fast-Tracking Data Warehousing & Business Intelligence Projects via Intelligent Data Modeling. Sponsored by:

January 2010. Fast-Tracking Data Warehousing & Business Intelligence Projects via Intelligent Data Modeling. Sponsored by: Fast-Tracking Data Warehousing & Business Intelligence Projects via Intelligent Data Modeling January 2010 Claudia Imhoff, Ph.D Sponsored by: Table of Contents Introduction... 3 What is a Data Model?...

More information

A Tipping Point for Automation in the Data Warehouse. www.stonebranch.com

A Tipping Point for Automation in the Data Warehouse. www.stonebranch.com A Tipping Point for Automation in the Data Warehouse www.stonebranch.com Resolving the ETL Automation Problem The pressure on ETL Architects and Developers to utilize automation in the design and management

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

MULTITENANCY AND THE ENTERPRISE DATA HUB:

MULTITENANCY AND THE ENTERPRISE DATA HUB: MULTITENANCY AND THE ENTERPRISE DATA HUB: Version: Q414-105 Table of Content Introduction 3 Business Objectives for Multitenant Environments 3 Standard Isolation Models of an EDH 4 Elements of a Multitenant

More information

A discussion of information integration solutions November 2005. Deploying a Center of Excellence for data integration.

A discussion of information integration solutions November 2005. Deploying a Center of Excellence for data integration. A discussion of information integration solutions November 2005 Deploying a Center of Excellence for data integration. Page 1 Contents Summary This paper describes: 1 Summary 1 Introduction 2 Mastering

More information

Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data

Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Transforming Data into Intelligence Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Big Data Data Warehousing Data Governance and Quality

More information

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics ESSENTIALS EMC ISILON Use the industry's first and only scale-out NAS solution with native Hadoop

More information

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS!

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS! The Bloor Group IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS VENDOR PROFILE The IBM Big Data Landscape IBM can legitimately claim to have been involved in Big Data and to have a much broader

More information

Establish and maintain Center of Excellence (CoE) around Data Architecture

Establish and maintain Center of Excellence (CoE) around Data Architecture Senior BI Data Architect - Bensenville, IL The Company s Information Management Team is comprised of highly technical resources with diverse backgrounds in data warehouse development & support, business

More information

www.ducenit.com Analance Data Integration Technical Whitepaper

www.ducenit.com Analance Data Integration Technical Whitepaper Analance Data Integration Technical Whitepaper Executive Summary Business Intelligence is a thriving discipline in the marvelous era of computing in which we live. It s the process of analyzing and exploring

More information

90% of your Big Data problem isn t Big Data.

90% of your Big Data problem isn t Big Data. White Paper 90% of your Big Data problem isn t Big Data. It s the ability to handle Big Data for better insight. By Arjuna Chala Risk Solutions HPCC Systems Introduction LexisNexis is a leader in providing

More information

Investor Presentation. Second Quarter 2015

Investor Presentation. Second Quarter 2015 Investor Presentation Second Quarter 2015 Note to Investors Certain non-gaap financial information regarding operating results may be discussed during this presentation. Reconciliations of the differences

More information

Whitepaper. Data Warehouse/BI Testing Offering YOUR SUCCESS IS OUR FOCUS. Published on: January 2009 Author: BIBA PRACTICE

Whitepaper. Data Warehouse/BI Testing Offering YOUR SUCCESS IS OUR FOCUS. Published on: January 2009 Author: BIBA PRACTICE YOUR SUCCESS IS OUR FOCUS Whitepaper Published on: January 2009 Author: BIBA PRACTICE 2009 Hexaware Technologies. All rights reserved. Table of Contents 1. 2. Data Warehouse - Typical pain points 3. Hexaware

More information

WHITE PAPER LOWER COSTS, INCREASE PRODUCTIVITY, AND ACCELERATE VALUE, WITH ENTERPRISE- READY HADOOP

WHITE PAPER LOWER COSTS, INCREASE PRODUCTIVITY, AND ACCELERATE VALUE, WITH ENTERPRISE- READY HADOOP WHITE PAPER LOWER COSTS, INCREASE PRODUCTIVITY, AND ACCELERATE VALUE, WITH ENTERPRISE- READY HADOOP CLOUDERA WHITE PAPER 2 Table of Contents Introduction 3 Hadoop's Role in the Big Data Challenge 3 Cloudera:

More information

Data virtualization: Delivering on-demand access to information throughout the enterprise

Data virtualization: Delivering on-demand access to information throughout the enterprise IBM Software Thought Leadership White Paper April 2013 Data virtualization: Delivering on-demand access to information throughout the enterprise 2 Data virtualization: Delivering on-demand access to information

More information

Information-Driven Transformation in Retail with the Enterprise Data Hub Accelerator

Information-Driven Transformation in Retail with the Enterprise Data Hub Accelerator Introduction Enterprise Data Hub Accelerator Retail Sector Use Cases Capabilities Information-Driven Transformation in Retail with the Enterprise Data Hub Accelerator Introduction Enterprise Data Hub Accelerator

More information

High Performance Data Management Use of Standards in Commercial Product Development

High Performance Data Management Use of Standards in Commercial Product Development v2 High Performance Data Management Use of Standards in Commercial Product Development Jay Hollingsworth: Director Oil & Gas Business Unit Standards Leadership Council Forum 28 June 2012 1 The following

More information

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE Current technology for Big Data allows organizations to dramatically improve return on investment (ROI) from their existing data warehouse environment.

More information

Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO. Big Data Everywhere Conference, NYC November 2015

Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO. Big Data Everywhere Conference, NYC November 2015 Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO Big Data Everywhere Conference, NYC November 2015 Agenda 1. Challenges with Risk Data Aggregation and Risk Reporting (RDARR) 2. How a

More information

Actian SQL in Hadoop Buyer s Guide

Actian SQL in Hadoop Buyer s Guide Actian SQL in Hadoop Buyer s Guide Contents Introduction: Big Data and Hadoop... 3 SQL on Hadoop Benefits... 4 Approaches to SQL on Hadoop... 4 The Top 10 SQL in Hadoop Capabilities... 5 SQL in Hadoop

More information

Five Technology Trends for Improved Business Intelligence Performance

Five Technology Trends for Improved Business Intelligence Performance TechTarget Enterprise Applications Media E-Book Five Technology Trends for Improved Business Intelligence Performance The demand for business intelligence data only continues to increase, putting BI vendors

More information

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases DATAMEER WHITE PAPER Beyond BI Big Data Analytic Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches

Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches Powerful Duo: MapR Big Data Analytics with Cisco ACI Network Switches Introduction For companies that want to quickly gain insights into or opportunities from big data - the dramatic volume growth in corporate

More information

MDM and Data Warehousing Complement Each Other

MDM and Data Warehousing Complement Each Other Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There

More information

www.sryas.com Analance Data Integration Technical Whitepaper

www.sryas.com Analance Data Integration Technical Whitepaper Analance Data Integration Technical Whitepaper Executive Summary Business Intelligence is a thriving discipline in the marvelous era of computing in which we live. It s the process of analyzing and exploring

More information

CAS Seminar on Ratemaking! "! ###!!

CAS Seminar on Ratemaking! ! ###!! CAS Seminar on Ratemaking $%! "! ###!! !"# $" CAS Seminar on Ratemaking $ %&'("(& + ) 3*# ) 3*# ) 3* ($ ) 4/#1 ) / &. ),/ &.,/ #1&.- ) 3*,5 /+,&. ),/ &..- ) 6/&/ '( +,&* * # +-* *%. (-/#$&01+, 2, Annual

More information

Data Integration for the Real Time Enterprise

Data Integration for the Real Time Enterprise Executive Brief Data Integration for the Real Time Enterprise Business Agility in a Constantly Changing World Overcoming the Challenges of Global Uncertainty Informatica gives Zyme the ability to maintain

More information

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments

Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Cloudera Enterprise Reference Architecture for Google Cloud Platform Deployments Important Notice 2010-2015 Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, Impala, and

More information

Business Intelligence and Big Data Analytics: Speeding the Cycle from Insights to Action Four Steps to More Profitable Customer Engagement

Business Intelligence and Big Data Analytics: Speeding the Cycle from Insights to Action Four Steps to More Profitable Customer Engagement white paper Business Intelligence and Big Data Analytics: Speeding the Cycle from Insights to Action Four Steps to More Profitable Customer Engagement»» Summary For business intelligence analysts the era

More information

Bringing the Power of SAS to Hadoop. White Paper

Bringing the Power of SAS to Hadoop. White Paper White Paper Bringing the Power of SAS to Hadoop Combine SAS World-Class Analytic Strength with Hadoop s Low-Cost, Distributed Data Storage to Uncover Hidden Opportunities Contents Introduction... 1 What

More information

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP Business Analytics for All Amsterdam - 2015 Value of Big Data is Being Recognized Executives beginning to see the path from data insights to revenue

More information

How to avoid building a data swamp

How to avoid building a data swamp How to avoid building a data swamp Case studies in Hadoop data management and governance Mark Donsky, Product Management, Cloudera Naren Korenu, Engineering, Cloudera 1 Abstract DELETE How can you make

More information

COURSE OUTLINE. Track 1 Advanced Data Modeling, Analysis and Design

COURSE OUTLINE. Track 1 Advanced Data Modeling, Analysis and Design COURSE OUTLINE Track 1 Advanced Data Modeling, Analysis and Design TDWI Advanced Data Modeling Techniques Module One Data Modeling Concepts Data Models in Context Zachman Framework Overview Levels of Data

More information

CDH AND BUSINESS CONTINUITY:

CDH AND BUSINESS CONTINUITY: WHITE PAPER CDH AND BUSINESS CONTINUITY: An overview of the availability, data protection and disaster recovery features in Hadoop Abstract Using the sophisticated built-in capabilities of CDH for tunable

More information

A Service-oriented Architecture for Business Intelligence

A Service-oriented Architecture for Business Intelligence A Service-oriented Architecture for Business Intelligence Liya Wu 1, Gilad Barash 1, Claudio Bartolini 2 1 HP Software 2 HP Laboratories {name.surname@hp.com} Abstract Business intelligence is a business

More information

E-Guide THE CHALLENGES BEHIND DATA INTEGRATION IN A BIG DATA WORLD

E-Guide THE CHALLENGES BEHIND DATA INTEGRATION IN A BIG DATA WORLD E-Guide THE CHALLENGES BEHIND DATA INTEGRATION IN A BIG DATA WORLD O n one hand, while big data applications have eliminated the rigidity of the data integration process, they don t take responsibility

More information

Are You Big Data Ready?

Are You Big Data Ready? ACS 2015 Annual Canberra Conference Are You Big Data Ready? Vladimir Videnovic Business Solutions Director Oracle Big Data and Analytics Introduction Introduction What is Big Data? If you can't explain

More information

Three Open Blueprints For Big Data Success

Three Open Blueprints For Big Data Success White Paper: Three Open Blueprints For Big Data Success Featuring Pentaho s Open Data Integration Platform Inside: Leverage open framework and open source Kickstart your efforts with repeatable blueprints

More information

Integrating SAP and non-sap data for comprehensive Business Intelligence

Integrating SAP and non-sap data for comprehensive Business Intelligence WHITE PAPER Integrating SAP and non-sap data for comprehensive Business Intelligence www.barc.de/en Business Application Research Center 2 Integrating SAP and non-sap data Authors Timm Grosser Senior Analyst

More information

The Role of the Analyst in Business Analytics. Neil Foshay Schwartz School of Business St Francis Xavier U

The Role of the Analyst in Business Analytics. Neil Foshay Schwartz School of Business St Francis Xavier U The Role of the Analyst in Business Analytics Neil Foshay Schwartz School of Business St Francis Xavier U Contents Business Analytics What s it all about? Development Process Overview BI Analyst Role Questions

More information

Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep. Neil Raden Hired Brains Research, LLC

Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep. Neil Raden Hired Brains Research, LLC Data Catalogs for Hadoop Achieving Shared Knowledge and Re-usable Data Prep Neil Raden Hired Brains Research, LLC Traditionally, the job of gathering and integrating data for analytics fell on data warehouses.

More information

Business Intelligence & IT Governance

Business Intelligence & IT Governance Business Intelligence & IT Governance The current trend and its implication on modern businesses Jovany Chaidez 12/3/2008 Prepared for: Professor Michael J. Shaw BA458 IT Governance Fall 2008 The purpose

More information

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform...

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform... Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure Requirements... 5 Solution Spectrum... 6 Oracle s Big Data

More information

White Paper: Enhancing Functionality and Security of Enterprise Data Holdings

White Paper: Enhancing Functionality and Security of Enterprise Data Holdings White Paper: Enhancing Functionality and Security of Enterprise Data Holdings Examining New Mission- Enabling Design Patterns Made Possible by the Cloudera- Intel Partnership Inside: Improving Return on

More information

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata

BIG DATA: FROM HYPE TO REALITY. Leandro Ruiz Presales Partner for C&LA Teradata BIG DATA: FROM HYPE TO REALITY Leandro Ruiz Presales Partner for C&LA Teradata Evolution in The Use of Information Action s ACTIVATING MAKE it happen! Insights OPERATIONALIZING WHAT IS happening now? PREDICTING

More information

Adobe Insight, powered by Omniture

Adobe Insight, powered by Omniture Adobe Insight, powered by Omniture Accelerating government intelligence to the speed of thought 1 Challenges that analysts face 2 Analysis tools and functionality 3 Adobe Insight 4 Summary Never before

More information

WHITE PAPER. Hadoop and HDFS: Storage for Next Generation Data Management. Version: Q414-102

WHITE PAPER. Hadoop and HDFS: Storage for Next Generation Data Management. Version: Q414-102 Storage for Next Generation Data Management Version: Q414-102 Table of Content Storage for the Modern Enterprise 3 The Challenges of Big Data 5 Data at the Center of the Enterprise 6 The Internals of HDFS

More information

ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION

ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION ORACLE BUSINESS INTELLIGENCE, ORACLE DATABASE, AND EXADATA INTEGRATION EXECUTIVE SUMMARY Oracle business intelligence solutions are complete, open, and integrated. Key components of Oracle business intelligence

More information

Buying vs. Building Business Analytics. A decision resource for technology and product teams

Buying vs. Building Business Analytics. A decision resource for technology and product teams Buying vs. Building Business Analytics A decision resource for technology and product teams Introduction Providing analytics functionality to your end users can create a number of benefits. Actionable

More information

Offload Enterprise Data Warehouse (EDW) to Big Data Lake. Ample White Paper

Offload Enterprise Data Warehouse (EDW) to Big Data Lake. Ample White Paper Offload Enterprise Data Warehouse (EDW) to Big Data Lake Oracle Exadata, Teradata, Netezza and SQL Server Ample White Paper EDW (Enterprise Data Warehouse) Offloads The EDW (Enterprise Data Warehouse)

More information

Accelerate BI Initiatives With Self-Service Data Discovery And Integration

Accelerate BI Initiatives With Self-Service Data Discovery And Integration A Custom Technology Adoption Profile Commissioned By Attivio June 2015 Accelerate BI Initiatives With Self-Service Data Discovery And Integration Introduction The rapid advancement of technology has ushered

More information

Automated Business Intelligence

Automated Business Intelligence Automated Business Intelligence Delivering real business value,quickly, easily, and affordably 2 Executive Summary For years now, the greatest weakness of the Business Intelligence (BI) industry has been

More information

Discovering Business Insights in Big Data Using SQL-MapReduce

Discovering Business Insights in Big Data Using SQL-MapReduce Discovering Business Insights in Big Data Using SQL-MapReduce A Technical Whitepaper Rick F. van der Lans Independent Business Intelligence Analyst R20/Consultancy July 2013 Sponsored by Copyright 2013

More information

FROM DATA STORE TO DATA SERVICES - DEVELOPING SCALABLE DATA ARCHITECTURE AT SURS. Summary

FROM DATA STORE TO DATA SERVICES - DEVELOPING SCALABLE DATA ARCHITECTURE AT SURS. Summary UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS Working paper 27 February 2015 Workshop on the Modernisation of Statistical Production Meeting, 15-17 April 2015 Topic

More information

Best Practices for Deploying Managed Self-Service Analytics and Why Tableau and QlikView Fall Short

Best Practices for Deploying Managed Self-Service Analytics and Why Tableau and QlikView Fall Short Best Practices for Deploying Managed Self-Service Analytics and Why Tableau and QlikView Fall Short Vijay Anand, Director, Product Marketing Agenda 1. Managed self-service» The need of managed self-service»

More information

Big Data and Big Data Modeling

Big Data and Big Data Modeling Big Data and Big Data Modeling The Age of Disruption Robin Bloor The Bloor Group March 19, 2015 TP02 Presenter Bio Robin Bloor, Ph.D. Robin Bloor is Chief Analyst at The Bloor Group. He has been an industry

More information

Data Management, Analytics and Business Intelligence

Data Management, Analytics and Business Intelligence TECHNOLOGY TRANSFER PRESENTS Rome, June 25-26 2015 Residenza di Ripetta Via di Ripetta, 231 INTERNATIONAL SUMMIT 2 0 1 5 Data Management, Analytics and Business Intelligence A B O U T T H E S U M M I T

More information

THE DEVELOPER GUIDE TO BUILDING STREAMING DATA APPLICATIONS

THE DEVELOPER GUIDE TO BUILDING STREAMING DATA APPLICATIONS THE DEVELOPER GUIDE TO BUILDING STREAMING DATA APPLICATIONS WHITE PAPER Successfully writing Fast Data applications to manage data generated from mobile, smart devices and social interactions, and the

More information

In-Database Analytics

In-Database Analytics Embedding Analytics in Decision Management Systems In-database analytics offer a powerful tool for embedding advanced analytics in a critical component of IT infrastructure. James Taylor CEO CONTENTS Introducing

More information

Blazent IT Data Intelligence Technology:

Blazent IT Data Intelligence Technology: Blazent IT Data Intelligence Technology: From Disparate Data Sources to Tangible Business Value White Paper The phrase garbage in, garbage out (GIGO) has been used by computer scientists since the earliest

More information

The IBM Cognos Platform

The IBM Cognos Platform The IBM Cognos Platform Deliver complete, consistent, timely information to all your users, with cost-effective scale Highlights Reach all your information reliably and quickly Deliver a complete, consistent

More information

Improving your Data Warehouse s IQ

Improving your Data Warehouse s IQ Improving your Data Warehouse s IQ Derek Strauss Gavroshe USA, Inc. Outline Data quality for second generation data warehouses DQ tool functionality categories and the data quality process Data model types

More information