TOTAL DATA INTEGRATION

Transcription

1 The Impact of Big Data on Integration and Governance Big data and Total Data have the potential to change the face of the data integration market. This report outlines the key drivers shaping this sector today and in coming years, and highlights the role that integration and governance technologies have to play in enabling the analysis of structured, semi-structured and unstructured data. KEY FINDINGS Data integration technologies have a fundamental role to play in delivering on the potential of big data by ensuring that emerging data processing and analysis technologies work well with each other, and also with more established approaches. The roles of established data management approaches such as data governance, data quality and master data management gain even greater significance given the desire to process and analyze data from previously ignored sources. The off-loading of the ELT process to Hadoop from a data warehouse can not only remove bottlenecks, but also free up a data warehouse s capacity for user queries. By adopting an approach we are calling Total Data Integration, a company not only gets a complete picture of its business, customers and products, but it can derive new insights that weren t possible when only structured data was brought into the integration mix. NOVEMBER 2013

2 REPORT SNAPSHOT TITLE Total Data Integration ANALYSTS RELEASE DATE November 2013 LENGTH Krishna Roy, Analyst, BI and Analytics Matthew Aslett, Research Director, Data Management and Analytics 33 pages ABOUT THIS REPORT Data integration technologies have a critical role to play in delivering on the potential of big data, by ensuring that emerging data processing and analysis technologies work well with each other, and also with more established approaches. The roles of established data management techniques such as data governance, data quality and master data management gain even greater significance given the desire to process and analyze data from previously ignored sources. By adopting an approach we call Total Data Integration, a company not only gets a complete picture of its business, customers and products, but it can derive new insights that weren t possible when only structured data was brought into the mix. This report outlines the key drivers shaping this sector and highlights the role that integration and governance technologies will play in enabling the analysis of structured, semi-structured and unstructured data. It explains how established vendors and emerging startups alike are responding to the increased complexity of Total Data analytics by enabling integration of these various data sources. The report also explores new approaches to data integration, including the role of Hadoop.

3 TABLE OF CONTENTS SECTION 1: EXECUTIVE SUMMARY KEY FINDINGS METHODOLOGY SECTION 2: DATA INTEGRATION IN A WORLD OF TOTAL DATA 4 Figure 1: Total Data Integration Glossary Figure 2: Traditional Analytics Stack (Simplified) Figure 3: Total Data Analytics Stack (Simplified) Figure 4: Role of the Total Data Hub (Simplified) The Business Case for Total Data Integration Riding the Elephant The Rise of the Machine and NoSQL Big-Data Integration Upstarts HADOOP AS AN INTEGRATION ENGINE Putting the T in ETL Enabling Exploratory Analysis ETL Replacement Beyond Transformation Limitations APPROACHES TO Addressing the Data Deluge To ETL Or Not to ETL? Data Virtualization and Other Integration Methods TOTAL DATA GOVERNANCE Identifying Dirty Data Products for Dirty Data Using a Mask Governing and Mastering Big Data THE TOTAL DATA HUB - RECONSIDERED Existing Players Refine Their Strategies New Players Enter the Arena

4 2.5.3 Defining the Total Data Hub Figure 5: Total Data Hub SECTION 3: FUNDING AND M&A 26 Figure 6: Potential Investment Opportunities Figure 7: Recent Data-Integration-Related Acquisitions SECTION 4: 20 VENDORS TO WATCH 29 INDEX OF COMPANIES 34

5 ABOUT 451 RESEARCH 451 Research is a leading global analyst and data company focused on the business of enterprise IT innovation. Clients of the company at end-user, service-provider, vendor and investor organizations rely on 451 Research s insight through a range of syndicated research and advisory services to support both strategic and tactical decision-making Research, LLC and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication, in whole or in part, in any form without prior written permission is forbidden. The terms of use regarding distribution, both internally and externally, shall be governed by the terms laid out in your Service Agreement with 451 Research and/or its Affiliates. The information contained herein has been obtained from sources believed to be reliable. 451 Research disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although 451 Research may discuss legal issues related to the information technology business, 451 Research does not provide legal advice or services and their research should not be construed or used as such. 451 Research shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.