Empowering Operational Business Intelligence with Data Replication

Transcription

1 Empowering Operational Business Intelligence with Data Replication A Whitepaper Rick F. van der Lans Independent Business Intelligence Analyst R20/Consultancy April 2013 Sponsored by

2 Copyright 2013 R20/Consultancy. All rights reserved. Trademarks referenced in this document are the sole property of their respective owners.

3 Table of Contents 1 Management Summary 1 2 Operational BI and the Need for Low-Latency Data 2 3 Big Data and Operational Business Intelligence 3 4 Different Forms of Operational Business Intelligence 4.1 Examples of Operational Reporting 4.2 Examples of Operational Analytics 4.3 Embedded Operational Reporting and Analytics 5 Limitations of Current Business Intelligence Systems 7 6 Data Replication to the Rescue 10 7 Data Replication to Enable Operational Business Intelligence 7.1 Data Replication for Simple OBI Reporting 7.2 Data Replication for OBI reporting on Multiple Databases 7.3 Data Replication for Consistent OBI and CBI Reporting 7.4 Data Replication for OBI Reporting Using Historical Data 8 HVR Software Solutions 17 9 High Volume Replicator s Features for Operational Business Intelligence 9.1 High-Speed Replication 9.2 Minimal Source System Interference 9.3 Easy to Use, Maintain, and Monitor 9.4 Heterogeneous Data Sources and Platforms 9.5 Transformation Capabilities 9.6 Reliability and Recovery 9.7 Multiple Topologies 9.8 Database Refresh and Compare 9.9 Interfacing with ETL 9.10 Cloud 10 Two Case Studies 25 About the Author Rick F. van der Lans 27 About HVR Software Copyright 2013 R20/Consultancy, all rights reserved.

4 Copyright 2013 R20/Consultancy, all rights reserved.

5 Empowering Operational Business Intelligence with Data Replication 1 1 Management Summary Operational business intelligence (OBI) is about supporting decision-making processes that require access to data that is 100% up to date, or close to 100% up to date. In other words, OBI involves delivering zero-latency data. Developing OBI systems is, however, a technological challenge. It demands realtime integration of data, processing of massive amounts of data, 24x7 availability of data, realtime analytics of streaming data, and large groups of users. This whitepaper describes how data replication technology can be a key building block for developing OBI systems. Implementing OBI within an existing BI system is not straightforward. Most BI systems are based on a network of databases in which data is copied from one database to the next. The last databases in this network, the so-called data marts, are the ones accessed by the reports. In many current BI systems, ETL solutions are used to copy the data. These widely used tools are not designed for realtime data feed. Usually, ETL jobs are scheduled to run periodically (every few hours, at night, or in the weekend), because using ETL continuously would impact the performance and scalability of production systems negatively. The biggest trend in the BI industry is big data. Many big data projects can be classified as OBI projects, because the data must be analyzed when it comes in. The big difference with classic forms of OBI is the amount of data that must be processed. Implementing OBI can already be a technological challenge, due to the low-latency demand, OBI extended with big data raises the technological bar even higher. Nevertheless, organizations see opportunities for deploying big data-based OBI. Organizations need architectures in which OBI reports can access low-latency data and where interference on the operational systems is minimal or zero. This is where data replication technology comes to the rescue. Instead of periodically copying a batch of data, data replication keeps a Operational BI extended with big data raises the technological bar. Data replication is the technological missing link needed to implement OBI. database up to date by constantly trickle feeding data. In fact, it s the technological missing link needed to implement OBI. And it s this technology that is currently missing in most BI systems. This whitepaper has two purposes. One, it describes different ways of how data replication can be embedded in BI systems to deliver zero-latency data. Second, the whitepaper lists the features a data replicator must support that are crucial for developing OBI reports. The whitepaper also describes how well High Volume Replicator supports these requirements. This data integrator, offered by HVR Software, offers all the standard features to be expected from such a product, plus a number of quite unique ones. HVR supports: Log-based change data capture for providing non-intrusive data replication. High-speed architecture for fast and reliable delivery and processing of data. Data refresh for synchronization of multiple databases. Data compare for comparing two databases (even if they are from different vendors). File replication for robust and efficient transmitting files from one environment to another. Easy to use, manage, and monitor for improving productivity and simplifying maintenance. Copyright 2013 R20/Consultancy, all rights reserved.

6 Empowering Operational Business Intelligence with Data Replication 2 Heterogeneous system support for supporting a large set of source and target systems, ranging from classic SQL databases to high-end appliances, and cloud-based applications. Copyright 2013 R20/Consultancy, all rights reserved.