Open Source Data Warehousing and Business Intelligence

Size: px

Start display at page:

Download "Open Source Data Warehousing and Business Intelligence"

Ruby Manning
8 years ago
Views:

1 Open Source Data Warehousing and Business Intelligence Lakshman Bulusu CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup, an infonna business AN AUERBACH BOOK

Raton London New York CRC Press is an imprint of the

2 Contents Foreword Introduction What Does This Book Cover? Who Should Read This Book? Why a Separate Book? Acknowledgements About the Author xvii xix xxii xxii xxiii xxvii xxix Chapter 1 Introduction 1 Chapter In This Chapter Data Warehousing and Business Intelligence: What, Why, How, When, When Not? Taking IT Intelligence to Its Apex 3 '1.3 Open Source DW and Bl: Much Ado about Anything-to-Everything DW and Bl, When Not, and Why So Much Ado? Taking Business Intelligence to Its Apex: Intelligent Content for Insightful Intent Summary 13 Data Warehousing and Business Intelligence: An Open Source Solution t In This Chapter What Is Open Source DW and Bl, and How "Open" Is This Open? 17 VII

2 Data Warehousing and Business Intelligence: What, Why, How, When, When Not? 2 1.2.1 Taking IT Intelligence to Its Apex 3 '1.

3 viii Contents 2.3 What's In, What's Not: Available and Viable Options for Development and Deployment Semantic Analytics \ Testing for Optimizing Quality and Automation Accelerated! Business Rules, Real-World Perspective, Social Context Personalization Through Customizable Measures Leveraging the Cloud for Deployment The Foundations Underneath: Architecture, Technologies, and Methodologies Open Source versus Proprietary DW and Bl Solutions: Key Differentiators and Integrators Open Source DW and Bl: Uses and Abuses An Intelligent Query Accelerator Using an Open Cache In, Cache Out Design Summary 31 Chapter 3 Open Source DW & Bl: Successful Players and Products In This Chapter Open Source Data Warehousing and Business Intelligence Technology Licensing Models Followed Community versus Commercial Open Source The Primary Vendors: Inventors and Presenters ' Oracle: MySQL Vendor PostgreSQL Vendor Infobright Pentaho: Mondrian Vendor Jedox: Palo Vendor EnterpriseDB Vendor Dynamo Bl and Eigenbase: LucidDB Vendor i GreenPlum Vendor ' Hadoop Project HadoopDB Talend 44

4 The Foundations Underneath: Architecture, Technologies, and Methodologies 21 2.5 Open Source versus Proprietary DW and Bl Solutions: Key Differentiators and Integrators 27 2.

4 Contents ix The Primary Products and Tools Set: Inclusions and Exclusions Open Source Databases N Open Source Data Integration Open Source Business Intelligence Open Source Business Analytics The Primary Users: User, End-User, Customer and Intelligent Customer MySQL PostgreSQL Mondrian Customers Palo Customers EnterpriseDB Customers LucidDB Customers Greenplum Customers Talend Customers Summary Reference; Chapter 4 Chapter 5 Analysis, Evaluation and Selection In This Chapter Essential Criteria for Reauirements Analvsis of an 99 Open Source DW and Bl solution Key and Critical Deciding Factors in Selecting a Solution The Selection-Action Preview Raising your BIQ: Five Things Your Company Can Do Now Evaluation Criteria for Choosing a Vendor- Specific Platform and Solution The Final Pick: An Information-Driven, Customer-Centric Solution, and a Best-of-Breed Product/Platform and Solution Convergence Key Indicator Checklist Summary References? 118 Design and Architecture: Technologies and Methodologies by Dissection In This Chapter

1 4.2 In This Chapter Essential Criteria for Reauirements Analvsis of an 99 Open Source DW and Bl solution 100 4.3 Key and Critical Deciding Factors in Selecting a Solution 102 4.3.1 The Selection-Action Preview 103 4.

5 x Contents 5.2 The Primary Aspects of DW and Bl from a Usability Perspective: Strategic Bl, Pervasive Bl, Operational Bl, and Bl On-Demand x : Design and Architecture Considerations for the Primary Bl Perspectives The Case for Architecture as a Precedence Factor Information-Centric, Business-Centric, and Customer-Centric Architecture: AThree-in-One Convergence, for Better or Worse Open Source DW and Bl Architecture Pragmatics and Design Patterns Components Why and How an Open Source Architecture Delivers a Better Enterprisewide Solution Open Source Data Architecture: Under the Hood Open Source Data Warehouse Architecture: Under the Hood Open Source Bl Architecture: Under the Hood The Vendor/Platform Product(s)/Tools(s) That Fit into the Open DW and Bl Architecture Information Integration, Usability and Management (Across Data Sources, Applications and Business Domains) EDW: Models to Management Bl: Models to Interaction to Management to Strategic Business t Decision Support (via Analytics and. Visualization) Best Practices: Use and Reuse Summary 147 Chapter 6 Operational Bl and Open Source In This Chapter Why a Separate Chapter on Operational Bl and Open Source? J Operational Bl by Dissection Design and Architecture Considerations for Operational Bl Operational Bl Data Architecture: Under the Hood 157

4 Information-Centric, Business-Centric, and Customer-Centric Architecture: AThree-in-One Convergence, for Better or Worse 123 5.5 Open Source DW and Bl Architecture 125 5.5.1 Pragmatics and Design Patterns 126 5.

6 Contents xi 6.6 A Reusable Information Integration Model: From Real- Time to Right Time Operational Bl Architecture: Under the Hood Fitting Open Source Vendor/Platform Product(s)/ Tools(s) into the Operational Bl Architecture Talend Data Integration expressor 3.0 Community Edition Advanced Analytics Engines for Operational Bl Astera's Centerprise Data Integration Platform Actuate BIRT BI Platform JasperSoft Enterprise Pentaho Enterprise Bl Suite KNIME (Konstanz Information Miner) Pervasive DataRush Pervasive DataCloud Best Practices: Use and Reuse Summary 169 Chapter 7 Development and Deployment In this Chapter Introduction Development Options, Dissected Deployment Options, Dissected Integration Options, Dissected 182 t 7.6 Multiple Sources, Multiple Dimensions DW and Bl Usability and Deployment: Best Solution versus Best-Fit Solution Leveraging the Best-Fit Solution: Primary Considerations Better, Faster, Easier as the Hitchhiker's Rule Dynamism and Flash Real Output in Real Time in the Real World Interactivity Better Responsiveness, User Adoptability, and Transparency Fitting the Vendor/Platform Product(s)/tTools(s): A Development and Deployment Standpoint Best Practices: Use and Reuse Summary 204

8.4 Astera's Centerprise Data Integration Platform 165 6.8.5 Actuate BIRT BI Platform 165 6.8.6 JasperSoft Enterprise 166 6.8.7 Pentaho Enterprise Bl Suite 166 6.8.8 KNIME (Konstanz Information Miner) 167 6.

7 xii Contents Chapter 8 Best Practices for Data Management In This Chapter, Introduction ; Best Fit of Open Source in EDW Implementation Best Practices for Using Open Source as a Bl-Only Methodology for Data/Information Delivery Mobile Bl and Pervasive Bl Best Practices for the Data Lifecycle in a Typical EDWLifecycle Data Quality, Data Profiling, and Data Loss Prevention Components The Data Integration Component Best Practices for the Information Lifecycle as It Moves into the Bl Lifecycle The Data Analysis Component: The Dimensions of Data Analysis in Terms of Online Analytics vs. Predictive Analytics vs. Real-Time Analytics vs. Advanced Analytics Data to Information Transformation and Presentation Best Practices for Auditing Data Access, as It Makes Its Way via the EDW and Directly (Bypassing the EDW) to the Bl Dashboard Best Practices for Using XML in the Open Source EDW/BI Space ', Best Practices for a Unified Information Integrity and Security Framework Object to Relational Mapping: A Necessity or Just a Convenience? Synchrony Maintenance Dynamic Language Interoperability Summary 262 Chapter 9 Best Practices for Application Management J In This Chapter Introduction Using Open Source as an End-to-End Solution Option: How Best a Practice Is It? 266

5 Best Practices for the Data Lifecycle in a Typical EDWLifecycle 210 8.5.1 Data Quality, Data Profiling, and Data Loss Prevention Components 212 8.5.2 The Data Integration Component 219 8.

8 Contents xiii Chapter Accelerating Application Development: Choice, Design, and Suitability Aspects Visualization of Content: For Better or Best Fit Best Practices for Autogenerating Code: A Codeless Alternative to Information Presentation Automating Querying: Why and When How Fine Is Fine-Grained? Drawing the Line between Representation of Data at the Lowest Level and a Best-Fit Metadata Design and Presentation Best Practices for Application Integrity Sharing Data between EDW and the Bl Tiers: Isolation or a Tightrope Methodology Breakthrough Bl: Self-Serviceable Bl via a Self-Adaptable Solution Data-in, Data-Out Considerations: Data-to-lnformation I/O Security Inside and Outside Enterprise Parameters: Best Practices for Security beyond User Authentication Best Practices for Intra- and Interapplication Integration and Interaction Continuous Activity Monitoring and Event Processing 286 t Best Practices to Leverage Cloud-Based Methodologies Best Practices for Creative Bl Reporting Summary 297 Best Practices Beyond Reporting: Driving Business Value In This Chapter Introduction. j Advanced Analytics: The Foundation for a Beyond-Reporting Approach (Dynamic KPI, Scorecards, Dynamic Dashboarding, and Adaptive Analytics) 300

5 Best Practices for Application Integrity 275 9.5.1 Sharing Data between EDW and the Bl Tiers: Isolation or a Tightrope Methodology 278. 9.5.2 Breakthrough Bl: Self-Serviceable Bl via a Self-Adaptable Solution 279 9.

9 xiv Contents 10.4 Large Scale Analytics: Business-centric and Technology-centric Requirements and Solution Options "\ Business-centric Requirements Technology-centric Requirements Accelerating Business Analytics: What to Look for, Look at, and Look Beyond Delivering Information on Demand and Thereby Performance on Demand Design Pragmatics Demo Pragmatics Summary 329 Chapter 11 EDW/BI Development Frameworks In This Chapter Introduction From the Big Bang to the Big Data Bang: The Past, Present, and Future A Framework for Bl Beyond Intelligence Raising the Bar on Bl Using Embeddable Bl and Bl in the Cloud Raising the Bar on Bl: Good to Great to Intelligent Raising the Bar on the Social Intelligence Quotient (SIQ) Raising the Bar on Bl by Mobilizing Bl: Bl on the Go, A Pragmatic Framework for a Customer-Centric EDW/BI Solution A Next-Generation Bl Framework Taking EDW/BI to the Next Level: An Open Source Model for EDW/BI-EPM Open Source Model for an Open Source DW-BI/EPM Solution Delivering Business Value f Open Source Architectural Framework for a Best-Fit Open Source BI/EPM Solution Value Proposition The Road Ahead

7 Summary 329 Chapter 11 EDW/BI Development Frameworks 331 11.1 In This Chapter 331 11.2 Introduction 332 11.3 From the Big Bang to the Big Data Bang: The Past, Present, and Future 332 11.

10 Contents xv 11.7 A Bl Framework for a Reusable Predictive Analytics Model A Bl Framework for Competitive Intelligence: Time, Technology, and the Evolution of the Intelligent Customer Summary 360 Chapter 12 Best Practices for Optimization In This Chapter Accelerating Application Testing: Choice, Design, and Suitability Best Practices for Performance Testing: Online and On Demand Scenarios A Fine Tuning Framework for Optimality Looking Down the Customer Experience Trail, Leaving the Customer Alone: Customer Feedback Management (CFM)-Driven and APM-Oriented Tuning Codeful and Codeless Design Patterns for Business-Savvy and IT-Friendly QOS Measurements and In-Depth Impact Analysis Summary 375 Chapter 13 Open Standards for Open Source: An EDW/BI Outlook Introduction - ' Summary ' References 385 Index 387

3 Best Practices for Performance Testing: Online and On Demand Scenarios 366 12.4 A Fine Tuning Framework for Optimality 369 12.

Open Source Data Warehousing and Business Intelligence

Open Source Data Warehousing and Business Intelligence Lakshman Bulusu K10843_FM.indd 3 7/10/12 11:32 AM CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742