Optimization of Analytic Data Flows for Next Generation Business Intelligence Applications

Transcription

1 Optimization of Analytic Data Flows for Next Generation Business Intelligence Applications Umeshwar Dayal, Kevin Wilkinson, Alkis Simitsis, Malu Castellanos, Lupita Paz HP Labs Palo Alto, CA, USA 1 Copyright Copyright 2010 Hewlett-Packard 2010 Hewlett-Packard Development Development Company, Company, L.P. L.P.

2 OUTLINE Next Generation Business Intelligence Analytic Data Flows Challenges in Analytic Data Flow Design and Optimization QoX Framework QoX-driven Optimization of Data Integration Flows Micro-benchmarks Open Problems Summary

3 TRADITIONAL BUSINESS INTELLIGENCE ARCHITECTURE BI: Technologies, tools, and practices for collecting, integrating, analyzing, and presenting large volumes of information to enable better decision making Focus on enterprise OLTP data sources Batch ETL (Extract-Transform-Load) pipeline Front-end analytics for strategic decision making

4 EMERGENCE OF AN INFLECTION POINT Sensors, real time events, unstructured data and web contents have the promise of transforming the way we manage our customers, resources, environments, health, The shift to the web makes available unprecedented quantities of structured and unstructured databases about human activities and sensing technologies are making available great quantities of data that provide unprecedented scope and resolution the availability of rich streams of data, create(s) an inflection point.. for generating insights and guiding decision making To date, we have only scratched the surface of the potential for learning from these large-scale data sets.. - From Data to Knowledge to Action A Global Enabler for the 21 st Century, Computing Community Consortium, August 2010 Analytics, and specifically technologies that enable real-time analysis of large data volumes, social media analytics, and mobile BI will continue to dominate the enterprise agenda - Computer World, Jan

5 Live BI: Decisions at any time scale Executives Model Builders Enterprise Operational systems Business Control Systems Operations staff Operational Data Stores Enterprise Data Warehouse Decreasing time scales for response (decision latencies) Traditional Business Intelligence 5 5 Increasing need for automation Live Business Intelligence

6 ANALYTICS DATA FLOWS End-to-end Analytics Data Flow Front-end Analytics Back-end Analytics Analysis querying, reporting, data mining, modeling, analytics, visualization operations Loading Transformation Cleaning, de-noising, entity identification, etc. Information extraction, access Streaming Challenges Long development cycles Little support for optimization or automation Possibly many objectives: Correctness, Fault-tolerance, Scalability, Maintainability, Cost Many types of data sources: Structured/ Unstructured, Stored/ Streaming, Internal/ External Complex flows, involving ETL, stream processing, queries, analytics, visualization operations Possibly executed on multiple engines

7 NEXT-GENERATION: LIVE BI Development & Deployment-time Execution-time Interaction and Delivery Data Flow Composer and Editor Federation and Orchestration Optimizer Execution Engines RDBMS Hadoop ETL engine Massively parallel, large memory cluster Stream engine Analytics Engine Data Stores RDB Unstructured File Data Cloud Storage other Streaming data sources (sensor data, event logs, social media, web data )

8 EXAMPLE FLOW 1: STRUCTURED DATA -- TPC-H SOURCES Store 1 Lineitem Union Store n Lineitem Surrogate key generate Lineitem Orders Compute sales = quantity * unitcost Filter on date range Join on orderkey (get relevant sales for date range) Filter on cmpnkey (get products, date range) Join on prodkey (get relevant sales for products) 9 CmpnDim Copyright 2010 Hewlett-Packard Development Company, L.P. Rollup sales on prodkey, cmpnkey RptSalesByProdCmpn Sales of products in response to marketing campaign

9 EXAMPLE FLOW 2: UNSTRUCTURED DATA SENTIMENT ANALYSIS 10 Copyright 2010 Hewlett-Packard Development Company, L.P.

10 EXAMPLE FLOW 3 (STRUCTURED & UNSTRUCTURED DATA) Tweets Filter on any mention of enterprise or its products Relevant tweets Sentiment analysis for products by tweet RptSAbyTwtProd CmpnDim Surrogate key generate Filter on cmpnkey (get products, date range) Filter on date range (get tweets during campaign) Join on prodkey (get tweets for products) 11 RptSalesByProdCmpn Copyright 2010 Hewlett-Packard Development Company, L.P. Rollup sentimentscore on prodkey, cpmnkey, attribute (aggregate sentiment scores) Join on prodkey, cmpnkey RptSAbyProdCmpn Correlate Sales and Sentiments for products in response to marketing campaign

11 EXAMPLE FLOW 4: EVENT AND STREAM ANALYTICS Sensor data, external event Streams Stream pre-processing Detect Patterns/Events Events Event Stream Event Correlation Diagnosis & Root Cause Analysis Complex Events Diagnoses Contextual data (structured, unsructured) Event History Prediction Action Assessment Predicted Events Recommended Actions Event Discovery and Creation Event Analysis 12

12 document batch file Data Sources Preprocessing (Adaptor) (Doc Info, Text body) vertical slice profile info profile info Token, POS Tag, Negation Flag, Attribute Flag, Sentiment Flag] Relate Sentiment to Attribute (Doc Info, Text body) attribute sentiment value Doc Info = (Doc id, Time Stamp, Source type, Source URL, Author Profile, GeoTag, Score) Author Profile = (id, name, , location, gender, ) Splitter on Doc Id, hash partition, 3x parallel Sentiment Word Detection sentiment word dict. Attribute, Sentiment Value] (Doc Info, Text body) Token, POS Tag, Negation Flag, Attribute Flag] Join on doc Id profile info Remove Docs from Blacklisted Authors (Doc Info, Text body) Attribute Extraction attribute dict. Attribute, Sentiment Value] Sentence Detector Token, POS Tag, Negation flag] Aggregate by Doc document sentiment value Sentence] Negation Detection negation dict. (Doc Info, Attribute, Sentiment Value]) Tokenizer Token, POS Tag], order <Doc Id, Sentence Id, Token Loc> Token id, Token], order <Doc Id, Sentence Id, Token Loc> Compound Phrases phrase dict. DASHBOARD POS Tagging Token id, Token], order <Doc Id, Sentence Id, Token Loc> (Doc Info, [ Token id, Token, POS Tag]) recovery point Merger on Doc id, 3x parallel create recovery point FROM DESIGN TO EXECUTION [Dayal, et al., EDBT 09] [Wilkinson, Simitsis ER 10] Flow design editor xlm Logical Data Flow Graph Performance Fault-tolerance Maintainability Freshness Scalability Availability Cost QoX Objectives Optimizer Optimized Physical Data Flow Graph Sources Hadoop Scripts xlm Enginespecific codegen tool-specific execution instructions Scripts DBMS ETL Tool Target 13

13 Quality objectives functional correctness the optimization of the initial flow does not affect the flow semantics or correctness performance measures the flow completion in terms of time and resources used freshness the latency between the occurrence of an event at a source system and the reflection of that event at the target system reliability the ability of flow to complete successfully within the specified time window despite failures 14 14

14 QUALITY OBJECTIVES Performance: usually throughput, total execution time Reliability: probability that a flow will complete correctly (~ mean time between failures) Recoverability: ability to restore the flow after a failure (~mean time to recover) Scalability: ability to process higher volumes of data or higher data rates Freshness: latency between source data and data in the DW Consistency: accuracy (correctness) of the data in the DW Availability: probability that the data is available for queries Maintainability: ability to operate at specified service levels Flexibility: ability to respond to evolving requirements Affordability: cost of integration project Traceability: ability to track lineage (provenance) of data Auditability: ability to satisfy legal/regulatory compliance requirements

15 TRADEOFFS AMONG QoX OBJECTIVES Fastest Recoverable REC REC Perf Rec Rel Cost w/o RPs Performance w/ 2 RPs STATE=AZ REC REC Perf Rec Rel Cost STATE=CA REC REC w/o RPs STATE=NV REC REC w/ 6 RPs Reliable Perf Rec Rel Cost w/o RPs + n/a

16 QoX Optimization [Simitsis, et al., ICDE 10 Simitsis, et al., SIGMOD 09] Logical ] Optimization: Flow restructuring Parallelization, partitioning Collapsing chains Reordering operators Inserting recovery (persistence) points Replication Physical optimization Implementations of operators Materialize vs. Virtualize Choice of execution engines Data shipping vs.function shipping Optimization for performance and faulttolerance Given a data flow F, objective function OF w n: input recordset size w: execution time window k: number of faults to be tolerated

17 STATE SPACE SEARCH initial state min-cost state 18

18 OPTIMIZATION TACTICS: TRANSITIONS IN THE STATE SPACE GRAPH factorize/ distribute swap addrp partition replicate 19

19 OPTIMIZATION FOR MULTI-ENGINE EXECUTION Characterize the execution of operations on different execution engines Implement micro-benchmarks for individual operators Learn cost functions to plug into optimization objective Interpolate from micro-benchmarks Compose costs for individual operations into cost of flow Account for resource contention Extend transitions considered by the optimizer to include implementation and engine choices Extend heuristics for state space search

20 Example: Sentiment Analysis Flow Logical Model Data Sources Doc Info = (Doc id, Time Stamp, Source type, Source URL, Author Profile, GeoTag, Score) Author Profile = (id, name, , location, gender, ) Other Operators Preprocessing (Adaptor) (Doc Info, Text body) document batch file (Doc Info, Attribute, Sentiment Value]) Aggregate by Doc Sentence Detector Attribute, Sentiment Value] Sentence] Remove Docs from Blacklisted Authors Tokenizer Token id, Token], order <Doc Id, Sentence Id, Token Loc> Attribute, Sentiment Value] POS Tagging Relate Sentiment to Attribute (Doc Info, [ Token id, Token, POS Tag]) Token, POS Tag, Negation Flag, Attribute Flag, Sentiment Flag] phrase dict. Compound Phrases Sentiment Word Detection Token, POS Tag], order <Doc Id, Sentence Id, Token Loc> Token, POS Tag, Negation Flag, Attribute Flag] negation dict. Negation Detection Token, POS Tag, Negation flag] Attribute Extraction document sentiment value attribute sentiment value sentiment word dict. attribute dict. 21

21 Example: Sentiment Analysis Flow Physical Model Sources Data Sources Doc Info = (Doc id, Time Stamp, Source type, Source URL, Author Profile, GeoTag, Score) Author Profile = (id, name, , location, gender, ) document batch file Scripts Preprocessing (Adaptor) (Doc Info, Text body) vertical slice profile info profile info (Doc Info, Text body) Splitter on Doc Id, hash partition, 3x parallel Hadoop (Doc Info, Text body) Remove Docs from Blacklisted Authors (Doc Info, Text body) Sentence Detector Sentence] Tokenizer Token id, Token], order <Doc Id, Sentence Id, Token Loc> POS Tagging Token id, Token], order <Doc Id, Sentence Id, Token Loc> recovery point Scripts Merger on Doc id, 3x parallel create recovery point DBMS Token, POS Tag, Negation Flag, Attribute Flag, Sentiment Flag] Sentiment Word Detection sentiment word dict. Token, POS Tag, Negation Flag, Attribute Flag] Attribute Extraction attribute dict. Token, POS Tag, Negation flag] Negation Detection negation dict. Token, POS Tag], order <Doc Id, Sentence Id, Token Loc> Compound Phrases phrase dict. (Doc Info, [ Token id, Token, POS Tag]) ETL Tool Relate Sentiment to Attribute Attribute, Sentiment Value] Join on doc Id Attribute, Sentiment Value] Aggregate by Doc (Doc Info, Attribute, Sentiment Value]) Target DASHBOARD attribute sentiment value profile info document sentiment value 22

22 MICRO-BENCHMARKS FOR CONVENTIONAL (ETL, RELATIONAL) OPERATORS Experiments on sort, join, surrogate-key, etc. Example: Sort os-sort home-grown distributed sort implementation hd-sort Hadoop-based sort (used Pig Latin) pdb-sort used a commercial, parallel DBMS Dataset TPC-H lineitem SF Size (GBs) Rows (x10 6 )

23 Sort (Comparison) Hadoop vs. Custom script vs. pdb (with and without loading) Pig script (hd-sort): sf$sf = load 'lineitem.tbl.sf$sf' using PigStorage(' ') as (f1,f2,...,f17); ord1_$nd = order sf$sf by f1 parallel $nd;... ord10_$nd = order ord9_$nd by f10 parallel $nd; store ord$op_$nd into 'res_sf$sf_ord$op-$nd.dat using PigStorage(' '); C and shell scripts (os-sort): split file: filesize/#nodes (leftovers go to chunk #1) transfer data sort chunks on nodes transfer back and merge (hierarchical) (to be fair with hd-sort and etl-sort, avoid pipelining) 24 (TPC-H data SF:1,10,100)

24 Sort (Tuning) Hadoop e.g., #reducers Shell scripts e.g., analyze sort internal phases 25

25 MICRO-BENCHMARKS FOR UNCONVENTIONAL OPERATORS Experiments on Sentiment Analysis operators individual operators the whole sentiment analysis flow treated as a single operator Implementations Single node Distributed, scripts Distributed, Hadoop Datasets Small- to medium-sized document corpus: up to 200,000 documents Large corpus: ~30 million documents

26 MICRO-BENCHMARKS FOR UNCONVENTIONAL OPERATORS (SENTIMENT ANALYSIS) Single node, small to mediumsized corpus Use linear regression to learn interpolation function 27 Copyright 2010 Hewlett-Packard Development Company, L.P. Single node, large corpus; effect of limited memory 12-node Hadoop cluster

27 SENTIMENT ANALYSIS (COMPARISON) Single node vs. Distributed with Hadoop vs. Distributed without Hadoop

28 OPEN PROBLEMS: FUTURE WORK Micro-benchmarks of other operators: ETL, relational, content analytics, front-end analytics, stream processing Cost functions: interpolation, composition Benchmarks and objective functions for other QoX measures Optimization strategies for physical optimization Assign segments of the flow to execution engines Where, when, and what to materialize

29 SUMMARY Design of data flows for BI applications is a very expensive, largely manual activity: little support for automation, optimization Situation gets worse with next gen BI: fuse back-end and front-end operations into end-to-end analytic data flows Many different types of sources, more complex operators, multiple execution engines Layered methodology: QoX-driven optimization allows trading off different quality objectives Need benchmarks and micro-benchmarks to drive the design and optimization Many challenges remain

30 31 Copyright 2010 Hewlett-Packard Development Company, L.P. Thank You