Traditional vs Big Data Analytics

Transcription

1 Webminar Business Analytics and Big Data Traditional vs Big Data Analytics Luis Reina 1

2 Some Data History. Operational Systems Relational Model 1970 OLTP Databases Edgar Frank Codd Internal Data Informational Systems Structured Data Stable Data Models Historical Data 2

3 But The World have changed becoming more INSTRUMENTED INTERCONNECTED INTELLIGENT The resulting explosion of information creates a need for a new kind of intelligence 3

4 12+ TBs of tweet data every day 50 billion RFID tags (1.3B in 2005) data every day Driving to the Data Revolution 4.6 billion camera phones world wide? TBs of 25+ TBs of log data every day 80% Of world s data is unstructured 76 million smart meters in M by s of millions of GPS enabled devices sold annually 2+ billion people on the Web by end

5 Data Volume Traditional vs Big Data Analytics Ignorance Zone Grows Everyday Time 5

6 Modeling Big Data Data is the new oil. Clive Humby We have for the first time an economy based on a key resource [Information] that is not only renewable, but self-generating. Running out of it is not a problem, but drowning in it is. John Naisbitt 6

7 Traditional vs Big Data Analytics Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Business Users Determine Questions Analyzed Information IT Team Delivers Data On Flexible Platform Analyzed IT Team Information Builds System To Answer Known Questions Available Information Capacity constrained down sampling of available information Carefully cleanse a small information before any analysis Analyze ALL Available Information Whole population analytics connects the dots Analyzed Information Business Users Explore and Ask Any Question Analyze information as is & cleanse as needed & existing repeatable 7

8 Traditional vs Big Data Analytics Traditional Analytics Structured & Repeatable Structure built to store data Hypothesis Question? Big Data Analytics Iterative & Exploratory Data is the structure Data All Information Exploration Analyzed Information Answer Data Start with hypothesis Test against selected data Actionable Insight Correlation Data leads the way Explore all data, identify correlations Analyze after landing Analyze in motion 8

9 Stream Computing: New Paradigm Traditional Computing Stream Computing Historical fact finding with data-at-rest Batch paradigm, pull model Query-driven: submits queries to static data Relies on Databases, Data Warehouses Real time analysis of data-in-motion Streaming data Stream of structured or unstructured data-in-motion Stream Computing Analytic operations on streaming data in real-time Queries Data Results Data Queries Results 9

10 Big Data: Most of the Data is unstructured Most of the data is unstructured o semistructured. In social networks there are a lot of discussion about products and services. A lot of organizations information is kept on LOBs, description fields, this information is never used. How can I get metrics from unstructured text? I had an iphone, but it's (I've no idea where it's)!want a blackberry im moving to miami in 3 months. i look foward to the new lifestyle I'm at Mickey's Irish Pub Downtown (206 3rd St, Court Ave, Des Moines) w/ 2 others 10

11 The Magic of Analyzing Unstructured Data Mobile Distribution I like Android a lot I am going to buy a new pone soon.... Twitters Magic Analyze Obtain the relevant information from the nonstructured data. Eliminate noise. Format relevant information in a format that it can be: Analyzed, Enriched, Correlated with other information. Convert nonstructured data in structured data! 11

12 World Cup 2010 Football World Cup 2010, one team distinguished well from the rest winning the final. Early in the second half, Netherlands striker, Arjen Robben, had a chance to score, but the awesome keeper for Spain, Iker Casillas made the save. Winner superiority was reflected when Winger Andres Iniesta scored for Spain for the win. World Cup 2010 Highlights Arjen Robben Iker Casillas Andres Iniesta Striker Keeper Winger Netherlands Spain Spain 12

13 What is Big Data and What is not? The Limit is not 100% clear TRADITIONAL DATA BIG DATA Relational Database OLTP ERP. Social Networks Sensor Data. 13

14 What is Big Data and What is not? DATA that because of its VOLUME, VELOCITY or VARIETY (format) Is DIFICULT o UNPRACTICAL ANALYZE With TRADITIONAL means. 14

15 New Technologies and not so new Zoo keeper PIG 15

16 Some Conclusions Traditional Data Warehouse will remain. But Analyzing Traditional Data could not be enough. Analyzing Big Data can be a competitive Advantage. New Technologies make analyzing Big Data cost effective. 16