Turning Big Data into Big Decisions Delivering on the High Demand for Data Michael Ho, Vice President of Professional Services Digital Government Institute s Government Big Data Conference, October 31, 2013, Washington, DC
What is big data? 2
Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable 3
Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable 4
What does Structured Valuable Large Zettabytes? What is big data? Exabytes? Various Complex Fast-moving Petabytes? Terabytes? Gigabytes? Variable Unstructured mean? 5
What does Structured Valuable Large Complex What is big data? Various Unstructured Fast-moving Variable mean? 6
What does Valuable Large Complex Structured What is big data? Various Fast-moving Unstructured Variable mean? 7
So what actually makes data big? When the amount of data is too large to utilize within its window of relevance. 8
There is no formula. Data sources Data velocity Amount of data Window of relevance 9
There is no formula. Window of relevance Data sources Amount of data Data velocity are constantly shifting. 10
Why the hype, why now? Demand 2008 The number of internet-enabled devices exceeds the total world population. 2012 2020 An estimated 2.8 zettabytes, or over 3 trillion gigabytes, of data is collected. Experts predict 40 zettabytes of data will be created by the year 2020. 11
Why the hype, why now? Cost effectiveness Cost per gigabyte of RAM 1965 2010 2025 12
The big [data] picture 13
The big [data] picture Getting the right data 14
The big [data] picture Getting the right data In front of the right people 15
The big [data] picture Getting the right data In front of the right people At the right time 16
The big [data] picture Saves time 17
The big [data] picture Saves time Saves money 18
The big [data] picture Saves time Saves money Saves lives 19
The big [data] picture Revolutionary or Evolutionary? 20
The big [data] picture and Revolutionary or Evolutionary 21
Big data technologies are the natural evolution of data collection and processing. 1970s - First RDBMS and SQL developed 2003 Ehcache created 1991 WWW becomes publicly available 1995 GPS operationalized 2005 Hadoop created 2009 Terracotta creates Enterprise Ehcache 1998 Google founded 2003 Wikipedia started 1970 1980 1990 2000 2010 22
Big data technologies are the natural evolution of data collection and processing. Shrinking Decision Windows Decision Time ENTERPRISE EHCACHE Data Size Data Growth 1970 1980 1990 2000 2010 23
Technology evolving to meet demand ENTERPRISE EHCACHE Traditional database or data warehouse Parallel computing, distributed processing Faster on demand access, interaction, and analytics Better, faster decision making 24
We ve seen this before. Personal computer Parallel computing, distributed processing Faster on demand access, interaction, and analytics Better, faster decision making 25
Evolving with your enterprise A good big data solution builds upon the solid technology foundation you have already laid. 26
The best big data solution will revolutionize the way you and your enterprise interact with data and each other. 27
Use cases revolutionized by big data solutions Real-time Credit & Market Risk in Banks A/B Testing Fraud Detection (Credit Card) & Financial Crimes (AML) in Banks Event-based Marketing in Financial Services and Telecomms Household Scoring Data Velocity Claims and Tax Fraud in Public Sector Predictive Maintenance in Aerospace Social Media Sentiment Analysis Demand Forecasting in Manufacturing Traditional Data Warehousing Disease Analysis on Electronic Health Records Market Sizing Video Surveillance & Analysis Batch Structured Semi-structured Unstructured Data Variety 28
Use cases revolutionized by big data solutions Real-time Credit & Market Risk in Banks A/B Testing Fraud Detection (Credit Card) & Financial Crimes (AML) in Banks Event-based Marketing in Financial Services and Telecomms Household Scoring Data Velocity Claims and Tax Fraud in Public Sector Predictive Maintenance in Aerospace Social Media Sentiment Analysis Demand Forecasting in Manufacturing Traditional Data Warehousing Disease Analysis on Electronic Health Records Market Sizing Video Surveillance & Analysis Batch Structured Semi-structured Unstructured Data Variety 29
A real-time data revolution Consume Analyze Visualize ENTERPRISE EHCACHE Volume Velocity Variety Historical Data Big Data Live Data 30
Visual tools facilitate real time analysis of disparate data Step 1 Step 2 Step 3 Consume User- Driven Analyze User- Driven Visualize 31
Real-time Analytics allows you to get the right data in front of the right people at the right time in ways you couldn t before. 32
Real-time Analytics Use Case #1: Situational Awareness 33
Real-time Analytics Use Case #2: Cyber Defense 34
Real-time Analytics Use Case #3: Financial Planning 35
Real-time Analytics Use Case #3: Financial Planning 36
Real-time Analytics Use Case #4: Data Transparency 37
Real-time Analytics Use Case #1: Situational Awareness 39
Real-time Analytics Use Case #1: Situational Awareness 40