!!! The Fallacy of Big Data! Brian Fine and Con Menictas!

Transcription

1 !!! The Fallacy of Big Data! Brian Fine and Con Menictas! 1!

2 What is Big Data?! Big data is a vague term for a massive phenomenon that has rapidly become an obsession with entrepreneurs, scientists, governments and the media Tim Harford, FT Magazine, 2014! Big data is an all-encompassing term for any collection of data sets so large and complex that it becomes difficult to process using traditional data processing applications. Wikipedia, 2014!

3 Why is Big Data a fallacy?!! There is no magic in Big Data on its own, it needs context!! We still require business issues to guide our investigation!! Strategic management/fusion of multiple data sources needed!! Primary research is required to focus into the why?!

4 What is Big Data?!! The power of Big Data is what we do with it!! Big Data usually associated with parallelized deep learning algorithms because it is different data from different sources.!! Big Data can be termed BIG just because of its size, e.g., 10M+ rows.!

5 What is Big Data?! Data! Sizes! Work With! Data In! Computer Storage! Example! Units of Sale! Small! <10GB! Med! 10GB to 1TB! Big! >1TB! Excel! Memory! 1000 s! 1 Database! 1 Disk Drive! 1,000,000 s! >1 Database! >1 Disk Drive! 1000,000,000 s! 5!

6 50! Growth of Big Data Worldwide! 37.5! 44! 25! 12.5! 0! 4! 2014! 2020! Zetabytes (Trill GBs)! 6!

7 60! US Big Data Market Revenue Forecast in Billion USD! 45! 45! 50! 30! 29! 38! 15! 19! 12! 7! 0! 2011! 2012! 2013! 2014! 2015! 2016! 2017! 7!

8 Twitter Interactions between Monster Energy Drinks and Call of Duty [Piptook]! 8!

9 Share Price Manipulation [Piptook]! 9!

10 Big Data in Australia (Accenture 2014)! 1. Just over 50 per cent of Australian organisations believe big data can be of significant value to their operations - compared with 82 percent globally! 2. Only 58 per cent of Australian organisations believe big data has the power to deliver value compared to 82 per cent of global organisations! 3. Australian organisations are taking longer to see big data as a driver of transformation, ranking last of all 19 countries surveyed! 4. Only 36 per cent of Australian organisations (versus 57 per cent in Asia Pacific) stated their leadership extensively understood and supported big data strategies! 10!

11 Some Problems of Using Big Data! o If there are data errors, they are amplified, because of the volume of data! o Because big data comes from various sources, it is harder to understand! o We may not be able to merge and append to it! o Understanding what to do with the data is in most instances challenging! 11!

12 Big data predictions may not do so well! After four years of intensive analysis, Silver concludes that big data predictions are not actually going very well. Whether the field is economics or finance, medical science or political science, most predictions are either entirely wrong or else sufficiently wrong as to be of minimal value.! (Nate Silver 2014)! 12!

13 Instances of Big Data Prediction Failures!! Terror attack on the US in 9/11!! GFT overestimating the prevalence of flu in the and seasons by more than 50% (Google 2014)!! Too many big data projects are structured like boil-theocean experiments (Infochimps CEO, Jim Kaskade, 2014)!! Economics is easily the single most important failure of the application of Big Data (Hillary Mason 2013)! 13!

14 Comparing Big Data to MR and Transactions! Data Source! Market Research! Transaction! Big Data! Understanding! H! M! L! Quality! M! H! L! Outlier Control! H! H! M! Displaying! H! M! L! Overall Quality! H! M! L! 14!

15 Why is Big Data potentially problematic! 1. Big Data looks at the what! 2. Big Data does not know the why! 3. The why can only be attained through market research! 15!

16 What market research brings to Big Data:! an understanding of the population! 1. Psychographics! 2. Needs! 3. Values! 4. Lifestyles! 16!

17 Example of a Geodemographic Segmentation (GeoTribes, 2014)! 17!

18 Case Study: Improving Big Data Propensity Modelling at Qantas with Choice Models! 18!

19 Case Study: Improving Segment Profiles with Needs to better understand segments! 19!

20 Conclusions!! Market research is needed to add meaning and structure to Big Data! Big Data Database! Draw an Affordable Sample! Research the Sample for Needs and Value Drivers! Predict the Sample Findings back onto the Database!! Market Research also helps us understand Big Data!! 20!

21 Big Data can t stand on its own!! Big Data on its own looks at the what.!! Big Data needs to be brought down to small data problems! For developing and understanding products and! services, Big Data needs market research, otherwise it cannot provide insights into why people behave the way they do! 21!

22 ! Big Data is just the starting point!! Thank you! 22!