Big Data Buzzword or Real Opportunity?

Transcription

1 CENTRAL EASTERN EUROPEAN SMART CARD ASSOCIATION Big Data Buzzword or Real Opportunity? Zoran Bohaček Zadar, September 17 th, 2013

2 Agenda Big Data's 4 V's Few bits of history How big is BIG? What to do with it?

3 History 50's credit scoring (correlation) 70's behaviour scoring 80's adaptive control systems (ACS) Nowadays ACSs manage a vast majority of credit cards worldwide Letters (collection) Limits Authorisation Cross-selling

4 Changing world Barely 50 years since first use of computers to analyse "big" data to evaluate risk of credit loss Analytical methods improved and optimised decision making in every field of human activity Even the name changed Operations research Data mining Analytics Data science "Science of better"

5 Internet of Things (2008) For fun see: The IoT Comic Book (Autumn, 2011)

6 Internet of everything Internet of Things Almost fully dependent on data introduced by people More information about ideas than things If working with consumers Products and services, regulation and protection Internet of People? Instead of huge quantities of sensor data about things, we take huge quantity of sensor and other (social networks?) data about individuals

7 Information growth Definition of "information" ANYTHING EVER produced, stored or communicated 300 B.C. Ptolemaeus II gathered ALL of world's knowledge in Alexandrian library copy of every record ever produced In 50 years ( ) about 8 mill. books were printed, more than all scribes and copiers produced from founding of Constantinopolis Nowadays doubling every THREE years!

8 Scary quantities of information Example from astronomy Sloan Digital Sky Survey (2000) in first few weeks more data than in the entire history of astronomy Untill 2010 collected information: 140 TB New telescope in Chile from 2016 will collect that much in FIVE days Digitalisation 2000 about ¼ information was digital 2007 about 7% analog out of total 300 EB 2013 less than 2% analog total 1,2 ZB

9 Some real world Big Data Text analysis Fast growing complex technology where it is not evident to discover predictive signals in unstructured data (text) Instead of rules, brute force text analysis (measured in Terawords), learning by example and calculation of probabilities Card industry Only in ,812,365,400,000 card transactions were recorded Estimated value: $440,404,782,000,000 It looks like Very Big Data

10 Card world example Some license data to 3 rd parties Some process it and then sell information to others MC Advisors aggregates and analyzes 65 bill. transactions from 1,5 bill. cardholders in 210 countries If you fill up gas around 4 p.m. quite likely to spend $35-50 in the next hour at a grocery store or restaurant

11 Open data Largest collectors of data on mass scale are governments Private sector is likely to be more effective and innovative in using it Open Data initiative US ,000 datasets from 172 agencies Australia, Brazil, Chile, Kenya EU initiative

12 Some examples Driver's score Health data Sensors monitoring activities and sleep Food diaries - calculating calories, proteins, carbs etc.... or smarter ways of tracking food habits Devices that follow heartbeat, blood pressure even realtime ECG (through clothes) Credit behaviour data predict who would stay in phyiscal activity programme Insurance companies use credit reports for many years ReCAPTCHA

13 Other examples Amazon Walmart Financial services Taking medicines Income verification Medicals before insurance policy Datification Posture when driving Twitter stock market trading Traffic around big stores estimate results Floor sensors number of people, control, help

14 Famous examples Flu spreading Institutional analysis in two weeks or Google 50 mill. most common search term 450 mill. mathematical models to test them A combination of 45 terms highly correlated with official numbers but almost real time! "Premmies" in Canada Dangerous infections can be cured if detected early enough More than 1,000 readings per second Findings contrary to expectations and medical logic

15 Big Data revolution Huge turnaround we don't need to define or design an analytical method to get answers to a question and explain reasons why Big Data can give answers to questions we haven't even asked Correlation replaces causality Hypotheses are not needed Is this the "End of Theory"?

16 Conclusion Buzzword? Opportunity? It is really up to you whether you will seize it in your line of business...