1 Big Data for Development: What May Determine Success or failure? Emmanuel Letouzé OECD Technology Foresight 2012 Paris, October 22
3 Swimming in Ocean of data Data deluge Algorithms Drowning in Digital smoke signals Pattern recognition Petabytes Digital footprints Correlations Data exhaust Digital signatures Proxy indicators Unstructured data Sentiment analysis Digital crumbs Noise Physical sensors
4 Background and Question(s) 1. Complex, hyperconnected, volatile world. Shocks, risks, vulnerability, resilience, agility, have become mainstream. Policies need to be more agile. 2. Big Data. Big excitement. Big skepticism. New Industrial Revolution? New oil that needs to be refined? Big enough data speak for themselves? Big Data, Big Deal? Worse: a dangerous path? (think conflict settings)? Can it help / change development how? What is the opportunity, what are the challenges? A better question is: What will determine whether Big Data for Development succeeds or fails?
5 Big Data for Development=Success Enables Proximate determinants Requires 1. Right Intent 2. Right Capacities Enables Underlying drivers Recognizing/communicating around: 1. Potential and applications 2. Challenges and Risks 3. Necessary requirements Requires
6 From Big data Big Data for development + purpose (intent) + computing tools, systems (capacity) It s relative It s contextual
7 Big data for development: Global Pulse taxonomy Open Web Data Web content such as news media and social media interactions (e.g. blogs, Twitter), news articles obituaries, e- commerce, job postings; sensor of human intent, sentiments, perceptions, and want. Data Exhaust passively collected transactional data from people s use of digital services like mobile phones, purchases, web searches, etc., these digital services create networked sensors of human behavior; Citizen reporting information actively produced or submitted by citizens through mobile phone-based surveys, hotlines, user-generated maps, etc; Physical Sensors satellite or infrared imagery of changing landscapes, traffic patterns, light emissions, urban development and topographic changes, etc; remote sensing of changes in human activity
8 Potential applications according to Global Pulse Early warning: early detection of anomalies in how populations use digital devices and services can enable faster response in times of crisis ; Digital awareness: Big Data can paint a fine-grained and current representation of reality which can inform the design and targeting of programs and policies; Real-time feedback: monitor a population in real time makes it possible to understand where policies and programs are failing and make adjustments.
9 Intent Capacity Early warning digital smoke signals -syndromic surveillance; anomaly detection, extreme values.. Enables Requires Digital awareness Digital signatures, patternsunderstanding human ecosystems
10 Ex: Conflict prevention (highly simplified) Operational prevention, i.e. conflict early warning and response systems. Detect and respond to anomalies Structural prevention; addressing structural drivers of conflict (e.g., poverty, inequality, elite capture). understand the ecosystem through digital patterns/signatures
11 Examples Early warning, syndromic surveillance and proxies= pick up / model anomalies Twitter-based vs. Official Influenza Rate in the U.S. «You Are What You Tweet: Analyzing Twitter for Public Health. M. J. Paul and M. Dredze,
12 Modeling & predicting food riots
13 Finding proxy indicators (Global Pulse and SAS project) Tweets about the price of rice vs. actual price of rice in Indonesia
14 Tweets per day about meals during Ramadan Start of Ramadan End of Ramadan
15 Digital awareness = understand ecosystem Tea Party on Twitter Occupy Wall Street on Twitter The Tea Party appears as a far more tight-knit group (..) whereas Occupy is made up of looser clusters with a few high-profile accounts. In the lower right-hand corner = matrix of "isolates:" People who are talking about the ideas of Occupy or the Tea Party, but ( ) don't have connections to others on the graph. For Occupy, the number of isolates is greater, which (..) could indicate a larger potential for growth and stronger brand cachet. Source: M. Smith cited in FP What if we could do the same thing for militias, rebel groups not today, but in N years, where N=5, 10, 15..?
18 Summing up potential / applications i. digital signatures that help better understand human ecosystems ii. digital smoke signals for anomaly detection may be especially relevant in risky/poor places high rates of growth of tech penetration with typically poor / little more traditional data But also gives a sense of some of the risks and challenges posed by Big Data for Development
19 Recognizing the difficulties and requirements
20 1. Data challenges Data: How do we access data and protect their producers? Challenge #1: Availability: Depends on technological penetration and use. But mobile phone use is increasing fast and supports/will support Internet access. Challenge #2: Reliability. Attempts at playing the system(s), plus suppressing signals? Especially prevalent in conflict settings.
21 1. Data challenges Challenges #3: Access. Not all data produced in easily accessible, storable (e.g. Twitter). This is compounded in poor countries. How to access data from a mobile phone carrier? Challenge #4: Privacy. Because it s accessible does not make it ethical or safe. True in all settings. In conflict/crisis settings, the privacy challenge may soon become a security risk.
22 2. Analytical challenges Challenge #1: Arrogance/naivety. Believing that the truth is in the (big) data, that the (big enough) data speak for themselves, that all it takes is to beep digging and mining to unveil the truth, that more is always better etc. Apophenia seeing patterns where none exists. If you torture the data long enough, it will eventually start talking. S&P index and butter production in Bangladesh? (Leinweber, 2007)
23 2. Analytical challenges Challenge #2: Making sense of the data.. Not having the right computing capacities. Unstructured data. Sentiment analysis. Translation. Sample bias. No understanding of context.
24 3. Operational/systemic challenges Challenge #1: Legal. Data privacy? Intellectual property? Challenge #2: changing systems, policies, frameworks, decision making processes
25 2. Risks Risk #1: Security and privacy of individual and communities may be jeopardize dark side of Big Data. Conflict zones! Risk #2: Creating new digital divides, between the Small and the Big world, as well as within them, between communities. Who has access, who has the knowledge? Those who can answer the questions are often those who ask the questions. Risk #3: Respond foolishly. i.e. in a way that may exacerbate tensions, poverty..
26 What to avoid Vertical down top down a-theoritical no questions asked a-contextual automated, autarchic, unresponsive to the recommendations of practitioners and social scientists
27 Desirable principles and policies
28 Right intent, sound principles: Big Data is a tool that can help answer QUESTIONS start from questions Contextualization is key especially when lives are in the line. Rely on local insights to make sense of the data. Responsibility: do not harm. Privacy and security: privacy preserving data analytics Build on existing systems and knowledge, e.g. thinking around strategic peacebuilding Think incremental, complementary, iterative and long term
29 Right Capacities Policies and systems: Bring together data and social scientists, private sector and public sector, traditional statisticians and Big Data people Set up Small World-Big World interdisciplinary facilities and agreements to ask questions, test and exchange, draw lessons Create work flows as feedback loops Engage in debates on privacy, security, responsibility, design legal frameworks
30 Research on big data sample bias correction exists You are where you Using Data to Estimate International Migration Rates llustrative relationship between the correction factor for selection bias and Internet penetration rates for different choices of the parameter k.
33 Where may this take us? Lessons from the Technology Sector
35 Agile Global Development?
36 2000 Traditional planning-monitoring policymaking cycle Policymaking/De velopment cycle
37 2020 Responsive policymaking cycle?? Big Data Policymaking/De velopment cycle!
38 2020 Agile policymaking cycle?? All Data? All Data? All Data!!!
39 10 years ahead? Data streams will continue to have grown in volume, velocity and variety. It is very unlikely that Big Data won t be playing a significant role in Development. Question is: what do we want to achieve and how? Big Data for Development should start small, and grow.
BIG DATA IN ACTION FOR DEVELOPMENT This volume is the result of a collaboration of World Bank staff (Andrea Coppola and Oscar Calvo- Gonzalez) and SecondMuse associates (Elizabeth Sabet, Natalia Arjomand,
a report by harvard business review analytic services The New Conversation: Taking Social Media from Talk to Action Sponsored by Conventional marketing wisdom long held that a dissatisfied customer tells
Compliments of 2nd IBM Limited Edition Business Analytics in Retail Learn to: Put knowledge into action to drive higher sales Use advanced analytics for better response Tailor consumer shopping experiences
AAPOR Report on Big Data AAPOR Big Data Task Force February 12, 2015 Prepared for AAPOR Council by the Task Force, with Task Force members including: Lilli Japec, Co-Chair, Statistics Sweden Frauke Kreuter,
July 2013 Contents 1. Introduction 3 2. What is Big Data? 4 3. Big Data Adoption 5 4. Drivers and Barriers 11 5. Opportunities for Digital Entrepreneurship 14 5.1. Supply-side Business opportunities 14
J U L Y 2 0 1 2 OpenText Enterprise Information Management CIOs are under siege Do more with less is no longer an ideal, it s a mandate. With growing volumes and a host of information formats to manage
SOFTWARE ENGINEERING Key Enabler for Innovation NESSI White Paper Networked European Software and Services Initiative July 2014 Executive Summary Economy and industry is experiencing a transformation towards
Convergence of Social, Mobile and Cloud: 7 Steps to Ensure Success June, 2013 Contents Executive Overview...4 Business Innovation & Transformation...5 Roadmap for Social, Mobile and Cloud Solutions...7
An Oracle White Paper March 2013 Big Data Analytics Advanced Analytics in Oracle Database Advanced Analytics in Oracle Database Disclaimer The following is intended to outline our general product direction.
Foresight, Competitive Intelligence and Business Analytics Tools for Making Industrial Programmes More Efficient Jonathan Calof, Gregory Richards, Jack Smith 9 φ β X Creating industrial policy and programmes,
Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com W H I T E P A P E R B i g D a t a : W h a t I t I s a n d W h y Y o u S h o u l d C a r e Sponsored
Surfing the Data Deluge How Retailers Can Turn Big Data into Big Profits Author Ronald van Zanten Contributors Paul Schottmiller Shaun Kirby Richard Medcalf August 2012 Cisco Internet Business Solutions
Big data and open data as sustainability tools A working paper prepared by the Economic Commission for Latin America and the Caribbean Supported by the Project Document Big data and open data as sustainability
GSR discussion paper Big Data - Opportunity or Threat? Work in progress, for discussion purposes Comments are welcome! Please send your comments on this paper at: email@example.com by 20 June 2014. The views
Home Digital Revolution I Combating Evasion & Fraud I Future Compliance Model I Required Capabilities I Our Solution: Trouve I Getting Started I Capgemini & SAS Trouve Our Solution to Combat Tax & welfare
CHAPTER 1.5 Managing the Risks and Rewards of Big Data MATT QUINN CHRIS TAYLOR TIBCO One of the biggest challenges of the term big data is deciding on a standard definition of what those words really mean.
Internet of Things Next-Generation Business and the Internet of Things Opportunities and Challenges Created by a Connected and Real-Time World Table of Contents 3 The Internet of Things Is Redefining Enterprise
journey 2018 the 3 rd digital revolution agility and fragility Your business technologists. Powering progress Welcome The Scientific Community anticipates and crafts the Atos vision of the major trends
Introduction.... 1 Emerging Trends and Technologies... 3 The Changing Landscape... 4 The Impact of New Technologies... 8 Cloud... 9 Mobile... 10 Social Media... 13 Big Data... 16 Technology Challenges...
THE LYTX ADVANTAGE: USING PREDICTIVE ANALYTICS TO DELIVER DRIVER SAFETY BRYON COOK VP, ANALYTICS www.lytx.com 1 CONTENTS Introduction... 3 Predictive Analytics Overview... 3 Drivers in Use of Predictive
Big Data How it can become a differentiator Contents Definition of Big Data 3 Opportunity space 4 Key Players 6 Leading Industries taking advantage 7 of the Big Data trend Big Data in the Financial Industry
Eindhoven, August 2014 Big Data Opportunities for the Retail Sector A Model Proposal by M.G.H. (Marcel) van Eupen BSc Industrial Engineering & Management Science TU/e 2014 Student identity number 0715154
CHAPTER 1.8 The Wisdom of the Cloud: Hyperconnectivity, Big Data, and Real-Time Analytics MIKAEL HAGSTRÖM NEENA GILL SAS In a hyperconnected world, transactions and communication do not happen in a vacuum.
ICC CYBER SECURITY GUIDE FOR BUSINESS ICC CYBER SECURITY GUIDE FOR BUSINESS Acknowledgements The ICC Cyber security guide for business was inspired by the Belgian Cyber security guide, an initiative of
2013 UNDER EMBARGO until 17.10.2013 00:01 GMT World Disasters Report Focus on technology and the future of humanitarian action www.ifrc.org Saving lives, changing minds. The International Federation of
Making the Big Data Connection Business & Network Consulting A Huawei White Paper by Peter Massam February, 2014 CONTENTS 1 Executive Summary...1 2 The Big Data Challenge...4 2.1 Preparing for the Challenge...5