BIG DATA, SMALL DATA, OPEN DATA opportunities and challenges for governance BETHSIMONENOVECK Belk School of Business, Charlotte, North Carolina MAY132014
http://online.northcarolina.edu/unconline/courses.php 2
Governing Smarter with Data Why Data for Governing? Big Data Small Data Open Data Challenges Missed Opportunities 3
Data for Accountability 4
5
USASpending.gov 6
http://www.theguardian.com/commentisfree/2013/jun/28/transparency-restore-trust-nhs 7
Data for Efficiency 8
http://www.mastodonc.com/ 9
Data for Problem-Spotting 10
The Mayor s Geek Squad, New York Times, http://www.nytimes.com/2013/03/24/nyregion/mayor-bloombergs-geek-squad.html 11
Urban Data Sources Organic data flows Administrative records (census, permits, ) Transactions (sales, communications, ) Operational (traffic, transit, utilities, health system, ) New and social media (Twitter feeds, blog posts, Facebook, ) Sensors Personal (location, activity, physiological) Fixed in situ sensors Crowd sourcing (mobile phones, ) Choke points (people, vehicles) Opportunities for novel sensor technologies Visible, infrared and spectral imagery RADAR, LIDAR Gravity and magnetic Seismic, acoustic Ionizing radiation, biological, chemical 12
1/3 of large police forces equip patrol cars with automatic license plate readers that can check 1,000 plates per minute ~ 400,000 ATMs record video of all transactions 4,214 red-light cameras 761 speed trap cameras 30 million commercial surveillance cameras 300 million mobile phones 494,151 cell towers NYU CUSP, The Promise of Urban Informatics. 13
What can cities do with the data? Optimize operations traffic flow, utility loads, services delivery, Monitor infrastructure conditions bridges, potholes, leaks, Infrastructure planning zoning, public transit, utilities Improve regulatory compliance ( nudges, efficient enforcement) Public health Nutrition, epidemiology, environmental impacts Abnormal conditions Hazard detection, emergency management Data-driven formulation of data-driven policies and investments Low-Income Home Energy Assistance Program, road pricing and congestion charging, Better inform the citizenry Enhance economic performance and competitiveness NYU CUSP, The Promise of Urban Informatics. 14
15
Sentiment Analysis http://grade.dc.gov/ 16
http://www.durkheimproject.org/ http://www.marist.edu/ 17
Data for Problem Solving 18
http://strata.oreilly.com/2012/06/predictive-data-analytics-big-data-nyc.html 19
Data for Transformation 20
http://www.healthdata.gov/ 21
Data for Collaboration 22
http://charlotte-nc.meetup.com/ 23
BIGDATA, SMALLDATA, OPENDATA Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6, 2013 24
Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6, 2013 25
Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6, 2013 26
27
Big data a growing torrent $600 to buy a disk drive that can store all of the world s music 5 billionmobile phones in use in 2010 30 billion pieces of content shared on Facebook every month 40%projected growth in global data generated Big data: The next frontier for innovation, competition, and productivity - June 2011 - http://www.mckinsey.com/insights/mgi 28
29
Small Data 30
Small data is... the data we generate implicitly and explicitly across a myriad of systems, activities and encounters: from mobile, cable TV, purchases, browsing, apps, entertainment, games, email, posts, texts and tweets... Deborah Estrin 31
Beyond mobile... Leveraging digital traces from diverse consumer services Your rows of their matrices Deborah Estrin 32
Mining small data to fuel health(ier) behaviors Lets build small-data fueled apps, engagements, and games that measure, motivate, and mitigate behavior change Martha s food choices, daily stretches, yoga, to help her reduce dependence on pain meds Sean s media consumption to help him rebalance his sleep habits Deborah Estrin 33
Example Consumer transaction patterns as small data The goal is to transform purchasing patterns Since we have a progression of spending patterns over time into a consumption model (a categorical distribution of restaurant, fast food, drug store, etc. purchases) on which descriptive statistics can be computed. we can examine how the resulting distributions change and plot the statistics for each time frame, resulting in a time series which can then be correlated against other time series. Deborah Estrin 34
Example personalized advice and feedback on weekly grocery shopping to promote healthier home food environment Deborah Estrin 35
Open Data 36
37
38
Transparency Government should be transparent. Transparency promotes accountability and provides information for citizens about what their Government is doing. Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use. Executive departments and agencies should harness new technologies to put information about their operations and decisions online and readily available to the public. Executive departments and agencies should also solicit public feedback to identify information of greatest use to the public. Source: Memorandum on Transparency and Open Government, January 21, 2009 39
http://sunlightfoundation.com/clearspending/ 40
Joel Gurin, GovLab, Senior Advisor and Author, Open Data Now 41
Why Open Data Matters 42
Health Education Energy Use Financial Services Transportation 43
BIGDATA, SMALLDATA, OPENDATA Weather Prediction Builds a $1 Billion Business Climate Corporation offices in San Francisco 44
What s the Value of Open Data? McKinsey study: $3 trillion annually worldwide 30 to 140 billion euros for Europe s public sector data 2 to 9 billion British pounds $30 billion for U.S. weather data Tens of billions for U.S. GPS data Hundreds of billions for U.S. health data 45
46
47
48
A Kayak for Everything 49
Help for K-12 Households Bill Jackson, CEO 50
51
Ogi Kavazovic VP Marketing & Strategy 52
Consumer Data Drives Corporate Concern Dara O Rourke, Founder/CEO 53
54
55
Challenges 56
Challenge Privacy ta REPORT TO THE PRESIDENT BIG DATA AND PRIVACY: A TECHNOLOGICAL PERSPECTIVE Executive Office of the President President s Council of Advisors on Science and Technology May 2014 57
Challenge Knowing What Works 58
Challenge Accidental Release of PII http://www.theguardian.com/commentisfree/2014/apr/18/open-data-whitehall-government-editorial 59
Challenge 60
Challenge Which License? 61
Challenge Designing with the Citizen At the Center 62
Challenge 63
Missed Opportunities 64
Missed Opportunities A/B Testing Presenting users with distinct versions of the same content and measuring differences in use is being utilized in the private sector to optimize functionalities. Big names in business from tech-focused ones like Amazon, Google and Bing (which has over 30 billion variants) to Marriott are using this parallel experimentation to improve sales. 65
Missed Opportunities We lack the opportunity in policymaking to experiment and discover workable arrangements through an open-ended process of trial and error. Jim Manzi, author of Uncontrolled (2012) 66
Missed Opportunities Treating Citizens as the Subject not the Object 67
Missed Opportunities 68
Missed Opportunities Recognizing Data 69
Data without collaboration is information without knowledge 70