BIG DATA, SMALL DATA, OPEN DATA

Transcription

1 BIG DATA, SMALL DATA, OPEN DATA opportunities and challenges for governance BETHSIMONENOVECK Belk School of Business, Charlotte, North Carolina MAY132014

2 2

3 Governing Smarter with Data Why Data for Governing? Big Data Small Data Open Data Challenges Missed Opportunities 3

4 Data for Accountability 4

5 5

6 USASpending.gov 6

7 7

8 Data for Efficiency 8

9 9

10 Data for Problem-Spotting 10

11 The Mayor s Geek Squad, New York Times, 11

12 Urban Data Sources Organic data flows Administrative records (census, permits, ) Transactions (sales, communications, ) Operational (traffic, transit, utilities, health system, ) New and social media (Twitter feeds, blog posts, Facebook, ) Sensors Personal (location, activity, physiological) Fixed in situ sensors Crowd sourcing (mobile phones, ) Choke points (people, vehicles) Opportunities for novel sensor technologies Visible, infrared and spectral imagery RADAR, LIDAR Gravity and magnetic Seismic, acoustic Ionizing radiation, biological, chemical 12

13 1/3 of large police forces equip patrol cars with automatic license plate readers that can check 1,000 plates per minute ~ 400,000 ATMs record video of all transactions 4,214 red-light cameras 761 speed trap cameras 30 million commercial surveillance cameras 300 million mobile phones 494,151 cell towers NYU CUSP, The Promise of Urban Informatics. 13

14 What can cities do with the data? Optimize operations traffic flow, utility loads, services delivery, Monitor infrastructure conditions bridges, potholes, leaks, Infrastructure planning zoning, public transit, utilities Improve regulatory compliance ( nudges, efficient enforcement) Public health Nutrition, epidemiology, environmental impacts Abnormal conditions Hazard detection, emergency management Data-driven formulation of data-driven policies and investments Low-Income Home Energy Assistance Program, road pricing and congestion charging, Better inform the citizenry Enhance economic performance and competitiveness NYU CUSP, The Promise of Urban Informatics. 14

15 15

16 Sentiment Analysis 16

17 17

18 Data for Problem Solving 18

19 19

20 Data for Transformation 20

21 21

22 Data for Collaboration 22

23 23

24 BIGDATA, SMALLDATA, OPENDATA Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6,

25 Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6,

26 Richard Price, PulsePoint Citizen Responder CPR/AED Mobile App: When Minutes Matter, September 6,

27 27

28 Big data a growing torrent $600 to buy a disk drive that can store all of the world s music 5 billionmobile phones in use in billion pieces of content shared on Facebook every month 40%projected growth in global data generated Big data: The next frontier for innovation, competition, and productivity - June

29 29

30 Small Data 30

31 Small data is... the data we generate implicitly and explicitly across a myriad of systems, activities and encounters: from mobile, cable TV, purchases, browsing, apps, entertainment, games, , posts, texts and tweets... Deborah Estrin 31

32 Beyond mobile... Leveraging digital traces from diverse consumer services Your rows of their matrices Deborah Estrin 32

33 Mining small data to fuel health(ier) behaviors Lets build small-data fueled apps, engagements, and games that measure, motivate, and mitigate behavior change Martha s food choices, daily stretches, yoga, to help her reduce dependence on pain meds Sean s media consumption to help him rebalance his sleep habits Deborah Estrin 33

34 Example Consumer transaction patterns as small data The goal is to transform purchasing patterns Since we have a progression of spending patterns over time into a consumption model (a categorical distribution of restaurant, fast food, drug store, etc. purchases) on which descriptive statistics can be computed. we can examine how the resulting distributions change and plot the statistics for each time frame, resulting in a time series which can then be correlated against other time series. Deborah Estrin 34

35 Example personalized advice and feedback on weekly grocery shopping to promote healthier home food environment Deborah Estrin 35

36 Open Data 36

37 37

38 38

39 Transparency Government should be transparent. Transparency promotes accountability and provides information for citizens about what their Government is doing. Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use. Executive departments and agencies should harness new technologies to put information about their operations and decisions online and readily available to the public. Executive departments and agencies should also solicit public feedback to identify information of greatest use to the public. Source: Memorandum on Transparency and Open Government, January 21,

40 40

41 Joel Gurin, GovLab, Senior Advisor and Author, Open Data Now 41

42 Why Open Data Matters 42

43 Health Education Energy Use Financial Services Transportation 43

44 BIGDATA, SMALLDATA, OPENDATA Weather Prediction Builds a $1 Billion Business Climate Corporation offices in San Francisco 44

45 What s the Value of Open Data? McKinsey study: $3 trillion annually worldwide 30 to 140 billion euros for Europe s public sector data 2 to 9 billion British pounds $30 billion for U.S. weather data Tens of billions for U.S. GPS data Hundreds of billions for U.S. health data 45

46 46

47 47

48 48

49 A Kayak for Everything 49

50 Help for K-12 Households Bill Jackson, CEO 50

51 51

52 Ogi Kavazovic VP Marketing & Strategy 52

53 Consumer Data Drives Corporate Concern Dara O Rourke, Founder/CEO 53

54 54

55 55

56 Challenges 56

57 Challenge Privacy ta REPORT TO THE PRESIDENT BIG DATA AND PRIVACY: A TECHNOLOGICAL PERSPECTIVE Executive Office of the President President s Council of Advisors on Science and Technology May

58 Challenge Knowing What Works 58

59 Challenge Accidental Release of PII 59

60 Challenge 60

61 Challenge Which License? 61

62 Challenge Designing with the Citizen At the Center 62

63 Challenge 63

64 Missed Opportunities 64

65 Missed Opportunities A/B Testing Presenting users with distinct versions of the same content and measuring differences in use is being utilized in the private sector to optimize functionalities. Big names in business from tech-focused ones like Amazon, Google and Bing (which has over 30 billion variants) to Marriott are using this parallel experimentation to improve sales. 65

66 Missed Opportunities We lack the opportunity in policymaking to experiment and discover workable arrangements through an open-ended process of trial and error. Jim Manzi, author of Uncontrolled (2012) 66

67 Missed Opportunities Treating Citizens as the Subject not the Object 67

68 Missed Opportunities 68

69 Missed Opportunities Recognizing Data 69

70 Data without collaboration is information without knowledge 70