Reality Mining of Mobile Communications: Toward a New Deal on Data

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Reality Mining of Mobile Communications: Toward a New Deal on Data"

Transcription

1

2 CHAPTER 1.6 Reality Mining of Mobile Communications: Toward a New Deal on Data ALEX PENTLAND, Massachusetts Institute of Technology (MIT) Within just a few years people data will be 90% of the world s collective data. Jeff Nick, CTO of EMC, personal communication We have enough water, enough food, enough money; we have enough of everything except the ability to agree and move forward. Abdul Kalam, former President of India, personal communication 1.6: Reality Mining of Mobile Communications Around the world, many of us live our lives in digital networks.we wake up in the morning, check our , make a quick phone call, commute to work, and buy lunch. Many of these transactions leave digital breadcrumbs tiny records of our daily experiences. 1 Reality mining, which pulls together these crumbs using statistical analysis and machine learning methods, offers increasingly extensive information about our lives, both individually and collectively.this technology s potential to transform our understanding of ourselves, our organizations, and our society is the reason that MIT s Technology Review recently identified reality mining as one of 10 Emerging Technologies That Will Change the World An emerging and truly global nervous system The ability to understand the patterns of human life by analyzing the digital traces that we leave behind will transform poor nations even more than rich nations. Ten years ago, half of humanity had never made a phone call and only 20 percent had regular access to communications.today almost 4 billion people have a mobile telephone, and because remanufactured mobile phones cost US$10 in the developing world and incoming messages are free, almost every stratum of society is now connected. 3 For the first time, the majority of humanity is linked and has a voice. The most important changes may come, however, from the fact that these same mobile phones can be used for reality mining.their reality mining functionality is mostly latent at this point, but already these devices are being used to measure population flows into cities and slums, to map the movement of populations during emergencies, to identify neighborhoods where social services are inadequate (see Figure 4), and to manage automobile traffic congestion (see Figure 1). The reality mining functionality of mobile phone networks is what governments in countries such as India are using to track terrorists, and they claim that the vast This article builds on papers co-authored with David Lazer at the J. F. Kennedy School of Government at Harvard University and with my current and former students, as well as on continual interactions and conversations with my colleagues at Sense Networks and at the MIT Media Laboratory.

3 1.6: Reality Mining of Mobile Communications Figure 1: Traffic congestion predicted using mobile phone GPS data Latitude Longitude 0 Source: Dong, majority of captured terrorists have been identified through mobile phone transactions.the ability of mobile phone networks to identify unusual patterns of movement and communication are also how public health officials and disaster relief teams are scanning for outbreaks of diseases such as severe acute respiratory syndrome (SARS) and emergencies such as tidal waves. It seems that the human race suddenly has the beginnings of a working nervous system. Like some world-spanning living organism, automobile traffic systems, security sensors, and especially mobile telephone networks are all becoming intelligent, reactive systems with sensors serving as their eyes and ears. Moreover, the evolution of this nervous system will continue at a quickening speed because of the exponential progress in computing and communication technologies as well as basic economics. Networks will become faster, devices will have more sensors, and techniques for modeling human behavior will become more accurate and detailed. From individuals to societies To date, the vast majority of research on the human condition has relied on single-shot, self-report data on relationships: a yearly census, public polls, focus groups, and the like. Reality mining offers a remarkable, secondby-second picture of group interactions over extended periods of time, providing dynamic structural and content information. Perhaps the simplest example of reality mining is the analysis of automobile traffic congestion by using the global positioning system (GPS) data collected from the mobile telephones carried by the automobile drivers. These data provide minute-by-minute updates on traffic flow, allowing for more accurate predictions of driving time. Congestion patterns can be predicted days in advance, and traffic jams detected hours before they become serious, as illustrated by the congestion map of city streets shown in Figure 1. Similar to using reality mining to understand traffic within a city, we can also use reality mining of mobile phone GPS data, call logs, and records to better understand the traffic within an organization. Analysis of these digital traces allows a detailed picture of faceto-face, voice, and digital communication patterns. These patterns, in turn, allow us a new level of insight into the problems of industry and government, including building customer relationships, resource management, transportation, and public health. Figures 2 and 3 show two recent but very different applications of reality mining of people data. Figure 2 shows the pattern of face-to-face and communication within a German bank.the thickness of the top arcs (in black) shows the amount of face-to-face communication, while the thickness of the bottom arcs (in blue) shows the amount of communication. In our studies of many different types of companies, we have found that the tradeoff between face-to-face and communication is a critical

4 Figure 2: Patterns of communication within departments of a bank Manager Development Sales Support Customer Service 1.6: Reality Mining of Mobile Communications Source: Pentland, predictor of both productivity and job satisfaction. In fact, variations in these patterns can account for more than 30 percent of the differences in productivity between different parts of the same company! Patterns of change within these communication networks are also a critical predictor of creative productivity, accounting for up to 40 percent of differences in creative output. 4 Figure 3 shows a plot of regional communication diversity and the corresponding index of deprivation, a socioeconomic status measurement that is a combination of metrics such as average income levels, access to health care, and education.these data, drawn from town councils across the United Kingdom, show that it is possible to use patterns of communication to identify information ghettos that have serious social problems. This capability can allow government services to be far more responsive to citizen needs than is possible by using census or survey data. In short, reality mining is beginning to provide the capacity to collect and analyze data about people with a breadth and depth that was previously inconceivable. Current work using reality mining techniques is underway in a variety of applications. For example, reality mining studies of voice, communications, and mobility patterns have already demonstrated the ability to screen for depression, infer quality-of-life metrics, and develop financial indexes for individual neighborhoods. 5 Beyond demographics Advertising, commercial development, and government services all currently rely on demographic data to guide them. Such data can quickly become out of date and, of course, good demographic data simply do not exist in many parts of the world. More importantly, however, we have found that demographics are a relatively poor predictor of behavior, and behavior is what we really need to understand in order to reach the right people. The fact that mobile phones have GPS means that we can leap beyond demographics directly to measuring behavior.where do people eat? Work? Hang out? How does word of mouth spread? Analysis of travel patterns using mobile phone GPS data, for instance, allows discovery of the independent subgroups within a city. Figure 4a shows movement patterns with popular hang outs coded by the different subgroups that populate these destinations, where the subgroups are defined by both their demographics and, more importantly, by their behaviors. Figure 4b shows that the mixing between these different behavior groups is surprisingly small. Knowledge of the hang outs of different subgroups and the mixing among groups can provide great improvements in advertising, commercial development, public education, and policy interventions. 77

5 Usefulness for society Reality mining of behavior data is just beginning. For instance, the correlation of behavior data with medication data from millions of people could make drug therapies more effective and help medical professionals detect drug interactions more quickly. If behavior data were correlated with medical conditions, the data could illuminate the etiology and preconditions of disease far more powerfully than is possible today and, further, serve as an early warning system for epidemic diseases such as SARS. Comparing the medical data with genomic and protonomic data from different population samples could provide a powerful method for understanding complex gene and environment interactions. For society, the hope is that we can use this new in-depth understanding of individual behavior to increase the efficiency and responsiveness of industries and governments. For individuals, the attraction is the possibility of a world where everything is arranged for your convenience your health checkup is magically scheduled just as you begin to get sick, the bus comes just as you get to the bus stop, and there is never a line of waiting people at city hall. Data ownership and privacy Perhaps the greatest challenge posed by this new ability to sense the pulse of humanity is creating a new deal around questions of privacy and data ownership. Many of the network data that are available today are freely offered because the entities that control the data have difficulty extracting value from them. As we develop new analytical methods, however, this will change. Moreover, not all people who want access to the data do so for altruistic motives, and it is important to consider how to keep the individuals who generate this information safe. Advances in analysis of network data must be approached in tandem with understanding how to create value for the producers and owners of the data while at the same time protecting the public good. Clearly, our notions of privacy and ownership of data need to evolve in order to adapt to these new challenges. This raises another important question: how do we design institutions to manage the new types of privacy issues that will emerge with these new reality mining capabilities? Digital traces of people are ubiquitously preserved within our private and public organizations location patterns, financial transactions, public transportation, phone and Internet communications, and so on. Certainly new types of regulatory institutions are required to deal with this information, but what form should they take? Companies will have a key role in this new deal for privacy and ownership. One suggestion is that there is an incentive system, one that gives added value to the users. Market mechanisms appear to be a particularly interesting avenue of exploration, since they may allow people to give up their data for monetary or service rewards. Ideally, this would be put into place in order to gain approval from the majority of the population to use data collected from their digital interactions. Other important considerations revolve around data anonymity.the use of anonymous data should be enforced, and analysis at the group level should be preferred over that at the individual level. Robust models of collaboration and data sharing need to be developed; guarding both the privacy of consumers as well as corporations legitimate competitive interests are vital here. What must be avoided is either the retreat into secrecy, so that these data become the exclusive domain of private companies and remain inaccessible to the Common Good, or the development of a big brother model, with government using the data but denying the public the ability to investigate or critique its conclusions. Neither scenario will serve the long-term public interest in having a transparent and efficient government. The new deal on data The first step toward open information markets is to give people ownership of their data.the simplest approach to defining what it means to own your own data is to go back to Old English Common Law for the three basic tenets of ownership, which are the rights of possession, use, and disposal: 1. You have a right to possess your data. Companies should adopt the role of a Swiss bank account for your data.you open an account (anonymously, if possible), and you can remove your data whenever you d like. 2. You, the data owner, must have full control over the use of your data. If you re not happy with the way a company uses your data, you can remove it. All of it. Everything must be opt-in, and not only clearly explained in plain language, but with regular reminders that you have the option to opt out. 3. You have a right to dispose or distribute your data. If you want to destroy it or remove it and redeploy it elsewhere, it is your call. Ownership seems to be the minimal guideline for the new deal on data. There needs to be one more principle, however which is to adopt policies that encourage the combination of massive amounts of anonymous data to promote the Common Good. Aggregate and anonymous location data can dramatically improve society. Patterns of how people move around can be used for early identification of infectious disease outbreaks, protection of the environment, and public 1.6: Reality Mining of Mobile Communications 79

6 1.6: Reality Mining of Mobile Communications safety. It can also help us measure the effectiveness of various government programs, and improve the transparency and accountability of government and nonprofit organizations. Conclusions Revolutionary new measurement tools provided by mobile telephones and other digital infrastructures are providing us with a God s eye view of ourselves. For the first time, we can precisely map the behavior of large numbers of people as they go about their daily lives. These distributed sensor networks have given us a new, powerful way to understand and manage human groups, corporations, and entire societies. As these new abilities become refined by the use of more sophisticated statistical models and sensor capabilities, we could well see the creation of a quantitative, predictive science of human organizations and human society. At the same time, these new tools have the potential to make George Orwell s vision of an all-controlling state into a reality. What we do with this new power may turn out to be either our salvation or our destruction. 80 Notes 1 See index.shtml for amazing videos of the time evolution of automobiles, airplanes, telephone calls, and shipping activity patterns. 2 Technology Review Eagle Pentland Pentland References Dong, W Influence Modeling of Complex Stochastic Processes. July. Master s Thesis in Media Arts and Sciences. Available at Eagle, N (in preparation). Mobile Phones as Social Sensors. The Handbook of Emergent Technologies in Social Research. New York: Oxford University Press. Pentland, A Honest Signals: How They Shape our World. Cambridge, MA: MIT Press. Sense Networks Technology tab at networks.com. Technology Review Emerging Technologies That Will Change the World. Available at review.com/article/13060/.

Supporting Knowledge Creation and Sharing in Social Networks

Supporting Knowledge Creation and Sharing in Social Networks Organizational Dynamics, Vol. 30, No. 2, pp. 100 120, 2001 ISSN 0090-2616/01/$ see frontmatter 2001 Elsevier Science, Inc. PII S0090-2616(01)00046-8 www.organizational-dynamics.com Knowing What We Know:

More information

Deciding with data How data-driven innovation is fuelling Australia s economic growth

Deciding with data How data-driven innovation is fuelling Australia s economic growth September 2014 Deciding with data How data-driven innovation is fuelling Australia s economic growth www.digitalinnovation.pwc.com.au Contents 01 Executive summary 19 Value to be unlocked from data 02

More information

Making Smart IT Choices

Making Smart IT Choices Making Smart IT Choices Understanding Value and Risk in Government IT Investments Sharon S. Dawes Theresa A. Pardo Stephanie Simon Anthony M. Cresswell Mark F. LaVigne David F. Andersen Peter A. Bloniarz

More information

THE INFORMATION GENERATION TRANSFORMING THE FUTURE, TODAY Outlook Report

THE INFORMATION GENERATION TRANSFORMING THE FUTURE, TODAY Outlook Report THE INFORMATION GENERATION TRANSFORMING THE FUTURE, TODAY Outlook Report SPONSORED BY 2015 Institute for the Future for EMC. All rights reserved. SR-1782 1 THE INFORMATION GENERATION TRANSFORMING THE FUTURE,

More information

The Way We Pay. Transforming the Canadian Payments System

The Way We Pay. Transforming the Canadian Payments System The Way We Pay Transforming the Canadian Payments System Table of contents Message from the Chair... 2 Introduction... 3 Chapter 1: Engaging dialogue... 4 Chapter 2: Falling behind... 8 Chapter 3: Fundamental

More information

CREATING AN ENGAGED WORKFORCE

CREATING AN ENGAGED WORKFORCE Research report January 2010 CREATING AN ENGAGED WORKFORCE CREATING AN ENGAGED WORKFORCE FINDINGS FROM THE KINGSTON EMPLOYEE ENGAGEMENT CONSORTIUM PROJECT This report has been written by: Kerstin Alfes,

More information

building a performance measurement system

building a performance measurement system building a performance measurement system USING DATA TO ACCELERATE SOCIAL IMPACT by Andrew Wolk, Anand Dholakia, and Kelley Kreitz A Root Cause How-to Guide ABOUT THE AUTHORS Andrew Wolk Widely recognized

More information

fs viewpoint www.pwc.com/fsi

fs viewpoint www.pwc.com/fsi fs viewpoint www.pwc.com/fsi 02 15 19 21 27 31 Point of view A deeper dive Competitive intelligence A framework for response How PwC can help Appendix Where have you been all my life? How the financial

More information

TRANSFORM YOUR CITY THROUGH INNOVATION THE INNOVATION DELIVERY MODEL FOR MAKING IT HAPPEN

TRANSFORM YOUR CITY THROUGH INNOVATION THE INNOVATION DELIVERY MODEL FOR MAKING IT HAPPEN TRANSFORM YOUR CITY THROUGH INNOVATION THE INNOVATION DELIVERY MODEL FOR MAKING IT HAPPEN JANUARY 2014 CONTENTS INTRODUCTION... 1 THE IMPERATIVE FOR INNOVATION... 2 WHAT IS THE INNOVATION DELIVERY MODEL?....

More information

Data driven What students need to succeed in a rapidly changing business world. February 2015

Data driven What students need to succeed in a rapidly changing business world. February 2015 Data driven What students need to succeed in a rapidly changing business world February 2015 Contents Abstract 1 The changing business world 2 A world in flux 2 CEOs are bracing for change 3 How data

More information

THE JOURNEY TOWARD CASH LITE

THE JOURNEY TOWARD CASH LITE THE JOURNEY TOWARD CASH LITE Addressing Poverty, Saving Money and Increasing Transparency by Accelerating the Shift to Electronic Payments Commissioned by BFA acknowledges the helpful comments received

More information

E x t r a c t i n g Value from Chaos

E x t r a c t i n g Value from Chaos I D C I V I E W E x t r a c t i n g Value from Chaos June 2011 By John Gantz and David Reinsel Sponsored by EMC Corporation Content for this paper is excerpted directly from the IDC iview "Extracting Value

More information

Doing more with Digital

Doing more with Digital STRAITÉIS DHIGITEACH NÁISIÚNTA NATIONAL DIGITAL STRATEGY Doing more with Digital Digital Engagement July 2013 Department of Communications, Energy and Natural Resources Roinn Cumarsáide, Fuinnimh agus

More information

FEDERAL HEALTH IT STRATEGIC PLAN 2015 2020

FEDERAL HEALTH IT STRATEGIC PLAN 2015 2020 FEDERAL HEALTH IT STRATEGIC PLAN 2015 2020 Prepared by: The Office of the National Coordinator for Health Information Technology (ONC) Office of the Secretary, United States Department of Health and Human

More information

Are you innovation ready? Plotting your journey on the Innovation Readiness Model

Are you innovation ready? Plotting your journey on the Innovation Readiness Model Are you innovation ready? Plotting your journey on the Innovation Readiness Model Message from Andy Green CEO, Logica I believe that the future success of our global economy relies on building a dynamic

More information

Making a difference in your neighborhood

Making a difference in your neighborhood Making a difference in your neighborhood A Handbook for Using Community Decision-Making to Improve the Lives of Children, Youth and Families Acknowledgements Making a Difference in Your Neighborhood: Using

More information

ANALYSE THIS PREDICT THAT. How institutions compete and win with data analytics

ANALYSE THIS PREDICT THAT. How institutions compete and win with data analytics ANALYSE THIS PREDICT THAT How institutions compete and win with data analytics Foreward 4 1.0 Executive summary 5 2.0 Major competitive growth forces 8 2.1 From the information age, to the personalisation

More information

Making the right connections across Australia s innovation ecosystem to support our future growth and international competitiveness

Making the right connections across Australia s innovation ecosystem to support our future growth and international competitiveness Making the right connections across Australia s innovation ecosystem to support our future growth and international competitiveness Welcome 4 Introduction 5 Research findings and recommendations 9 Next

More information

Climate Surveys: Useful Tools to Help Colleges and Universities in Their Efforts to Reduce and Prevent Sexual Assault

Climate Surveys: Useful Tools to Help Colleges and Universities in Their Efforts to Reduce and Prevent Sexual Assault Climate Surveys: Useful Tools to Help Colleges and Universities in Their Efforts to Reduce and Prevent Sexual Assault Why are we releasing information about climate surveys? Sexual assault is a significant

More information

MONEY MORE FOR MORE GOOD. Bob Ottenhoff & Greg Ulrich

MONEY MORE FOR MORE GOOD. Bob Ottenhoff & Greg Ulrich MORE MONEY FOR MORE GOOD Bob Ottenhoff & Greg Ulrich Acknowledgements & Copyrights We would like to thank the following people for their invaluable contribution to the creation of this guide: Hope Neighbor,

More information

Transforming the citizen experience One Stop Shop for public services

Transforming the citizen experience One Stop Shop for public services pwc.com.au Transforming the citizen experience One Stop Shop for public services February 2012 What do you value? Contents 04 07 14 Getting it right: putting Core Principles of Operating Model the citizen

More information

Digital Inside Out Creating a digital-first Britain

Digital Inside Out Creating a digital-first Britain Digital Inside Out Creating a digital-first Britain 1 Foreword When Tim Berners-Lee invented the World Wide Web in 1989, I wonder if he realised the immeasurable impact it would have in such a short space

More information

Operational Intelligence: What It Is and Why You Need It Now

Operational Intelligence: What It Is and Why You Need It Now April 2013 Operational Intelligence: What It Is and Why You Need It Now Sponsored by Splunk Contents Introduction 1 What Is Operational Intelligence? 1 Trends Driving the Need for Operational Intelligence

More information

CITIZENS AS SENSORS: THE WORLD OF VOLUNTEERED GEOGRAPHY

CITIZENS AS SENSORS: THE WORLD OF VOLUNTEERED GEOGRAPHY CITIZENS AS SENSORS: THE WORLD OF VOLUNTEERED GEOGRAPHY Michael F. Goodchild 1 ABSTRACT In recent months there has been an explosion of interest in using the Web to create, assemble, and disseminate geographic

More information

Why we need better ethics for emerging technologies

Why we need better ethics for emerging technologies Ethics and Information Technology (2005) 7:111 119 Ó Springer 2006 DOI 10.1007/s10676-006-0008-0 Why we need better ethics for emerging technologies James H. Moor Department of Philosophy, Dartmouth College,

More information

The AcademyHealth Listening Project: Improving the Evidence Base for Medicare Policymaking

The AcademyHealth Listening Project: Improving the Evidence Base for Medicare Policymaking The AcademyHealth Listening Project: Improving the Evidence Base for Medicare Policymaking February 2014 Table of Contents Executive Summary 3 Introduction 9 Methods 9 Sample Population 9 Instrument and

More information

The Internet of Things

The Internet of Things The Internet of Things How the Next Evolution of the Internet Is Changing Everything Author Dave Evans April 2011 Cisco IBSG 2011 Cisco and/or its affiliates. All rights reserved. 04/11 The Internet of

More information

At the Big Data Crossroads: turning towards a smarter travel experience. Thomas H. Davenport. Visiting Professor at Harvard Business School

At the Big Data Crossroads: turning towards a smarter travel experience. Thomas H. Davenport. Visiting Professor at Harvard Business School At the Big Data Crossroads: turning towards a smarter travel experience Thomas H. Davenport Visiting Professor at Harvard Business School Amadeus IT Group is committed to minimizing its carbon footprint.

More information

Privacy and Electronic Communications Regulations. Guidance on the rules on use of cookies and similar technologies

Privacy and Electronic Communications Regulations. Guidance on the rules on use of cookies and similar technologies Privacy and Electronic Communications Regulations Guidance on the rules on use of cookies and similar technologies Contents 1. Introduction 2. Background 3. Consumer awareness of cookies 4. Terminology

More information

a report by harvard business review analytic services The New Conversation: Taking Social Media from Talk to Action Sponsored by

a report by harvard business review analytic services The New Conversation: Taking Social Media from Talk to Action Sponsored by a report by harvard business review analytic services The New Conversation: Taking Social Media from Talk to Action Sponsored by Conventional marketing wisdom long held that a dissatisfied customer tells

More information