People centric sensing Leveraging mobile technologies to infer human activities. Applications. History of Sensing Platforms

Transcription

1 People centric sensing People centric sensing Leveraging mobile technologies to infer human activities People centric sensing will help [ ] by enabling a different way to sense, learn, visualize, and share information about ourselves, friends, communities, the way we live, and the world we live in A.T. Campbell et al The Rise of People Centric Sensing Dr. Christos Efstratiou Computer Laboratory, University of Cambridge Applications History of Sensing Platforms Individual activity sensing: fitness applications, behavioural suggestions. Group activity sensing: groups to sense common activities and help achieving group goals. Eg: assess neighbourhood safety, collective recycling efforts. Community sensing: large scale sensing, where large number of people have the same application installed. E.g., tracking speed of disease across a city, congestion in city. Building sensors Computer vision On body accelerometers MSP Nicholas D. Lane, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, Andrew T. Campbell, A Survey of Mobile Phone Sensing, IEEE Communications Magazine, September, instrumenting the environment instrumenting the person instrumenting the mobile phone 1

2 People centric sensing in the Computer Laboratory, Cambridge Explore the potential of mobile phones as a platform for peoplecentric sensing applications. Explore the potential of instrumented environments with sensors to detect human activities. Mobile Phone Sensing Microphone Camera GPS Accelerometer Compass Gyroscope WiFi Bluetooth Proximity Light NFC (near field communication) Phone Sensing vs Sensor Networks Mobile Phone Sensing Sensor Networks Well suited for sensing the environment Specialized hardware designed to accurately monitor specific phenomena All resources dedicated to sensing High cost of deployment and maintenance (regular recharging thousands of sensor nodes) Phone Sensing Well suited for sensing human activities General purpose hardware, often not well suited for accurate sensing of the target phenomena Multi tasking OS. Main purposed of the device is to support other applications Low cost of deployment and maintenance (millions of potential users where each user charges their own phone) But not sure if users will keep you app on their device! The mobile phone sensing domain is filled with hacks, and imaginative techniques that were used to circumvent the limitations of a platform that was designed for a different purpose. However, manufacturers have started to change direction In the near future we expect the release of New hardware platforms that facilitate back ground sensing New OS frameworks that incorporate a general purpose sensing middleware 2

3 Development Design Patterns Resources Collect data High sampling rate Label with ground truth (e.g. user walking data set) Inference pipeline Use collected data for training Sensing is resource intensive BATTERY CPU MEMORY STORAGE Mobile Sensing App Feed back to the user Sensing Feature extraction Classification {walking} The mobile phone s purpose is to support multiple applications A mobile phone sensing application needs to maintain a balance between The amount of resources needed to operate The accuracy of the detection that is achieved Applications Detecting Emotions Adaptive Duty Cycling Inference: Emotional state, location and co location with others Sensors used: Microphone, bluetooth, GPS Map speaking features to emotional state Source: EmotionSense: A Mobile Phones based Adaptive Platform for Experimental Social Psychology Research Ubicomp 10 3

4 Adaptive Duty Cycling Applications Detecting Workplace Behaviour 100 Accuracy [%] 250 Energy (joules) Inference: social behaviour and work performance Sensors: Integrating phones with sensors in the environment continuous 50% duty learning 0 continuous 50% duty learning Fusing mobile phones and sensor networks Research Question: Can we improve the performance of mobile phone sensing by linking it withsensing in the environment? METIS Sensing Offloading Reduce energy consumptions on mobile devices Opportunistic offloading of sensing to the environment Support continuous sensing METIS: Sensing offloading Mobile Phone Social Sensing Application METIS Social Sensing API Sensing Task Distribution Local Sensing Remote Sensing Inference Sensor Mapping Component Sensor Mapping and Inference Plugins Sensing Infrastructure Sensor Network Infrastructure Communication Access Point Phone Sensors Network Interface 4

5 Fusing mobile phones and sensor networks Detecting informal interactions in the work place Detecting informal interactions in the work place Location tracking C ti dt ti Conversation detection Detecting meetings Conversation Patterns Detected Calendar Time(Houroftheday) Detecting collaborations Detecting collaborations Level 2 communities Level 1 communities Level 2 communities Level 1 communities 3/04567$#$ 3/04567$&$!"#,$!"+$!"'$!" $!"&$!"##$!")$!"*$ 3/04567$' $. /012$#$!"($. /012$&$!"#$!"%$ Ground truth 5

6 People centric sensing in construction Applying the same techniques Detecting individual activities in the workplace Fusing data to understand collaborative activities Applications Work practice monitoring and understanding Health & Safety Real time work scheduling and efficiency Challenges Commodity mobile phones not widely used Specialised sensing technologies for people tracking Construction Site THANK YOU 6