Using a Virtual Environment to Test a Mobile App for the Ambient Assisted Living Paolo Calvaresi 1, Aldo Franco Dragoni 1, Matteo Pierdicca 1, Davide Calvaresi 2, Paolo Sernani 1 1 Università Politecnica delle Marche 2 Scuola Superiore Sant Anna 4th International Workshop on Artificial Intelligence and Assistive Medicine (AI-AM/NetMed 2015) Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 1 / 24
Outline 1 Introduction 2 A 3D Simulator for the AAL 3 Implementation Mobile application Virtual Caregiver Human activities 4 Conclusions Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 2 / 24
Smart homes: tests Tests should be conducted: with real human inhabitants with different kinds of impairments in different environmental situations in diverse economical conditions Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 3 / 24
Smart homes: tests Thus, the development and the assessment of software prototypes is extremely expensive! hardware for a great number of tests modify or redesign software components test again Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 4 / 24
Solution Testing smart systems in a 3D Virtual Environment: to speed up the development of software systems (e.g. by decoupling hardware from software) to easily migrate software systems from the simulator to the real world to execute economically sustainable tests Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 5 / 24
Based on MORSE Modular OpenRobots Simulation Engine: to simulate sensors, actuators and robots to manage rendering and physics of the simulation (via the Blender Game Engine and the Bullet Physics Library) to allow extensions (e.g. new sensors) Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 6 / 24
Implementation Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 7 / 24
Implementation Sensors: temperature motion RFID Tags Actuators: lights elevator automatic doors/windows Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 8 / 24
Implementation The interactions between the mobile app (or the virtual caregiver) and the simulation are based on TCP/IP sockets: to retrieve value from the sensors to send commands to actuators Easy migration, if the sensors offer the same interface as off-the shelf sensors Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 9 / 24
Mobile application Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 10 / 24
Mobile application Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 10 / 24
Example: lights & doors Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 11 / 24
Example: wheelchair control Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 12 / 24
Virtual Caregiver An expert system modeling a distributed, reliable and modular sensor network, being able to: interact with the assisted person control the environment around him monitor his health conditions Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 13 / 24
Virtual Caregiver Daily living scenarios: interaction with lights doors/windows Emergency scenarios: gas leak/flooding heartbeat sleep apnea Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 14 / 24
Example: lights & doors Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 15 / 24
Example: flooding Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 16 / 24
Videos Videos (for both the mobile app and the virtual carer) available at: https://www.youtube.com/user/airtlab https://www.youtube.com/watch?v=ibuyzfokmy8 https://www.youtube.com/watch?v=zxepshrnguo Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 17 / 24
Human activities We defined needs, intents and actions 1 needs: variables describing hunger, thirst, tiredness, boredom intent: high level actions associated to needs (e.g. eating ) actions: low level (atomic) actions associated to intents (e.g. go to the kitchen, take an apple, eat the apple ) 1 B. Kormanyos and B. Pataki, Multilevel simulation of daily activities: Why and how?, in 2013 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), 2013, pp. 1-6. Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 18 / 24
Human activities: actions We trigger intents based on needs value, implementing an interrupt mechanism: needs increase their value according to time each intent is associated to a priority value: its importance change proportionally to needs, and an ageing factor avoids starvation each action is associated to four modifiers, one for each need. Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 19 / 24
Human activities: avatar s movements We added the capability to explore and look for objects in the environment using the Semantic Camera provided by MORSE. Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 20 / 24
Conclusions A 3D virtual environment, based on MORSE: to represent a smart home to allow the interaction with software systems, as the mobile application and the virtual caregiver to allow a transparent migration, via TCP/IP sockets A 3D simulator can speed up the testing phase of IT systems for the AAL (without completely replace it). Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 21 / 24
Ongoing works Animation for the actions of the human avatar Integration of the human activities simulations with the software system simulations Serious games to: learn configuration possibilities increase acceptance test software systems Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 22 / 24
Serious Games Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 23 / 24
Q&A Thank you for your attention. Email: p.sernani@univpm.it Web: http://airtlab.dii.univpm.it/ Calvaresi et al. AI-AM/NetMed @ AIME June 20, 2015 24 / 24