W3C MMI Architecture as a Basis for Enhanced Interaction for Ambient Assisted Living António Teixeira, Nuno Almeida, Carlos Pereira, Miguel Oliveira e Silva ajst@ua.pt
Overview Our motivations AAL Interaction and goals Information on usage of MMI architecture in AAL projects University of Aveiro/IEETA is involved AAL4ALL Smartphones for Seniors AAL Paelife Some movies and demo(s)
Motivations: Multimodal Interaction Speech centric Interaction for AAL
Human interaction Dialog Speech Speech + gestures + expressions 5 5
Long term goal Development of Robust, Useful and Easy to Use Human- Machine Interaction Using speech and language, Supporting Portuguese native speakers We have been involved in spoken interaction since the 90s 6 6
AAL Ambient Assisted Living
Motivations - Active Ageing Based on the recognition of the human rights of older people and the United Nations (UN) principles of independence, participation, dignity, care and self-fulfillment; Is the process of optimizing opportunities for health, participation and security in order to enhance quality of life as people age; Refers to continuing participation in social, economic, cultural, spiritual and civic affairs Allows people to realize their potential for physical, social, and mental well-being throughout the life course and to participate in society, while providing them with adequate protection, security and care when they need 8
Motivations Tech Innovation The introduction of technological solutions can facilitate the daily life of the elderly, fighting isolation and exclusion, increasing their pro-activity, work capacity and autonomy. Addressing Accessibility There is a need to overcome the gap between an individual's capabilities and the technological resources, Addressing Usability a term used to denote the ease with which people can employ a particular tool or other humanmade object in order to achieve a particular goal. 9
AAL and multimodality Strong focus on contextual usage, usability, pervasiveness, and the user Leading to highly heterogeneous environments Multimodality can provide an important push in shortening the gap between the user and its ambient AAL challenges: necessity of including a wide number of devices distributed focus of AAL requirement of constant connection 10
W3C MMI architecture The architecture has the potential to solve some of our issues for interaction in AAL: Heterogeneity now a smaller issue Autonomy is increased Usability through choice
Information on concrete uses
Example 1 - AAL4ALL AAL4ALL project has a specific task (under the leadership of University of Aveiro) to define the architecture and develop multimodal support for AAL Applications W3C Architecture Adoption within the task followed several phases First phase» Some experiments with EMMA» Creation of an MMI interoperability test report Second phase» Tests with Interaction Manager based on SCXML (on the right) Third phase» First blackbox modalities supporting Lifecycle Events communication ASR, TTS with Interaction Manager iteration» Test framework based only on EMMA and Lifecycle Events» First integrated demo Ongoing» Developing and integrating more modalities (ex: avatar)» Work on fusion and fission capabilities 13
AAL4ALL Demonstrator - July 2012 To prove the potential of W3C multimodal architecture, a cooperation was made between University of Aveiro and Microsoft Language Development Center (in Lisbonne). Development of a simple tecnological demonstrator making use of input and output modalities developed in scope of the project. The demo consisted in an multimodal integrated water heater controller. It was presented at project general meeting in July 2012, Porto, Portugal 15
AAL4ALL Demonstrator - July 2012 16
Example 2 Smartphones for Seniors System Architecture Windows Phone 7 Web services to overweight features Several modules to control modalities and intelligence 19
Multimodal Output Graphics and Text Panoramic view Medication images generation Medication details Speech TTS (e.g. The side effects of medication are ) Use of Natural Lang Generation Ex: You should take one tablet of Aspirin 20
The movie 21
Example 3 PaeLife Personal Assistant Goal: Accessible and easy-to-use Assistant that integrates a set of services aiming at increasing social activity and socialintegration of the elderly, making use of multimodal interaction. 22
System architecture overview Platform Project adopted a system composed of both a stationary main unit and a portable unit. Home Portable Unit Portable Unit (Premium Tablet) Portable Unit (Low Cost Tablet) Internet Main Unit Screen PLA Database Home computer authentification services kinect Automatic Speech Recognition Office 365 Text to Speech Voice Search 23
System architecture overview Platform: main unit Main unit integrates the mentioned big screen supporting graphical output modalities and also sound and speech output Home computer And input devices such as Microsoft Kinect providing the means for the body gestures input modality 24
System architecture overview Platform: portable unit Integrated multi-touch support in the operating system makes possible the touch input modality Integrated microphone and speakers will enable the speech input and output modalities The display will support graphical output modalities 25
Interaction with PLA Since the PLA will be composed of two units each one being independent yet simultaneously interoperable The following types of interaction are possible: Only through the main unit Only through the portable unit Both through the main and portable units 26
Architecture The Paelife multimodal framework is directly based on the standards defined by the W3C This choice is justified by the architecture s open standard nature. This architecture provides an answer to the previous requirements easing the creation and integration of new modules 27
Modalities Speech input/output modalities were already created Using Microsoft s speech platform Easy support for the several languages and future language packs for Elderly Speech ASR Input Modality for Body Gestures Uses the PaeLife Kinect Framework built for the PaeLife project for recognition of user gestures 28
Proof-of-concept demonstration of the Framework To serve as a test for the produced framework, a demo application was created. The application recreates a news reader. DEMO 29
Acknowledgments This work presented is part of the COMPETE - Programa Operacional Factores de Competitividade and the European Union (FEDER) under projects AAL4ALL (www.aal4all.org), Smartphones for Seniors (www.martphones4seniors.org) and Living Usability Lab for Next Generation Networks (http://www.livinglab.pt/). Authors also acknowledge the funding for project PaeLife (www.paelife.eu) from AAL JP and national agencies: Part of the work presented was funded by FEDER, COMPETE and FCT in the context of AAL/0015/2009 and IEETA Research Unit funding FCOMP-01-0124-FEDER-022682 (FCT-PEstC/EEI/UI0127/2011). Special thanks to all our partners