How To Help People With Disabilities With A Computer Program

Transcription

1 Faculty of Informatics Computer-Human Interfaces Personal information technologies András Lőrincz Neural Information Processing Group

2 Back from Columbia University Research and academy Thanks are due to my to my group Now, finishing Company Viktor Dani Balázs II András Szityú Zsolt Ákos Marci Kati Melinda Zoli II Balázs I Gyula Gábor Zoli I Gergő

3 Content 1 Main activites Personal information example Typical & specific Dictionaries and their integration

4 Main activites Machine learning Computer-human interaction (TÁMOP projects) Natural language processing (US Air Force Research Lab) Machine vision Map-Reduce optimization (Morgan-Stanley) Analysis of customers behavior (Hungarian Telekom) Neuroscience & psychology learning from the structure of the brain

5 Content 1 Main activites Personal information example Typical & specific Dictionaries and their integration

6 Motivation Non-speaking, but speech understanding children with special needs They can barely interact and communicate Their intelligence, knowledge, and personality need interaction augmentative and alternative communication We develop enabling ICT tools for them for communication and control

7 Stone Age for alternative communication

8 Motivation

9 Practicing Zozó and the first results Monitor 9

10 The first three trials The first three trials showed reasonable improvement 10

11 The sequence of 20 trials cursor trajectories target positions on screen Not perfect, but good for the purpose 11

12 Development: Bencus story Highly imprecise cursor control: barely controlled head swings A learner In three months he learned how to control 12

13 Motivation webcam mike speaker Aibo

14 It is for real Laptops and webcams RF-MEMS motes Practicing at the Alternative and Augmentative Communication Center Budapest Sophie L. Kálmán ultrasonic RF-MEMS distance estimation

15 Using Dasher and playing Load Balancing game Emotional and cognitive monitoring

16 We / they need Intelligent computer-human interface Cognitive and emotional profiling Intelligent dialogue system Learning and development virtual environments edutainment serious games Leipzig 12/Oct/2010

17 We / they also need Data collection and example based recommendation systems personalization Interaction with other people depersonalization social networks, social games, social involvement Viktor Gyenes Leipzig 12/Oct/2010

18 Social Gaming

19 EU Future and Emerging Technology Exhibition April 2009, Prague: Science Beyond Fiction gesturing playing anger webcam emotional monitoring Not sufficient artificial intelligence Leipzig 12/Oct/2010

20 Education Two Classrooms Budapest Node

21 Under planning Nurse HomeCare Budapest Node

22 Under planning Elderly Apartment Houses: Gold Sunset Törökbálint Újpalota Zugló Budapest Node

23 In sum We need to collect large databases about previous events (i.e., movies) results of tests development of interactions emotions and cognitive profiles subject to privacy in order to have a predictive recommendation system for typical events annotation through machine recognition for specific events novelty recognition through machine learning

24 Content 1 Motivation 2 3 Personal information example Typical & specific Recommendations 4 Outlook: Dictionaries and their integration

25 Dot patterns Basic forms Squire July 08, 2010, & Gatsby Knowlton

26 Dot patterns Basic forms Study items Squire July 08, 2010, & Gatsby Knowlton

27 Dot patterns Test items Squire July 08, 2010, & Gatsby Knowlton

28 Dot patterns Test items Test results on category learning control hippocampal subject Squire July 08, 2010, & Gatsby Knowlton

29 Dot patterns Test results on recognition control hippocampal subject Test results on category learning control hippocampal subject Squire July 08, 2010, & Gatsby Knowlton

30 Discrimination vs. generalization normal child, age four years and two months autistic child, age three and half years

31 Autism Individuals with autism have difficulty abstracting subtle spatial information that is necessary for the formation of a mean prototype, for categorizing faces and objects.

32 Nine dots in autism discrimination / visual acuity Autistic group is less accurate Number of prototype formers is smaller International Meeting for Autism Research May 7-9, 2009, Gastgeb et al. Univ. Pittsburgh Eagle-Eyed Visual Acuity Ashwin et al., Biol. Psych Cambridge Univ.

33 Content 1 Main activites Personal information example Typical & specific Dictionaries and their integration

34 Route to dictionary integration Annotated databases: Wikipedia, WordNet, OpenCyc ontologies Map ontologies onto each other recommendation system map typical parts Map to other modalities LabelMe: segmented images and textual information segmented images and movies model of the self: interaction control optimization

35 Route to dictionary integration Texts, documents Dialogue system Images Image understanding through texts Movies Motion understanding and control

36