Faculty of Informatics Computer-Human Interfaces Personal information technologies András Lőrincz Neural Information Processing Group http://nipg.inf.elte.hu
Back from Columbia University Research and academy Thanks are due to my to my group Now, finishing Company Viktor Dani Balázs II András Szityú Zsolt Ákos Marci Kati Melinda Zoli II Balázs I Gyula Gábor Zoli I Gergő
Content 1 Main activites 2 3 4 Personal information example Typical & specific Dictionaries and their integration
Main activites Machine learning Computer-human interaction (TÁMOP projects) Natural language processing (US Air Force Research Lab) Machine vision Map-Reduce optimization (Morgan-Stanley) Analysis of customers behavior (Hungarian Telekom) Neuroscience & psychology learning from the structure of the brain
Content 1 Main activites 2 3 4 Personal information example Typical & specific Dictionaries and their integration
Motivation Non-speaking, but speech understanding children with special needs They can barely interact and communicate Their intelligence, knowledge, and personality need interaction augmentative and alternative communication We develop enabling ICT tools for them for communication and control
Stone Age for alternative communication
Motivation
Practicing Zozó and the first results Monitor 9
The first three trials The first three trials showed reasonable improvement 10
The sequence of 20 trials cursor trajectories target positions on screen Not perfect, but good for the purpose 11
Development: Bencus story Highly imprecise cursor control: barely controlled head swings A learner In three months he learned how to control 12
Motivation webcam mike speaker Aibo
It is for real Laptops and webcams RF-MEMS motes Practicing at the Alternative and Augmentative Communication Center Budapest Sophie L. Kálmán ultrasonic RF-MEMS distance estimation
Using Dasher and playing Load Balancing game Emotional and cognitive monitoring
We / they need Intelligent computer-human interface Cognitive and emotional profiling Intelligent dialogue system Learning and development virtual environments edutainment serious games Leipzig 12/Oct/2010
We / they also need Data collection and example based recommendation systems personalization Interaction with other people depersonalization social networks, social games, social involvement Viktor Gyenes Leipzig 12/Oct/2010
Social Gaming
EU Future and Emerging Technology Exhibition April 2009, Prague: Science Beyond Fiction gesturing playing anger webcam emotional monitoring Not sufficient artificial intelligence Leipzig 12/Oct/2010
Education Two Classrooms Budapest Node
Under planning Nurse HomeCare Budapest Node
Under planning Elderly Apartment Houses: Gold Sunset Törökbálint Újpalota Zugló Budapest Node
In sum We need to collect large databases about previous events (i.e., movies) results of tests development of interactions emotions and cognitive profiles subject to privacy in order to have a predictive recommendation system for typical events annotation through machine recognition for specific events novelty recognition through machine learning
Content 1 Motivation 2 3 Personal information example Typical & specific Recommendations 4 Outlook: Dictionaries and their integration
Dot patterns Basic forms Squire July 08, 2010, & Gatsby Knowlton
Dot patterns Basic forms Study items Squire July 08, 2010, & Gatsby Knowlton
Dot patterns Test items Squire July 08, 2010, & Gatsby Knowlton
Dot patterns Test items Test results on category learning control hippocampal subject Squire July 08, 2010, & Gatsby Knowlton
Dot patterns Test results on recognition control hippocampal subject Test results on category learning control hippocampal subject Squire July 08, 2010, & Gatsby Knowlton
Discrimination vs. generalization normal child, age four years and two months autistic child, age three and half years
Autism Individuals with autism have difficulty abstracting subtle spatial information that is necessary for the formation of a mean prototype, for categorizing faces and objects.
Nine dots in autism discrimination / visual acuity Autistic group is less accurate Number of prototype formers is smaller International Meeting for Autism Research May 7-9, 2009, Gastgeb et al. Univ. Pittsburgh Eagle-Eyed Visual Acuity Ashwin et al., Biol. Psych. 2008 Cambridge Univ.
Content 1 Main activites 2 3 4 Personal information example Typical & specific Dictionaries and their integration
Route to dictionary integration Annotated databases: Wikipedia, WordNet, OpenCyc ontologies Map ontologies onto each other recommendation system map typical parts Map to other modalities LabelMe: segmented images and textual information segmented images and movies model of the self: interaction control optimization
Route to dictionary integration Texts, documents Dialogue system Images Image understanding through texts Movies Motion understanding and control