Designing Kinect based games for children with Autism

Transcription

1

2 Designing Kinect based games for children with Autism Aug 13 May 15, Georgia Institute of Technology Faculty Mentors: Dr. Agata Rozga, Dr. Brian Magerko, Dr. Gregory Abowd Student Collaborators: Mirko Gelsomini, Patricia Perez These games are a fun interactions with one and two player modes to encourage social and motor skills in children with Autism. Designing: The games are prototyped based on user centered design process with the teachers at the Lionheart School, Alpharetta. The prototypes are iterated based on their requirements and feedback. There are two modes of representation of the child - 2 D Skeleton figures, Green Screen Image. Evaluation: The requirement data is qualitatively analyzed using affinity modelling and the impact of the games on social and motor behavior is quantitatively and qualitatively analyzed by videocoding the childrens game play sessions.

3 Other technology probes included: A freeform interactions that also mapped the Kinect s skeleton joints to 2D characters like iron man, princess and Mr. Bean. A story telling application with simple 2D animations for various movements of animals like flying, jumping and hopping and also explored gesture-detection with Kinect. Technology: The front end application was a web application designed using Html, css, javascript and jquery. The Kinect joints were detected by a C# application which sent the skeleton information to the web application using socket communication.

4 Video Game Design Aug 14 Dec 14, Georgia Institute of Technology I am working on Unity-3D to create mini games as a part of the assignments I am working on in my Video Game Design Class. This is a link to the blog page that hosts. My blog alias is wolverine. You will need to install Unity Web Player to play these games. hor/wolverine/ I am currently working on creating more games for children with Autism on the Unity platform with Kinect plugins.

5 The Worker Helper Feb 14 May 14, Georgia Institute of Technology Human Computer Interaction Student Collaborators: Varun Gatne, Keerthi Amurgam, David StoneCypher The Worker helper is a prototype of a smart phone application designed for custodians (janitors) to facilitate communication, eliminate extensive paper work required to file complaints and order supplies. User Study: We approached 10 custodians in 5 buildings at Georgia Tech, out of which 6 agreed to try our prototype and take our survey. 66% of them were positive that the app will be an useful in their work.

6 Low Fidelity Prototyping: Designed 4 main features into our app, text and voice chat, complaint and order. Android phones were taken to the janitors and a survey was taken after the usability testing.

7 Survey Design: Analyzed the demography and the response of the surveys. We also had qualitative input on changing certain names and features as per their daily usage and requirement for further iteration.

8 Posture Sensing Garment - NASA Jan 14 May 14, Georgia Institute oftechnology Mobile and Ubiquitous Computing Student Collaborators: Shailesh Lohia, Louis Fischer, Parth Saigal We used flex sensors on to a stretchable unitard garment and programmed the out put on to a teensy board to map the voltage change due to stretching of the sensor to the angular displacement of the joint angle. Nine sensors were sewed on and connected using conductive fiber for both the upper and the lower body. This was presented at the Wearable Computing Symposium, NASA in February 2014.

9 Kinect based peer to peer Yoga Trainer Sept 13 Nov 13, Georgia Institute of Technology Principles of User Interface Design Student Collaborators: Meghna Mehta Desktop application: Kinect posture detection was done using Microsoft SDK for skeleton tracking in C#. 3D modelling: and mapping of skeleton joint data to the application UI for feedback. Data transmission: Joint data transferred over remote network using php request response and temporary storage in Mysql database.

10 Low Fidelity Prototyping Figure shows a prototyped redesign of the coffee vending machine in Axure. Other methods used in various projects include story-boarding, task analysis, paperprototyping, balsamiq, html-css prototyping, presentations and sequence modelling.

11 A 3 by 3 Matrix game for Eye-tracking technique using Artificial Neural Networks July 12 April 13, Mumbai, India B.E. Project Student Collaborators: Pranita Desai, Aarti Gupta Faculty Mentor: Sharmila Sengupta Image processing: done in Matlab to detect pupil and track eye-movement. Game UI: Created simple matrix games in flash to detect words. Artificial neural networks: used feed forward back propagation method to do training and mapping the eye coordinates to the desired gaze location on the screen.

12 Automated Lip Reading July 11 April 12, Mumbai, India Student Collaborators: Pranita Desai, Aarti Gupta Faculty Mentor: Sharmila Sengupta Image Processing: simulated simple words like "hello", "yellow", "yes","no" for lip contour detection and pattern storage in the form of image sets of 4 by 4 frames. Verified test cases for false negatives and false positives. Extended its application to speech for the disabled and automated password detection.

13 Mobile App for predicting Home-Budget Mumbai, India Android Application : programmed in eclipse. Used php graph and Mysql database to store the user's data and retrieve it in graph interface. Prediction algorithm: allowed the user to predict the amount of time that the user would take to save up a desired amount depending on the user's past expenses and monthly income.

14 CraYon: A Campaign Website against Child Abuse Mumbai, India Student Collaborator: Jayati Ambekar Web Application: Front end designing: Used Html, css and javascript. Created forms and form validation using javascript. Back end designing: Used JSP and JDBC connectivity to store and retrieve data from SQL server. Pattern recognition and matching using sql queries enabled for site admin to flag and delete spam or abusive comments.

15