VIRTUAL DRONE SIMULATOR Practical Study 2015/2016 Supervisors Valérie GOURANTON Ronan GAUGNE Reviewer Bertrand COUASNON Team members Agathe DUBOUE Anthony GUILLET Raphaël LE GORANDE Corentin VIVES 1
3/26 TABLE OF CONTENTS 1.Project presentation 2.Technologies used 3.Solutions found 4.Prototype demonstration 5.Conclusion OUR GOAL 1. Provide a way to pilot the drone with gestures 2. Suggest a virtual training program 4/26 2
5/26 SPECIFICATIONS Kinect Catch gestures Already existing tools Gesture processing Jakopter Control API Training program will be implemented into a virtual reality simulator High school students VirtualJakopter Virtual Drone TECHNOLOGIES USED 3
7/26 KINECT Microsoft, 2010 Motion capture device Infrared camera + webcam Kinect v2? (2013) JAKOPTER 8/26 Kinect Catch gestures Gesture processing Jakopter Control API VirtualJakopter Virtual Drone 4
9/26 JAKOPTER Jakopter Piloting scripts platform Initiated as ISTIC Works with Parrot ARDrone Provides modularity and simplicity VIRTUALJAKOPTER Kinect Catch gestures 10/26 Gesture processing Jakopter Control API VirtualJakopter Virtual Drone 5
14/01/2016 11/26 VIRTUALJAKOPTER. 1 VirtualJakopter IRISA Internship Project Controlling a drone in V-R environment A few bugs Presentation Technologies Solutions Prototype Conclusion 12/26 VIRTUALJAKOPTER. 2 Presentation Technologies Solutions Prototype Conclusion 6
UNITY3D 13/26 Cross platform game engine Enables fast development of 3D games (C#, JavaScript) MIDDLEVR Interface between software and hardware Easy and complete One extra software needed 14/26 7
SOLUTIONS FOUND FAAST 16/26 Kinect Catch gestures FAAST Gesture processing Jakopter Control API VirtualJakopter Virtual Drone 8
17/26 FAAST. 1 The Flexible Action and Articulated Skeleton Toolkit Project from USC (University of Southern California) Bind gestures to action easily 18/26 FAAST. 2 9
19/26 COMPARISON Accessible Easy to use Interface Predefined gestures Advanced capabilities Well documented FAAST OpenKinect (libkinect) Kinect SDK (C#) VRPN 20/26 Kinect Catch gestures Gesture processing VRPN Jakopter Control API VirtualJakopter Virtual Drone 10
VRPN 21/26 Network protocol designed for VR Transfer of information between devices Analog and digital channels Built in FAAST and embedded in Jakopter HOW IT WORKS Kinect Catch gestures 22/26 U SB VRPN Server FAAST Gesture processing VRPN VRPN Client Jakopter Control API L U A MiddleVR Unity3D VirtualJakopter Drone 11
23/26 TIMELINE Software installation Running existing project Gestures defining Hands-off Kinect How to capture gestures? How to bind them to actions? S6 Training program PROTOTYPE DEMONSTRATION 12
25/26 DEMONSTRATION VIDEO IN A NUTSHELL 26/26 Full drone control by Kinect gestures FAAST What s next? Training program Improve our knowledge of virtual reality Experience of team work 13