Interaction devices and sensors. EPFL Immersive Interaction Group Dr. Nan WANG Dr. Ronan BOULIC

Transcription

1 Interaction devices and sensors EPFL Immersive Interaction Group Dr. Nan WANG Dr. Ronan BOULIC

2 Outline 3D interaction tasks Action capture system Large range Short range Tracking system With markers Markerless Orientation sensor

3 3D interaction tasks Navigation Selection and manipulation System control Symbolic input Two-handed interaction

4 Action capture system Large range (1:2) Wiimote controller (Nintendo) 1*speaker Infrared optical sensor 12* Digital buttons Rumble motor Motion sensing» Vertical axis» Left-right horizontal axis» Horizontal rotation Measurements are 148 mm long, 36.2 mm wide, and 30.8 mm thick Accurate pointing device up to 5 meters

5 Action capture system Wiimote - NunChunk (Nintendo) 1* Control stick 2*Digital buttons Motion sensing Vertical axis Left-right horizontal axis Horizontal rotation Wiimote game resort: Nunchuk measures 113 mm long, 38 mm wide, and 37 mm thick related

6 Action capture system Connectivity Sensor Bar (Infra-red) Bluetooth How it works? Accelerometer: Recording the direction and strength of force Data converter converts analog to digital data IR filter on wiimote Triangulate its position on the screen Class 2: ~10 meters Motion sensoring Pointer system Accurate pointing in 5 meters away from the sensor bar

7 Action capture system Actions tracking in VR: Wiimote Object manipulation Steering a vehicle (forward, back, turning right and left) Target-based (using buttons) NunChunk Navigation (joystick) Desktop VR tracking

8 Actions capture system Large range(2:2) PlayStation Move for PlayStation 3 (PS3 Sony) Motion sensing 3-axis accelerometer 3-axis angular rate Location tracking Magnetometer Object recognition (via PlayStation Eye) 1 * Analog trigger 8 * Button Range of distance : 25cm - (in wide angle view)field of view56 0 (standard view) (wide angle view) FOV zoom lens External dimension : Approx. 200mm 46mm (height diameter) Best range : 0.6-3meter from camera

9 Actions capture system PlayStation Move Navigation Controller: Analog stick (for navigation) D-pad (thumb-operated directional control) 1 * Analog trigger 5 * Buttons Connectivity: Class 2: ~10 meters External dimension : Approx. 138mm 42mm (height diameter) Sony PlayStation Move trailer:

10 Actions tracking in VR: PS Move Object manipulation/selection PlayStation Move Navigation Controller Navigation

11 Actions capture system Short range: Razer Hydra by Sixense Entertainment Absolute position and orientation detection in magnetic field per controller 6 degrees of freedom (6DOF) Forward/back Up/down Left/right Each controller: 5 * Digital buttons 1 * analog stick/button 1 * bumper button 1 * analog trigger Connectivity: Razer Hydra motion capture Approximate Size : 120 mm / 4.92 (Length) x 120 mm / 4.92 (Width) x 100 mm / 3.94 (Height) Best range: meter radius around the base station. (from )

12 STEM System Magnetic field, 1 kit contains 1 base station 2 hand hold controllers 3 controllers 1 head position tracking 2 feet position tracking

13 Actions capture system Comparison between above devices Wiimote PSMove Razer Hydra STEM Wireless Yes Yes No Yes Sub controller cable Motion sensoring in Sub Controller Yes No Yes No, two controllers Yes No Yes YES Depth movement precision Movement detection Position detection Low High - YES Accelerometer Magnetometer Magnetic field Yes Yes With sensor bar Yes With PS eye Yes Yes Occlusion sensitive Yes Yes No No

14 Actions capture system Software Wiimote and NunChunk : Wiiyourself PSMove : Sony official Move.Me Razer Hydra : Sixense SDK STEM system: official SDK

15 Actions capture system (gesture) Leapmotion Usb 2.0/3.0 connection Height Width Depth Weight eo?yid=gby6hgzb3ww 13 mm 13 mm 76 mm 45 grams

16 Tracking system With markers Active Passive Markerless

17 Tracking system (Head tracking) With markers: Active motion capture system PhaseSpace optical motion capture Motion tracking Position sensing LEDs Dimensions: 20 mm x 14 mm x 3.2 mm Weight: 4.5 grams Each LED modulates at a unique frequency resulting in a unique digital ID. LEDs are available in Red visible and Infra-red versions. Cameras Dimensions: 108 mm x 92 mm x 57 mm Weight: 380 grams Each camera achieves an Optical Resolution of 3600 x 3600 (12 Megapixel) using two linear detectors with 16-bit dynamic range. Onboard processors produce an impressive Subpixel

18 Tracking system (Head tracking) Active motion capture system Only Head Tracking P1 V1 y P2 V2 P3 x z

19 Tracking system (Head and hand tracking) Passive tracking system Reflect incoming IR radiation back in the direction of the incoming light (ART tracking maxi distance = 7m)

20 Tracking system Passive tracking system architecture

21 Tracking system Passive tracking system TrackIR for Head tracking Tracking Camera Markers Distance: 0.6m~1.2m Sensor resolution: FPS: 120Hz DOF: 6

22 Tracking system IMU motion capture Xsens Perception Neuron Rs

23 Tracking system Markerless Motion capture system Kinect sensor Natural user Interface using Gestures and Spoken commands Connectivity: + power cable (for PC)

24 Tracking system Kinect sensors: RGB camera Depth sensor(ir Emitter and IR Depth Sensor) Multi-array microphone Functionality: Full-body 3D motion capture Facial recognition Voice recognition

25 Tracking system Kinect sensor features : Sensor outputs video at 30Hz RGB video 8-bit VGA Resolution (640*480 pixels/hd) Depth sensing in VGA resolution (320*240pixels/512*424pixels) Ranging limit : Distance 4.5m Angular field of view : 57 horizontally / vertically / 60

26 Tracking system New Kinect Sensor Improved field of view results in much larger play space. RGB stream is higher quality and higher resolution. See in the dark Biometric scanning Muscle+force Expression engagement

27 Tracking system Kinect motion tracking: Head tracking Hand tracking Upper body Full-body tracking

28 Tracking system Active Passive Markerless (Kinect) Precision High High Low Invasiveness High Medium Low Measurement Marker pos(/ori) 1 Marker pos(/ori) Pos Preparation Complex Complex Easy Battery Yes No No Calibration Yes Yes Yes Marker recognition Yes No - Price High Very High (e.g Low VICON) 1: at least, 3 markers are needed for orientation measurement

29 Tracking system Orientation sensor Sensor we are going to use: 9DOF (accelerometer, magnetometer and gyroscope) sensor bar 9DOF Razor IMU (inertial measurement unit) Dimensions: 1.37x0.42 (35 x 10mm) Dimensions: 1.1" x 1.6" (28 x 41mm)

30 Tracking system Configuration: Wired sensor bar + USB cable + Arduino board Wireless Bluetooth Modem + battery + Razor IMU

31 Head Mounted Display (HMD) 3D display Few buttons

32 Head Mounted Display (HMD) HTC vive One Display 2 IR Cameras 2 wireless controllers Active trackers

33 HMD comparison Features PlaystationVR Oculus Commercial version PC requirements SONY Playstation console Nvidia GTX970 AMD 290 I5, 8G+DDR3 HTC Vive Nvidia GTX970 AMD 290 I5, 4G+ DDR3 Connection HDMI+USB HDMI 1.3 3USB 3.0 HDMI USB 3.0 Sensor Types Acc, Gyro, Mag PS Eye Tracking Acc, Gyro, Mag,Tracking Sesnor Array Acce, Gyro, Front- Facing Camera, Laser sensor Field of View Appr. 100 degrees Appr. 110 degrees Appr. 110 degrees Resolution 1920* * *1200 Refresh Rate 120hz 90hz 90hz 90hz $$$ TBA 599 USD 799 USD

34 Question?

35 TPs 8 Groups on 4 different tasks 2 groups on 2 (similar/same) configurations Take turns next week TP1: Navigation with active markers (CAVES and stereo server) TP2: Object manipulation TP3: Kinect character control and interactive scene TP4: Google Cardboard

36 TP1 : Navigation with active markers CAVE/Stereo 4-sided CAVE/Stereo Screen Stereo glasses + emitters 3D projectors Motion capture system Left projector Top projector Front projector Right projector

37 TP2 : Object manipulation PCs Playstation PS Move / PS Eye

38 TP3 : Kinect character control and Oculus Display Projector if needed Kinect PCs

39 TP4 : Google Cardboard Google Cardboard PC