3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving

Transcription

1 3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving AIT Austrian Institute of Technology Safety & Security Department Christian Zinner Safe and Autonomous Systems

2 Outline 3D AIT Vision technology components for land vehicles 3D sensing Visual odometry mapping path planning Showcase I: Autonomous off-road vehicle Showcase II: Driver assistance system for trams 2

3 3D AIT First 3D Vision concepts Darpa Grande Challenge participation Research & Development Indoor robotic (robots@home) Darpa Urban Challenge participation dental scanner presentation at International Dental Show Cologne Obstacle detection for autonomous trucks and trains Driver assistance systems dental Autonomous scanner driving prototype trains, tramways, trucks, industrial 3D object mobile machines reconstruction Exploitation Cooperation with Auburn University Cooperations with TU Vienna and ETH Zürich Cooperation with a.tron3d dental scanner Vision Award 2011 intelligent production safety surveillance Cooperations with industry companies for developing assistive and autonomy functions (e.g. Bombardier) 3 3

4 AIT 3D Vision Application Areas Land & aerial vehicles Local railways Streetcars, tram Trucks Construction vehicles Agricultural vehicles UAV Tram on Demand Industrial Automation Safety & Security Dental / Medical 4

5 Vision Technology for Land Vehicles Real-time stereo vision Trinocular & wide baseline stereo 3D point cloud registration surface modeling Vision based offroad terrain mapping Enhanced stereo calibration Night vision stereo matching Dense motion stereo Visual odometry object detection, tracking, classification obstacle avoidance autonomous path planning Autonomous path planning High precision observation of hull volume in front of trains 5

6 AIT Real Time Stereo Vision Standard high volume cameras AIT stereo matching engine Resolution up to 1920 x 1200 (Sony low noise sensor) Large baseline and trinocular camera configuration Long distance 3D data (>100m) Efficient computation of high resolution depth data Windshield integration without connecting structure enabled by online calibration refinement Large HFOV 80 cm 20 cm 90 6

7 Other Visual Depth Sensor Modalities Night vision stereo Thermal infrared imaging Bolometer array sensors Noisy + low resolution images AIT dense motion stereo Structure-from-motion in real-time with moving mono camera Side-viewing single camera with wide angle optics Consecutive images built up stereo pairs Each pair must be rectified individually Depth image computation with stereo matching engine v 7

8 Visual Odometry Recognition and tracking of image features Reconstruction of the relative vehicle movement based on the displacement of image elements Tracks 6 DOFs (location and pose) Drift rate <1% when using stereo cameras Stabilizes self-localization (e.g. during GPS dropouts) 8

9 AIT Mapping Module Depth Data fusion and accumulation Fusion of point clouds from various depth sensors in real-time Sensors may run asynchronously Sensors deliver different data characteristics and quality Real-time stereo vision LIDAR Depth from moving mono camera Data accumulation over time Environment model becomes more complete and more accurate Inherent plausibility checks remove artifacts 3D sensor information is transformed into a map representation Basis for autonomous path planning 9

10 AIT Mapping Module 3D Environment Model Online-creation of geo-referenced map from sensor data Map stores various data modalities / attributes elevation texture terrain roughness visual cues Methods for visual self localization of a vehicle within a pre-recorded map The key to GPS-independent navigation Living map: each ride generates new sensor data for map update 11

11 Autonomous Path Planning and Self-localization Input: Accessibility Map Rough sequence of waypoints (mission plan) Result: Detailed path under consideration of the actual environment (terrain, obstacles, ) simulation real-world path according to mission plan > > > > > > > real path based on sensor data 12

12 Showcase I: Autonomous off-road Vehicle Driverless operation in unregulated/unstructured Environment Missions in crisis & disaster management Visual mapping and self-localization Towards GPS independent navigation 3D-Vision based obstacle avoidance and terrain modelling Solely camera based mode of operation (no need for laser scanners) 13

13 Autonomous off-road Vehicle Technology components Actuators Steering Throttle Braking Autopilot & Cruise control Tracking Processing Hardware Path planning Mission planning / Route definition Mapping 3D - Sensors Daylight Stereo camera Night vision Stereo camera Laser scanner Odometry / Localisation Visual Odometry Inertial sensor Magnetometer Wheel speed GPS Receiver Trajectory filter 14 14

14 Autonomous off-road Vehicle Field Experiment 15

15 Showcase II: Advanced Driver Assistance System for Trams Cooperation between and Collision avoidance for light rail vehicles Observation of vehicle envelope in front of the vehicle Stereo vision sensor Detect and track obstacles Identify collision risks Issue warnings or trigger brakes 16

16 Advanced Driver Assistance System for Trams Driver Stereo camera braking maneuver, ringing the bell Warning signals to driver Image data Vehicle Control Unit Power supply light functions, etc braking signals Computation unit (image analysis, scene interpretation, assistance function) 17

17 Advanced Driver Assistance System for Trams AIT 3D Vision technology for advanced collision prevention for urban trains Trinocular wide baseline stereo vision wide field-of-view >80 precise 3D data from close-range to more than 80 m Object detection, localization, tracking Speed and trajectory estimation of own vehicle and objects Field-prototypes running in German cities 18

18 AIT Austrian Institute of Technology your ingenious partner Contact Christian Zinner Thematic Coordinator 3D Vision and Modeling Safety & Security Department Safe and Autonomous Systems AIT Austrian Institute of Technology