Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite
|
|
|
- Milton Jackson
- 7 years ago
- Views:
Transcription
1 Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite Philip Lenz 1 Andreas Geiger 2 Christoph Stiller 1 Raquel Urtasun 3 1 KARLSRUHE INSTITUTE OF TECHNOLOGY 2 MAX-PLANCK-INSTITUTE IS 3 TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO Philip Lenz (KIT) Andreas Geiger (MPI-IS) Christoph Stiller (KIT) Raquel Urtasun (TTI-C)
2 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
3 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
4 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
5 The Challenge of Autonomous Cars 3D Laserscanner State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
6 The Challenge of Autonomous Cars Depth/reflectance map for 3D localization Detailed road map for path planning State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
7 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
8 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
9 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
10 The Challenge of Autonomous Cars State of the art Localization, path planning, obstacle avoidance Heavy use of 3D laser scanner and detailed maps Problems for computer vision Stereo, optical flow, localization Object detection, recognition and tracking Semantic segmentation, 3D scene understanding P. Lenz: The KITTI Vision Benchmark Suite 2
11 The KITTI Vision Benchmark Suite Two stereo rigs ( px, 54 cm base, 90 opening) Velodyne laser scanner, GPS+IMU localization 6 hours of recordings, 10 frames per second P. Lenz: The KITTI Vision Benchmark Suite 3
12 The KITTI Vision Benchmark Suite Two stereo rigs ( px, 54 cm base, 90 opening) Velodyne laser scanner, GPS+IMU localization 6 hours of recordings, 10 frames per second P. Lenz: The KITTI Vision Benchmark Suite 3
13 The KITTI Vision Benchmark Suite Two stereo rigs ( px, 54 cm base, 90 opening) Velodyne laser scanner, GPS+IMU localization 6 hours of recordings, 10 frames per second P. Lenz: The KITTI Vision Benchmark Suite 3
14 Sensor Calibration Challenges Camera camera calibration Velodyne camera registration GPS Velodyne registration } Geiger et al., ICRA 2012 } ICP + Hand-eye calibration P. Lenz: The KITTI Vision Benchmark Suite 4
15 Sensor Calibration Challenges Camera camera calibration Velodyne camera registration GPS Velodyne registration } Geiger et al., ICRA 2012 } ICP + Hand-eye calibration P. Lenz: The KITTI Vision Benchmark Suite 4
16 Sensor Calibration Challenges T GPS T C T C T Velodyne Camera camera calibration Velodyne camera registration GPS Velodyne registration } Geiger et al., ICRA 2012 } ICP + Hand-eye calibration P. Lenz: The KITTI Vision Benchmark Suite 4
17 Data Annotation Challenges 3D object labels: 22 Annotators Occlusion labels: Mechanical Turk P. Lenz: The KITTI Vision Benchmark Suite 5
18 Data Statistics 150k Number of Labels 100k 50k 0 Car Van Truck Pedestrian Person (sitting) Cyclist Tram Misc P. Lenz: The KITTI Vision Benchmark Suite 6
19 Data Statistics 150k Number of Labels 100k 50k 0 Car Van Truck Pedestrian Person (sitting) Cyclist Tram Misc P. Lenz: The KITTI Vision Benchmark Suite 6
20 Data Statistics P. Lenz: The KITTI Vision Benchmark Suite 6
21 Novel Challenges Middlebury Stereo Evaluation Version 2 Average errors: 2 3% (non-occluded regions) P. Lenz: The KITTI Vision Benchmark Suite 7
22 Novel Challenges Middlebury Stereo Evaluation Version 2 Average errors: 2 3% (non-occluded regions) P. Lenz: The KITTI Vision Benchmark Suite 7
23 Novel Challenges Fast guided cost-volume filtering (Rhemann et al., CVPR 2011) Middlebury, Errors: 2.7% Error threshold: 1 px (Middlebury) / 3 px (KITTI) P. Lenz: The KITTI Vision Benchmark Suite 8
24 Novel Challenges Fast guided cost-volume filtering (Rhemann et al., CVPR 2011) Middlebury, Errors: 2.7% KITTI, Errors: 46.3% Error threshold: 1 px (Middlebury) / 3 px (KITTI) P. Lenz: The KITTI Vision Benchmark Suite 8
25 Novel Challenges So what is the difference? Middlebury KITTI Laboratory Lambertian Rich in texture Medium-size label set Largely fronto-parallel Moving vehicle Specularities Sensor saturation Large label set Strong slants P. Lenz: The KITTI Vision Benchmark Suite 9
26 Novel Challenges So what is the difference? Middlebury KITTI Laboratory Lambertian Rich in texture Medium-size label set Largely fronto-parallel Moving vehicle Specularities Sensor saturation Large label set Strong slants P. Lenz: The KITTI Vision Benchmark Suite 9
27 Novel Challenges So what is the difference? Middlebury KITTI Laboratory Lambertian Rich in texture Medium-size label set Largely fronto-parallel Moving vehicle Specularities Sensor saturation Large label set Strong slants P. Lenz: The KITTI Vision Benchmark Suite 9
28 Novel Challenges So what is the difference? Middlebury d = 16 px KITTI d = 0 px d = 50 px d = 150 px Laboratory Lambertian Rich in texture Medium-size label set Largely fronto-parallel Moving vehicle Specularities Sensor saturation Large label set Strong slants P. Lenz: The KITTI Vision Benchmark Suite 9
29 Novel Challenges So what is the difference? Middlebury KITTI Laboratory Lambertian Rich in texture Medium-size label set Largely fronto-parallel Moving vehicle Specularities Sensor saturation Large label set Strong slants P. Lenz: The KITTI Vision Benchmark Suite 9
30 Stereo Evaluation 200 training images / 200 test images P. Lenz: The KITTI Vision Benchmark Suite 10
31 Stereo Evaluation 200 training images / 200 test images P. Lenz: The KITTI Vision Benchmark Suite 10
32 Stereo Evaluation Particle Convex Belief Propagation (PCBP): Best Results Errors: 0.5% Errors: 0.5% Natural scenes, lots of texture, no objects A couple of wrong pixels at poles P. Lenz: The KITTI Vision Benchmark Suite 10
33 Stereo Evaluation Particle Convex Belief Propagation (PCBP): Worst Results Errors: 19.5% Errors: 21.1% Inner city scenes, lots of objects Textureless surfaces, sensor saturation, reflections P. Lenz: The KITTI Vision Benchmark Suite 10
34 Optical Flow Evaluation Second Order Total Generalized Variation: Best Results Errors: 0.5% Errors: 0.5% City scenes with slow motion (intersections) Small flow vectors (< 30 px) P. Lenz: The KITTI Vision Benchmark Suite 11
35 Optical Flow Evaluation Second Order Total Generalized Variation: Worst Results Errors: 56.5% Errors: 58.8% Difficult lighting conditions, highway driving Large flow vectors (> 150 px) P. Lenz: The KITTI Vision Benchmark Suite 11
36 SLAM Evaluation 22 sequences 40 kilometers P. Lenz: The KITTI Vision Benchmark Suite 12
37 SLAM Evaluation 11 training sequences / 11 test sequences P. Lenz: The KITTI Vision Benchmark Suite 12
38 Challenges: Visual Odometry Ground truth from GPS+IMU Metric: For all frame combinations (i, j): P. Lenz: The KITTI Vision Benchmark Suite 13
39 Challenges: Visual Odometry Ground truth from GPS+IMU Metric: For all frame combinations (i, j): P. Lenz: The KITTI Vision Benchmark Suite 13
40 Challenges: Visual Odometry Ground truth from GPS+IMU Metric: For all frame combinations (i, j): Translation Error [%] VISO2-S VO3ptLBA VO3pt VOFSLBA VOFS VISO2-M GT V O3pt Rotation Error [deg/m] VISO2-S VO3ptLBA VO3pt VOFSLBA VOFS VISO2-M Speed [km/h] Speed [km/h] P. Lenz: The KITTI Vision Benchmark Suite 13
41 Object Detection and Orientation Image Selection for Training and Test Sequences are added exclusively to training or testset Maximize no. of non-occluded objects Images with many objects Diversity in object orientation Entropy Maximization [ ] X X argmax α noc(x) + 1 C H c (X x) x C X : current set x: image from the whole dataset c=1 noc(x): no. of non-occluded objects in an image C: no. of object classes H c: entropy of class c with respect to the orientation P. Lenz: The KITTI Vision Benchmark Suite 14
42 Object Detection and Orientation Image Selection for Training and Test Sequences are added exclusively to training or testset Maximize no. of non-occluded objects Images with many objects Diversity in object orientation Entropy Maximization [ ] X X argmax α noc(x) + 1 C H c (X x) x C X : current set x: image from the whole dataset c=1 noc(x): no. of non-occluded objects in an image C: no. of object classes H c: entropy of class c with respect to the orientation P. Lenz: The KITTI Vision Benchmark Suite 14
43 Object Detection and Orientation Image Selection for Training and Test Sequences are added exclusively to training or testset Maximize no. of non-occluded objects Images with many objects Diversity in object orientation Entropy Maximization [ ] X X argmax α noc(x) + 1 C H c (X x) x C X : current set x: image from the whole dataset c=1 noc(x): no. of non-occluded objects in an image C: no. of object classes H c: entropy of class c with respect to the orientation P. Lenz: The KITTI Vision Benchmark Suite 14
44 Object Detection Evaluation Evaluation Criteria difficulty height occlusion truncation easy > 40px fully visible < 0.15 moderate > 25px partly occluded < 0.30 hard > 25px mostly occluded < 0.50 Overlap o Criterion per Class class overlap car > 0.7 pedestrian > 0.5 cyclist > 0.5 o = GT DET GT DET Object Detector used as baseline for the benchmark Discriminatively Trained Deformable Part Models v4 [1] [1] P. Felzenszwalb et. al. Cascade Object Detection with Deformable Part Models. CVPR P. Lenz: The KITTI Vision Benchmark Suite 15
45 Object Detection Evaluation Evaluation Criteria difficulty height occlusion truncation easy > 40px fully visible < 0.15 moderate > 25px partly occluded < 0.30 hard > 25px mostly occluded < 0.50 Overlap o Criterion per Class groundtruth class overlap car > 0.7 pedestrian > 0.5 cyclist > 0.5 o = GT DET GT DET detection Object Detector used as baseline for the benchmark Discriminatively Trained Deformable Part Models v4 [1] [1] P. Felzenszwalb et. al. Cascade Object Detection with Deformable Part Models. CVPR P. Lenz: The KITTI Vision Benchmark Suite 15
46 Object Detection Evaluation Evaluation Criteria difficulty height occlusion truncation easy > 40px fully visible < 0.15 moderate > 25px partly occluded < 0.30 hard > 25px mostly occluded < 0.50 Overlap o Criterion per Class groundtruth class overlap car > 0.7 pedestrian > 0.5 cyclist > 0.5 o = GT DET GT DET detection Object Detector used as baseline for the benchmark Discriminatively Trained Deformable Part Models v4 [1] [1] P. Felzenszwalb et. al. Cascade Object Detection with Deformable Part Models. CVPR P. Lenz: The KITTI Vision Benchmark Suite 15
47 Object Orientation Evaluation Evaluation Metric: Orientation Average Orientation Similarity (AOS) AOS = 1 11 max s( r) r: r r r {0,0.1,..,1} Normalized Cosine Similarity s(r) groundtruth detection s(r) = 1 D(r) i D(r) 1 + cos (i) θ δ i 2 D(r): set of all object detections at recall rate r (i) θ : difference in angle between estimated and ground truth orientation of detection i δ i = 1 if detection i has been assigned to a ground truth bounding box δ i = 0 if no ground truth has been assigned P. Lenz: The KITTI Vision Benchmark Suite 16
48 Object Orientation Evaluation Evaluation Metric: Orientation Average Orientation Similarity (AOS) AOS = 1 11 max s( r) r: r r r {0,0.1,..,1} Normalized Cosine Similarity s(r) groundtruth detection s(r) = 1 D(r) i D(r) 1 + cos (i) θ δ i 2 D(r): set of all object detections at recall rate r (i) θ : difference in angle between estimated and ground truth orientation of detection i δ i = 1 if detection i has been assigned to a ground truth bounding box δ i = 0 if no ground truth has been assigned P. Lenz: The KITTI Vision Benchmark Suite 16
49 Object Detection Evaluation Car Easy Moderate Hard Precision Recall P. Lenz: The KITTI Vision Benchmark Suite 17
50 Object Detection Evaluation Car Easy Moderate Hard Orientation Similarity Recall P. Lenz: The KITTI Vision Benchmark Suite 17
51 Object Detection Evaluation Pedestrian Easy Moderate Hard Precision Recall P. Lenz: The KITTI Vision Benchmark Suite 18
52 Object Detection Evaluation Pedestrian Easy Moderate Hard Orientation Similarity Recall P. Lenz: The KITTI Vision Benchmark Suite 18
53 Which cars are hard to see? Bad illumination conditions P. Lenz: The KITTI Vision Benchmark Suite 19
54 Which cars are hard to see? Bad illumination conditions P. Lenz: The KITTI Vision Benchmark Suite 19
55 Which cars are hard to see? Occlusions P. Lenz: The KITTI Vision Benchmark Suite 19
56 Which cars are hard to see? Occlusions P. Lenz: The KITTI Vision Benchmark Suite 19
57 Which pedestrians are hard to see? Children P. Lenz: The KITTI Vision Benchmark Suite 20
58 Which pedestrians are hard to see? Carried Items P. Lenz: The KITTI Vision Benchmark Suite 20
59 Conclusion Where are we now? Realistic dataset with 3D ground truth Stereo Optical flow SLAM Object detection / orientation estimation Object Tracking Recognition meets Reconstruction Challenge Complement existing benchmarks / reduce overfitting Submit your results: P. Lenz: The KITTI Vision Benchmark Suite 21
60 Conclusion Where are we now? Realistic dataset with 3D ground truth Stereo Optical flow SLAM Object detection / orientation estimation Object Tracking Recognition meets Reconstruction Challenge Complement existing benchmarks / reduce overfitting Submit your results: P. Lenz: The KITTI Vision Benchmark Suite 21
61 Conclusion Where are we now? Realistic dataset with 3D ground truth Stereo Optical flow SLAM Object detection / orientation estimation Object Tracking Recognition meets Reconstruction Challenge Complement existing benchmarks / reduce overfitting Submit your results: P. Lenz: The KITTI Vision Benchmark Suite 21
62 Conclusion Where are we now? Realistic dataset with 3D ground truth Stereo Optical flow SLAM Object detection / orientation estimation Object Tracking Recognition meets Reconstruction Challenge Complement existing benchmarks / reduce overfitting Submit your results: P. Lenz: The KITTI Vision Benchmark Suite 21
63 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps Lenz et al., IV 2011 P. Lenz: The KITTI Vision Benchmark Suite 22
64 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps Williams et al., RAS 2009 P. Lenz: The KITTI Vision Benchmark Suite 22
65 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps Agarwal et al., ICCV 2009 P. Lenz: The KITTI Vision Benchmark Suite 22
66 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps Wojek et al., ECCV 2008 P. Lenz: The KITTI Vision Benchmark Suite 22
67 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps Geiger et al., CVPR 2011 and NIPS 2011 P. Lenz: The KITTI Vision Benchmark Suite 22
68 Conclusion But KITTI is much more... 3D object tracking Loop closure (SLAM) Structure-from-Motion Semantic segmentation (class labels) 3D scene understanding (layout and objects) Use of maps OpenStreetMap The free Wiki World Map P. Lenz: The KITTI Vision Benchmark Suite 22
69 Related Datasets and Benchmarks Stereo / Optical Flow setting #images ground truth EISATS synthetic 500 dense Middlebury laboratory 40 dense Make3D Stereo real % Ladicky real 70 manual KITTI real % SLAM setting length metric TUM RGB-D indoor 0.4 km New College outdoor 2.2 km Malaga 2009 outdoor 6.4 km Ford Campus outdoor 5.1 km KITTI outdoor 39.2 km P. Lenz: The KITTI Vision Benchmark Suite 23
70 Related Datasets and Benchmarks Object Detection #cat. #labels/cat. occlusion 3D orientation Caltech MIT StreetScenes 9 3k LabelMe ETHZ Pedestrian 1 12k PASCAL k Daimler 1 56k Caltech Pedestrian 1 350k COIL discrete EPFL Multi-View discrete Caltech 3D discrete KITTI 3 1k - 40k continuous P. Lenz: The KITTI Vision Benchmark Suite 23
Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite
Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite Andreas Geiger and Philip Lenz Karlsruhe Institute of Technology {geiger,lenz}@kit.edu Raquel Urtasun Toyota Technological Institute
3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving
3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving AIT Austrian Institute of Technology Safety & Security Department Manfred Gruber Safe and Autonomous Systems
3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving
3D Vision An enabling Technology for Advanced Driver Assistance and Autonomous Offroad Driving AIT Austrian Institute of Technology Safety & Security Department Christian Zinner Safe and Autonomous Systems
Pedestrian Detection with RCNN
Pedestrian Detection with RCNN Matthew Chen Department of Computer Science Stanford University mcc17@stanford.edu Abstract In this paper we evaluate the effectiveness of using a Region-based Convolutional
Chapter 10: Third Working Phase
LEARNING AND INFERENCE IN GRAPHICAL MODELS Chapter 10: Third Working Phase Dr. Martin Lauer University of Freiburg Machine Learning Lab Karlsruhe Institute of Technology Institute of Measurement and Control
Vision meets Robotics: The KITTI Dataset
Geiger, Andreas ; Lenz, Philip ; Stiller, Christoph ; Urtasun, Raquel: Vision meets Robotics: The KITTI Dataset. 1 International Journal of Robotics Research (IJRR) 32 (213), pp. 1229 1235 Vision meets
Traffic Monitoring Systems. Technology and sensors
Traffic Monitoring Systems Technology and sensors Technology Inductive loops Cameras Lidar/Ladar and laser Radar GPS etc Inductive loops Inductive loops signals Inductive loop sensor The inductance signal
3D Model based Object Class Detection in An Arbitrary View
3D Model based Object Class Detection in An Arbitrary View Pingkun Yan, Saad M. Khan, Mubarak Shah School of Electrical Engineering and Computer Science University of Central Florida http://www.eecs.ucf.edu/
VEHICLE LOCALISATION AND CLASSIFICATION IN URBAN CCTV STREAMS
VEHICLE LOCALISATION AND CLASSIFICATION IN URBAN CCTV STREAMS Norbert Buch 1, Mark Cracknell 2, James Orwell 1 and Sergio A. Velastin 1 1. Kingston University, Penrhyn Road, Kingston upon Thames, KT1 2EE,
The KITTI-ROAD Evaluation Benchmark. for Road Detection Algorithms
The KITTI-ROAD Evaluation Benchmark for Road Detection Algorithms 08.06.2014 Jannik Fritsch Honda Research Institute Europe, Offenbach, Germany Presented material created together with Tobias Kuehnl Research
Object Recognition. Selim Aksoy. Bilkent University saksoy@cs.bilkent.edu.tr
Image Classification and Object Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Image classification Image (scene) classification is a fundamental
Point Cloud Simulation & Applications Maurice Fallon
Point Cloud & Applications Maurice Fallon Contributors: MIT: Hordur Johannsson and John Leonard U. of Salzburg: Michael Gschwandtner and Roland Kwitt Overview : Dense disparity information Efficient Image
Tracking and integrated navigation Konrad Schindler
Tracking and integrated navigation Konrad Schindler Institute of Geodesy and Photogrammetry Tracking Navigation needs predictions for dynamic objects estimate trajectories in 3D world coordinates and extrapolate
Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report
Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 69 Class Project Report Junhua Mao and Lunbo Xu University of California, Los Angeles mjhustc@ucla.edu and lunbo
Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection
CSED703R: Deep Learning for Visual Recognition (206S) Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection Bohyung Han Computer Vision Lab. bhhan@postech.ac.kr 2 3 Object detection
Automatic Labeling of Lane Markings for Autonomous Vehicles
Automatic Labeling of Lane Markings for Autonomous Vehicles Jeffrey Kiske Stanford University 450 Serra Mall, Stanford, CA 94305 jkiske@stanford.edu 1. Introduction As autonomous vehicles become more popular,
The Visual Internet of Things System Based on Depth Camera
The Visual Internet of Things System Based on Depth Camera Xucong Zhang 1, Xiaoyun Wang and Yingmin Jia Abstract The Visual Internet of Things is an important part of information technology. It is proposed
Robot Perception Continued
Robot Perception Continued 1 Visual Perception Visual Odometry Reconstruction Recognition CS 685 11 Range Sensing strategies Active range sensors Ultrasound Laser range sensor Slides adopted from Siegwart
Real-Time Tracking of Pedestrians and Vehicles
Real-Time Tracking of Pedestrians and Vehicles N.T. Siebel and S.J. Maybank. Computational Vision Group Department of Computer Science The University of Reading Reading RG6 6AY, England Abstract We present
Vehicle Tracking by Simultaneous Detection and Viewpoint Estimation
Vehicle Tracking by Simultaneous Detection and Viewpoint Estimation Ricardo Guerrero-Gómez-Olmedo, Roberto López-Sastre, Saturnino Maldonado-Bascón, and Antonio Fernández-Caballero 2 GRAM, Department of
Tracking in flussi video 3D. Ing. Samuele Salti
Seminari XXIII ciclo Tracking in flussi video 3D Ing. Tutors: Prof. Tullio Salmon Cinotti Prof. Luigi Di Stefano The Tracking problem Detection Object model, Track initiation, Track termination, Tracking
High speed 3D capture for Configuration Management DOE SBIR Phase II Paul Banks Paul.banks@tetravue.com
High speed 3D capture for Configuration Management DOE SBIR Phase II Paul Banks Paul.banks@tetravue.com Advanced Methods for Manufacturing Workshop September 29, 2015 1 TetraVue does high resolution 3D
Colorado School of Mines Computer Vision Professor William Hoff
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Introduction to 2 What is? A process that produces from images of the external world a description
automatic road sign detection from survey video
automatic road sign detection from survey video by paul stapleton gps4.us.com 10 ACSM BULLETIN february 2012 tech feature In the fields of road asset management and mapping for navigation, clients increasingly
T-REDSPEED White paper
T-REDSPEED White paper Index Index...2 Introduction...3 Specifications...4 Innovation...6 Technology added values...7 Introduction T-REDSPEED is an international patent pending technology for traffic violation
The goal is multiply object tracking by detection with application on pedestrians.
Coupled Detection and Trajectory Estimation for Multi-Object Tracking By B. Leibe, K. Schindler, L. Van Gool Presented By: Hanukaev Dmitri Lecturer: Prof. Daphna Wienshall The Goal The goal is multiply
Wii Remote Calibration Using the Sensor Bar
Wii Remote Calibration Using the Sensor Bar Alparslan Yildiz Abdullah Akay Yusuf Sinan Akgul GIT Vision Lab - http://vision.gyte.edu.tr Gebze Institute of Technology Kocaeli, Turkey {yildiz, akay, akgul}@bilmuh.gyte.edu.tr
Part-Based Pedestrian Detection and Tracking for Driver Assistance using two stage Classifier
International Journal of Research Studies in Science, Engineering and Technology [IJRSSET] Volume 1, Issue 4, July 2014, PP 10-17 ISSN 2349-4751 (Print) & ISSN 2349-476X (Online) Part-Based Pedestrian
Pallas Ludens. We inject human intelligence precisely where automation fails. Jonas Andrulis, Daniel Kondermann
Pallas Ludens We inject human intelligence precisely where automation fails Jonas Andrulis, Daniel Kondermann Chapter One: How it all started The Challenge of Reference and Training Data Generation Scientific
Optical Flow. Shenlong Wang CSC2541 Course Presentation Feb 2, 2016
Optical Flow Shenlong Wang CSC2541 Course Presentation Feb 2, 2016 Outline Introduction Variation Models Feature Matching Methods End-to-end Learning based Methods Discussion Optical Flow Goal: Pixel motion
Learning Detectors from Large Datasets for Object Retrieval in Video Surveillance
2012 IEEE International Conference on Multimedia and Expo Learning Detectors from Large Datasets for Object Retrieval in Video Surveillance Rogerio Feris, Sharath Pankanti IBM T. J. Watson Research Center
Behavior Analysis in Crowded Environments. XiaogangWang Department of Electronic Engineering The Chinese University of Hong Kong June 25, 2011
Behavior Analysis in Crowded Environments XiaogangWang Department of Electronic Engineering The Chinese University of Hong Kong June 25, 2011 Behavior Analysis in Sparse Scenes Zelnik-Manor & Irani CVPR
Announcements. Active stereo with structured light. Project structured light patterns onto the object
Announcements Active stereo with structured light Project 3 extension: Wednesday at noon Final project proposal extension: Friday at noon > consult with Steve, Rick, and/or Ian now! Project 2 artifact
3D Scanner using Line Laser. 1. Introduction. 2. Theory
. Introduction 3D Scanner using Line Laser Di Lu Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute The goal of 3D reconstruction is to recover the 3D properties of a geometric
Real-time Visual Tracker by Stream Processing
Real-time Visual Tracker by Stream Processing Simultaneous and Fast 3D Tracking of Multiple Faces in Video Sequences by Using a Particle Filter Oscar Mateo Lozano & Kuzahiro Otsuka presented by Piotr Rudol
Organizers: Cristóbal Curio (Reutlingen University, Germany) and Christoph Stiller (KIT, Germany)
Interaction of Automated Vehicles with other Traffic Participants ITSC 2015, Workshop 14, 15. September, Las Palmas, Gran Canaria Organizers: Cristóbal Curio (Reutlingen University, Germany) and Christoph
3D/4D acquisition. 3D acquisition taxonomy 22.10.2014. Computer Vision. Computer Vision. 3D acquisition methods. passive. active.
Das Bild kann zurzeit nicht angezeigt werden. 22.10.2014 3D/4D acquisition 3D acquisition taxonomy 3D acquisition methods passive active uni-directional multi-directional uni-directional multi-directional
Tracking performance evaluation on PETS 2015 Challenge datasets
Tracking performance evaluation on PETS 2015 Challenge datasets Tahir Nawaz, Jonathan Boyle, Longzhen Li and James Ferryman Computational Vision Group, School of Systems Engineering University of Reading,
Semantic Recognition: Object Detection and Scene Segmentation
Semantic Recognition: Object Detection and Scene Segmentation Xuming He xuming.he@nicta.com.au Computer Vision Research Group NICTA Robotic Vision Summer School 2015 Acknowledgement: Slides from Fei-Fei
Multi-view Intelligent Vehicle Surveillance System
Multi-view Intelligent Vehicle Surveillance System S. Denman, C. Fookes, J. Cook, C. Davoren, A. Mamic, G. Farquharson, D. Chen, B. Chen and S. Sridharan Image and Video Research Laboratory Queensland
Situated Visualization with Augmented Reality. Augmented Reality
, Austria 1 Augmented Reality Overlay computer graphics on real world Example application areas Tourist navigation Underground infrastructure Maintenance Games Simplify Qualcomm Vuforia [Wagner ISMAR 2008]
National Performance Evaluation Facility for LADARs
National Performance Evaluation Facility for LADARs Kamel S. Saidi (presenter) Geraldine S. Cheok William C. Stone The National Institute of Standards and Technology Construction Metrology and Automation
ACCURACY ASSESSMENT OF BUILDING POINT CLOUDS AUTOMATICALLY GENERATED FROM IPHONE IMAGES
ACCURACY ASSESSMENT OF BUILDING POINT CLOUDS AUTOMATICALLY GENERATED FROM IPHONE IMAGES B. Sirmacek, R. Lindenbergh Delft University of Technology, Department of Geoscience and Remote Sensing, Stevinweg
Practical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006
Practical Tour of Visual tracking David Fleet and Allan Jepson January, 2006 Designing a Visual Tracker: What is the state? pose and motion (position, velocity, acceleration, ) shape (size, deformation,
Crater detection with segmentation-based image processing algorithm
Template reference : 100181708K-EN Crater detection with segmentation-based image processing algorithm M. Spigai, S. Clerc (Thales Alenia Space-France) V. Simard-Bilodeau (U. Sherbrooke and NGC Aerospace,
LIBSVX and Video Segmentation Evaluation
CVPR 14 Tutorial! 1! LIBSVX and Video Segmentation Evaluation Chenliang Xu and Jason J. Corso!! Computer Science and Engineering! SUNY at Buffalo!! Electrical Engineering and Computer Science! University
Two-Frame Motion Estimation Based on Polynomial Expansion
Two-Frame Motion Estimation Based on Polynomial Expansion Gunnar Farnebäck Computer Vision Laboratory, Linköping University, SE-581 83 Linköping, Sweden gf@isy.liu.se http://www.isy.liu.se/cvl/ Abstract.
A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA
A PHOTOGRAMMETRIC APPRAOCH FOR AUTOMATIC TRAFFIC ASSESSMENT USING CONVENTIONAL CCTV CAMERA N. Zarrinpanjeh a, F. Dadrassjavan b, H. Fattahi c * a Islamic Azad University of Qazvin - nzarrin@qiau.ac.ir
Removing Moving Objects from Point Cloud Scenes
1 Removing Moving Objects from Point Cloud Scenes Krystof Litomisky klitomis@cs.ucr.edu Abstract. Three-dimensional simultaneous localization and mapping is a topic of significant interest in the research
Cloud Based Localization for Mobile Robot in Outdoors
11/12/13 15:11 Cloud Based Localization for Mobile Robot in Outdoors Xiaorui Zhu (presenter), Chunxin Qiu, Yulong Tao, Qi Jin Harbin Institute of Technology Shenzhen Graduate School, China 11/12/13 15:11
How does the Kinect work? John MacCormick
How does the Kinect work? John MacCormick Xbox demo Laptop demo The Kinect uses structured light and machine learning Inferring body position is a two-stage process: first compute a depth map (using structured
ARC 3D Webservice How to transform your images into 3D models. Maarten Vergauwen info@arc3d.be
ARC 3D Webservice How to transform your images into 3D models Maarten Vergauwen info@arc3d.be Overview What is it? How does it work? How do you use it? How to record images? Problems, tips and tricks Overview
Position Estimation Using Principal Components of Range Data
Position Estimation Using Principal Components of Range Data James L. Crowley, Frank Wallner and Bernt Schiele Project PRIMA-IMAG, INRIA Rhône-Alpes 655, avenue de l'europe 38330 Montbonnot Saint Martin,
Flow Separation for Fast and Robust Stereo Odometry
Flow Separation for Fast and Robust Stereo Odometry Michael Kaess, Kai Ni and Frank Dellaert Abstract Separating sparse flow provides fast and robust stereo visual odometry that deals with nearly degenerate
Vehicle Tracking System Robust to Changes in Environmental Conditions
INORMATION & COMMUNICATIONS Vehicle Tracking System Robust to Changes in Environmental Conditions Yasuo OGIUCHI*, Masakatsu HIGASHIKUBO, Kenji NISHIDA and Takio KURITA Driving Safety Support Systems (DSSS)
Tracking People and Their Objects
Tracking People and Their Objects Tobias Baumgartner Dennis Mitzel Bastian Leibe Computer Vision Group, RWTH Aachen University tobias.baumgartner@rwth-aachen.de, {mitzel,leibe}@vision.rwth-aachen.de Abstract
Robust Pedestrian Detection and Tracking From A Moving Vehicle
Robust Pedestrian Detection and Tracking From A Moving Vehicle Nguyen Xuan Tuong a, Thomas Müller b and Alois Knoll b a Department of Computer Engineering, Nanyang Technological University, Singapore b
Automatic parameter regulation for a tracking system with an auto-critical function
Automatic parameter regulation for a tracking system with an auto-critical function Daniela Hall INRIA Rhône-Alpes, St. Ismier, France Email: Daniela.Hall@inrialpes.fr Abstract In this article we propose
Towards License Plate Recognition: Comparying Moving Objects Segmentation Approaches
1 Towards License Plate Recognition: Comparying Moving Objects Segmentation Approaches V. J. Oliveira-Neto, G. Cámara-Chávez, D. Menotti UFOP - Federal University of Ouro Preto Computing Department Ouro
Multi-View Object Class Detection with a 3D Geometric Model
Multi-View Object Class Detection with a 3D Geometric Model Joerg Liebelt IW-SI, EADS Innovation Works D-81663 Munich, Germany joerg.liebelt@eads.net Cordelia Schmid LEAR, INRIA Grenoble F-38330 Montbonnot,
Limitations of Human Vision. What is computer vision? What is computer vision (cont d)?
What is computer vision? Limitations of Human Vision Slide 1 Computer vision (image understanding) is a discipline that studies how to reconstruct, interpret and understand a 3D scene from its 2D images
A Genetic Algorithm-Evolved 3D Point Cloud Descriptor
A Genetic Algorithm-Evolved 3D Point Cloud Descriptor Dominik Wȩgrzyn and Luís A. Alexandre IT - Instituto de Telecomunicações Dept. of Computer Science, Univ. Beira Interior, 6200-001 Covilhã, Portugal
Real-time 3D Scanning System for Pavement Distortion Inspection
Real-time 3D Scanning System for Pavement Distortion Inspection Bugao Xu, Ph.D. & Professor University of Texas at Austin Center for Transportation Research Austin, Texas 78712 Pavement distress categories
Mobile Robot FastSLAM with Xbox Kinect
Mobile Robot FastSLAM with Xbox Kinect Design Team Taylor Apgar, Sean Suri, Xiangdong Xi Design Advisor Prof. Greg Kowalski Abstract Mapping is an interesting and difficult problem in robotics. In order
3D Vehicle Extraction and Tracking from Multiple Viewpoints for Traffic Monitoring by using Probability Fusion Map
Electronic Letters on Computer Vision and Image Analysis 7(2):110-119, 2008 3D Vehicle Extraction and Tracking from Multiple Viewpoints for Traffic Monitoring by using Probability Fusion Map Zhencheng
EFFICIENT VEHICLE TRACKING AND CLASSIFICATION FOR AN AUTOMATED TRAFFIC SURVEILLANCE SYSTEM
EFFICIENT VEHICLE TRACKING AND CLASSIFICATION FOR AN AUTOMATED TRAFFIC SURVEILLANCE SYSTEM Amol Ambardekar, Mircea Nicolescu, and George Bebis Department of Computer Science and Engineering University
Convolutional Feature Maps
Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection Kaiming He Microsoft Research Asia (MSRA) ICCV 2015 Tutorial on Tools for Efficient Object Detection Overview
LabelMe: Online Image Annotation and Applications
INVITED PAPER LabelMe: Online Image Annotation and Applications By developing a publicly available tool that allows users to use the Internet to quickly and easily annotate images, the authors were able
Machine vision systems - 2
Machine vision systems Problem definition Image acquisition Image segmentation Connected component analysis Machine vision systems - 1 Problem definition Design a vision system to see a flat world Page
Online Learning for Offroad Robots: Using Spatial Label Propagation to Learn Long Range Traversability
Online Learning for Offroad Robots: Using Spatial Label Propagation to Learn Long Range Traversability Raia Hadsell1, Pierre Sermanet1,2, Jan Ben2, Ayse Naz Erkan1, Jefferson Han1, Beat Flepp2, Urs Muller2,
A Robust Multiple Object Tracking for Sport Applications 1) Thomas Mauthner, Horst Bischof
A Robust Multiple Object Tracking for Sport Applications 1) Thomas Mauthner, Horst Bischof Institute for Computer Graphics and Vision Graz University of Technology, Austria {mauthner,bischof}@icg.tu-graz.ac.at
Interactive person re-identification in TV series
Interactive person re-identification in TV series Mika Fischer Hazım Kemal Ekenel Rainer Stiefelhagen CV:HCI lab, Karlsruhe Institute of Technology Adenauerring 2, 76131 Karlsruhe, Germany E-mail: {mika.fischer,ekenel,rainer.stiefelhagen}@kit.edu
Blender 3D Animation
Bachelor Maths/Physics/Computer Science University Paris-Sud Digital Imaging Course Blender 3D Animation Christian Jacquemin Introduction to Computer Animation Animation Basics animation consists in changing
Synthetic Sensing: Proximity / Distance Sensors
Synthetic Sensing: Proximity / Distance Sensors MediaRobotics Lab, February 2010 Proximity detection is dependent on the object of interest. One size does not fit all For non-contact distance measurement,
Can we calibrate a camera using an image of a flat, textureless Lambertian surface?
Can we calibrate a camera using an image of a flat, textureless Lambertian surface? Sing Bing Kang 1 and Richard Weiss 2 1 Cambridge Research Laboratory, Compaq Computer Corporation, One Kendall Sqr.,
Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery
Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery Abstract. The recovery of tissue structure and morphology during robotic assisted surgery is an important step towards
VECTORAL IMAGING THE NEW DIRECTION IN AUTOMATED OPTICAL INSPECTION
VECTORAL IMAGING THE NEW DIRECTION IN AUTOMATED OPTICAL INSPECTION Mark J. Norris Vision Inspection Technology, LLC Haverhill, MA mnorris@vitechnology.com ABSTRACT Traditional methods of identifying and
The Advantages of Using a Fixed Stereo Vision sensor
Proc. of International Conference on Industrial & Engineering Applications of Artificial Intelligence & Expert Systems (IEA/AIE), 2005 Real-Time People Localization and Tracking through Fixed Stereo Vision
Feature Tracking and Optical Flow
02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve
Sensor Modeling for a Walking Robot Simulation. 1 Introduction
Sensor Modeling for a Walking Robot Simulation L. France, A. Girault, J-D. Gascuel, B. Espiau INRIA, Grenoble, FRANCE imagis, GRAVIR/IMAG, Grenoble, FRANCE Abstract This paper proposes models of short-range
Robotics. Lecture 3: Sensors. See course website http://www.doc.ic.ac.uk/~ajd/robotics/ for up to date information.
Robotics Lecture 3: Sensors See course website http://www.doc.ic.ac.uk/~ajd/robotics/ for up to date information. Andrew Davison Department of Computing Imperial College London Review: Locomotion Practical
Discovering objects and their location in images
Discovering objects and their location in images Josef Sivic Bryan C. Russell Alexei A. Efros Andrew Zisserman William T. Freeman Dept. of Engineering Science CS and AI Laboratory School of Computer Science
Aguantebocawertyuiopasdfghvdslpmj klzxcvbquieromuchoanachaguilleanic oyafernmqwertyuiopasdfghjklzxcvbn mqwertyuiopasdfghjklzxcvbnmqwerty
Aguantebocawertyuiopasdfghvdslpmj klzxcvbquieromuchoanachaguilleanic oyafernmqwertyuiopasdfghjklzxcvbn mqwertyuiopasdfghjklzxcvbnmqwerty GOOGLE CAR uiopasdfghjklzxcvbnmqwertyuiopasdf Gonzalo Ghigliazza
Open-Set Face Recognition-based Visitor Interface System
Open-Set Face Recognition-based Visitor Interface System Hazım K. Ekenel, Lorant Szasz-Toth, and Rainer Stiefelhagen Computer Science Department, Universität Karlsruhe (TH) Am Fasanengarten 5, Karlsruhe
SIMULTANEOUS LOCALIZATION AND MAPPING FOR EVENT-BASED VISION SYSTEMS
SIMULTANEOUS LOCALIZATION AND MAPPING FOR EVENT-BASED VISION SYSTEMS David Weikersdorfer, Raoul Hoffmann, Jörg Conradt 9-th International Conference on Computer Vision Systems 18-th July 2013, St. Petersburg,
MetropoGIS: A City Modeling System DI Dr. Konrad KARNER, DI Andreas KLAUS, DI Joachim BAUER, DI Christopher ZACH
MetropoGIS: A City Modeling System DI Dr. Konrad KARNER, DI Andreas KLAUS, DI Joachim BAUER, DI Christopher ZACH VRVis Research Center for Virtual Reality and Visualization, Virtual Habitat, Inffeldgasse
ROBUST VEHICLE TRACKING IN VIDEO IMAGES BEING TAKEN FROM A HELICOPTER
ROBUST VEHICLE TRACKING IN VIDEO IMAGES BEING TAKEN FROM A HELICOPTER Fatemeh Karimi Nejadasl, Ben G.H. Gorte, and Serge P. Hoogendoorn Institute of Earth Observation and Space System, Delft University
What more can we do with videos?
What more can we do with videos? Occlusion and Mo6on Reasoning for Tracking & Human Pose Es6ma6on Karteek Alahari Inria Grenoble Rhone- Alpes Joint work with Anoop Cherian, Yang Hua, Julien Mairal, Cordelia
Snapshot of the online application for image annotation.
1 LabelMe: online image annotation and applications Antonio Torralba, Bryan C. Russell, and Jenny Yuen Abstract Central to the development of computer vision systems is the collection and use of annotated
A Methodology for Obtaining Super-Resolution Images and Depth Maps from RGB-D Data
A Methodology for Obtaining Super-Resolution Images and Depth Maps from RGB-D Data Daniel B. Mesquita, Mario F. M. Campos, Erickson R. Nascimento Computer Science Department Universidade Federal de Minas
INTRODUCTION TO RENDERING TECHNIQUES
INTRODUCTION TO RENDERING TECHNIQUES 22 Mar. 212 Yanir Kleiman What is 3D Graphics? Why 3D? Draw one frame at a time Model only once X 24 frames per second Color / texture only once 15, frames for a feature
Urban Vehicle Tracking using a Combined 3D Model Detector and Classifier
Urban Vehicle Tracing using a Combined 3D Model Detector and Classifier Norbert Buch, Fei Yin, James Orwell, Dimitrios Maris and Sergio A. Velastin Digital Imaging Research Centre, Kingston University,
Local features and matching. Image classification & object localization
Overview Instance level search Local features and matching Efficient visual recognition Image classification & object localization Category recognition Image classification: assigning a class label to
THE CONTROL OF A ROBOT END-EFFECTOR USING PHOTOGRAMMETRY
THE CONTROL OF A ROBOT END-EFFECTOR USING PHOTOGRAMMETRY Dr. T. Clarke & Dr. X. Wang Optical Metrology Centre, City University, Northampton Square, London, EC1V 0HB, UK t.a.clarke@city.ac.uk, x.wang@city.ac.uk
Finding people in repeated shots of the same scene
Finding people in repeated shots of the same scene Josef Sivic 1 C. Lawrence Zitnick Richard Szeliski 1 University of Oxford Microsoft Research Abstract The goal of this work is to find all occurrences
Module 5. Deep Convnets for Local Recognition Joost van de Weijer 4 April 2016
Module 5 Deep Convnets for Local Recognition Joost van de Weijer 4 April 2016 Previously, end-to-end.. Dog Slide credit: Jose M 2 Previously, end-to-end.. Dog Learned Representation Slide credit: Jose
Digital Image Increase
Exploiting redundancy for reliable aerial computer vision 1 Digital Image Increase 2 Images Worldwide 3 Terrestrial Image Acquisition 4 Aerial Photogrammetry 5 New Sensor Platforms Towards Fully Automatic
PHOTOGRAMMETRIC TECHNIQUES FOR MEASUREMENTS IN WOODWORKING INDUSTRY
PHOTOGRAMMETRIC TECHNIQUES FOR MEASUREMENTS IN WOODWORKING INDUSTRY V. Knyaz a, *, Yu. Visilter, S. Zheltov a State Research Institute for Aviation System (GosNIIAS), 7, Victorenko str., Moscow, Russia
Recognizing Cats and Dogs with Shape and Appearance based Models. Group Member: Chu Wang, Landu Jiang
Recognizing Cats and Dogs with Shape and Appearance based Models Group Member: Chu Wang, Landu Jiang Abstract Recognizing cats and dogs from images is a challenging competition raised by Kaggle platform
Aligning 3D Models to RGB-D Images of Cluttered Scenes
Aligning 3D Models to RGB-D Images of Cluttered Scenes Saurabh Gupta Pablo Arbeláez Ross Girshick Jitendra Malik UC Berkeley Universidad de los Andes, Colombia Microsoft Research UC Berkeley sgupta@ pa.arbelaez@