SIMULTANEOUS LOCALIZATION AND MAPPING FOR EVENT-BASED VISION SYSTEMS David Weikersdorfer, Raoul Hoffmann, Jörg Conradt 9-th International Conference on Computer Vision Systems 18-th July 2013, St. Petersburg, Russia
Outline 1. Event-based Vision with Dynamic Vision Sensors 2. Event-based Particle Filtering 3. Event-based Simultaneous Localization and Mapping 4. Applications and Results 2013/07/18 Page 2
Classic vs. Event-based Vision 2013/07/18 Page 3
Event-based Vision Pixels individually react to change in illumination Stream of pixel coordinates No frames! Sparse data stream Efficient algorithms Page 4
Dynamic Vision Sensors
Dynamic Bayesian Network Temporal development of random variable Based on observations void Page 6
Dynamic Bayesian Network Temporal development of random variable Based on observations void Condensation Algorithm (Isard and Black, 1998) Page 7
Dynamic Bayesian Network Temporal development of random variable Based on observations void Condensation Algorithm (Isard and Black, 1998) Page 8
Event-based Particle Filtering Event-based vision from edvs Page 9
Event-based Particle Filtering Problem 1: Too many measurements ~20,000 events/sec Page 10
Event-based Particle Filtering Problem 1: Too many measurements ~20,000 events/sec Solution: Do not resample after every event Page 11
Event-based Particle Filtering Problem 2: Measurements not independent Page 12
Event-based Particle Filtering Problem 2: Measurements not independent Solution: Use exponential decay Page 13
Event-based Particle Filtering Problem 2: Measurements not independent Solution: Use exponential decay D. Weikersdorfer, J. Conradt, Event-based particle filtering for robot self-localization, ROBIO 2012 Page 14
Event-based SLAM Simultaneous Localization and... Page 15
Event-based SLAM Simultaneous Localization and Mapping Page 16
Event-based Mapping Dynamic map over 2D map space : Occurrence map : Normalization map Occurrence map Normalization map Final map Page 17
Event-based Mapping Dynamic map over 2D map space Occurrence map: : Gaussian normal distribution : Projection of event onto map given a state estimate Occurrence map Normalization map Final map Page 18
Event-based Mapping Dynamic map over 2D map space Occurrence map: : Gaussian normal distribution : Projection of event onto map given a state estimate Occurrence map Normalization map Final map Page 19
Event-based Mapping Dynamic map over 2D map space Normalization map: : Total state estimate at step k : Projection of map location onto retina given a state Occurrence map Normalization map Final map Page 20
Event-based Mapping Dynamic map over 2D map space Normalization map: : Total state estimate at step k : Projection of map location onto retina given a state Occurrence map Normalization map Final map Page 21
Results: Event-based SLAM (2D) Results from event-based SLAM Comparison to ground truth (Red: estimated path, Blue: overhead tracker) Position and rotation error Page 22
Results: Event-based SLAM (2D) Results from event-based SLAM Map estimate from SLAM Comparison to ground truth Map comparison Position and rotation error Map ground truth (photo of ceiling) Page 23
Results: Event-based SLAM (2D) Page 24
Results: Autonomous Exploration Page 25
Conclusion Dynamic Vision Sensors Sparse data stream of pixel events Event-based Particle Filtering Adaptation of Condensation to event-based vision Event-based SLAM Low bandwidth for embedded systems 2013/07/18 Page 26
References D. Weikersdorfer, J. Conradt, Event-based particle filtering for robot selflocalization, ROBIO 2012 D. Weikersdorfer, R. Hoffmann, J. Conradt, Simultaneous Localization and Mapping for event-based Vision Systems, ICVS 2013 R. Hoffmann, D. Weikersdorfer, J. Conradt, Autonomous Indoor Exploration with an Event-Based Visual SLAM System, ECMR 2013 Contact: David Weikersdorfer, weikersd@in.tum.de Neuroscientific System Theory (NST) 2013/07/18 Page 27