Fast field survey with a smartphone

Transcription

1 Fast field survey with a smartphone A. Masiero F. Fissore, F. Pirotti, A. Guarnieri, A. Vettore CIRGEO Interdept. Research Center of Geomatics University of Padova Italy cirgeo@unipd.it 1

2 Mobile Mapping with Smartphones Use of embedded sensors: Camera is used as imaging sensor of the observed environment via photogrammetry (e.g. SfM) Device position estimated by integrating information provided by the embedded sensors spatial referring - Low cost, fast w.r. to other techniques (e.g. TLS) - Limited resources: stringent restrictions on the computational power, limited battery life... Goal: exploit information provided by the navigation system to improve the reconstruction procedure 2 2

3 Navigation system achieved by integrating information: GNSS inertial sensors (embedded in the device, they provide good local estimates of position variations but drift in long time intervals if used alone) WiFi signal strength Barometer Geometry of the environment Nonlinear filtering 3 3

4 Particle filtering Information fusion of PDR (Pedestrian Dead Reckoning), WiFi, building map... Particle filtering Device position is expressed as average position of N particles v u Dynamic equation of each particle: qt+i =q t +s t [ ] sin αt cos αt it exploits measured step length and heading direction 4 4

5 Particle filtering Advantage: simple to introduce non-linear constraints (and to deal with multiple hypothesis) in position estimation Neglegted, and resampled High accuracy for large N, but computational burden issues! [Masiero 2014] proposed a revised version of [Widyawan 2012] in order to increase accuracy for small N (N 100) and uncalibrated sensors For further accuracy improvement: - good sensor calibration - exploiting landmarks 5 5

6 Particle filtering Information fusion of PDR (Pedestrian Dead Reckoning), WiFi, building map... Particle filtering - [Widyawan 2012]: Particle filter for PDR - [Masiero 2014]: revised version of the particle filter in [Widyawan 2012] in order to increase accuracy for small N (number of particles N 100) and uncalibrated sensors Magnetometer & accelerometer simultaneous calibration [Masiero MMT2015] Barometer altitude variation - linear model to describe the relation between pressure and altitude variations (precision 0.2m). 6 6

7 3D photogrammetric reconstruction Reconstruction outline Compute feature locations (e.g. Harris feature detector) Compute feature descriptors (e.g. SIFT) Feature matching (Best Bin First Kd tree search) Remove outliers (epipolar geometry, RANSAC or its variants) Bundle adjustment (optimize parameter values) Projective reconstruction Control points are used to obtain Euclidean reconstruction and for georeferencing 7 7

8 3D photogrammetric reconstruction Reconstruction outline Compute feature locations (e.g. Harris feature detector) Compute feature descriptors Feature matching (Best Bin First Kd tree search) Remove outliers (epipolar geometry, RANSAC or its variants) Bundle adjustment (optimize parameter values) take into account of affine transformations Projective reconstruction Control points are used to obtain Euclidean reconstruction and for georeferencing 8 8

9 Feature matching Typically done by using SIFT (Scale-invariant feature transform, [Lowe 1999]) matchings [Vedaldi 2008] SIFT deals well with rotations with respect to rotations along the optical axis 9 9

10 Feature matching However, issues can occur when considering other rotations (as typical with generic changes of the point of view) 10 10

11 Feature matching ASIFT [Morel 2011] increases SIFT robustness with respect to such rotations by modelling their effect by means of affine transformations. However, in ASIFT 32 affine transformations of each feature are computed comparisons between each couple of features in two different images. Goal: reducing computational complexity of ASIFT while ensuring increase of matchings with respect to SIFT in the critical cases (e.g. previously described changes of the point of view...) 11 11

12 Feature matching Appearance of a feature seen by camera j depends on the point of view and on the spatial orientation of the feature Information by the navigation system change of the point of view transformation (translation + rotation) approximately known Uncertainty in the spatial orientation of the feature 12 12

13 Feature matching Image plane Surface of the real object Image plane 13 13

14 Feature matching Appearance of a feature seen by camera j depends on the point of view and on the spatial orientation of the feature Information by the navigation system change of the point of view transformation (translation + rotation) approximately known Uncertainty in the spatial orientation of the feature Compensate for this uncertainty by simulating the effect of 20 possible orientations (on a semi-sphere...) Thanks to information provided by the navigation system: - ASIFT: comparisons (per feature couple) - Our approach: 20 comparisons (per feature couple) 14 14

15 Matches with SIFT Images of this example available from the internet [Lhuillier and Quan, 2005] 15 15

16 Matches with the proposed method Images of this example available from the internet [Lhuillier and Quan, 2005] 16 16

17 Number of correct matches vs (difference of) observation angle SIFT: Blue x-marks Our approach: red circles 17 17

18 3D photogrammetric reconstruction Reconstruction outline Compute feature locations (e.g. Harris feature detector) Compute feature descriptors Feature matching Remove outliers (epipolar geometry, RANSAC or its variants) Bundle adjustment (optimize parameter values) take into account of affine transformations Use approximate epipolar constraints to discard false matchings Projective reconstruction Control points are used to obtain Euclidean reconstruction and for georeferencing 18 18

19 3D photogrammetric reconstruction Reconstruction outline Compute feature locations (e.g. Harris feature detector) Compute feature descriptors Feature matching Remove outliers (epipolar geometry, RANSAC or its variants) Bundle adjustment (optimize parameter values) Control points take into account of affine transformations Use approximate epipolar constraints to discard false matchings Use positions provided by the navigation system and calibrated camera to ease/improve Euclidian reconstruction 19 19

20 Bibliography - Hol, Sensor Fusion and Calibration of Inertial Sensors, Vision, Ultra-Wideband and GPS. PhD. Thesis, Linkoping University, The Institute of Technology. - Masiero et al, A particle filter for smartphone-based indoor pedestrian navigation. Micromachines 5(4), pp Masiero and Vettore, Towards mobile mapping with smartphones. MMT Morel and Yu, Is SIFT scale invariant? Inverse Problems and Imaging 5(1), pp Lhuillier and Quan, A quasi-dense approach to surface reconstruction from uncalibrated images. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(3), Liu et al, Novel calibration algorithm for a three-axis strapdown magnetometer. Sensors 14(5), pp Lowe, Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV), Vol. 2, pp vol.2. - Vedaldi and Fulkerson, VLFeat: An open and portable library of computer vision algorithms. - Widyawan et al, Virtual lifeline: Multimodal sensor data fusion for robust navigation in unknown environments. Pervasive and Mobile Computing 8(3), pp

21 1) Estimation of the structure of the scene: camera positions and 3D positions of certain features 1a) More robust estimation results by exploiting a priori information provided by the navigation system on camera positions (and orientations) 1b) Feature matching for estimating geometry of the scene: feature matching issues when points of view are quite different. Goal: provides more robust feature matching (and more feature matches) 2) Dense reconstruction: (usually) greed algorithm for increasing the point cloud based on local matches 21 21

22 Selection of orientation based on information by the navigation system 22 22

23 Calibration with bundle adjustment Convex optimization methods based on photogrammetry - Feature extraction Feature matching Reconstruction via triangulation (Bundle adjustment) From CRCSI webpage 23 23