Matching of Satellite, Aerial and Close-range Images

Transcription

1 Matching of Satellite, Aerial and Close-range Images Luigi Barazzetti Politecnico di Milano, ABC Departement Lab. via Ponzio 31, Milan Lab. IC&T, Corso Premessi Sposi, Lecco -

2 2 The image matching problem What stuff in the left image matches with stuff on the right? Stuff what does it mean? points, lines, areas,?

3 3 Image acquisition platforms (Photogrammetry and Remote Sensing) Satellite images > 700 km Aerial images UAVs Close-range images Waterproof equipment < 1 m

4 4 Matching is easy! Why? Point of view Illumination Good texture Absence of occlusions Figure by Noah Snavely

5 5 Harder case! Why? An important consideration: human operators can easily extract corresponding points, objects, Figure by Noah Snavely

6 Introduzione al controllo statico delle strutture 6 Harder? (Nasa Mars Rover images) Figure by Noah Snavely

7 7 Harder? SIFT key-points (automatic) Figure by Noah Snavely

8 8 Applications Photogrammetry and Remote Sensing fiducial mark measurement (target detection and matching) tie point extraction DTM/DSM generation image registration Computer Vision object recognition multiple view analysis (3D reconstruction, panoramic images, HDR images, medical data alignment, ) motion capture, object tracking too many to be exhaustively listed

9 9 Motion capture: tracking a limited number of points

10 10 2.5D vs 3D models from aerial images

11 11 Dense image matching and oblique imagery

12 12 Example: Apple ios Maps (the Cathedral of Milan)

13 13 Example: Apple ios Maps The Statue of Liberty that was missing before now shows a 3D imagery of the landmark and the details of the structures surrounding it

14 14 Shaded and textured model

15 15 Point matching Goal: extraction of good points How can we define good and bad candidates? Moving the window in any direction gives a big change flat region: no change in all directions edge : no change along the edge direction corner : significant change in all directions Slide adapted from Darya Frolova, Denis Simakov, and Noah Snavely

16 16 Classification of point matching techniques Intensity-based matching: image data is used in form of grey values. Most prominent methods are cross-correlation and least squares matching (LS-matching). Also called "area-based" matching. Give sub-pixel accuracy, in extreme cases 1/100 pixel and better Feature-based matching: requires the extraction of basic image features, like blobs, corners, junctions, edges, etc. first. Matching is performed between these features. Features are sometimes more stable with regard to reflectance characteristics Relational matching: uses geometrical or other relations between features and structures (combination of features). Correspondence is established by tree-search techniques. These methods are not very accurate but usually robust

17 17 Relational matching: uses geometrical or other relations between features and structures (combination of features). These methods are not very accurate but usually robust. They do not require good approximation Structural description: set of primitives and their inter-relationships

18 18 Intensity-based matching: Normalized Cross-Correlation The solution (best match) is found at max(r(x, y)). r is limited to the region [-1, 1]. False or weak matches are indicated by small (r 0,5). However, large r do not always indicate good, stable matches (e.g. in case of multiple solutions or in cases of weak signal in template and patch) r =

19 19 Feature-based matching: Foerstner operator

20 Feature-based matching: modern operators New trend based on local features with detectors and descriptors Computer Vision methods: their use in Photogrammetry and RS is attractive Example with the SIFT operator: panoramic photography Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2:

21 Feature-based matching: modern operators Picture with a Compact Camera (FOV) 50 x 35 Do you want a 360 reconstruction? Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2:

22 Feature-based matching: modern operators Several pictures with a Compact Camera (FOV) Do you want a 360 reconstruction? Panoramic images (panoramas) Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2:

23 plus constraints (geometry) Take pictures with a rotating camera

24 Feature-based matching: modern operators Extract SIFT features: detector / descriptor Geometrically invariant to similarity transforms Some robustness to affine change Automated methods outliers!!! Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2:

25 Feature-based matching: modern operators Outlier rejection is fundamental RANSAC LS line

26 Feature-based matching: modern operators Outlier rejection is fundamental RANSAC RANSAC line

27 Feature-based matching: modern operators Outlier rejection is fundamental RANSAC (homography)

28 Feature-based matching: modern operators Final radiometric correction Extension to N images and final blending Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2:

29 Feature-based matching and satellite images Similar approach but different operator (SURF) Input data over Las Vegas: 1 ASTER image 5627x5001 pixels resolution 15 m 1 LANSAT TM image 7811x7011 pixels resolution 30 m

30 Feature-based matching and satellite images Image interest points Image interest points Descriptor comparison 502 matches (outliers??)

31 Feature-based matching and satellite images Hypothesis: similarity transformation Xn a b Xo c cosα sinα Xo DXo * S * * Yn = + = + b a Yo d sinα cosα Yo DYo Y0 Yn Yn Y0 Yn X0 Y0 Xn Xn X0 Xn X0 Translation Scaling Rotation

32 Feature-based matching and satellite images RANSAC: 395 inliers (from 502 matches) Final Least Squares adjustment

33 References Brown, M. and Lowe, D.G., Recognizing Panoramas. International Conference on Computer Vision, 2: Brown, M. and Lowe, D.G., Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1): Gruen, A., Adaptative least squares correlation: a powerful image matching technique. South African Journal of Photogrammetry, Remote Sensing and Cartography, 14(3): Gruen A., Geomatics in the 21th Century State of the art and future perspectives, Course at Politecnico di Torino Gruen A., Image Matching for DSM Generation, Compact Course at Politecnico di Milano Some slides and pictures from A. Gruen, M. Scaioni, S. Seitz, R. Szeliski, N. Snavely, M Brown, D. Lowe, D. Frolova, D. Simakov