From Ideas to Innovation

Transcription

1 From Ideas to Innovation Selected Applications from the CRC Research Lab in Advanced Geomatics Image Processing Dr. Yun Zhang Canada Research Chair Laboratory in Advanced Geomatics Image Processing (CRC-AGIP Lab) Department of Geodesy and Geomatics Engineering (GGE) University of New Brunswick (UNB) CRC-AGIP 1

2 Selected Research Applications (1) 3D Building Mapping: Optical-Elevation Data Co-registration - Alaeldin Suliman (PhD Candidate), Dr. Yun Zhang (2) 3D Building Mapping: Off-terrain Objects Disparity Mapping - Alaeldin Suliman (PhD Candidate), Dr. Yun Zhang (3) EarthView3D Seeing the Earth in Stereo - Dr. Yun Zhang, David Fraser (MScE), William Liu (MScE), and others (4) Multi-Epipolar Images of VHR Satellite images with Elevation Proportional Disparities for Several Products - Alaeldin Suliman (PhD Candidate), Dr. Yun Zhang (5) Urban Change Detection using off Nadir Images - Dr. Shabnam Jabari (Post-Doctoral Fellow), Dr. Yun Zhang (6) Target Detection Using High Resolution Panchromatic Images - Mohammed Rezaee (PhD Candidate), Dr. Yun Zhang 2

3 Registration Using Conventional Registration Using (1) Optical-Elevation Data Co-registration for Better 3D Representation and Building Mapping Optical Data Elevation Data Registration Result 3D Representation Conventional Approach Ortho DSM Off-nadir VHR imagery Conventional DSM Misregistration 3D Representation of the integrated datasets CRC-AGIP Approach line-of-sight DSM Off-nadir VHR imagery line-of-sight DSM Accurate Registration 3D Representation of the integrated datasets Description: The algorithm uses sensor information to adjust the orthographic DSM to have the perspective characteristics of any off-nadir and perspective imagery and also allows for sub-pixel level registration. The algorithm provides an improvement in rooftop building detection of almost 12% in comparing to conventional techniques (without incorporating sensor model info.). Improvement is due to the accurate registration achieved. This algorithm is robust and it is currently not available in any commercial software product.

4 (2) Off-terrain Objects Disparity Mapping for Better 3D Representation and Building Detection Conventional Approach CRC-AGIP Approach Matching-based Stereo information Filling occlusion gaps by interpolation Stereo VHR images over dense urban area Epipolar rectified images followed by registrationbased terrain subtraction to keep off-terrain objects DSM Extracted Terrain by conventional techniques DTM Off-terrain Disparity Map Mapped Buildings Missed off-terrain objects Normalized DSM 3D Disparity representation (flat terrain) Description: The algorithm forces the matching techniques to map only the disparities of building rooftops by subtracting the terrain level features in the image space. It also takes the advantages of along-track stereo images of opposite directions to exclude building facades. The product of this algorithm is a disparity map that contains only the building rooftops. The algorithm is well suited for dense urban areas since the terrain variation is reasonable.

5 (3) EarthView3D - Viewing the Earth s Surface in Stereo EarthView3D (Anaglyph Mode): Figure shows an anaglyph image of Honolulu, USA. Description: In Fall 2015, CRC-AGIP lab launched EarthView3D - an online application for viewing satellite images and line maps in 3D. With EarthView3D, we can use 3D glasses (active shutter glasses or anaglyph glasses) to see the earth s surface. The approach used by EarthView3D is different than that used by applications such as Sketchup and CityEngine where monocular cues (e.g. relative size, linear perspective, etc.) are used to give a perspective a 3D perception. Continued on next slide. 5

6 (3) EarthView3D, cont. Available for download from EarthView3D.net Landsat 7/8 Stereo Coverage - Nepal, US and Canada - Coming soon: - Europe (Jan 2016), - Asia, South America, (2016/2017) NAIP Stereo Coverage (early Nov 2015) - Sections of US/Mexico border EarthView 3D NAIP Mosaic Arizona/Mexico border Sample Applications - Resource Management - Environmental Monitoring - Security - Site Planning - Education - Grand Canyon Proposed route in Google Earth of the PennEast Pipeline near the Swan Creek reserve. Integration with ArcGIS - Possible, currently not under development - Display stereo images in ArcMap and ArcScene Vancouver, Canada 6

7 (4) Multi-Epipolar Images of VHR Satellite images with Elevation Proportional Disparities for Several Products In Progress A Epipolar images A Same pixels lie on the same row B C This property is useful for generating: C Proportional Disparity Shown by Straight lines Shifted rooftop in offnadir imagery Corrected rooftop location 1. Dense disparity map 2. One-Step Dense DSM 3. 3-D Representation 4. True-orthoimage via disparity info. Description: The algorithm utilizes sensor information to construct epipolar images of multi along-track stereo images. The images are processed to develop disparity values proportional to each other as well as to the ground elevations. This robust algorithm tremendously facilitates the following: (1) Disparity values transferability from image domain to another one. (2) Occlusion gap-filling for and mismatches detection and correction. (3) Direct DSM generation without triangulation and bundle adjustment. (4) True Ortho-image generation by disparity data (without DSM ).

8 (5) Urban Change Detection Using off Nadir Images Co-registration of off-nadir Satellite and Airborne images using a Patch-Wise Co-Registration (PWCR) approach Co-registration Problem For change detection: Original Segmentation in base image Segments transferred to the target image using conventional method (using the segments from the base image as thematic layer) Segments transferred to the target image using our PWCR Description: The main problem in urban change detection using off-nadir satellite or airborne imagery is to find similar objects in bi-temporal images (base image and target image) regardless of the attitudes and acquisition angles of the imagery. This task is challenging due to relief displacements of elevated objects towards different directions in different imagery. We proposed a Patch-Wise Co-Registration (PWCR) method to overcome this problem and to find the corresponding objects in bi/multi-temporal images. We tested the PWCR method on IKONOS, QuickBird, GeoEye-1, WV-2, and Airborne imagery and the results are very consistent which shows the robustness of the work. Continued on next slide.

9 (5) Urban Change Detection Using off Nadir Images, cont. Co-registration Results Image 1: Samples of urban buildings Imagery Courtesy Google Earth Image 2: WorldView2 image acquired in 2011 Off-Nadir Angle: 17 Image 3: WorldView2 image acquired in 2013 Off-Nadir Angle: 27 Image 4: Airborne image acquired in 2005 In the above images, Image 1 shows 3D representations of sample buildings in Google Earth imagery. Image 2 is the borders of the building roof generated manually in a WV-2 image. We used this image as a base image for change detection. Using PWCR, the building roof borders are transferred to a high off-nadir satellite image (Image 3) and an airborne image (Image 4). These latter images are target images so that the transferred borders fit the actual building borders closely. These Images highlight the ability of the PWCR method to find the exact position of the objects (in this case building roof) regardless of the attitude and geometry of the images. As can be seen, there is no change between Image 3 (target) and Image 2 (base images), but one of the buildings in image 4 is a new construction and the borders of the roof fall on the bare ground since the building was not built yet. A comparison method such as correlation Analysis or MAD Transform can then be applied to these results to determine whether or not an object has changed. 9

10 (6) Complex-shape Target Detection Using High Resolution Panchromatic Images Start with a database that contains one or more images of the desired target (e.g. airplane) Generate a dictionary for that target Find the location of the target (e.g. airplanes) in different images that is, search a collection of images for matches with the information contained in the dictionary Database Airplane & Background Dictionary Result Continued on next slide.

11 (6) Complex-shape Target Detection Using High Resolution Panchromatic Images, cont. Methodology used to create target dictionary Image(s) Extracted features descriptors Feature extraction Advantages Invariant to rotation and scale Dictionary is saveable for further use Applicable to transportation (Ship, plane) Resistant to occlusion Quantization (coding) Codes Spatial Pyramid Pooling Concatenation Detection histogram Features with high frequency Features with low frequency