Unmanned Airship Based High Resolution Images Acquisition and the Processing

Unmanned Airship Based High Resolution Images Acquisition and the Processing 1 Qian Yang, 2 Shengbo Chen *, 3 Peng Lu, 4 Mingchang Wang, 5 Qiong Wu, 6 Chao Zhou, 7 Yanli Liu 1 College of Geo-Exploration Science and Technology, Jilin University,yangqian10@mails.jlu.edu.cn *2 College of Geo-Exploration Science and Technology, Jilin University, chensb@jlu.edu.cn 3, 4, 5, 6, 7 College of Geo-Exploration Science and Technology, Jilin University, lupeng@jlu.edu.cn, wangmc@jlu.edu.cn, wuqiong@jlu.edu.cn, zhouc0129@163.com, hm501611020@126.com Abstract Low altitude high resolution aerial images captured by Unmanned Airship (UA) have great potentials in application of environment investigation and large scale mapping. Taken Miyun, Beijing as the test area, a UA-based imaging system was employed and 640 aerial images at 370m and 430 images at 170m were captured. Then photogrammetric processing of these images, including preprocessing, image matching, aerial triangulation and panorama stitching, are described in detail. The ground resolution of final panorama is 15 mm, and space precision is 0.00136, approximately 0.25 pixels. To assess the image quality quantitatively, remote sensing images on SPOT and Quickbird are compared with unmanned airships by swiping horizontally and vertically. What s more, the correction error of overlap is less than 0.5 pixels. Therefore, the images captured by UA, valuable complements of satellite remote sensing, are reliable and capable of meeting practical demands. Keywords: Unmanned Airship, Low -Altitude Remote Sensing, Photogrammetric Processing. 1. Introduction In recent decades, unmanned aerial vehicles (UAV) have been widely used as the platform to study information collection, environmental monitoring, intelligent control and navigation [1]. Due to the demands of rapid data acquisition, UAV-based systems equipped with digital cameras could provide low-cost, high-resolution and large-scale images. Compared with satellite and aerial platforms, UVAs could fly below the clouds and make it possible to obtain the information obscured by cloud and fog on other platforms. UVAs have a host of advantages: low cost, superior resolution, reclaimable, longer duration as well as greater operation flexibility [2]. Among aircrafts used as UAVs, Unmanned Airship (UA) is eligible for low-altitude and low-speed flight, and can be used as the platform to obtain highresolution remote sensing images. UAs take off and land vertically without a runway, which makes common flat areas, like sport fields or city square, suitable for the flight field. Besides, it is safe and easy to operate on, especially suitable for data acquisition in urban city and complicated topography [3-6]. Another practical demand for UA-based systems is to provide quick and accurate processing technology for the aerial photos. The current geometry rectification is mainly to calibrate nonlinear camera distortion, image rotation and projection distortion caused by unfixed flight altitudes. When camera focus is fixed, the camera distortion is considered as a systematic error and exerts same influence on every image, which is just opposite to image rotation and projection distortion. Thus the predefined relationship between adjacent strips can not be strictly guaranteed. There are often large overlap variations and large rotation angles between adjacent images, and also large parallax discontinuities between features above the ground, which makes the low-altitude remote sensing differ from traditional photogrammetry [7-9]. To accelerate processing speed of image captured by UA, it is necessary to develop a set of quick processing workflow for better application in environment motoring and information extraction [10-13]. International Journal of Advancements in Computing Technology(IJACT) Volume5,Number3,February 2013 doi:10.4156/ijact.vol5.issue3.44 379

Motivated by applying low-cost and high resolution imagery by UA into environment motoring, we introduce a set of data acquisition and processing method for aerial images captured by UA, and compare remote sensing images on satellite platform and low-altitude platform respectively, which illustrates the reliability and applicability of low-altitude remote sensing images captured by UA. 2. Data acquisition The UA-based imaging system is composed of an airship, autopilot system, cloud platform, and digital camera for data acquisition and other auxiliary equipments. The airship s technological parameters are listed as follows: UA has a length of 12.9 m, cruising speed is about 20-30 km/h, maximum flight speed is 80 km/h, maximum payload capacity is about 11 kg, and anti-wind performance is 4-5 degree. Besides, a non-metric Nikon D300 is employed, the pixel size of which being 0.0055mm (2848pixel 4288pixel), the focus being 14 mm. On Dec.02, 2009, the flights at 370m and 170m had been carried out in Miyun, 65 km northeast of Beijing, China. The weather condition is bright with a fresh breeze. And the longitudinal and lateral overlap designed is 85% and 55% respectively, covering a total area of 3 km 4 km. Totally, 403 aerial images at 7 flight strips for 170m and 640 images of 8 flight strips for 370m were taken by UA. Fig.1 shows two aerial photos contrast between 170m (left) and 370 m (right). Relatively, the lower the flight height is, the more serious the deformation will be and the smaller the size will be. However, higher resolution images could be obtained through low altitude flight. To point out, the aerial images at 370m are employed during the following processing. 3. Image processing Figure 1. Image contrast of 170 m (left) and 370 m (right) In this paper, a data processing system for UA image has been developed to process images by Jilin University and Wuhan University jointly. The photogrammetric processing of images consists of image preprocessing, image matching, aerial triangulation, panorama stitching and DOM generation in sequence [7-9], as shown in Fig.2. Traditionally, sufficient number ground control points and large scale topographical map are used during the processing, which consuming labor and resources and is hard to obtain when disasters occur. To explore a fast method for UA images stitching and processing, no control points are involved in the processing, that s why the absolute orientation is not presented here. 3.1 Preprocessing For aerial photos captured by UA, inconsistent grayscale, high inclination and irregular after overlap are the three major problems affecting the image quality. Thus, the preprocessing, including color matching, distortion rectification and image rotation, is conducted every image to guarantee the congruity of texture, brightness, contrast, grayscale and saturation [12]. Then, the flight strips are established according to the shooting order and actual conditions. 380

3.2 Image matching Aerial Photos Image Pre-processing Distortion Correction Color Matching Image Ratotion Image Enhancement Combined Adjustment NO Add Conjugate Points by Manual Delete Matching Erros Yes Low Precision DSM Yes Continue NO Panoramic Stitching Yes DSM Matching NO Yes Manual Adjustment of Match Lines Fast Mosaic NO Misconnection of Ground Object DOM Generation Figure 2. Flow chart of image processing How to find out optimum conjugate points is the key to mosaic the low-altitude images. Image matching includes three steps [8-9]: feature extraction, overall matching and fine matching. In this paper, feature points are automatically extracted by Harris corner detector. Conjugate points are generated from feature points with great interest. If enough conjugate points have been matched, the relative orientation process will begin. 3.3 Aerial triangulation The precision of three dimensional spot coordinate data is the key problem for low-altitude remote sensing system. The UA images have advantage of higher forward and side overlaps, more redundant observations than traditional photogrammetry so that the precision of UA images is superior to that of traditional photogrammetry. In our work, aerial triangulation consists of relative orientation, model link and sparse combined bundle adjustment. Relative orientation could determine the orientation parameters within one stereo pairs among which one image s parameters have been known and the process is important to calibrate the lens distortion caused by non-metric camera. In one strip, all the models link automatically after image matching; in adjacent strips, orientation parameters should be transformed through common points in adjacent strips. The sparse combined bundle adjustment is performed to eliminate the discrepancies among camera parameters and model coordinate of conjugate points. Then six parameters (position and altitude angles) of orientation points are obtained. The bundle adjustment doesn t finish until the new RMS of 381

observations is smaller than last one of iteration. Finally, the number of iteration is 3, and the RMS is 0.001555 and the RMS of projection is 0.001998. Table 1 shows the precision of triangulation: Table 1. Precision of aerial triangulation ITEM RMS MIN MAX X Y Z 0.0116 0.0226 0.085-0.0015-0.0043 0.0035-0.0218 0.0396 0.1687 Fig.3 shows the relationship between aerial photos and orientation points through aerial triangulation. The chromatic cones represent the position and altitude of camera, and the yellow points represent the orientation points used in aerial triangulation. The more stable the altitude of cones is, the better the flight altitude will be. As shown in the figure, this flight is of good quality. 3.4 Panorama stitching Figure 3. Result of aerial triangulation The panorama is the direct stitching result of aerial photos, which could be produced quickly from the homonymy points after matching in no need of ground control points. To reduce edge errors in adjacent photos, the aerial triangulation is carried out before this step. What s more, the panorama is good at real-time and precision that could satisfy the demands of status survey and quick response to geological disasters. Low-precision Digital Surface Model (DSM) is generated on the basis of pass points after aerial triangulation. Considering DSM and project transformation relation, each photo has been corrected and editable match lines for correction are automatically generated. Fig. 4 shows the relationship between match lines and every photos, the red line stands for these lines and the green points stands for conjugate points. There often exist obvious edge errors around linear objects on the image, which could be modified through match lines by hand. Figure 4. Match lines interface 382

Fig.5 shows the panorama result at 370m. After 7 hours image processing, a panorama with 30000 *15583 pixels is generated. There are 82189 orientation points and 323199 image points are involved to produce it. The ground resolution is 15cm. The precision of image space is 0.00136, approximately 0.25 pixels. 3.5 DOM generation Figure 5. Panorama of flight with 370 m Digital Elevation Model (DEM) and Digital Orthophoto Map (DOM) could be generated as soon as panorama stitching is finished. Using elements of orientation after adjustment, numerous homonymy points through DSM matching are transformed into 3D point clouds by forward intersection method. DEM images could be generated by means of man-computer interactive editing and then DOM images are produced. 4 Image evaluation Satellite images, including SPOT and Quickbird, are swiped horizontally and vertically to observe what is different from images captured by UA. Fig.6 and Fig.7 show the horizontal (1) and vertical (2) image comparison with SPOT and Quickbird, respectively. According to the results, correction errors of overlap area are less than 0.5 pixels. Judging by overlap of decipherable objects like road and building, the precision of UA image could be determined directly. Generally, the matching degree of images is higher in middle of overlap part, but lower around the edges. There lies some dislocation within a narrow range around the edges. As shown in Fig.6 and Fig.7, the UA image enjoys higher definition than others obviously. The high resolution images from UA express the features of spatial structure and surface texture better, which could identify minor internal structures of ground objects. What s more, their edges information is clearer. The UA-based platform could capture the information obscured by cloud and fog, it is easy to find the influence exerted by cloud on satellite image, especially on the Quickbird image. Thus, the high resolution image from UA could satisfy the needs of large scale mapping and further interpretation [14-15]. 383

(1)Horizontal Swiping (2) Vertical Swiping Figure 6. Image Comparison with SPOT (1) Horizontal Swiping 384

5 Conclusions (2) Vertical Swiping Figure 7. Image Comparison with Quickbird The UA-based system could provide affordable, current and accurate remote sensing images. The low-altitude aerial image sequences have advantages of high overlap and great ground resolution, which could meet the demands of photogammetry in terms of data acquisition and processing. In our research, data acquisitions at different height have been achieved and high resolution panorama has been generated, which prove the workflow and processing is feasible. Thus, the images captured by UA, valuable complements of satellite remote sensing, are reliable and capable of meeting the need of geological investigation and mapping in large scale, especially suitable for up-to-date data acquisition for small areas. Acknowledgments Thanks are due to Wuhan University and Capital Normal University for their help during data acquisition and image processing. Thanks are also due to anonymous reviewers for their valuable comments to improve the paper. The research was sponsored by Program for Investigation of Chinese National Land and Resources (1212011087112) and Chinese National Natural Science Foundation Committee Project (40974047). References [1] J.J. Rao, Z.B., Gong, J, Luo, S.H. Xie, A flight control and navigation system of a small unmanned airship, In Proceeding(s) of the IEEE International Conference on Mechatronics & Automation, pp.1491-1496, 2005. [2] H.Y. Zhao, Z.Y. Gou, P.Q. Gao, No ground control point making the orthophoto for UAV Remote Sensing System, In Proceeding(s) of the SPIE International Symposium on Photoelectronic Detection and Imaging 2007: Related Technologies and Applications, pp. 66250C, 2007. [3] H. Lu, Y.S. Li, J. He, Ren Z.M, Capture and processing of low altitude remote sensing images by UAV, Engineering of Surveying and Mapping, vol. 20, no. 1, pp.52-54, 2011. [4] J.J. Rao, Z.B. Gong, J. Luo, S.H, Unmanned airships for emergency management, In Proceeding(s) of the 2005 IEEE International Workshop on Safety, Security and Rescue Robotics, pp.125-130, 2005. [5] J. Wu, H.S. Yang, Z.G Liu., Automatic registration of high-resolution multispectral imageries from band-reconfigurable imaging system on board unmanned airship, In Proceeding(s) of 5th International Conference on Image and Graphics, pp. 613-617, 2009. 385

[6] K. George, W. David, P.O James, Collection of environmental data from an airship platform, In Proceeding(s) of the SPIE Conference on Sensor Fusion and Decentralized Control IV, pp. 76-81, 2001. [7] E. Alberto, S.B Samuel, B. Marcel, A semi-autonomous robotic airship for environmental monitoring missions, In Proceeding(s) the 1998 IEEE International Conference on Robotics & Automation, pp. 3449-3455, 1998. [8] Y. J. Zhang, Geometric processing of low altitude remote sensing images captured by unmanned airship, Geomatics and Information Science of Wuhan University, vol. 34, no. 3, pp285-288, 2009. [9] Xi Chen, Dingwen Wang, Qinglin Zhang, Guihui Xie, "A Novel Fast-view System for High Resolution Remote Sensing Camera", JDCTA: International Journal of Digital Content Technology and its Applications, Vol. 6, no. 11, pp. 1-7, 2012. [10] Y.J. Zhang, Photogrametiric processing of low altitude image sequences by unmanned airship, In Proceeding(s) of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp751-757, 2008. [11] G. Bitelli, V. A. Girelli, M. A. Tini, L. Vittuari, Low-height aerial imagery and digital photogrammetrical processing for archaeological mapping, In Proceeding(s) of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 8-11, 1999. [12] Y.H. Wang and Y.L, Chen, Automatic matching and panorama generation of low altitude aerial images, Journal of Geomatics, vol. 34, no. 2, pp. 47-49, 2009. [13] Z.X. Zhang, Orthophoto Technique, Surveying and Mapping Publication, CHN, 1981. [14] Chengfan Li, Jingyuan Yin, Junjuan Zhao, Feiyue Ye, Lan Liu, "A New Method for Urban Road Extraction based on High Resolution Remote Sensing Images", JDCTA: International Journal of Digital Content Technology and its Applications, Vol. 5, no. 4, pp. 178-186, 2011. [15] K.R Harvey, J.E. G Hill, Vegetation mapping of a tropical freshwater swamp in the northern territory, Australia: a comparison of aerial photography, International Journal of Remote Sensing, Taylor & Francis, vol.22, no.15, pp. 2911-2925, 2001. 386