CHAPTER 1 INTRODUCTION

Transcription

1 CHAPTER 1 INTRODUCTION 1.1 Motivations Vision is the process of discovering from images what is present in the world and where it is (Marr [88], p. 3). Due to the apparent simplicity of the action of seeing, however, the underlying difficulties of visual information processing had not been realized until Marr s pioneer work on computational vision. According to Marr, the ultimate task of any computer vision system is essentially to transform an array of input numerical values into meaningful description that a human normally perceives. Figure 1.1(a) shows an image which consists of two texture regions. However, the texture regions are not obvious at all from the numerical values, where a small portion is shown in Figure 1.1(c). While the true transformation employed by humans is not known, any algorithm for solving vision problems attempts to approximate the transformation based on different assumptions and constraints with respect to the problems to be solved. Existing approaches can thus be categorized according to the problems to be solved and their assumptions. Due to the complexities of the vision process, four problems are widely studied in computer vision relatively independently: edge detection, stereo 1

2 (a) (b) 67 6d a 73 6c 6a f 7b 7c 7d 7e 87 7b 86 bd a6 85 8e b4 8d 95 8c a d 69 6b 63 6f 6c 69 6b 6c 70 6e 6c 6b d 93 8b 7e 7f 81 b7 8f c9 a9 c d 80 7e 8e 7e 8d d 6b 71 6c 6e 6b 6e bb ee ac 81 b8 b6 9c 57 7a 8c 5d d0 8b ab b2 8d d0 bd 8f 84 7a 86 7c e a 6c 62 6d 6a 6d 6e 73 6c 69 b8 9e a6 6b 4e a a7 79 7e 59 ad 80 7e 7a 70 b9 7b d e d a b0 e5 ce 5c 8e 7d 64 6c 98 bd 88 9a e 3d 9b 8c cc 7d 85 7c 87 8d b 65 6a 6e 67 6c dd 9e e e 6b 87 b9 a4 ae 9b 9f b 92 5f a6 7f a8 ab 7c b 8a 8f 8d c a8 a7 a9 a2 5f a c a8 a0 97 9c a d 9d 8a 9f a7 ac b e 93 8b a0 73 6c 70 6b ef e7 d1 a5 82 8e c5 40 7f 9f 9f a2 a3 a6 a0 9d 9e 8f a ca c e e 72 6c 6c 6d dd e0 c6 c4 c2 b0 b6 8e f 9a 9f a3 99 a0 9d a1 70 8a 95 eb b9 7b 84 8a 82 8a b c 77 6e 6d fb d0 da 9f 8f be c0 a8 aa a0 9f a1 a4 9d 9e a d 9e ce 8e c d 9a 8f 96 9a 71 6f e3 e0 dd da c0 a4 d7 a1 a f 98 a0 af a5 9e a7 8b 4d 9c 86 a7 a e 8e d 6d c e6 cf d8 c8 da ad a1 ca b1 fb a4 5f 92 a2 b4 b5 a3 9b 7f 82 b2 4f dd dd ff 88 8d e f d f5 af e3 d0 c0 c2 b7 a7 b2 a2 a6 7f 79 c8 a5 a2 a6 a3 9a b6 d9 99 8a 8a e 9a 97 9a 71 6e 6b 6f fc f5 e0 b1 a3 cc a2 d0 a0 ae a0 93 8f ab be a5 a9 a1 a2 87 7e 8a b1 8f a 8d 8c d 6c 79 6f 6d 73 df d4 e2 9b 84 c0 b0 9d a0 ce 9b ac bb 9b ac aa 9a a c c 88 8c f 78 6d 6f c cf f3 ce bb a2 c0 94 c9 bf 94 9a 9d a0 ad ad b3 a6 ad a3 86 7f b7 92 7a c 8f 8c e 99 9a ec bd cb c5 d8 94 a7 a0 b c8 a7 ab a8 aa bb af bc 8d 94 6e dc 8c 8d 88 8f 8c a 6d 6d 6c c ff d9 df fa d2 b8 ad 8b b1 b7 cd c8 ad a8 b6 af b4 bd d 8a b4 8a 8b 8f 8c d 94 9b 99 6e 6c c 6e ff f4 f0 d8 c5 db 8f a1 b1 c3 f6 d5 95 a8 b5 aa b1 e9 e4 78 7d 8c f 93 8e c 6e 73 6a 71 6f 72 5e dd f4 ef d5 ba bb aa ab aa ae ae ae 96 ab c1 f9 90 c2 f3 ed c4 81 7f d 86 8a c e 6d 6c ff da e6 dc cc b4 ae b5 ae 9e d5 ae ba af 93 a4 b5 ea b1 d3 e7 ff e b 8b 8f 8b a a f f7 c1 97 d8 c4 bb ab ac a7 af a3 aa b1 c5 e8 c3 af 94 9e b2 d6 cd ee de f b d 63 5d e5 d8 cb bb b8 ae aa a9 a3 9b a9 ca e3 ff b5 b6 8f 9b af aa b6 eb c1 80 7e 87 8d 8b d cc d7 c0 b3 bf b6 b3 ad a6 92 a3 c8 de ff a7 bd b5 9b 9c 98 9b d0 a6 e a 8a e 6c 6e 6e 76 6c 6f ff e9 c5 b9 b4 b3 c8 c8 b0 98 a9 b9 d2 ff af aa d1 d9 7b e 87 8e e 6d 63 5f e1 c9 ba ba b3 f6 cd c2 b4 a4 b9 d3 e8 ff 7e 7e 7b 8a 8d 96 7f 9b b3 d3 d7 6e b 8d 8e 96 6c a 71 6b 76 6c 61 db bd b6 a6 ee d9 d4 c4 b5 ad ac c7 d9 ea a b5 8b a5 a8 ad c0 dc d d c c3 a9 a9 99 de e5 d5 bd ae a5 ab bb c8 d7 e3 80 7a 7a d2 b f a0 ad d9 7b 7b 7f d 6a 5b 58 5e 5c 59 5a 63 8f 9c 76 db de c0 bd b9 b0 a9 b1 c2 bd c9 d4 7a f e c2 ee 7a 7f f 5f 5b f e9 cf b5 b0 ae ae b6 b0 bd c6 bb c9 cb a 87 8e e ad d5 e3 7d a 5e d6 bd ba b3 aa af ad b4 b7 bd c3 b9 bf c5 d3 cc 75 8f 9a c 9d a9 dc f a d 5c 68 5d 8d d0 b9 ab a9 a7 ac af b5 c2 c3 bd b2 af b5 c2 c0 d2 d4 b7 b9 97 9a f ae ff a f 65 5c 5f 59 4f 6d c5 bd ae ab a0 a9 a3 b1 b7 b9 c2 c8 c0 a9 9c a5 ba c6 d4 dc da c1 c6 c8 8a b3 c7 ff 7d c 5c 5a cb be bb b8 ba aa a8 a2 92 9c 9a a1 bc ba c5 d5 b9 a0 93 a4 b6 c5 de ee ee de c1 cc d9 b2 b7 ac b2 c3 da bd 5f 5c 54 d0 bd b5 b1 ad a7 9f a3 9a 94 9b 8c 98 a5 b6 c7 d2 e6 c7 9c 8b 94 b4 c2 e2 fe f0 e5 dc e7 de df b4 c1 cc d4 f6 f d0 c1 b8 b2 a6 a4 9e 9b 9d 97 8e 8e ac b5 cc e4 ef c8 9e 8c 91 b1 c5 ef ff fd f1 e7 ee f1 dd d4 cf cc dc e5 e2 82 ca c4 ba ab a6 9e 9f 9b 99 9a 8e a1 b4 cc ea ff c ae c0 f5 ff f4 f6 e9 e6 e2 df d3 d9 dd e3 ea d5 d0 c3 b3 a6 b1 a2 a4 9b 94 8f f e b5 cf ee fb ce 98 8c 90 a7 c4 e9 ff f4 eb e4 d9 d0 d8 d1 d7 e2 d6 e6 d9 ce bd b7 ac a7 a0 a3 9a 92 8e a b 9d ba d7 f2 ff d4 9a a7 c6 e3 f9 f9 f9 e4 d9 d7 de de ec ee ea f3 e4 (c) Figure 1.1: A texture image and the corresponding numerical arrays. (a) A texture image with size (b) A small portion with size of (a) centered at pixel (64, 37), which is on the boundary between the two texture regions. (c) Numerical values of (b). To save space, the values are displayed in hexadecimal format. 2

3 matching and motion analysis, image segmentation and perceptual organization, and pattern recognition. Roughly speaking, the techniques for the first three problems are primarily data-driven, or called bottom-up processes, and pattern recognition is model-driven, or called top-down process. Early techniques with successful applications are classification techniques [25], which map a given input into one of the pre-defined classes according to a distance measure. However, all the possible classes and their variations we normally perceive are too gigantic to be implemented effectively in any system. The attention of computer vision was then shifted to derive more generic features for arbitrary images. From information and encoding theories, edges, i.e., discontinuities in images, carry more information and exhibit nice properties such as invariance to luminance changes. Motivated by neurophysiological findings [53], many edge detection algorithms were proposed and studied. Segmentation techniques try to solve the same problem by segmenting an image into homogeneous regions, where edges and region contours can be obtained straightforwardly and more robustly. These approaches were claimed to be unified [92] through what is called Mumford-Shah Segmentation energy functional [94] (See Chapters 4 and 5). Common to these approaches, the images are assumed to be piece-wise smooth regions with additive Gaussian noise, resulting in efficient algorithms. To improve the performance for real images, multiple scales are generally needed and linear and nonlinear scale spaces are thus proposed and studied. Chapter 2 studies segmentation for range images. Chapter 3 studies a new nonlinear smoothing algorithm and addresses some of problems in nonlinear scale spaces. Chapter 7 applies a nonlinear smoothing algorithm to hydrographic object extraction from remotely sensed images. 3

4 While there are useful applications of edge detection and segmentation algorithms, the underlying assumption limits their successes in dealing with natural images. As shown in Figure 1.1, texture regions neither are piece-wise smooth nor can be modeled with additive Gaussian noise. Figure 1.2 demonstrated that a pure linear system is not sufficient for natural image modeling [63], where the spatial relationships among pixels are more prominent in characterizing the texture regions than individual pixels. Clearly piece-wise smooth regions with additive Gaussian noise are not sufficient and more sophisticated models are needed to deal with texture images. Supported by neurophysiological and psychophysical experiments [12] [23], The early processes in the human vision system can be abstractly modeled by filtering with a set of frequency and orientation tuned filters. However, as demonstrated in Figure 1.2, purely linear filtering is not sufficient, nonlinearity beyond filtering must be incorporated [87]. Spectral histograms integrate the responses of a chosen bank of filters through marginal distributions [148] [149] [150] [147]. As demonstrated in Figure 1.2, spectral histograms are nonlinear. Chapters 4 and 5 apply spectral histograms to modeling [147], classification, and segmentation of texture as well as intensity images. While edge detection and segmentation techniques are very fruitful, there are perceptual phenomena that cannot be explained by purely data-driven processes. Classical examples include virtual contours, which are widely studied by Gestaltists. The long-range order grouping is known as perceptual organization. Chapter 6 studies perceptual organization through temporal dynamics. Because many of the meaningful objects cannot be characterized well using intensity values or even textures, such as a human face, the relationships among some 4

5 (a) (b) (c) Figure 1.2: Demonstration of nonlinearity for texture images. (a) A regular texture image. (b) The image in (a) was circularly shifted left and downward for 2 pixels at each direction. (c) The pixel-by-pixel average of (a) and (b). The relative variance defined in (3.20) between (a) and (b) is 137, and between (a) and (c) is 69. The distance between the spectral histograms defined in Chapter 4 between (a) and (b) is and between (a) and (c) is

6 primitives need to be modeled. This leads to the need of top-down processes such as recognition. Clearly, the four problems studied are sub-problems of vision process and the integration among them is critical for a complete vision system. The interaction between different modules is briefly discussed in Chapter Thesis Overview As discussed above, we study vision problems at different organizational levels in this dissertation. In Chapter 2, we study the segmentation problem for range images. Depth is most important cue for visual perception and range image segmentation has a wide range of applications. We propose a feature vector consisting of surface normal, mean and Gaussian curvatures and a similarity measure for range images. We implemented a system based on oscillatory correlation using a LEGION (locally excitatory globally inhibitory oscillator network) network. Experimental results demonstrate that our system is capable of handling different kinds of surfaces. With the unique properties of a temporal dynamic system, our approach may lead to a real-time approach for range image segmentation. In Chapter 3, we propose a new nonlinear smoothing algorithm by incorporating contextual information and geometrical constraints. Several nonlinear algorithms are derived as special cases of the proposed one. We have compared the temporal behavior and boundary detection results of several widely algorithms, including the proposed method. The proposed algorithm gives quantitatively good results and exhibits nice temporal behaviors such as quick convergence and robustness to noise. In Chapter 4, we propose spectral histograms as a generic statistic feature for texture as well as intensity images. We demonstrate the properties of spectral histograms 6

7 using image synthesis, image classification, and content-based image retrieval. We also compare with several widely used statistic features for textures and show that the distribution of local features is critically important for classification while mean and variance in general are not sufficient. We also propose a model for texture discrimination, which matches the existing psychophysical data well. Chapter 5 continues the work in Chapter 4. In Chapter 5, segmentation problem is studied extensively using spectral histograms. A new energy functional for segmentation is proposed by making explicit the homogeneity measures. An approximate algorithm is derived, implemented and studied under different assumptions. Satisfactory results have been obtained using natural texture images. Chapter 6 studies the problem of perceptual organization and long-range grouping, which is one level beyond the segmentation. By using a boundary-pair representation, we propose a figure-ground segregation network. Gestalt-like grouping rules are incorporated by modulating the connection weights in the network. The network can explain many perceptual phenomena such as modal and amodal completion, shape composition and perceptual grouping using a fixed set of parameters. Chapter 7 presents a computational framework for feature extraction from remote sensing images for map revision and geographic information extraction purposes. A multi-layer perceptron is used to learn the features to be extracted from examples. A locally coupled LEGION network is used to achieve accurate boundary localization. To increase the robustness of the system, a weight adaption method is used. Experimental results using DOQQ images show that our system can handle very large images efficiently and may have a wide range of applications. 7

8 Chapter 8 summarizes the contributions of the work presented in this dissertation and concludes this dissertation with discussions on the future work. 8