ONE of the most challenging problems addressed by the

Transcription

1 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER A Mutieve Context-Based System for Cassification of Very High Spatia Resoution Images Lorenzo Bruzzone, Senior Member, IEEE, and Lorenzo Carin, Student Member, IEEE Abstract This paper proposes a nove pixe-based system for the supervised cassification of very high geometrica (spatia) resoution images. This system is aimed at obtaining accurate and reiabe maps both by preserving the geometrica detais in the images and by propery considering the spatiacontext information. It is made up of two main bocks: 1) a nove feature-extraction bock that, extending and deveoping some concepts previousy presented in the iterature, adaptivey modes the spatia context of each pixe according to a compete hierarchica mutieve representation of the scene and 2) a cassifier, based on support vector machines (SVMs), capabe of anayzing hyperdimensiona feature spaces. The choice of adopting an SVM-based cassification architecture is motivated by the potentiay arge number of parameters derived from the contextua featureextraction stage. Experimenta resuts and comparisons with a standard technique deveoped for the anaysis of very high spatia resoution images confirm the effectiveness of the proposed system. Index Terms Hierarchica feature extraction, hierarchica segmentation, mutieve and mutiscae anaysis, spatia-context information, support vector machines (SVMs), very high spatia resoution images. I. INTRODUCTION ONE of the most chaenging probems addressed by the remote sensing community in current years is the deveopment of effective data processing techniques for images acquired with the ast generation of very high spatia resoution sensors. The deveopment of these kinds of techniques appears even more important in ight of recenty aunched commercia sateites (e.g., Ikonos and Quickbird), with on-board sensors characterized by very high geometrica resoution (from 2.5 to 0.60 m). The avaiabiity of images acquired by these sensors eads to a new set of possibe appications, which require mapping the Earth surface both with great geometrica precision and a high eve of thematic detai. In this context, great attention is devoted to the anaysis of urban scenes, with appications such as road network extraction and road map updating, transportation infrastructure management, the monitoring of growth in urban areas, and discovering buiding abuse [1], [2]. Other appications are reated to the monitoring of forests, ike the definition of seective cutting panning and the anaysis of forest status heath [3], [4]. In addition, high-resoution remote sensing images can be used by pubic administrations to monitor, manage, and prevent natura disasters, to anayze evacuation Manuscript received Juy 11, 2005; revised March 7, This work was supported by the Itaian Ministry of Education, University and Research (MIUR). The authors are with the Department of Information and Communication Technoogy, University of Trento, Trento, Itay (e-mai: orenzo. bruzzone@ing.unitn.it). Digita Object Identifier /TGRS panning in areas with high probabiity of foods or fires [5], etc. However, these are ony a few exampes of the wide range of potentia appications of high geometrica resoution data. The significant amount of geometrica detais present in a high-resoution scene competey changes the perspective of data anaysis compared with moderate-resoution images provided by previous-generation mutispectra sensors [such as the Thematic Mapper (TM) and Enhanced Thematic Mapper Pus (ETM+)]. In particuar, the improvement in spatia resoution simpifies the probem of mixed pixes 1 present in standard mutispectra images, but at the same time, it increases the interna spectra variabiity (intracass variabiity) of each andcover cass and decreases the spectra variabiity between different casses (intercass variabiity). Thus, on the one hand, the resuting high intracass and ow intercass variabiities ead to a reduction in the statistica separabiity of the different andcover casses in the spectra domain, which in turn invoves high cassification errors [6], [7]. In addition, the imited spectra resoution of very high geometrica resoution sensors, which depends on technica constraints, further increases the compexity of the cassification probem [6], [19]. On the other hand, due to the high spatia resoution of the images, the geometrica information of the scene can aso be considered in the cassification process according to proper feature-extraction methodoogies. In the recent iterature, many papers have addressed the deveopment of nove techniques for the cassification of highresoution remote sensing images. In [9], the authors present a technique for the identification of and deveopments across arge-scae regions. The proposed technique uses straight ines, statistica measures (ength, orientation, and periodicity of straight ine), and a spatia coherence constraint to identify three casses, namey: 1) urban; 2) residentia; and 3) rura. In [10], a standard maximum-ikeihood cassifier is used to discriminate four spectray simiar macrocasses. Subsequenty, each macrocass can be hierarchicay subdivided according to cass-dependent spatia features and a fuzzy cassifier. The main probem of these techniques is that they are highy probem dependent. This means that they cannot be considered as a genera operationa too. In [11], the authors anayze the effectiveness of the gray-eve cooccurrence matrix (GLCM) texture features in modeing the spatia context that characterizes high-resoution images. However, the fact that the anaysis 1 Mixed pixes are pixes that represent the spectra signature of more than one cass due to the insufficient geometrica resoution of the sensor (more than one and-cover cass is incuded in the ground-projected instantaneous fied of view (GIFOV) of the sensor) /$ IEEE

2 2588 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 depends on a square window and different heuristic parameters and the intrinsic inabiity to mode the shape of the objects do not yied satisfactory cassification accuracies. A more promising famiy of approaches to the anaysis of high spatia resoution images, which is inspired by the behavior of the human view system, is based on object-oriented anaysis and/or mutieve/mutiscae strategies. The rationae of these approaches is that each image is made up of interreated objects of different shapes and sizes. Therefore, each object can be modeed both with shape and topoogica measures which can be used and integrated with spectra features to improve the cassification accuracy. Objects can be extracted from images according to one of the standard segmentation techniques proposed in the iterature [6], [12]. In greater detai, the main idea of mutieve anaysis is that for each eve of detai, it is possibe to identify different objects that are pecuiar to the considered eve and that shoud not appear in other eves. In other words, each object can be anayzed at its optima representation eve. Moreover, other aspects considered in this anaysis are: 1) that objects at the same eve are ogicay reated to each other and 2) that each object at a generic eve is hierarchicay reated to those at the higher and ower eves [7], [8], [13], [14]. For exampe, in the mutiscae anaysis of a high-resoution image, at finer eves, we can identify houses, gardens, streets, and singe trees; at higher eves, we can identify urban aggregates, groups of trees, and agricutura fieds; finay, at the coarser eve, we can identify towns and cities, forests, and agricutura areas as one singe object. The exporation of the hierarchica tree resuts in a precise anaysis of the reations of objects. For exampe, we can count the number of houses that beong to an urban area [13]. In [15], the authors propose an approach based on the anaysis of a high-resoution scene through a set of concentric windows. The concentric windows anayze the pixe under investigation and the effects of its neighbor system at different scaes of resoution. To reduce the computationa burden, the information contained in each anaysis window is compacted using a Gaussian pyramida resamping approach. The cassification task is accompished by a soft mutiayer perceptron neura network that can be used adaptivey as a pixe-based or an area-based cassifier. One of the imitations of this approach is the fixed shape and choice of size of the anaysis window. In [16], an object-based approach is proposed for cassification of dense urban areas from pan-sharpened mutispectra Ikonos imagery. This approach expoits a cascade combination of a fuzzy pixe-based cassifier and a fuzzy object-based cassifier. The fuzzy pixe-based cassifier uses spectra and simpe spatia features to discriminate between roads and buidings, which are spectray simiar. Subsequenty, a segmented image is used to mode the spectra and spatia heterogeneities and to improve the overa accuracy of the pixe-based thematic map. Shape features and other spatia features (extracted from the segmented image) as we as the previousy generated fuzzy cassification map are used as inputs to an object-based fuzzy cassifier. In [17], morphoogica operators (such as opening and cosing) are expoited within a mutiscae approach to provide image structura information for the automatic recognition of man-made structures. In greater detai, the structura information is obtained by appying morphoogica operators with a mutiscae approach and anayzing the residua images obtained as a difference between the mutiscae morphoogica images at successive scaes. A potentia probem of this technique is the arge feature space generated by the appication of a series of opening and cosing transforms. In [17], the authors overcome this probem by proposing the use of different feature-seection agorithms. An adaptive and supervised mode for object recognition is presented in [7], where a scae-space fitering process that modes a mutiscae anaysis for feature extraction is integrated in a unified framework within a mutiayer perceptron neura network. This means that the error backpropagation agorithm used to train the neura network aso identifies the most adequate fiter parameters. The main probems of this technique are reated to the choice of the number and type of fiters to be used in the input fitering ayer (first ayer) of the neura network. In [18], an agorithm based on seective region growing is proposed to cassify a high-resoution image. In the first step, the image is cassified by taking into account ony spectra information. In the second step, a cassification procedure is appied to the previous map by taking into account not ony spectra information but aso a pixe distance condition to aggregate neighbor pixes. By reiteration, neighbor pixes that beong to the same cass grow in a seective way, obtaining a fina cassification map. Nevertheess, at present, the few techniques specificay deveoped for the automatic anaysis of high spatia resoution images (compared with the very arge iterature on the cassification of moderate-resoution sensors) do not exhibit sufficient accuracy to meet end-user requirements in a appication domains. For this reason, it is important that the remote sensing community invests further efforts to define advanced effective methods for the cassification of the aforementioned type of data. In this paper, we propose a nove pixe-based approach to the cassification of very high spatia resoution images, which is based on two modues (see Fig. 1): 1) a feature-extraction modue that expoits an adaptive, mutieve, and compete hierarchica representation of the spatia context of each pixe in the scene under investigation and 2) a cassification modue based on support vector machines (SVMs). In greater detai, extending and deveoping concepts previousy presented in the iterature, a strategy for defining the spatia context of a pixe at different eves in an adaptive way is presented. The mutieve spatia-context information is then used to drive the feature-extraction phase. The resuting high-dimensiona feature vectors are then anayzed according to a proper SVMbased muticass architecture. The choice of the SVM depends on the effectiveness of this machine-earning methodoogy to manage cassification probems in hyperdimensiona feature spaces [21], [22]. It is worth noting that the contribution of this work concerning the importance of SVM in the cassification of very high resoution images goes beyond the specific methodoogies presented in this paper because the cassification of high-resoution images generay requires the anaysis of hyperdimensiona feature vectors (e.g., when mutiscae morphoogica fiters are used, we can obtain a arge feature set) and, thus, the expoitation of a cassification technique robust

3 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2589 Fig. 1. Bock scheme of the proposed approach. to the Hughes phenomenon. Unike other methods presented in the iterature, the proposed approach is genera 2 and can be appied to any kind of very high geometrica resoution image. Experimenta resuts, obtained on two different data sets made up of very high spatia resoution images acquired by the Quickbird sateite in significanty different scenes (i.e., urban and rura areas), point out the effectiveness of the proposed system. This paper is organized in five sections. Section II presents a detaied description of the proposed adaptive mutieve context-driven feature-extraction technique. Section III addresses the cassification modue and describes the adopted SVM-based cassification architecture. Section IV presents the data sets used for the experiments and reports on experimenta resuts. Finay, Section V provides a discussion on the proposed approach and draws the concusion of this paper. II. PROPOSED ADAPTIVE MULTILEVEL CONTEXT-DRIVEN FEATURE-EXTRACTION TECHNIQUE The rationae of the proposed feature-extraction technique consists of adaptivey modeing the spatia context of each pixe according to a mutieve strategy. Each context eve is defined according to predefined spectra and spatia constraints. A. Adaptive Definition of the Mutieve Spatia Context To adaptivey characterize the spatia context of each pixe by taking into account a compete hierarchica mutiscae context representation, extending and deveoping some concepts previousy presented in the iterature, we propose to decompose the scene under investigation from the pixe eve to the highest eves of representation of its spatia context. A compete hierarchica modeing aows to capture and expoit the entire information present in the scene by working with adaptive context/neighborhood systems at different scaes. This task is based on the appication of a segmentation technique with a set of propery defined parameters that take into account both spectra and spatia constraints. This decomposition resuts in a mutieve representation of the spatia context of each pixe in the investigated scene. To satisfy a tree-based hierarchica 2 It is worth noting that, in this paper, the words genera and probem independent mean that the proposed technique has not aprioriconstraints on the kinds of objects present in the scene, but it can be used with any kind of high-resoution image and in any appication domain. On the contrary, many techniques proposed in the iterature are not genera and are probem dependent as they are specificay deveoped for addressing particuar appications (e.g., anaysis of urban areas) and are based on feature-extraction procedures and processing agorithms that cannot be appied to other scenes. Fig. 2. Hierarchica mutieve segmentation appied to a mutispectra Quickbird image. From (a) to (c), we use three different sets of parameters in the segmentation agorithm to adaptivey mode (at different eves) the context of the pixes in the image in (d). The seected rue guarantees that precise hierarchica reations between different eves are estabished. requirement, this process is accompished according to a specific set of rues. In this way, precise hierarchica reationships between each pixe in the image and the regions that adaptivey define its context at different eves are estabished. In other words, we obtain a set of segmentation maps (one for each eve) that characterize the context of each spatia position in the image hierarchicay and in a nonambiguous way (Fig. 2). It is worth noting that, unike other approaches proposed in the iterature and briefy described in Section I, hierarchica segmentation does not aim to identify the best eve of representation of each object, but simpy modes the mutieve spatia context of each pixe. This shoud be considered as a preprocessing stage aimed at driving the feature-extraction phase. A forma definition of the adopted segmentation procedure is given in the foowing. Let I denote the investigated image and H the homogeneity predicate at the generic eve ( =1,...,L). Varying the homogeneity predicate means varying the eve of definition of the adaptive spatia context of the pixe. This homogeneity predicate is defined according to different spatia and spectra attributes at different eves. According to the iterature, the segmentation of I at a generic eve is a partition P in a set of N regions O i (i =1, 2,...,N ),

4 2590 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 such that N i=1 O i = I with O n O m = φ, n m (1) H ( O i) = true i (2) H ( O n O m) = fase O n and O m adjacent. (3) These three rues are vaid for objects at a generic eve. To estabish a precise hierarchy between the contexts of a pixe defined at different eves, we consider the foowing additiona constraint: Oi 1 = Oj. (4) O 1 O i j This simpe reation states that the adaptive neighborhood of a pixe at eve 1 cannot be incuded in more than one adaptive neighborhood at eve (it has ony one father node). It is worth noting that eve 1 represents the pixe eve, i.e., the pixe for which the context is hierarchicay defined. We woud ike to stress that the idea of using hierarchica segmentation to represent the objects that compose the scene at different eves of abstraction is not a new contribution of this paper, but the novety of this paper consists in the technique adopted for expoiting the resuts of the hierarchica segmentation. In this respect, any segmentation agorithm that satisfies the aforementioned constraints can be used in the proposed system (see, for exampe, [13] and [28]). The mutiresoution segmentation agorithm we adopted is a bottom-up region-merging technique starting from the pixe eve (at the first step, each pixe represents an object). In an iterative way, at each subsequent step, image objects are merged into bigger ones. The aim of this procedure is to minimize the homogeneity predicate when two different objects are merged together (this constraint must be vaid for a the coupe of objects in the image). If the smaest growth exceeds a threshod defined by the user (the so-caed scae parameter ), the process stops. As briefy mentioned before, the homogeneity predicate takes into account spectra and spatia constraints. In detai, it can be defined as foows: H ( Oi) = w spectra h ) spectra( O i + w shape h ( ) shape O i (5) where wspectra [0,...,1], w shape [0,...,1] are userdefined parameters and wshape =1 w spectra. The first part of (5) is a cost criterion for the spectra component of the objects, whereas the second part is a shape cost criterion. Hence, we can define an information oss function when two cosest objects Oi and Oj, at a certain eve, are fused together as Ci,j = H ( Oi Oj ) ( ) ( ) H O i H O j = wspectra C i,j,spectra+ ( 1 wspectra) C i,j,shape. (6) We can stop the segmentation agorithm when Ci,j H TH, where HTH is a eve-dependent user-defined threshod defined for each eve. The greater the vaue of HTH, the greater the dimension of obtained objects. (In other words, we decrease the sensibiity of the homogeneity predicate in the fusion of two adjacent regions.) The spectra information oss function Ci,j,spectra in (6) can be defined as C i,j,spectra = B d=1 [ ( w,d Ni,j σ,d i,j Ni σ,d i )] + Nj σ,d j where w,d, d =1,...,B (B is the number of spectra bands), represents the weight associated to the dth spectra channe at eve in the combination process, and B d=1 w,d =1; Ni,j represents the number of pixes of the object obtained by merging Oi and O j, and σ,d i,j represents its standard deviation on the spectra band d. Ni and N j represent the number of pixes that compose objects Oi and O j, respectivey, and σ,d i and σ,d j represent their standard deviations cacuated on the spectra band d. The spatia information oss function, Ci,j,shape in (6), takes into account the compactness and smoothness of the shape of the obtained object by merging Oi and O j. It is defined as C i,j,shape = w cmp C i,j,cmp + ( 1 w cmp) C i,j,smooth (8) where C i,j,cmp = N i,j e i,j N Ni,j i e i N i N j Ci,j,smooth = Ni,j e i,j ri,j Ni e i ri Nj e j rj e j N j (7) (9) (10) where wcmp [0,...,1] is a user-defined parameter to weight the smoothness of the obtained objects with respect to the compactness; e i,j represents the perimeter of the object obtained by merging Oi and O j, whereas r i,j represents the perimeter of the rectange containing it; e i and e j represent the perimeter of the objects Oi and O j, respectivey, whereas r i and r j represent the perimeter of the rectanges that contain Oi and Oj, respectivey. It is worth noting that the basic criteria of the aforementioned segmentation strategy are aso impemented in commercia software packages [28]. The choice of the range of variation of the parameters defining the homogeneity criterion affects the number of eves 3 in which the scene is decomposed. The number of eves to be used for characterizing the spatia context of each pixe depends on many factors. The most important issues to take into account are: 1) geometrica resoution of the image [e.g., given a specific scene, Quickbird images (GIFOV equa to 0.6 m) require higher numbers of decomposition eves than SPOT 5 images (GIFOV equa to 2.5 m)] and 2) size of the objects present in the scene. An empirica rue, for obtaining indications on the number of eves to use, consists in computing the mean size of the regions at different decomposition eves and comparing this size with 3 In this paper, we refer to a mutieve representation of the scene and not to a mutiscae decomposition. This is due to the use of a segmentation procedure, which is driven not ony on geometrica criteria but aso on spectra parameters, for accompishing the image decomposition task.

5 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2591 the expected average size of the objects in the scene, which can be defined by the end user. In greater detai, a the eves in the decomposition have to satisfy the foowing condition: 1 N N i=1 A i EA th (11) where N is the number of objects at eve, A i represents the area of object i at eve (in pixes), and EA th is a user-defined parameter that corresponds to the expected average size of the objects in the scene (in pixes). It is aso possibe to define the quantity EA th,m 2 = EA th GIFOV 2, which represents the expected average size of the objects in square meters. Accordingy, (11) can be rewritten as 1 N N A i GIFOV 2 EA th,m 2. (12) i=1 The definition of the vaue of EA th (or EA th,m 2)isreativey easy in homogeneous scenes (e.g., urban areas). In heterogeneous scenes, the probem can be addressed according to two different strategies: 1) consider a tradeoff between the average sizes of different casses of objects and 2) seect the aforementioned parameters according to the average size of the greatest cass of objects (impicity assuming the use of context information incuding more objects for the smaest components of the scene). Both strategies are consistent with the proposed approach. The choice of one of them shoud be based on end-user requirements. It is worth noting that, in the discussion above, we considered the pixe eve as the owest eve of the hierarchy, but it is aso possibe to define any eve of the mutiscae segmentation maps as the owest eve of the tree (in the atter case, we consider an object and its adaptive context). B. Mutieve Context-Driven Feature Extraction Given the hierarchica tree structure, it is then possibe to expoit the reationships between pixes and regions at different eves to extract an effective set of features that describe each pixe and its adaptive context at each eve. Depending on the eve considered, different kinds of features can be extracted to characterize the spatia context with the most reiabe attributes for the specific anayzed scae. We can extract spectra, spatia, or reationa features. Spectra features are derived anayzing directy the spectra information of a pixe and that of its adaptive neighborhood at different eves. Simpe spectra features (such as mean and standard deviation) or more compex measures (such as entropy and high-order statistics) can be easiy extracted to characterize both spaceinvariant and texture properties associated with the pixe. In addition, geometrica and reationa measures can be computed to characterize the shape, size, and interreation of the adaptive neighborhood of a pixe. In greater detai, geometrica features are reated to the description of the shape and size of the spatia context at different eves of anaysis (e.g., we can compute the area, the shape factor, and the perimeter of a generic region m Fig. 3. Exampe of the features extracted for objects that, at different eves, characterize the context of the pixe under investigation. Mean and StDev represent the mean vaue and the standard deviation of pixes in a generic object, respectivey. Area and SF represent the area and the shape factor (the ratio between ength and width) of an object, respectivey. Number of sub-objects represents the number of objects at eve 1 that make up an object at eve. at eve ). 4 Concerning reationa parameters, they can be expressed by a contextua anaysis of neighboring regions at the same eve or at different eves to mode the reation between the spatia context of a pixe at the same or at different eves. Thus, we can define the feature vector x i, which describes the pixes and, through the hierarchica tree, the spatia context (objects) in which the pixe is incuded. For a generic pixe i under anaysis, we can write { } x i = f i 1,fi 2,...,fi,...,fi (13) L where f i is the feature vector associated with the contextua information of pixe i at generic eve of the hierarchica tree, and L is the number of segmentation eves. It is worth noting that the components of f i are the features that characterize the 1 spatia position i in the image at pixe eve. The subvector f i is defined as { } f i = f i,1,...,f,j,...,f i i,nf (14) where f,j i is the jth feature that modes the context of pixe i at eve, and NF is the number of features extracted at eve. As stated before, the component f,j i can be a spectra, a geometrica, or a reationa feature. An important observation concerns the criterion to adopt for defining the set of features to be used at each eve. As shown in the exampe reported in Fig. 3, at the pixe eve, it is possibe to use ony the pixe spectra signature (there are no regions, and hence, it is not possibe to compute any geometrica feature). At intermediate eves, the regions are typicay sma and represent ony portions of the objects; thus, we recommend avoiding the use of geometrica features, as they do not contain reevant information about the geometry of the true objects present in the scene. At the higher eves, instead, the objects are better modeed by the 4 Exampes of these parameters are reported in Section IV-B.

6 2592 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 segmentation agorithm, and geometrica and reationa features can be propery used. It is worth noting that the main ideas of the proposed approach are that: 1) a the segmentation eves, which shoud be seected according to the aforementioned genera guideines, can be used to obtain a compete hierarchica representation of the spatia context of each pixe and 2) a the features extracted to characterize the context information of each pixe at different eves can be used as input to the cassification modue. However, athough this approach provides the cassifier with a arge amount of information, it has the disadvantage of eading to a very high dimensiona (hyperdimensiona) feature vector. This probem shoud be addressed in the cassification modue. III. SVM CLASSIFICATION APPROACH To achieve a good characterization of the spatia context of each pixe, we shoud use a sufficient number of segmentation eves. [The number of eves depends on the scene under anaysis; see the criterion defined in (11).] As mentioned earier, from the adaptive neighborhood of a pixe at each singe eve, we can extract a arge number of features that characterize the spectra, geometrica, and reationa attributes of the regions. Hence, the number of components of the feature vector extracted from the hierarchica tree may be very high. To obtain proper earning of the cassifier and to achieve a good generaization capabiity whie avoiding the course of dimensionaity probem (the so-caed Hughes phenomenon due to the sma ratio between the number of training sampes and the number of features [26]), we shoud coect a arge number of independent training set sampes to characterize a the possibe spectra variations of each and-cover cass. Athough it is quite simpe to coect ground truth sampes by photo interpretation on very high resoution images, it is rather time consuming. In addition, the spatia autocorreation of each sampe reduces the spectra information of the neighboring sampes and vioates the sampe-independent condition. This can ead to the so-caed unrepresentative sampe probem [24] that increases the compexity in the definition of training sampes. The foowing two possibe aternatives to the probem of coecting a very arge number of training sampes can be considered: 1) appying a feature-seection procedure and 2) using a cassifier intrinsicay robust to the Hughes phenomenon. Concerning feature seection, in the considered probem, it is quite difficut to define a criterion function (aimed at evauating the effectiveness of the considered subset of features) capabe of deaing with the heterogeneity of the statistica modes that characterize the different parameters extracted in the previous phase. Many feature-seection techniques assume Gaussian (or monomoda) distributions for the anayzed features, which do not fit some of the considered measures. For this reason, in the proposed approach, we prefer to avoid feature seection and to adopt a cassification technique intrinsicay ess sensitive to the high dimensionaity of the feature space. In particuar, we consider a machine-earning cassifier based on SVMs, which have been recenty proved to be effective in hyperdimensiona probems [21], [22]. Deveoped by Vapnik, SVMs are based on the structura risk minimization principe [27], and their popuarity within the remote sensing community is constanty on the increase [20], due to their properties and intrinsic effectiveness. In the foowing, we briefy describe the main concepts of the mathematica formuation of SVMs for binary cassification probems. Let us consider a binary cassification probem, with N training patterns in a d-dimensiona feature space. Each pattern is associated with a target y i {+1, 1}. The noninear SVM approach consists of mapping the data into a higher dimensiona feature space, i.e., Φ(x), where a separation between the two casses is ooked for by means of an optima hyperpane defined by a weight vector w and a bias b. The decision rue is defined by the function sign[f(x)], where f(x) represents the discriminant function of the hyperpane and is defined as f(x) =w Φ(x)+b. (15) The optima hyperpane is the one that minimizes a cost function that expresses a combination of two criteria, namey: 1) margin maximization and 2) error on training sampes minimization. It is defined as Ψ(w,ξ)= 1 N 2 w + C ξ i (16) i=1 and it is subject to the foowing constraints: { yi (w Φ(x)+b) 1 ξ i, i =1, 2,...,N ξ i 0, i =1, 2,...,N (17) where ξ i are caed sack variabes and are introduced to take into account nonseparabe data. The constant C represents a reguarization parameter that aows to tune the shape of the discriminant function. The above minimization probem can be reformuated through a Langrage functiona for which the Lagrange mutipiers can be found by means of a dua optimization eading to a quadratic programming soution. The fina resut is a discriminant function described (in the origina feature space) by the foowing equation: f(x) = i S α i y i K(x i,x)+b (18) where K(.,.) is a kerne function that shoud satisfy the Mercer s theorem. The set S is a subset of the indices {1, 2,..., N} corresponding to the nonzero Lagrange mutipiers α i. The training vectors associated with these mutipiers are caed support vectors. The soution of the dua-optimization probem avoids the probem of defining optima transformation from the origina to the hyperdimensiona feature space. The most widey used kerne functions adopted in the remote sensing probems are P oynomia kerne K(x i,x j )=(x i x j +1) d (19) Radia basis function (RBF ) kerne K(x i,x j ) = exp ( γ x i x j 2) (20)

7 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2593 where d is the order of the poynomia kerne function, and γ is the spread of the RBF kerne. To sove the muticass probem, we propose to define an architecture made up of as many binary SVMs as the number of information casses. Each singe SVM soves a one-againsta probem [20]. In greater detai, et Ω={ω 1,...,ω c } be the set of information casses that characterize the considered probem. The ith SVM soves a binary probem between casses ω A = ω i and ω B =Ω ω i (ω i Ω). A generic pattern x is abeed according to a winner-takes-a rue, i.e., x ω ω = arg max {f i(x)} (21) i=1,...,c where f i (x) is the output of the ith SVM. However, other muticass strategies coud be considered (see [22]). We refer the reader to [20], [22], [23], and [27] for greater detai on SVMs. IV. EXPERIMENTAL RESULTS To assess the effectiveness of the proposed approach, two different sets of experiments were conducted on two different data sets composed of Quickbird sateite images. The first data set represents a compex urban scene reated to the city of Pavia (Itay), whereas the second data set represents a rura area cose to the city of Trento (Itay). Fig. 4. Panchromatic image ( pixes) acquired by the Quickbird sateite on the city of Pavia (northern Itay). TABLE I NUMBER OF SAMPLES (IN PIXELS) IN THE TRAINING AND TEST SETS (PAVIA DATA SET) A. Pavia Data Set: Urban Area The image used in the experiments refers to the downtown area of the city of Pavia (northern Itay) and was acquired on June 23, 2002 from the Quickbird sateite. In particuar, we used a panchromatic image (Fig. 4) and a pan-sharpened mutispectra image obtained by appying a proper fusion technique to the panchromatic channe and the four bands of the mutispectra image. The adopted technique is based on the Gram Schmidt procedure impemented in the ENVI software package [25]. The fina data set is made up of a panchromatic image and four pan-sharpened mutispectra images of pixes with a spatia resoution of 0.7 m. It is worth noting that the mutiresoution fusion task artificiay increases the spatia resoution of the mutispectra channes on the one hand, whereas on the other, it may affect the spectra signatures of pixes. Nevertheess, this process was used both with the proposed method and with the standard approach adopted for comparison. We therefore do not expect it to affect the assessment of the effectiveness of the proposed approach compared with the standard method. To assess the effectiveness of the proposed method in chaenging cassification probems, we define casses in a very detaied way, by considering and covers with simiar spectra and/or geometrica attributes (e.g., buidings with different spectra signature). Tabe I shows the distribution of the sampes (in pixes) in the training and test sets among the eight and-cover casses that characterize the considered scene. These sampes have been coected by an accurate photo interpretation of the image for training and test sampes and according to the foowing guideines: 1) they have been extracted from different spatia positions in the image to propery represent casses in different portions of the scene and 2) training and test sampes have been seected from different regions to have patterns as more uncorreated as possibe. In addition, unike in standard accuracy assessment protocos, to better evauate the performance of the proposed system in both homogeneous and edge (or boundary) areas, we have spit the test set sampes in two subsets. This aows to better understand the effectiveness of the different cassification approaches in deaing with pixes with different properties in the image and resuts in a more precise accuracy assessment procedure. To evauate the effectiveness of the proposed approach, we conducted two different sets of experiments. One was aimed at assessing the effect of the number of context eves (segmentation eves) on cassification accuracy. In the other set of experiments, we compared the performances of the proposed

8 2594 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 TABLE II (a) KAPPA ACCURACY AND (b) OVERALL ACCURACY PROVIDED BY THE PROPOSED APPROACH ON EDGE, HOMOGENEOUS, AND GLOBAL TEST AREAS VERSUS THE NUMBER OF CONSIDERED CONTEXT LEVELS. THE OPTIMAL VALUES OF THE REGULARIZATION PARAMETER C AND OF THE SPREAD γ OF THE KERNEL FUNCTIONS ARE REPORTED (PAVIA DATA SET) Fig. 5. Representation of the hierarchica modeing of the context at different eves. (a) Four. (b) Five. (c) Six eves of segmentation (Pavia data set). system with those of a standard pixe-based cassifier and of an aternative technique based on a cassica feature-extraction method based on the generaized Gaussian pyramid decomposition of the image. 1) Experiment 1 Anaysis of the Effectiveness of the Proposed Approach: In our experiments, we carried out severa trias with hierarchies made up of a different number of eves (up to six eves). The first eve is the pixe eve, whereas the other five eves are obtained using the presented mutiscae hierarchica segmentation technique with different parameters to tune the homogeneity predicate. On the one hand, the eves between two and four are characterized by very sma regions. This means that from a genera point of view, objects in these eves are highy oversegmented, as shown in Fig. 5(a). In other words, a sma neighborhood system is adaptivey defined for the pixe. On the other hand, the eves between five and six are characterized by regions of medium size [Fig. 5(b) and (c)]. In particuar, at eve 6, a singe region defining the context of a pixe may contain different objects beonging to different information casses. Athough this modes the compex context of the object to which the pixe beongs, it may ead to cassification errors. We assessed the effectiveness of the proposed approach versus the number of eves considered (from two to six). The features extracted for the first eve (i.e., the pixe eve) were ony the vaues of each spectra channe and the panchromatic image. For eve 2, we ony considered the mean vaue of the digita numbers of pixes defining each region in each spectra band and the panchromatic image. From eves 3 to 6, for each region and for each band, we considered the mean vaue and the standard deviation of the digita numbers. On the whoe, 10, 20, 30, 40, and 50 features were considered for experiments with two, three, four, five, and six eves, respectivey. In the experiments, we used an SVM cassifier with RBF kernes, which have been proved effective in a number of different cassification probems. According to a proper mode seection technique [20], we have identified the best vaues of parameters (i.e., the reguarization parameter C and the spread factor of Gaussian kernes γ) using the training sampes and the goba test sampes (edge and homogeneous areas jointy) for vaidation. The highest accuracies obtained, as we as the reated parameters, are shown in Tabe II. These resuts confirm that the proposed cassification system aways exhibited a much greater overa accuracy compared with that obtained using ony the pixe eve. In detai, the greater increase in overa accuracy (i.e., about 13%) obtained with the presented approach reates to edge areas. Cassification accuracy increased aso on homogeneous areas, athough the improvement is significanty smaer (i.e., about 2%) than that in edge areas. The proposed criterion for the adaptive seection of the number of eves [see (11) and (12)] resuted in the choice of five eves. [The vaue of EA th used in (11) was reated to the expected average size of buidings present in the scene.] This confirms the effectiveness of this simpe criterion that seected the number of eves that provided the highest cassification accuracy on the goba test set. It is worth noting that the proposed approach provided stabe accuracies for a number of eves cose to the one identified by the automatic procedure (in the range between four and six eves), exhibiting Kappa vaues between 0.71 and 0.74 on edge areas and between 0.94 and 0.96 on homogeneous areas. This confirms its abiity to mode the spatia context of each anayzed pixe. As one can see from Tabe II, six context eves ead to a sight decrease of cassification accuracies. This behavior is due to the significant undersegmentation of rea objects at eve 6, which may affect cassification accuracy both on edge and homogeneous areas. In the second part of this experiment, according to the obtained resuts, we considered ony four, five, and six context eves, as they gave the highest cassification accuracy. To better evauate the performance of the proposed cassification system, we aso anayzed the cassification maps obtained in a trias. We report ony on a sma representative portion of the obtained maps, to present exampes that show both the advantages and the imitations of the proposed system. Fig. 6 shows a sma portion of the cassification maps obtained with (a) four, (b) five, and (c) six segmentation eves and (d) with ony the pixe eve. As can be seen, whereas the crossroad in the center of the images (within the red rectange) is we modeed in Fig. 6(b) and (c), the resuts are inaccurate in Fig. 6(a) because of high fragmentation in the modeing of the spatia context at the higher eve. In the map obtained using ony the pixe eve (without contextua information) reported in Fig. 6(d), we can

9 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2595 Fig. 6. Detai of cassification maps obtained with (a) four, (b) five, and (c) six segmentation eves and (d) with ony the pixe eve. The rea-coor Quickbird image is reported in (e). The red rectange shows an exampe of the effects of the different eves of the spatia context. The egend of the maps is reported in the caption of Fig. 8 (Pavia data set). see that the shape of the buidings is not we modeed, and in many cases, homogeneous areas are not correcty cassified. In addition, the crossroad is not propery recognized. On the other hand, in some areas, using fewer eves (i.e., very sma regions to characterize the adaptive neighborhood) eads to a better definition of sma detais. For exampe, in Fig. 7(a), sma roads are we cassified, whereas in Fig. 8(b) and (c), by expoiting more context eves, we obtain a poor representation of objects in the scene under investigation. It is worth noting that we carried out aso some trias by using geometrica (minimum rectanguar fit, width-to-ength ratio, etc.) and reationa (number of neighbors of an object and number of sub-objects that compose an object at the upper eve) features. These kinds of features were extracted ony for the higher eves of the representation (eves 5 and 6). The obtained resuts did not improve both the cassification accuracies and the quaity of the cassification maps. This behavior mainy depends on the criterion adopted for the definition of casses. Inasmuch as many casses share the same geometrica features (e.g., different kinds of buiding are discriminated ony on the basis of the spectra signature), in this case, the use of the geometrica and reationa information does not increase the separabiity among casses in the feature space. 2) Experiment 2 Comparisons With a Feature-Extraction Modue Based on a Generaized Gaussian Pyramid Decomposition: The aim of the second set of experiments is to compare the proposed system with a different approach to mutieve Fig. 7. Cassification maps obtained with (a) four, (b) five, and (c) six eves of segmentation and (d) with ony the pixe eve. The rea-coor Quickbird image is reported in (e). The egend of the maps is reported in the caption of Fig. 8 (Pavia data set). feature extraction of very high resoution images based on the generaized Gaussian pyramid decomposition. In detai, the panchromatic and pan-sharpened images are iterativey anayzed by a Gaussian kerne ow-pass fiter, with 5 5 square anaysis window, and are undersamped by a factor of 2. In this way, it is possibe to obtain a simpe mutiscae decomposition of the scene. In our experiments, we expoit five eves of pyramida decomposition (this is the number of eves that gives the highest accuracy between two and six) to characterize the spatia context of pixes and to abe each pixe of the scene under investigation. The extracted feature vector was made up of 25 spectra features. The SVM cassification modue was aso used in these trias. The best accuracies obtained for the proposed technique and for the reference feature-extraction technique are reported in terms of Kappa coefficient and overa accuracy in Tabe III. These resuts show that the proposed feature-extraction technique provided an accuracy higher than the reference method. The accuracy obtained on test edge areas confirms the greater abiity of the proposed approach (which increased Kappa vaues by 8.5% compared with the generaized Gaussian pyramid method) to mode the geometrica detais of objects in the scene, such as roofs and roads. A comparison between the accuracies obtained on homogeneous areas points out a gap of 2.4%. To better assess the effectiveness of the investigated methods, Fig. 8(a) and (b) shows the cassification maps obtained using the proposed cassification system and the reference system. A quaitative anaysis of the maps confirms the previous consideration based on the quantitative resuts. The adaptive and mutieve properties of the proposed feature-extraction

10 2596 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 TABLE III (a) KAPPA ACCURACY AND (b) OVERALL ACCURACY ON EDGE, HOMOGENEOUS, AND GLOBAL TEST AREAS VERSUS THE PROPOSED FEATURE EXTRACTION TECHNIQUE (ADAPTIVE HIERARCHICAL CONTEXT MODELING) AND THE REFERENCE TECHNIQUE (GENERALIZED GAUSSIAN PYRAMID REDUCTION)(PAVIA DATA SET) and compex structures accuratey. On the contrary, when a mutieve segmentation agorithm is used to adaptivey mode the neighborhood of a pixe, a proper representation of the edges of the objects is obtained. Fig. 8. Cassification maps obtained by an SVM cassifier (a) with the proposed feature extraction modue and (b) with a feature extraction agorithm based on a pyramida Gaussian decomposition (Pavia data set). technique can better mode the edge of objects in the scene, especiay in areas with sma detais. The great compexity of the anayzed scene, which incudes different types of buidings of different sizes and different types of roads, shows that the ow-pass fiter used in the generaized Gaussian pyramid decomposition is not suitabe to mode the boundaries of objects B. Trento Data Set: Rura Area The image used in these experiments refers to the rura area of Trento (northern Itay) and was acquired on March 30, 2004 from the Quickbird sateite. The data set consists of a panchromatic image (Fig. 9) and four pan-sharpened images of pixes with a spatia resoution of 0.7 m. The pan-sharpened images were obtained with the Gram Schmidt procedure [25]. Tabe IV shows the distribution of the sampes (in pixes) in the training and test sets among the eight and-cover casses that characterize the considered scene. We have seected the ground truth according to the guideines foowed in the previous data set on the Pavia area. As in the case of the Pavia data set, to evauate the effectiveness of the proposed approach, we conducted two different sets of experiments. The first one was aimed at assessing the effects of both the number of context eves and the different kinds of extracted features on the cassification accuracies. The second one compared the performances of the proposed system with those of the feature-extraction method based on the generaized Gaussian pyramid decomposition of the image. 1) Experiment 1 Anaysis of the Effectiveness of the Proposed Approach: In our experiments, we carried out severa trias with hierarchies made up of two to seven context eves. The first eve is the pixe eve, whereas the other six eves are obtained according to the presented mutiscae hierarchica segmentation technique with different parameters to tune the homogeneity predicate. As in experiments on the urban data set, eves between two and three are characterized by very sma regions. This means that from a genera point of view, objects in these eves are highy oversegmented. Leves 4 and

11 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2597 TABLE V (a) KAPPA ACCURACY AND (b) OVERALL ACCURACY PROVIDED BY THE PROPOSED APPROACH ON EDGE, HOMOGENEOUS, AND GLOBAL TEST AREAS VERSUS THE NUMBER OF CONSIDERED CONTEXT LEVELS. THE OPTIMAL VALUES OF THE REGULARIZATION PARAMETER C AND OF THE SPREAD γ OF THE KERNEL FUNCTIONS ARE REPORTED (TRENTO DATA SET) Fig. 9. Panchromatic image ( pixes) acquired by the Quickbird sateite on the city of Trento (northern Itay). TABLE IV NUMBER OF SAMPLES (IN PIXELS) IN THE TRAINING AND TEST SETS (TRENTO DATA SET) 5 are characterized by regions of medium size. Leves 6 and 7 contain regions that represent (or incude) the objects present in the scene. The features extracted for the first eve (i.e., the pixe eve) were ony the pixe vaues in a the spectra channes and the panchromatic image. For eve 2, we ony considered the mean vaue of the digita numbers of pixes defining each region in each spectra band and the panchromatic image. From eves 3 to 7, for each region and for each band, we considered the mean vaue and the standard deviation of the digita numbers. On the whoe, 10, 20, 30, 40, 50, and 60 features were considered for experiments with two, three, four, five, six, and seven eves, respectivey. In a the experiments, we used an SVM cassifier with RBF kernes. According to a proper mode seection technique [20], we identified the best vaues of the reguarization parameter C and the spread factor of Gaussian kernes γ using the training set sampes and the goba test set sampes for vaidation. The highest accuracies obtained, as we as the reated parameter vaues, are shown in Tabe V. These resuts confirm the effectiveness of the proposed cassification system, which aways exhibited a greater overa accuracy compared with that obtained using ony the pixe eve (much greater starting from three eves of context representation). In detai, the greatest increase in overa accuracy (i.e., about 10%) was obtained on edge test areas with six eves. With this number of eves, cassification accuracy increased aso on homogeneous test areas with an improvement of about 3%. It is worth nothing that these resuts confirm the effectiveness of the empirica criterion for the seection of the number of eves described in (11), which on this data set identified an optima number of decomposition eves equa to six. 5 By anayzing Tabe V, one can see that the proposed approach provided stabe accuracies versus the number of eves considered in the neighborhood of the optima number of scaes identified with the proposed empirica criterion (in the range between five and seven eves). In particuar, it exhibited Kappa vaues between 0.62 and 0.64 on edge areas and between 0.97 and 0.98 on homogeneous areas. This confirms the abiity of the presented methodoogy to mode the spatia context of each anayzed pixe. According to the previous resuts, in the second part of this experiment, we considered ony five and six eves, as they gave the highest cassification accuracies on the overa test set. To assess the importance of the use of geometrica features on this data set, we computed some geometrica parameters from the regions extracted at eves 5 and 6. We considered the foowing features. 1) Width-to-ength ratio. It can be cacuated as the ratio between the ength and the width of the bounding box that contains the object under investigation. 2) Shape index. It can be obtained by the ratio of the border ength of the object under anaysis and four times the square root of its area. 5 In this case, we used a hybrid approach for defining the EA th parameter, by considering an average of the size of the objects that compose the rura scene (i.e., buidings and crops).

12 2598 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 TABLE VI (a) KAPPA ACCURACY AND (b) OVERALL ACCURACY PROVIDED BY THE PROPOSED APPROACH ON EDGE, HOMOGENEOUS, AND GLOBAL TEST AREAS VERSUS THE NUMBER OF CONSIDERED CONTEXT LEVELS WHEN GEOMETRICAL FEATURES WERE ALSO CONSIDERED. THE OPTIMAL VALUES OF THE REGULARIZATION PARAMETER C AND OF THE SPREAD γ OF THE KERNEL FUNCTIONS ARE REPORTED (TRENTO DATA SET) TABLE VII (a) KAPPA ACCURACY AND (b) OVERALL ACCURACY ON EDGE, HOMOGENEOUS, AND GLOBAL TEST AREAS VERSUS THE PROPOSED FEATURE-EXTRACTION TECHNIQUE (ADAPTIVE HIERARCHICAL CONTEXT MODELING) AND THE REFERENCE TECHNIQUE (GENERALIZED GAUSSIAN PYRAMID REDUCTION)(TRENTO DATA SET) 3) Rectanguar fit. It can be obtained as the ratio between the area not covered by a rectange with the same area and proportion of the object under investigation, and the area of the object. On the whoe, 46 and 56 features were considered for experiments with five and six eves, respectivey. Tabe VI reports the best accuracies obtained with these features after performing a new mode seection for the SVM cassifier. These resuts point out that, on this data set, the use of geometrica features increases the accuracy on edge areas with respect to that obtained by using ony spectra features, at the expense of a sight decrease of accuracy in homogeneous areas. In greater detai, in the six-eve case, we obtained an increase of about 3% in terms of Kappa accuracy over edge areas and a decrease of 1% over homogeneous areas. This interesting resut, which does not seem intuitive, can be expained as foows. On the one hand, the use of geometrica features aows a better characterization of pixes cose to the border areas, which are better attracted from the geometry of objects to which they beong. On the other hand, oversegmentation errors sighty affect the accuracy on homogeneous areas, where spectra and textura features are sufficient for obtaining high accuracies. 2) Experiment 2 Comparisons With a Feature-Extraction Modue Based on a Generaized Gaussian Pyramid Decomposition: Aso, on this study area, the aim of the second set of experiments is to compare the proposed system with a mutieve feature extraction based on the generaized Gaussian pyramid decomposition. As in the previous case, we used five eves of pyramida decomposition to characterize the spatia context of pixes (five eves resuted in the highest accuracies on the goba test set) and adopted an SVM-based cassification modue. The extracted vector was made up of 25 features. The best resuts (in terms of Kappa and overa accuracies) obtained with the proposed technique (with six eves and the same feature vector extracted in the first part of the previous experiment) and with the generaized Gaussian pyramid technique are reported in Tabe VII. These resuts confirm the better capabiity of the proposed approach to mode the geometrica detais of objects in the scene. In greater detai, it increased the Kappa vaue on the edge areas by about 19% compared with the generaized Gaussian pyramid method. In addition, a sight increase of accuracy on homogeneous areas was obtained (i.e., about 2%). To better assess the effectiveness of the investigated methods, Fig. 10(a) and (b) shows the cassification maps obtained using the proposed cassification system and the system based on the Gaussian pyramid feature extraction. A quaitative anaysis of maps in Fig. 10 confirms the previous consideration based on the quantitative resuts. The adaptive and mutieve properties of the proposed feature-extraction technique can better mode the edge of objects in the scene. In greater detai, the map in Fig. 10(b) shows that the generaized Gaussian pyramid decomposition is not suitabe to accuratey mode the boundaries of objects and compex structures due to a burring effect. On the contrary, when the proposed mutieve segmentation agorithm is used to adaptivey mode the neighborhood of a pixe, a proper representation of the edges of the objects is obtained. V. D ISCUSSION AND CONCLUSION In this paper, a nove system for the cassification of very high resoution images has been presented. The system is made up of: 1) a feature-extraction modue that adaptivey modes the spatia context of each pixe according to a compete hierarchica mutieve representation of the scene under investigation and 2) a proper cassifier based on SVMs. In greater detai, a hierarchica segmentation is appied to the images to obtain segmentation resuts at different eves of resoution according to tree-based hierarchica constraints. In this way, precise hierarchica reationships are estabished between each pixe in the image and the regions that adaptivey define its context at different eves. Each pixe is characterized by a feature vector that incudes both the pixe-eve information in the spectra channes of the sensor and the attributes of a the regions, which represent the mutieve reationships of the pixe and define its spatia context adaptivey.

13 BRUZZONE AND CARLIN: PIXEL-BASED SYSTEM FOR VERY HIGH SPATIAL RESOLUTION IMAGE CLASSIFICATION 2599 Depending on the eve considered, different kinds of features are extracted to characterize the regions with the most reiabe attributes for the specific scae anayzed and the specific scene considered. It is worth noting that in our technique, unike other approaches proposed in the iterature, a features associated both with the pixe eve and a the region eves are jointy considered in the cassification phase to abe a pixe. This hierarchica representation aows to capture and expoit the entire information in the scene by working with adaptive regions at different scaes. To dea with the arge number of feature-vector components to be given to the cassifier as input, we used a machine-earning cassifier based on SVMs. This choice depends both on the effectiveness of SVMs cassifiers and on their capabiities to anayze a high-dimensiona feature space with a reduced effect of the Hughes phenomenon. Experimenta resuts, obtained on two very high geometrica resoution Quickbird images acquired on a compex urban area and on a rura area, confirm the effectiveness of the proposed cassification system. In detai, two main experiments have been carried out. In the first, we focused on the number of eves to be used to mode the context of a pixe. The resuts show that varying the number of eves used to characterize the spatia context adaptivey, in a range cose to the optima eve identified by the empirica criterion proposed in (11) and (12), does not criticay change the overa accuracy and the quaity of cassification maps. In the second set of experiments, we compared the proposed feature-extraction technique with a standard feature-extraction agorithm based on a generaized Gaussian decomposition pyramid. The SVM cassifier was aso used in these experiments. The experimenta resuts confirm that the proposed feature-extraction modue outperforms the reference method based on the Gaussian pyramida reduction. This is due both to the adaptive and to the mutieve nature of the proposed feature-extraction modue, which by expoiting a hierarchica segmentation agorithm can mode the objects (shapes and reationships at different eves of resoution) in the scene under investigation better, compared with the feature-extraction modue based on the generaized Gaussian pyramid decomposition. ACKNOWLEDGMENT The authors wish to thank A. Garzei (University of Siena, Itay) for providing the Quickbird image of Pavia, R. Rigon (University of Trento, Itay) for providing the Quickbird image of Trento (which was acquired in the framework of the project ASI 175/02 funded by the Itaian Space Agency), and A. Baradi (Joint Research Centre, Ispra, Itay) for usefu discussions about the segmentation agorithm. The authors are aso gratefu to the anonymous referees for their constructive comments. Fig. 10. Cassification maps obtained by an SVM cassifier (a) with the proposed feature extraction modue and (b) with a feature extraction agorithm based on a pyramida Gaussian decomposition (Trento data set). REFERENCES [1] F. Vope and L. Rossi, Quickbird high resoution sateite data for urban appication, in Proc. 2nd GRSS/ISPRS Joint Workshop Data Fusion and Remote Sens. Over Urban Areas, May 2003, pp [2] S. R. Repaka, D. D. Truax, E. Kostad, and C. G. O Hara, Comparing spectra and object based approaches for cassification and transportation feature extraction from high resoution mutispectra imagery, in Proc. ASPRS Annu. Conf., May 2004.

14 2600 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 [3] S. P. Lennartz and R. G. Congaton, Cassifying and mapping forest cover types using Ikonos imagery in the northeastern United States, in Proc. ASPRS Annu. Conf., May [4] L. M. Moska, Historica andscape visuaization of the Wison s creek nationabattefied based on object oriented tree detection method from Ikonos imagery, in Proc. ASPRS Annu. Conf., May [5] S. J. Goetz, R. K. Wright, A. J. Smith, E. Zineckerb, and E. Schaub, IKONOS imagery for resource management: Tree cover, impervious surfaces, and riparian buffer anayses in the mid-atantic region, Remote Sens. Environ., vo. 88, no. 1/2, pp , Nov [6] A. Career, O. Debeir, and E. Woff, Comparison of very high spatia resoution sateite image segmentation, in Proc. SPIE Conf. Image and Signa Processing Remote Sensing IX, vo. 5238, pp [7] E. Binaghi, I. Gao, and M. Pepe, A neura adaptive mode for feature extraction and recognition in high resoution remote sensing imagery, Int. J. Remote Sens., vo. 24, no. 20, pp , Oct [8] T. Baschke, Object-based contextua image cassification buit on image segmentation, in Proc. IEEE Workshop Adv. Tech. Ana. Remote Sensed Data, Oct. 2003, pp [9] C. Unsaan and K. L. Boyer, Cassifying and deveopment in highresoution panchromatic sateite images using straight-ine statistics, IEEE Trans. Geosci. Remote Sens., vo. 42, no. 4, pp , Apr [10] A. K. Shackeford and C. H. Davis, A hierarchica fuzzy cassification approach for high-resoution mutispectra data over urban areas, IEEE Trans. Geosci. Remote Sens., vo. 4, no. 9, pp , Sep [11] M. De Martinao, F. Causa, and S. B. Serpico, Cassification of optica high resoution images in urban environment using spectra and textura information, in Proc. IGARSS, Ju. 2003, vo. 1, pp [12] R. M. Haraick and L. G. Shapiro, Image segmentation techniques, Comput. Vis. Graph. Image Process., vo. 29, no. 1, pp , Jan [13] U. C. Benz, P. Hofmann, G. Wihauck, I. Lingenfeder, and M. Heynen, Muti-resoution, object-oriented fuzzy anaysis of remote sensing data for GIS-ready information, ISPRS J. Photogramm. Remote Sens., vo.58, no. 3/4, pp , Jan [14] C. Burnett and T. Baschke, A muti-scae segmentation/object reationship modeing methodoogy for andscape anaysis, Int. J. Eco. Mode. Syst. Eco., vo. 168, no. 3, pp , Oct [15] E. Binaghi, I. Gao, and M. Pepe, A cognitive pyramid for contextua cassification of remote sensing images, IEEE Trans. Geosci. Remote Sens., vo. 41, no. 12, pp , Dec [16] A. K. Shackeford and C. H. Davis, A combined fuzzy pixe-based and object-based approach for cassification of high-resoution mutispectra data over urban areas, IEEE Trans. Geosci. Remote Sens.,vo.41,no.10, pp , Oct [17] J. A. Benediktsson, M. Pesaresi, and K. Arnason, Cassification and feature extraction for remote sensing images from urban areas based on morphoogica transformations, IEEE Trans. Geosci. Remote Sens., vo. 41, no. 9, pp , Sep [18] C. Mott, T. Andresen, S. Zimmermann, T. Schneider, and U. Ammer, Seective region growing-an approach based on objectoriented cassification routine, in Proc. IGARSS, Jun. 2002, vo. 3, pp [19] R. A. Schowengerdt, Remote Sensing. Modes and Methods for Image Processing, 2nd ed. Norwe, MA: Academic, [20] N. Cristianini and J. Shaew-Tayor, An Introduction to Support Vector Machines and Other Kerne Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, [21] L. Bruzzone and F. Megani, Support vector machines for cassification of hyperspectra remote-sensing images, in Proc. IGARSS, Jun. 2002, vo. 1, pp [22] F. Megani and L. Bruzzone, Cassification of hyperspectra remotesensing images with support vector machines, IEEE Trans. Geosci. Remote Sens., vo. 42, no. 8, pp , Aug [23] T. Joachims, Making Large-Scae SVM Learning Practica. Advances in Kerne Methods Support Vector Learning, B. Schökopf, C. Burges, and A. Smoa, Eds. Cambridge, MA: MIT Press, [24] A. Baradi, L. Bruzzone, and P. Bonda, Bady-posed cassification of remotey sensed images An experimenta comparison of existing data mapping system, IEEE Trans. Geosci. Remote Sens., vo. 44, no. 1, pp , Jan [25] ENVI User Manua. Bouder, CO: RSI, [Onine.] Avaiabe: [26] G. Huges, On the mean accuracy of statistica pattern recognizers, IEEE Trans. Inf. Theory, vo. IT-14, no. 1, pp , Jan [27] V. Vapnik, Statistica Learning Theory. New York: Wiey, [28] Definiens Imaging, ecognition Professiona User Guide 4, 2003, Munich, Germany. [Onine]. Avaiabe: Lorenzo Bruzzone (S 95 M 98 SM 03) received the aurea (M.S.) degree in eectronic engineering (summa cum aude) and the Ph.D. degree in teecommunications from the University of Genoa, Genoa, Itay, in 1993 and 1998, respectivey. From 1998 to 2000, he was a Postdoctora Researcher at the University of Genoa. From 2000 to 2001, he was an Assistant Professor at the University of Trento, Trento, Itay, and from 2001 to 2005, he was an Associate Professor at the same university. Since March 2005, he has been a Fu Professor of teecommunications at the University of Trento, where he currenty teaches remote sensing, pattern recognition, and eectrica communications. He is currenty the Head of the Remote Sensing Laboratory in the Department of Information and Communication Technoogy, University of Trento. His current research interests are in the area of remote-sensing image processing and recognition (anaysis of mutitempora data, feature seection, cassification, regression, data fusion, and machine earning). He conducts and supervises research on these topics within the frameworks of severa nationa and internationa projects. Since 1999, he has been appointed Evauator of project proposas for the European Commission. He is the author (or coauthor) of more than 150 scientific pubications, incuding journas, book chapters, and conference proceedings. He is a Referee for many internationa journas and has served on the Scientific Committees of severa internationa conferences. Dr. Bruzzone ranked first pace in the Student Prize Paper Competition of the 1998 IEEE Internationa Geoscience and Remote Sensing Symposium (Seatte, Juy 1998). He was a recipient of the Recognition of IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING Best Reviewers in 1999 and was a Guest Editor of a Specia Issue of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING on the subject of the anaysis of mutitempora remote-sensing images (November 2003). He was the Genera Chair and Cochair of the First and Second IEEE Internationa Workshop on the Anaysis of Muti-tempora Remote-Sensing Images. Since 2003, he has been the Chair of the SPIE Conference on Image and Signa Processing for Remote Sensing. He is an Associate Editor of the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. He is a member of the Scientific Committee of the India Itay Center for Advanced Research. He is aso a member of the Internationa Association for Pattern Recognition and of the Itaian Association for Remote Sensing (AIT). Lorenzo Carin (S 06) received the aurea (B.S.) and Laurea Speciaistica (M.S.) degrees in teecommunication engineering (summa cum aude) from the University of Trento, Trento, Itay, in 2001 and 2003, respectivey. He is currenty working toward the Ph.D. degree in information and communication technoogies at the same university. He is currenty with the Pattern Recognition and Remote Sensing group at the Department of Teecommunication and Information Technoogies, University of Trento. His main research activity is in the area of pattern recognition appied to remote sensing images; in particuar, his interests are reated to cassification of very high resoution remote sensing images. He conducts research on these topics within the frameworks of severa nationa and internationa projects. He is a referee for the Itaian Journa of Remote Sensing (AIT). Mr. Carin is a referee for the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING.