An Edge-based Method for Registering a Graph onto an Image with Application to Cadastre Registration

Transcription

1 An Edge-based Method for Registering a Graph onto an Image with Application to Cadastre Registration Roger Trias-Sanz March 2004 Rapport technique 2004/1 du laboratoire SIP, CRIP5 Université de Paris 5 avec la collaboration de l Institut Géographique National

2

3 BIBTEX document author = {Trias-Sanz, Roger}, title = { An Edge-based Method for Registering a Graph onto an Image with Application to Cadastre Registration}, institution = {SIP-CRIP5 (Universit{\ e} de Paris 5) and Institut G{\ e}ographique National}, year = 2004, type = {technical report}, number = {2004/1}, address = {45 rue des Saints-P{\ e}res, Paris, France}, month = mar }

4 Abstract In the context of the development of a land use analysis system, we need to register cadastre edges onto color and near-infrared images at 50cm resolution. The mis-registration to be corrected exists not only because of geometrical deformation or acquisition errors, but because land use does not always follow cadastre divisions. We formalize this as a graph matching problem which can be solved by simulated annealing. As a side effect a score for each cadastre edge is obtained, which shows which edges can be found in the image and which cannot. 1. Introduction Salade is an ongoing project which aims at segmenting high-resolution (50cm) images into fields and other regions with very high reliability. Cadastre data helps by providing additional information about crop distribution and position. However, cadastre and image edges rarely match exactly, because land owners do not usually follow cadastre limits when growing their crops. We present a method for registering a cadastre graph onto an image. Registration of cadastres to images has been viewed as a non-rigid registration problem [2, 6]. The goal is to find the transformation within a class that best converts an image to a reference image. This approach works when the initial mis-registration is due to sensor deformation. In our case it is rather due to the two graphs representing data of different nature; we also want the cadastre to register onto image edges as much as possible. More precisely, we want to locally modify the cadastre graph so that its spatial structure is preserved (that is, the face topology of its planar representation) while incorporating the geometrical details of corresponding salient edges in the image. We approach this as a graph matching problem [4]: in this context we have two graphs representing the same physical reality, and the goal is to match the edges and nodes that correspond to the same part of that reality [1, 7]. In our case, we represent the salient edges in the image as a weighted graph, which we match with the cadastre graph. Tests show a 28.5% reduction in the average distance between the cadastre and a ground truth, and, more importantly for us, the registered portions of the resulting graph follow the edges of the image, so statistical analysis of regions in this graph will be less affected by adjacent regions. 2. Registration algorithm The cadastre input, which we simplify using mathematical morphology to remove thin faces, is already a graph. We create a graph (the segmentation graph) representing the salient edges in the image and their geometry (see section 2.1). This allows us to formalize this as a graph matching problem: We match the cadastre graph to the segmentation graph, and then transpose the geometrical information from the edges of the segmentation graph to the corresponding cadastre edges. This graph matching is asymmetric: the segmentation graph contains many more terrain edges than the cadastre graph, but they are shorter. We may assume that each terrain edge matches at most one cadastre edge, and that each cadastre edge may match several terrain edges. Most terrain edges will not have a match. In section 2.2 we formalize the representation of a solution, that is, a match. This will be a mapping from each cadastre edge to a chain of terrain edges. We can evaluate the quality or fitness of a match (section 2.4) and therefore use an optimization algorithm to find the optimal one. To avoid the combinatorial explosion associated with this kind of problem, we use simulated annealing to find a near-optimal solution. For that we also need a way of exploring the solution space, and an initial solution (section 2.5). However, we have found that, in order to register the cadastre onto image edges and at the same time preserve the spatial structure of the cadastre graph, we have to process separately the areas near cadastre nodes and the rest

5 of the problem. We first apply the simulated annealing optimization to the rest of the problem, and then an ad-hoc method (section 3) for the areas near cadastre nodes. The cadastre and segmentation graphs have a geometrical component: their edges and vertices exist in a space. Let G = (V, E) be a directed graph, with vertices V and edges E V 2. Let S be a topological space, for example R 2, or Z 2 with 4-connexity. The trace function, trc, maps each vertex to a point in S, and each edge to a directed connected curve in S, so that the trace of an edge e = (v 1, v 2 ) starts and ends at the traces of v 1 and v 2 respectively. We call G = (V, E, trc, S) a geometrical graph. The cadastre graph is a geometrical graph, C = (V c, E c, trc, Z 2 ) Image over-segmentation To obtain the segmentation graph, we need to extract the topology and geometry of the salient edges in an image. We could do that by calculating a watershed segmentation of an image and attributing each edge in the segmentation with, say, the module of the image gradient in that edge. This measure of edge saliency would be local and single-scale. However, different meaningful structures appear at different scales [5]. Several authors have proposed multi-scale algorithms to solve this. We use Guigues scalesets algorithm [3] because it makes the segmentation criterion and the scale parameter λ explicit: For each λ this algorithm gives a partition of the image p(λ); the set of values of λ for which a region exists in p(λ) turns out to be an interval, [λ min, λ max [. We flatten these results to obtain the segmentation graph: We build a geometrical weighted graph, the terrain graph T = (V t, E t, trc, Z 2, w) (w the weight function), whose edges follow the boundaries of the regions given by p( ). We find for each edge e the highest λ min of all the regions whose boundary contains e, λ min (e) (see figure 3, 3rd image). To improve processing time, we discard edges with small λ min. We sort the rest as λ min (e s0 ) > λ min (e s1 ) > > λ min (e sn 1 ) and attribute each edge with the apparition weight w, w(e si ) = e αi/n Solution representation We represent a solution to our registration problem as a relation between cadastre edges and terrain edges. In the backward representation, we label each terrain edge with its corresponding cadastre edge, or with to leave it unmatched (see [4] for a similar representation). We only allow labeling a terrain edge e i with cadastre edges that are close to it, N(e i ), N(e i ) = {e E c : min z trc e i z trc e z z < ɛ}, (1) and do not allow labeling with for terrain edges with only one near cadastre edge: this disables the optimization process for these edges, leaving only the shortest-path search described in section 2.3. A backward solution is a mapping S from each terrain edge e i to N x (e i ), S : e i N x (e i ) where { N(e i ) { } if N(e i ) 1, N x (e i ) = N(e i ) if N(e i ) = From backward to forward representations The backward representation does not give a single terrain chain matched to a cadastre edge, but rather an unordered set of edges. We convert it to a forward representation, in which we map each cadastre edge to a possibly empty oriented chain of terrain edges, as follows: Let s focus on cadastre edge e k E c, to which the terrain edges m k = {e i E t : S(e i ) = e k } are mapped. We create a graph T k which contains

6 the edges m k and the nodes at their ends, the nodes n 0 and n 1 at the ends of e k, N 0, the set of terrain nodes which are close to n 0, and N 1, the set of terrain nodes which are close to n 1. We attribute each edge e T k with a weight (the path matching weight) that depends on its length, its apparition weight, and the average distance between it and e k. T k contains several connected components. We create a graph T k with the edges and nodes from T k and additional edges with straight traces (which we call connecting edges), which join every two connected components in T k by their closest nodes. We attribute them with a path matching weight that depends on their length. Using the path matching weight as the length of an edge, we find on T k the shortest path s k from any node in N 0 {n 0 } to any node in N 1 {n 1 } using Dijkstra s algorithm. The forward solution is then the mapping of each cadastre edge e k to the path s k. Thanks to the connecting edges, there is always such a path; however, the path matching weight for connecting edges is high, to discourage their use. Note that since we allow these shortest paths to start and end not only on the endpoints of cadastre edges (n 0, n 1 ) but also on nearby terrain nodes (N 0, N 1 ), this will not register the cadastre in the areas near cadastre nodes. We show this in figure 1. Left to right: a cadastre edge e k ; terrain graph E t (edge weight shown by line darkness); subgraph T k, with N 0 and N 1 (hollow dots), and n 0 and n 1 (solid dots); shortest path, including connecting edges. Figure 1. From backward to forward solution Calculating a solution s fitness The sum of path matching weights in the paths s k from section 2.3 gives the quality of a registered cadastre edge. To get the fitness of the whole solution, we add these lengths and include two penalties: First, many terrain edges not labeled with are not actually used in any shortest path; to simplify the solutions, we impose a penalty on their quantity. Second, since the path matching weights are set independently for each cadastre edge, in some cases chains may cross each other; we add a penalty to these crossings Finding a similar and an initial solution To solve a problem using simulated annealing we need a way to find solutions which are close to a given solution, so as to explore the solution space, and an initial solution. We obtain one neighbor solution S from S by modifying the label of a certain number of terrain edges, but only for those edges close to two or more cadastre edges (eq. 1). There is a fixed probability p that the new label

7 be : p ( S (e i ) = ) = p p ( S (e i ) = e k ) = 1 p N(e i ), for all e k N(e i ). We can obtain an initial solution S 0 by labeling each terrain edge with the closest cadastre edge, or with if the closest cadastre edge is farther than a certain threshold. 3. Registration near cadastre nodes The optimization process alone (described in section 2) cannot at the same time register the cadastre onto salient image edges and preserve the spatial structure of the cadastre graph: To preserve the spatial structure in areas of the image with a low density of image edges we need to allow the use of connecting edges, which do not follow salient image edges. In these same low density areas, strictly following image edges may modify the spatial structure of the cadastre, splitting or deleting faces in the cadastre graph, for example. Additionally, adding connecting edges may make the graph non-planar. The underlying reason is that it is not really possible to register cadastre edges which do not have a corresponding image edge. These problems tend to occur at the areas near nodes of the cadastre graph. For this reason, the optimization procedure does not attempt to register the cadastre there (section 2.3). After the optimization phase, we process these areas as follows to complete the registration: For the cadastre node n, we have a set of cadastre edges incident to it, I. We split I into subsets J 1, J 2,..., J m, grouping those edges whose endpoints can be connected by following only terrain edges close to n; we then connect these endpoints using these terrain edges. Finally, we join the sets J 1,..., J m, by adding appropriate PSfrag straight replacements edges to the registered cadastre graph (see figure 2: J 1 = {a}, J 2 = {b, c}, J 3 = {d, e}; dotted curves are unused terrain edges near n). b J 2 b J 2 J 1 a n e J 3 c d J 1 a n e J 3 c d Figure 2. Before (left) and after (right) registration near cadastre node n. 4. Registration ratio For each cadastre edge c we calculate a quality measure, the registration ratio, which measures which portion of the edge was registered to real terrain edges instead of to connecting edges. For example, a cadastre edge which is fully registered to terrain edges has a registration ratio of 1, while a cadastre edge which corresponds to a single connecting edge from end to end has a registration ratio of 0. This appears to be to be confirmed in future research a good indicator of whether or not the cadastre edge follows a true terrain limit. We plan to use it to decide whether to merge adjacent cadastre regions.

8 5. Experiments and evaluation We have run the algorithm on a test site of 4 km 2, on which we defined a ground truth. We let the optimizer run for 2000 iterations, taking around 5 minutes. The segmentation was computed using the color components of an image (downsampled to 2 m per pixel to improve speed). In future work we plan to also use texture information and region shape for segmenting. We drew the registered cadastre graph and the ground truth, and calculated the average distance between pixels in the rasterized cadastre graph and pixels in the ground truth. We excluded from this average those cadastre edges for which the average distance to the ground truth was larger than 6 m, or which had a pixel that was farther than 20 m away from the ground truth these non-matchable edges probably do not have a corresponding image edge. We calculated the average distance between the unregistered cadastre graph and the ground truth in the same way. The results are summarized in table 1 (length does not include non-matchable edges). Figure 3 shows the procedure for a 0.5 km 2 region: the source image, the cadastre graph C, the segmentation graph T (darker edges have higher weights), the registered cadastre (darker edges have higher registration ratios), and a close-up (left: unregistered, right: registered). Edges which cannot be registered, such as the left-most one, are drawn as straight lines, which gives a strange visual effect; they will be removed in further processing steps anyway. unregistered registered area km km 2 ground truth length km km length 63.2 km 65.7 km mean distance m m Table 1. Evaluation results. The algorithm successfully registers the cadastre onto the image, and reduces the average distance between the cadastre graph and a ground truth from 2.35 m to 1.68 m, 28.5% less. The resulting graph is registered onto the segmentation edges, hence onto salient edges in the image statistical analysis of these regions will be less perturbed by adjacent regions. Visual inspection seems to show that the registration ratio indicates if a cadastre edge actually exists in the image. Using this information to delete cadastre edges that cannot be found in the image should be straightforward, and will be done in the near future. 6. Conclusion We have presented an asymmetric graph matching algorithm which can be used to register a cadastre graph onto an aerial image. We use a multi-scale segmentation of the image which is converted to a weighted graph. These graphs are asymmetrically matched by optimizing, using simulated annealing, the fitness of a solution. We use an ad-hoc method for the registration near cadastre nodes. Testing gives good results. As a side effect of the registration by simulated annealing we obtain a parameter, the registration ratio, which seems useful as an indicator of which cadastre edges actually exist in the image. The author would like to thank L. Guigues for his insightful comments on this work. References [1] S. Gautama and A. Borghgraef. Using graph matching to compare VHR satellite images with GIS data. In Proc. IGARSS 2003, July 2003.

9 [2] A. Goshtasby et al. Nonrigid image registration: guest editors introduction. Computer Vision and Image Understanding, 89(2/3): , Feb./Mar [3] L. Guigues, H. Le Men, and J.-P. Cocquerez. Scale-sets image analysis. In Proc. ICIP 2003, Sept [4] C. Hivernat and X. Descombes. Mise en correspondance et recalage de graphes. Technical Report RR-3529, INRIA, Oct [5] D. Marr. Vision. Freeman and Co., [6] J.-M. Viglino and L. Guigues. Géoréférencement automatique de feuilles cadastrales. In Proc. RFIA 2002, pp , Angers, France, Jan [7] V. Walter. Automatic classification of remote sensing data for GIS database revision. In IAPRS, vol. 32, pp , 1998.

10 Figure 3. Results for a portion of the test site.