BILATERAL SYMMETRY DETECTION BASED ON SCALE INVARIANT STRUCTURE FEATURE. Ibragim Atadjanov and Seungkyu Lee

Transcription

1 BILATERAL SYMMETRY DETECTION BASED ON SCALE INVARIANT STRUCTURE FEATURE Ibragim Atadjanov and Seungkyu Lee Dept. of Computer Engineering, Kyung Hee University, Republic of Korea ABSTRACT Symmetry is a salient visual pattern in images. Symmetrical structure attracts human eye more than other regions. Therefore, detecting symmetry in an image is one of the crucial tasks in pattern recognition and computer vision research. Sparse key point based symmetry detection methods have been proposed which are fast and robust to noise showing superior detection performance. However, such local appearance-based methods have difficulties in capturing structure based patterns mostly supported by edges and contours. In this paper, we propose a scale invariant structure feature which describes points on extremum curvature along edges. We propose to use a histogram of curvature responses at respective scale space for description. Experimental evaluation on public shape dataset and real world images show that our structure feature works better in detecting visually salient structure based symmetry patterns. Index Terms symmetry detection, structure feature, reflection 1. INTRODUCTION Humans try to parameterize and understand the visual world in order to recognize objects, scenes and dynamic events. Symmetry patterns out of the scenes attract human eye more and are recognized better than others. Symmetry is well defined mathematical theory, and describes repeated patterns under certain rules in images. There are a lot of symmetric objects in natural world such as a snow crystal, human face, flowers, butterfly, and most of all human-made objects. Therefore, symmetry detection is a valuable task for object recognition and understanding. However, it is not a trivial task with real world images and many researchers have devoted to practical and robust symmetry detection method under challenging real world environments [1]. Marola [2] introduces planar bilateral symmetry detection in Euclidean space. Prasad and Yegnanarayana [3] propose a gradient vector flow and a symmetry salient map for bilateral symmetry detection. Mitra et al. [4] defines general regularity in 3D geometry based on region based matching. Feature based methods like Loy and Eklundh [5] finds symmetry matches based on SIFT key points resulting fast and robust Fig. 1. Overall Framework of the Proposed Method: Structure Feature Detection, Description and Symmetry Detection performance with real world images. Part et al. [6] have provided a performance comparison of symmetry detection methods with public dataset and evaluation methodology. Recent years, competitions for symmetry detection from real wold images have been organized [7, 8]. In very last competition [8], methods using Gestalt theory [9, 10] and Multiscale Kernel Operators[11] are analyzed and compared. There are, also, state-of-art symmetry detection methods based on edges and contours in an image. Ming et al. [12] detects symmetry via contour grouping, while Wang et al. [13] uses locally affine invariant edge correspondence for this task. Appearance features like SIFT describe local region based on contrast of intensity and orientation of neighbor pixels with equal importance along the distance from a key point. Furthermore, some clear structures of an object mostly represented by combination of line edges and boundaries cannot be detected and described. However, these structure patterns are the most critical and visually salient features that attract human attention. Butterfly in figure 1 is a good example, where patterns on the wings hardly support the symmetric shape of the butterfly, however we easily notice a bilateral symmetry on it based on the global shape of the butterfly mostly supported by its contours. Detection result of this example is

2 shown in figure 7 A. In this paper, we propose a new structure-based key point detection and description method invariant to size, shape complexity and rotation angle based on edges and object contours. Unlike the local appearance based features, structure feature based on line edges and contours is robust to illumination and viewpoint changes. We construct the scale space of each edge line and investigate the curvature detecting local extrema points for key point detection. Histogram of curvature response is proposed to describe detected key points in the corresponding scale. Proposed key points are adopted for bilateral symmetry detection to verify its effectiveness in robust point description. Our proposed method is summarized in figure 1. The contributions of the proposed work includes: (1) a new scale invariant structure features, and (2) a novel structure description based bilateral symmetry detection algorithm. 2. SCALE INVARIANT STRUCTURE FEATURE Our scale invariant structure feature works on image contours and edges. Edge lines with high curvature are detected and described by our histogram of curvature responses in scale space. Even though our scale space is built on the orientation of edges not in image space, we follow the framework shown by Lindeberg [14] and Lowe [15]. Our approach for constructing descriptor is also similar to SIFT Feature Detection Figure 2 describes our structure based key point detection method. Basically, our method detects points on the edges where the curvature of the edge shape becomes local extremum. In order to calculate the curvature response along the edge lines we detect edges from noise eliminated input image first. At each edge line, we calculate the orientation of tangent line scanning from one end to the other end of the edge (Figure 2 bottom-left), with the size of L 1. For description of detected key point invariant to scale, we find them in scale space. For construction of scale-space, we follow the framework used in SIFT [15] except the fact that we construct our scale space in one-dimensional edge orientation space, not in the image space. Different scale provides different details of corresponding shape of each edge line. For instance, higher scale describes rough and global shape of the edge. Detected key points at this scale will be described based on the corresponding shape details of the scale. One difference from SIFT, we do not find local maxima or minima within neighbour scales. We simply include all extrema points from all scales. Note that we are constructing our scale space in one-dimensional edge orientation space, not in the image space in order to describe shape of given edges. Therefore, smoothing or scaling also is done in one-dimensional edge Fig. 2. Feature Detection orientation space. If we smooth our edge orientation, it removes high frequency of the edge shape not of intensity of the original image. Based on the base orientation response, we build multiple scales of orientation response by sub-sampling the orientation response L i = Li 1 2. At each size scale, derivative of the orientation is calculated respectively (Figure 2 bottom-right). Orientation φ of the i th edge line in scale level s and its derivative φ are defined as follows. φ(e i, s) = arctan y x φ (e i, s) = φ(ei, s) l where l is a step index along the edge line for the orientation calculation. We pick the points having local extrema value in φ (e i, s). With the derivative of the orientation response we can extract extremum curvature responses regardless of its original rotation angle in the image. p(e i, s) represents detected key point on i th edge line e i and scale level s Feature Description At each detected key point, we construct our key point description that is invariant to scale and rotation changes. Similar to SIFT [15] key point description, we build multiple histograms of curvature response along the edge (Figure 3). In order to build our histogram of curvature response for the j th key point p i j (ei, s i j ) on i th edge line, we calculate the derivative of orientation of the local edge segment in scale s j centered at the detected key point. And then, we divide the edge (1) (2)

3 Fig. 3. Feature Description using our Histogram of Curvature Response segment into s sub-segments and build respective histogram with b curvature bins each. Note that closer point to the key point describes edge curvature better, and we gives higher weight to closer histograms. Edge segment size is decided by the number of consecutive points on the edge line to be counted for single histogram calculation. Finally, the length of edge segment becomes s n stepsize. 3. BILATERAL SYMMETRY DETECTION In order to verify our structure based description, we apply this for bilateral symmetry detection. Instead of appearance feature based process in Loy and Eklundh [5] s method, we incorporate our key point detection and description method keeping overall symmetry detection framework. For a given image, our detected features and their mirrors are used to find reflectional matches in an image as illustrated in figure 4. Our description of mirrored edge shape corresponds to the mirrored histograms. The order of bins in each histogram is flipped. However, we keep the order of histograms, because the starting end of the edge line should be kept for symmetry shape matching. Symmetry axis is defined as θ = α+β 2. In order to be a symmetry pair, descriptor of one key point matches with mirrored descriptor of the other key point and two orientations of edge line α and β meets the condition ( α β π 2 ). For calculating global symmetry axis, we allow every possible match involved in the symmetry voting to keep potential good matches. Instead, each match is multiplied by new weight w s that is calculated to be proportional to the similarity of their descriptors. Along with the similarity weight factor, we have three more weight factors: angular constraint w a, scale w c, and distance w d weights. Based on these factors, we define the strength s x = w s w c w a w d of symmetry axis. Every symmetry axes are transferred into Hough space with the calculated symmetry strengths. And the points with maximum intensity have been chosen as candidates of global bilateral symmetry axis. Fig. 4. Bilateral Symmetry Detection using our Structure Feature Points 4. EXPERIMENTAL RESULTS In our experimental evaluation, our descriptor has 10 histograms with 36 bins for each key point. First, we test on MPEG-7 shape image dataset [16]. We have picked 30 classes having bilateral symmetry and have built a test dataset including randomly picked 120 shape images out of the classes. Bilateral symmetry detection result of Loy and Eklundh [5] and ours are compared. In each image, ground truth number of symmetry axis is set to 1 and performance is compared following the method used in [6]: S K = T P K F P GT and false positive rate R F P, where T P is the number of detected true positives, F P is the number of detected false positives, GT is the number of detected ground truth and K is penalization constant for the false positive. We also compare our method with previous method using real world images. Figure 7 shows sample detection results. In image A, our method detects full symmetry axis of the butterfly based on its clear object boundary, however [5] fails to detect lower part where patterns on the wings are different. Image B and C clearly show how symmetry axes are detected by both methods. Proposed method finds matched point pairs on the contours, on the other hand [5] detects points at inside the objects. As a result, our method finds more matched pairs from the clear object boundary. In image D, [5] detects more number of matched pairs, however our proposed method detects more meaningful matches on the clear structure finding more symmetry axes. In image E, multiple thin edges with the

4 Fig. 5. Bilateral Symmetry Detection Results on MPEG7 Shape Images: Lines are symmetry axes and points are supporting key point pairs background clutters can support symmetry correctly. In image F, segmented edges are irregular on the gate and matches are not detected enough to support center symmetry axis. In image I, our proposed method detects symmetry better based on the contour of the leaves. Image J is a good example that a symmetry cannot be detected based on the appearance based method due to its lack of patterns on the objects. However, our method detects the global symmetry axis. 5. CONCLUSION In this paper, we propose a scale invariant structure feature based on the key points detected on extremum curvature along edge lines. Our histogram of curvature responses describes structure feature of edges and contours at respective scale space. Experimental results show that the proposed method superior in detecting structure based bilateral symmetry. Based on the our effort, we expect that a unified framework for symmetry detection based on both appearance and structure can show better performance. Fig. 6. Quantitative Evaluation: S0 and false positive rate RF P are calculated as suggested in [6] for evaluation Fig. 7. Experimental Results on Real World Images

5 6. REFERENCES [1] Seungkyu Lee and Yanxi Liu, Curved glide-reflection symmetry detection, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 34, pp , Feb [2] Giovanni Marola, A technique for finding the symmetry axes of implicit polynomial curves under perspective projection, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp , March [3] V.S.N. Prasad and B. Yegnanarayana, Finding axes of symmetry from potential fields, Image Processing, IEEE Transactions on, vol. 13, pp , Dec [4] N. J. Mitra, L. Guibas, and M. Pauly, Partial and approximate symmetry detection for 3d geometry, ACM Transactions on Graphics (SIGGRAPH), vol. 25, pp , July [5] Gareth Loy and Jan-Olof Eklundh, Detecting symmetry and symmetric constellations of features, in Proceedings of the 9th European Conference on Computer Vision - Volume Part II. 2006, vol. 3952, pp , Springer-Verlag. [6] Minwoo Park, Seungkyu Lee, Po-Chun Chen, S. Kashyap, A.A. Butt, and Yanxi Liu, Performance evaluation of state-of-the-art discrete symmetry detection algorithms, in Computer Vision and Pattern Recognition, CVPR IEEE Conference on, June 2008, pp in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, June 2013, pp [11] S. Kondra, A. Petrosino, and S. Iodice, Multi-scale kernel operators for reflection and rotation symmetry: Further achievements, in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, June 2013, pp [12] Yansheng Ming, Hongdong Li, and Xuming He, Symmetry detection via contour grouping, in Image Processing (ICIP), th IEEE International Conference on, Sept 2013, pp [13] Zhaozhong Wang, Zesheng Tang, and Xiao Zhang, Reflection symmetry detection using locally affine invariant edge correspondence, Image Processing, IEEE Transactions on, vol. 24, no. 4, pp , April [14] Tony Lindeberg, Edge detection and ridge detection with automatic scale selection, International Journal of Computer Vision, vol. 30, no. 2, pp , [15] DavidG. Lowe, Distinctive image features from scaleinvariant keypoints, International Journal of Computer Vision, vol. 60, pp , [16] H K Kim Y S Kim W-Y Kim K Muller M Bober, J D Kim, Summary of the results in shape descriptor core experiment, MPEG-7, ISO/IEC JTC1/SC29/WG11/ MPEG99/M4869, [7] K. Brockelhurst S. Kashyap I. Rauschert, J. Liu and Y. Liu, Symmetry detection competition: A summary of how the competition is carried out,, in IEEE Conf. Comput. Vis. Pattern Recognit. Workshop Symmetry Detection Real World Images, 2011, p [8] Jingchen Liu, G. Slota, Gang Zheng, Zhaohui Wu, Minwoo Park, Seungkyu Lee, I. Rauschert, and Yanxi Liu, Symmetry detection from realworld images competition 2013: Summary and results, in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, June 2013, pp [9] V. Patraucean, R.G. von Gioi, and M. Ovsjanikov, Detection of mirror-symmetric image patches, in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, June 2013, pp [10] E. Michaelsen, D. Muench, and M. Arens, Recognition of symmetry structure by use of gestalt algebra,