MEAN SHIFT SEGMENTATION

Size: px

Start display at page:

Download "MEAN SHIFT SEGMENTATION"

Blaze Osborne
7 years ago
Views:

1 MEAN SHIFT SEGMENTATION Proseminar "Aufgabenstellungen der Bildanalyse und Mustererkennung 1 Thu Huong Nguyen Fakultät Informatik TU Dresden

2 CONTENTS 1. Overview about Mean Shift Segmentation 2. Mean Shift algorithm o o Kernel density estimation Kernel 3. Mean shift segmentation 4. Applications of Mean Shift o o o Clustering Filtering Visual tracking 5. Conclusions 2

3 OVERVIEW ABOUT MEAN SHIFT SEGMENTATION What is Mean Shift? For each data point, mean shift defines a window around it and computes the mean of data point. Then it shifts the center of window to the mean and repeats the algorithm till it convergens Mean shift is a nonparametric iterative algorithm or a nonparametric density gradient estimation using a generalized kernel approach Mean shift is the most powerful clustering technique Mean shift is used for image segmentation, clustering, visual tracking, space analysis, mode seeking... Mean shift segmentation is an advanced and vertisale technique for clustering based segmentation 3

4 MEAN SHIFT ALGORITHM Kernel density estimation Kernel density estimation is a non parametric way to estimate the density function of a random variable. It is a popular method for estimating probability density. This is usually called as the Parzen window technique. K(x) : kernel, h: bandwidth parameter (radius), n data point x i, i=1..n in d-dimension space R d Kernel density estimator for a given set of d- dimensional points is 4

5 MEAN SHIFT ALGORITHM The estimate of the density gradient is defined as the gradient of the kernel density estimate Setting it to 0 and define g(x) = -K (x) we have is called Mean shift vector (or sample mean shift) 5

6 MEAN SHIFT ALGORITHM The mean shift vector computed with kernel G is proportional to the normalized density gradient estimate obtained with the kernel K The mean shift algorithm seeks a mode or local maximum of density of a given distribution Mean shift can be sumed up like this For each point x i Choose a search window Compute the mean shift vector m(x it ) Repeat till convergence 6

7 MEAN SHIFT ALGORITHM 7

8 MEAN SHIFT ALGORITHM Kernel K is a fuction of k is called the profile of K The simplest kernel is the flat kernel

9 MEAN SHIFT ALGORITHM Shadow of the Kernel K is kernel H if is in the gradient direction at x of the density estimate using H

10 MEAN SHIFT ALGORITHM The most popularly kernel is Gaussian kernel And their shadows SK(x) = K(x) 10

11 MEAN SHIFT ALGORITHM Epanechikov kernel E(x) = 3 4 (1 x 2 ) if x 1 0 else and its shadow SE(x) = E(x) 11

12 MEAN SHIFT SEGMENTATION An advanced and versatile technique for clustering-based segmentation Let {x i } i=1 n be the original image points, {z i } i=1 n the points of convergence, and {L i } i=1 n a set of labels Mean Shift Segmentation For each i = 1 n run the mean shift procedure for x i and store the convergence point in z i. Identify clusters {C p } p=1 m of convergence points by linking together all z i which are closer than 0,5 from each other in the joint domain. For each i = 1 n assign L i = {p z i ϵ C p }. Optional: Eliminate spatial regions smaller than M pixels. 12

13 MEAN SHIFT SEGMENTATION Parameter of mean shift segmentation h s : Spatial resolution parameter Affects the smoothing, connectivity of segments Chosen depending on the size of the image, objects h r : Range resolution parameter Affects the number of segments Should be kept low if contrast is low M : Size of smallest segment Should be chosen based on size of noisy patches 13

14 SAMPLES OF MEAN SHIFT SEGMENTATION original (h s,h r ) = (8,4) (h s,h r ) = (8,7) 14

15 SAMPLES OF MEAN SHIFT SEGMENTATION (h s,h r ) = (8,8) (h s,h r ) = (8,7) original 15

16 SAMPLES OF MEAN SHIFT SEGMENTATION (h s,h r ) = (16,4) (h s,h r ) = (16,16) (h s,h r ) = (8,4) 16

17 OPTIMIZATION TECHNIQUES PROBLEM According to the mean shift algorithm, the complexity is O(Tn 2 ), the first step needs the most computationally expensive. It means finding the closet neighbors of a point is the most expensive operation of mean shift method The larger of h s, the slower of processing time. It s known multidimensional range searching (Mean Shift Based Clustering in High Dimensions: A Texture Classification Example.) 17

18 OPTIMIZATION TECHNIQUES - SOLUTIONS Speed up the computation of mean shift vector Decrease the number of mean shift vector Combination both ooo In Mean Shift Segmentation Evaluation of Optimization Techniques proposed 3 optimization techniques Bucket data structure Mean shift vector reutilization (medium speed up) Local neighborhood inclusion (high speed up) 18

19 OPTIMIZATION TECHNIQUES - EVALUATION The processing time is faster than the nonparametric mean shift The quantity of errors increased 19

20 APPLICATIONS OF MEAN SHIFT - FILTERING Image filtering is a process by which we can enhance, modify or multilate images. Filtering reduces the influence from noise to mode detection. Mean shift filtering can work with binary, gray scale, RGB and arbitrary multichanel images. Filtering is the first step of mean shift segmentation process. A second step is the clustering of filtered data point 20

21 APPLICATIONS OF MEAN SHIFT - FILTERING Let {x j } j=1 n the d-dimensional original and {z j } j=1 n the filtered image points in the spatialrange domain Mean shift filtering Initialize k=1 and y k = x j Compute convergence. Assign z j = (x jr,y r conv) till 21

22 APPLICATIONS OF MEAN SHIFT - FILTERING (h s,h r ) = (8,4) original (h s,h r ) = (8,7) 22

23 APPLICATIONS OF MEAN SHIFT - FILTERING (h s,h r ) = (8,4) original (h s,h r ) = (8,7) 23

24 APPLICATIONS OF MEAN SHIFT - FILTERING COMPARISON WITH GAUSSIAN FILTER Mean shift filter is nonlinear filter, while Gaussian filter is linear filter based on Gaussian fuction Gaussian filter is a linear smoothing filter commonly used in image processing applications. Gaussian filter is applied to multidimensional data, an explicit convolution range can be specified for each dimension. 24

25 APPLICATIONS OF MEAN SHIFT - FILTERING COMPARISON WITH GAUSSIAN FILTER (h s,h r ) = (8,4) Gaussian r = 1.5 Gaussian r = 3 25

26 APPLICATIONS OF MEAN SHIFT - FILTERING COMPARISON WITH GAUSSIAN FILTER Original (h s,h r ) = (8,7) Gaussian r = 2 26

27 APPLICATIONS OF MEAN SHIFT - CLUSTERING The most commonly and important application of mean shift is used for clustering Mean shift clustering is a powerful unsupervised data analysis technique which does not require prior knowledge of the number of clusters, and does not constrain the shape of the clusters 27

28 APPLICATIONS OF MEAN SHIFT - CLUSTERING COMPARISON WITH K-MEANS CLUSTERING : K-Means is one of most popular clustering algorithms. It is simple,fast and efficient. K-means makes two broad assumptions the number of clusters is already known. Mean shift is a non parametric algorithm, which does not assume anything about number of clusters. K-means is fast and has a time complexity O(knT) where k is the number of clusters, n is the number of points and T is the number of iterations. Classic mean shift is computationally expensive with a time complexity O(Tn 2 ) K-means is very sensitive to initializations, while Mean shift is sensitive to the selection of bandwidth h 28

29 APPLICATIONS OF MEAN SHIFT - VISUAL TRACKING Object tracking is an important task in computer vision Mean shift is recently used in tracking object (visual tracking) It requires a data of video frame to use the mean shift algorithm in visual (video) tracking Tracking methods: Camshift by Bradski Method by Allan Method by Comaniciu 29

30 APPLICATIONS OF MEAN SHIFT - VISUAL TRACKING 30

31 CONCLUSIONS MEAN SHIFT PROS AND CONS Pros Does not assume spherical clusters Just a single parameter (window size) Finds variable number of modes Robust to outliers Cons Output depends on window size Computationally expensive Does not scale well with dimension of feature space 31

32 REFERENCES Jens N. Kaftan and Andr`e A. Bell and Til Aach. Mean Shift Segmentation Evaluation of Optimization Techniques Cheng, Y. (1995). Mean Shift, Mode Seeking, and Clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8): KEINOSUKE FUKUNAGA, AND LARRY D. HOSTETLER. The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition. IERE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-21, NO. 1, JANUARY 1975 Comaniciu, D. and Meer, P. (1997). Robust Analysis of Feature Spaces: Color Image Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition. CVPR 1997, pages Comaniciu, D. and Meer, P. (1999). Mean Shift Analysis an Applications. In International Conference on Computer Vision. ICCV 1999, volume 2, pages Comaniciu, D. and Meer, P. (2002). Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5): K. Fukunaga, L.D. Hostetler, The estimation of the gradient of a density function, with applications in pattern recognition, IEEE Trans. Inf. Theory 21 (1975) Yiping Hong, Jianqiang Yi and Dongbin Zhao. Improved mean shift segmentation approach for natural images Kuo-Lung Wu and Miin-Shen Yang. Mean shift-based clustering Periklis Andritsos. Data Clustering Techniques. University of Toronto. Department of Computer Science Nicole M. Artner. A Comparison of Mean Shift Tracking Methods. Digital Media, Upper Austria University of Applied Sciences, Hagenberg, Austria Bogdan Georgescu, Ilan Shimshoni and Peter Meer. Mean Shift Based Clustering in High Dimensions: A Texture Classification Example. (2003) 32

Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006

Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006 More on Mean-shift R.Collins, CSE, PSU Spring 2006 Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent