# Mean shift-based clustering

Save this PDF as:

Size: px
Start display at page:

## Transcription

4 8 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) is used to find the modes of the density estimate fˆ (). SK In total, we resent three kernel classes with their shadows. These are Gaussian kernels G () with their shadows SG ()= G (), Cauchy kernels C () with their shadows SC () = C () and the generalized anechnikov kernels K () with their shadows SK () = K+ (). The behaviors of these kernels with different are shown in Fig.. The mean shift rocedures using these three classes of kernels can easily find their corresonding density estimates. The arameters β and, called the normalization and stabilization arameters, resectively, greatly influence the erformance of the mean shift rocedures for the kernel density estimate. We will discuss these in the net section.. Mean shift as a robust clustering A suitable clustering method should have the ability to tolerate noise and detect outliers in the data set. Many criteria such as the breakdown oint, local-shift sensitivity, gross error sensitivity and influence functions [] can be used to measure the level of robustness. Comaniciu and Meer [] discussed some relationshis of the mean shift and the nonarametric M-estimator. Here, we rovide a more detailed analysis of the robustness of the mean shift rocedures... Analysis of robustness The mean shift mode estimate m k () can be related to the location M-estimator ˆθ = arg min θ ρ( j θ), j= where ρ is an arbitrary loss measure function. ˆθ can be generated by solving the equation ( / θ) ρ( j θ) = j= φ( j θ) =, j= where φ( j θ) = ( / θ)ρ( j θ). If the kernel H is a reasonable similarity measure which takes values on the interval [, ], then ( H) could be a reasonable loss function. Thus, a location M-estimator can be found by ˆθ = arg min θ = arg ma θ j= j= ρ( j θ) = arg min θ [ h( θ j )] j= h( θ j ) = arg ma θ fˆ H (θ). This means that m k () is a location M-estimator if K is a kernel with its shadow H. The influence curve (IC) can hel us to assess the relative influence of an individual observation toward the value of an estimate. In the location roblem, we have the influence of an M-estimate with IC(y; F,θ) = φ(y θ) φ (y θ) df Y (y), where F Y (y) denotes the distribution function of Y. The M- estimator has shown that IC(y; F,θ) is roortional to its φ function []. If the influence function of an estimator is unbounded, an outlier might cause trouble where the φ function is used to denote the degree of influence. Suose that {,..., n } is a data set in the real number sace R s and SG () is a shadow of G (). Then, m G () is a location M-estimator with φ function defined to be φ G ( j ) = d d [ SG ()]= β ( j )G (). By alying the L Hosital s rule, we derive lim j ± φ G ( j ) =. Thus, we have the influence curve of m G () with IC( j ; F,) = when j tends to ositive or negative infinity. This means that the influence curve is bounded and the influence of an etremely large or small j on the mode estimator m G () is very small. We can also find the location of j which has a maimum influence on m G () by solving ( / θ)φ G ( j ) =. Note that both m C () and m () K also have the revious roerties where their φ functions are as follows: ( ) φ C ( j ) = ( j )C (), β ( + ) φ ( ( j )K K j ) = β () if ( j ) β, if ( j ) β. The φ function for the mean shift-based mode-seeking estimators with β = is shown in Fig.. The influence of an individual j on m () is zero if ( K j ) β and an etremely large or small j will have no effect on m K (). However, the influence of an etremely large or small j on m G () and m C () is a monotone decreasing function of the stabilization arameter. An outlier has no influence when becomes large. In this aer, we will choose these generalized anechnikov kernels for our mean shift-based clustering algorithm because they can suitably fit the data sets even with etremely large or small data oints... Bandwidth selection and stabilization arameter Bandwidth selection greatly influences the erformance of kernel density estimation. Comaniciu and Meer [] summarized four different techniques according to statistical analysisbased and task-oriented oints of view. Here, we give new consideration to first normalizing the distance measure by dividing the normalization arameter β and then focusing on estimating the stabilization arameter. The normalization arameter β is set to be the samle variance nj= j nj= j β = where =. n n

5 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) = The hi function of Gaussian kernels = = j - The hi function of Cauchy kernels = = =. - - The hi function of generalized anechnikon kernels = = j j = Fig.. The φ functions with different values: (a) Gaussian kernels, (b) Cauchy kernels and (c) generalized anechnikov kernels. Frequency f G () ^ 8 P= 7 6 P= P= = f C () ^ P= P= P= = fk () ^ 7 6 P= P= P= = Fig.. (a) The histogram of a three-cluster normal miture data. (b), (c) and (d) are its corresonding density estimates with different. ˆ f SG (), ˆ f SC () and fˆ SK () We have the roerties of and G() = G()/Ĝ(). We then have lim m G() = lim and nj= G () j nj= G () = nj= j n = () nj= lim m G () j G() = lim nj= G () = lim nj= [G() ] j nj= [G() ] = G() = j G() =. (6) lim m G() = lim m C() = lim m K () =. () This means that when tends to zero, the kernel density estimate has only one mode with the samle mean. Fig. shows the histogram of a three-cluster normal miture data with its corresonding density estimatesfˆ SG (), fˆ SC () and fˆ (). SK In fact, a small will lead to only one mode in the estimated density, as shown in Fig. for the case of =. However, a too larger will cause the density estimate to have too many modes as shown in Fig. for the case of =. This can be elained as follows. We denote Ĝ() = ma j G() This means that, as tends to infinity, the data oint which is the closest to the initial value will become the eak. Hence, we have lim m G() = lim m C() = lim m K (). (7) In this case, each data oint will become a mode in the blurring and nonblurring mean shift rocedures. The stabilization arameter can control the erformance of the density estimate. This situation is somehow similar to the bandwidth selection, but with different merits. The urose of the bandwidth selection is to find a suitable bandwidth

6 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) The increased shift for the stabilization arameter is M= suitable estinate The increased shift for the stabilization arameter is M= suitable estinate. The increased shift for the stabilization arameter is M=. The increased shift for the stabilization arameter is M=.9 suitable estinate.9 suitable estinate Fig.. The correlation values of { fˆ SK ( ),..., fˆ SK ( n )} and { fˆ +M ( SK ),..., fˆ +M ( n )}, where (a) M =, (b) M =, (c) M = and (d) M =. SK (covariance) for a kernel function of a data oint so that a suitable kernel function can induce a suitable density estimate. However, our new technique assigns a fied samle variance for the kernel function and then uses a suitable to induce a suitable density estimate. As shown in Fig., different corresonds to different shaes of the density estimates. A suitable will corresond to a satisfactory kernel density estimate so that the modes found by the mean shift can resent the dense area of the data set... A stabilization arameter estimation method A ractical bandwidth selection technique is related to the decomosition stability of the density shae estimates. The bandwidth is taken as the center of the largest oerating range over which the same number of clusters are obtained for the given data []. This means that the shaes of the estimated density are unchanged over this oerating range. Although this technique can yield a suitable bandwidth estimate, it needs to find all cluster centers (modes) for each bandwidth over the chosen oerating range. It thus requires a large comutation. The selected oerating range is also erformed case by case. In our stabilization arameter estimation method, we adot the above concet but ski the ste of finding all modes for each. We describe the roosed method as follows. We first define the following function: ˆ f H ( i ) = h( j ), i =,...,n, j=,...,n. (8) j= This function denotes the value of the estimated density shae on the data oints and is similar to the mountain function roosed by Yager and Filev [9] which is used to obtain the initial values for a clustering method. Note that the original mountain function is defined on the grid nodes. However, we define q. (8) on the data oints. In q. (8), we can set H to equal to SG, SC and SK. We now elain how to use q. (8) to find a suitable stabilization arameter. Suose that the density shae estimates are unchanged with = and =, the correlation value of { fˆ H ( ),..., fˆ H ( n )} between = and will be very close to. In this situation, = will be a suitable arameter estimate. In this way, we can ski the ste for finding the modes of the density estimate. In our eeriments, a good oerating range for always falls between and. This is because we normalize the kernel function by dividing the normalization arameter β. amle. We use the reviously described grahical method of correlation comarisons to find the suitable for the data set shown in Fig.. We have the correlation values of { fˆ ( SK ),..., fˆ ( SK n)} and { fˆ +M SK ( ),..., fˆ +M SK ( n )} as shown in Fig.. Figs. (a) (d) give the results of the cases with the increased shifts M =,, and, resectively. In Fig. (a), the y-coordinate of the first solid circle oint denotes the correlation value between = and. The y-coordinate of the second solid circle oint denotes the correlation value between = and. The st, nd, rd solid circle oints, etc., denote the correlation values of the resective airs ( =,= ), ( =,= ) and

7 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) = = Fig.. The data histograms and density estimates using generalized anechnikov kernel with =.. ( =,= ), etc. In Fig. (b), the y-coordinate of the first solid circle oint denotes the correlation value between = and = + =. The y-coordinate of the second solid circle oint denotes the correlation value between = and =. The st, nd, rd solid circle oints, etc., denote the correlation values of the resective airs ( =,= ), ( =,= ) and ( =,= 7), etc. The others can be similarly induced. Since the density shae will be unchanged when the correlation value is close to, we may choose with the solid circle oint close to the dotted line of value. In Fig. (a), with the increased shift M =, the th solid circle oint is very close to the dotted line (i.e. the correlation value between = and = is very close to ). Hence, = is a suitable estimate. We will elain why we do not choose the oint that lies on the dotted line later. In Fig. (b), with the increased shift M =, the sith solid circle oint is close to the dotted line (i.e. the correlation value between = and is close to ). Hence, = is a suitable estimate. In Fig. (c), with the increased shift M =, the correlation value between = and 6 is very close to. Hence, = is a suitable estimate. Similarly, = is a suitable estimate as shown in Fig. (d). Fig. illustrates the density estimates using fˆ () SK with =.. We find that these density estimates actually match the histogram of the data. The following is another eamle with two-dimensional data. amle. We use a two-dimensional data set in this eamle. Fig. 6(a) shows a 6-cluster data set where the data oints in each grou are uniformly generated from rectangles. Figs. 6(b) (d) are the correlation values with the increased shift M =., resectively. In Fig. 6(b), the th oint is close to the dotted line and hence = is a good estimate. In Fig. 6(c), the 7th oint is close to the dotted line and hence we choose =. Fig. 6(d) also indicates that = is a suitable estimate. In this eamle, our grahical method of correlation comarisons with different increased shift M resents all the same results. The density estimates using SK with =,, and are shown in Figs. 7(a), (b), (c) and (d), resectively. In Section., qs. () and () show that a small value will cause the kernel density estimate to have only one mode with the samle mean. Figs. and 7(a) also verify this oint. The selected = is between = and whose density shaes are shown in Figs. 7(c) and (d) that match the original data structure well. Our grahical method of correlation comarisons can find a good density estimate. This estimation method can accomlish the tasks where the bandwidth selection method can do it. However, our method skis the ste of finding all modes of the density estimate so that it is much simler and less comutational. We estimate a suitable stabilization arameter where its oerating range is always located between and. Note that, if the increased shift M is large such as M = or, then it may miss a good estimate for. However, a too small increased shift M, such as M<, may take too much comutational time. We suggest that taking M = for the grahical method of correlation comarisons may erform well in most simulations. We now elain why we did not choose the oint that lies on the dotted line according to the following two reasons. Note that the stabilization arameter is similar to the number of the bars drawn on the data histogram. The density shae with a large corresonds to the histogram with a large number of bars and hence has too many modes. qs. (6) and (7) also show this roerty. The stabilization arameter is the ower of the kernel function which takes values between and. ach data oint will have the value of q. (8) being close to with a large case and hence the correlation value becomes large. Figs. and 6 also show this tendency on the curvilinear tail. According to these two reasons, we suggest to choose the estimate of with the oint being very close to the dotted line... A mean shift-based clustering algorithm Since the generalized anechnikov kernel K has the robustness roerty as analyzed in Section., we use the K kernel in the mean shift rocedure. By combining the grahical method of correlation comarisons for the estimation of the stabilization arameter, we roose a mean shift-based clustering method (MSCM) with the K kernel, called MSCM(K ). The roosed MSCM(K ) rocedure is therefore constructed with four stes: () select the kernel K ; () estimate the stabilization ; () use the mean shift rocedures; () identify the clusters. The MSCM(K ) rocedure is with its diagram as shown in Fig. 8 and summarized as follows. The MSCM(K ) rocedure: Ste. Choose the K kernel. Ste. Use the grahical method of correlation comarisons for estimating. Ste. Use the nonblurring mean shift rocedure. Ste. Identify the clusters. We first imlement the MSCM(K ) rocedure for the data set shown in Fig.. According to Fig. (a), = is a suitable estimate for the data set in Fig. (a). The results of the MSCM(K ) rocedure are shown in Fig. 9. The curve

8 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) y The increased shift of is M= suitable estimate. The increased shift of is M=. The increased shift of is M=.9 suitable estimate.9 suitable estimate.9.8 Fig. 6. (a) The 6-cluster data set. (b), (c) and (d) resent the correlation values with the increased shift M =, and, resectively. = 6 y = 9 8 y = y = y Fig. 7. The density estimates of the 6-cluster data set where (a), (b), (c) and (d) are the fˆ SK values of each data oint with =,, and, resectively. reresents the density estimate using fˆ SK with =. The locations of the data oints are illustrated by the histograms. Figs. 9(a) (f) show these data histograms after imlementing the MSCM(K ) rocedure with the iterative times T =,,,, and 77. The MSCM(K ) rocedure is convergent when T = 77. The algorithm is terminated when

9 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) Data oints Select a kernel () Choose the kernel K P (used in this aer) () Choose the kernel G P or C P stimate the stabilization arameter. Use the roosed grahical method of correlation comarisons (used in this aer). Directly assign a value Use the mean shift rocedure. Nonblurring mean shift (used in this aer). General mean shift. Blurring mean shift Identify clusters. Merge data. Other methods (discuss in Section ) Fig. 8. The diagram of the roosed MSCM(K ) rocedure. Fig. 9. The density estimates using the MSCM(K ) rocedure with = and the histograms of the data oints where (a) T =, (b) T =, (c) T =, (d) T =, (e) T = and (f) T = 77. the locations of all data oints are unchanged when the iterative time achieves the maimum T =. Fig. 9(f) shows that all data oints are centralized to three locations, which are the modes of the density estimate fˆ. The SK MSCM(K ) rocedure finds that these three clusters do indeed match the data structure. We also imlement the MSCM(K ) rocedure on the data set in Fig. 6(a). According to the grahical method of correlation comarisons as shown in Fig. 6(b), = is a suitable estimate. The MSCM(K ) results after the iterative times T =, and are shown in Figs. (a), (b) and (c), resectively. The data histograms show that all data oints are centralized to 6 locations which match the data structure when the iterative time T =. We mention that no initialization roblems occurred with our method because the stabilization arameter is estimated by the grahical method of correlation comarisons. We can also solve the cluster validity roblem by merging the data oints which are similar to the many rogressive clustering methods [,7,8]. We have shown that the mean shift can be a robust clustering method by definition of the M-estimator and also discussed the φ function that denotes the degree of influence of an individual observation for the kernels G, C and K.Inthe comarisons of comutational time, our grahical method of correlation comarisons for finding the stabilization arameter is much faster than with the bandwidth selection method. Finding all cluster centers for the bandwidth selection method requires the comutational comleity O(n st ) for the first selected bandwidth where s and t denote the data dimension and the algorithm iteration number, resectively. Suose that

10 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) Fig.. (a), (b) and (c) are the histograms of the data locations after using the MSCM(K ) rocedure with iterative time T =, and, resectively. we choose the maimum bandwidths of the oerating range for the grahical method of correlation comarisons, then this would require the comutational comleity O(n s i= t i) for finding a suitable bandwidth. In the MSCM(K ) rocedure, the stabilization arameter is always in Refs. [,] because we normalize the kernel function. Thus, it requires O(n s) to find a suitable when the increased shift M =. Then, the MSCM(K ) rocedure should be faster if we set the increased shift M to be greater than. We finally further discuss the MSCM(K ) rocedure. The first ste is to choose a kernel function where we recommend the selection of K. You may also choose the kernels G and C. The second ste is to estimate the stabilization arameter using the roosed grahical method of correlation comarisons. Of course, the user can also directly assign a value for. In our eeriments, a suitable always fell between and. The third ste is to imlement the mean shift rocedure. Three kinds of mean shift rocedures mentioned in Section can be used. We suggest the nonblurring mean shift. The fourth ste is to classify all data oints. If the data set only contains the round shae clusters, the nonblurring mean shift can label the data oints by merging them. However, if the data set contains other cluster shaes such as line or circle structures, then the data oints may not be centralized to some small locations. In this case, merging the data oints will not work well. The net ste is to discuss this roblem and also rovide some numerical eamles and comarisons. We then aly these to image rocessing.. amles and alications Some numerical eamles, comarisons and alications are stated in this section. We also consider the comutational comleity, cluster validity and imrovements of the mean shift in large continuous, discrete data sets with its alication to the image segmentation... Numerical eamles with comarisons amle. In Fig. (a), a large cluster number data set with clusters is resented where we also add some uniformly noisy oints. The grahical lot of the correlation comarisons for the data set is shown in Fig. (b) where = is a suitable estimate. The MSCM(K ) results when T =,, and are shown in Figs. (c), (d), (e) and (f), resectively. We can see that all data oints are centralized to locations that suitably match the structure of data. The results are not affected by the noisy oints. The cluster number can easily be found by merging the data oints which are centralized to the same locations. Thus, the MSCM(K ) rocedure can be a simle and useful unsuervised clustering method. The sueriority of our roosed method to other clustering methods is discussed below. We also use the well-known FCM clustering algorithm [9] to cluster the same data set shown in Fig. (a). Suose that we do not know the cluster number of the data set. We adot the validity indees, such as the artition coefficient (PC) [], Fukuyama and Sugeno (FS) [], and Xie and Beni (XB) [], to solve the validity roblem when using FCM. Therefore, we need to rocess the FCM algorithm for each cluster number c =,,.... However, the first roblem is to assign the locations of the initial values in FCM. We simulate two cases of assignments with the random initial values and the designed initial values. Figs. (a) (c) resent the PC, FS and XB validity indees when the random initial values are assigned. FCM cannot detect those searated clusters very effectively, but it can when the designed initial values are assigned as shown in Fig.. Note that the PC inde has a large value tendency when the cluster number is small. Thus, a local otimal value of the PC inde curve may resent a good cluster number estimate []. In the case of the designed initials assignment, all validity measures should have an otimal solution with c =. However, this result is obtained only when the FCM algorithm can divide the data into well-searated clusters. If the initializations are not roerly chosen (for eamle, no centers are initialized in of these rectangles), we cannot ensure good artitions being found by the FCM algorithm. In fact, most clustering algorithms always have this initialization roblem. This is a more difficult roblem in a high-dimensional case. Here we find that our roosed MSCM(K ) rocedure has the necessary robust roerty for the initialization. We had mentioned that the comutational comleity of finding the stabilization arameter estimate in MSCM(K ) is O(n s). After estimating, the comutational comleity of imlementing the MSCM(K ) rocedure is O(n st K ) where t K denotes the number of iterations. In FCM, the comutational comleity is O(ncst C ), where t C is the number of iterations and c is the cluster number. For solving the validity roblem, we

11 K.-L. Wu, M.-S. Yang / Pattern Recognition (7). z 6 - y P= z y z y z y z y Fig.. (a) The data set. (b) The grah of correlation comarisons where = is a suitable estimate. (c), (d), (e), and (f) are the data locations after using the MSCM(K ) rocedure with iterative time T =,, and, resectively..6.. PC validity inde for FCM clustering algorithm with random initial values cluster number c c= - FS validity inde for FCM clustering algorithm with random initial values cluster number c c= XB validity inde for FCM clustering algorithm with random initial values cluster number c Fig.. (a), (b) and (c) resent the PC, FS and XB validity indees when the random initial values are assigned..6.. PC validity inde for FCM clustering algorithm with designed initial values c= cluster number c - FS validity inde for FCM clustering algorithm with designed initial values cluster number c c= XB validity inde for FCM clustering algorithm with designed initial values cluster number c c= Fig.. (a), (b) and (c) resent the PC, FS and XB validity indees when the designed initial values are assigned. need to rocess FCM from c = toc = C ma where C ma is the maimum number of clusters. Thus, the comutational comleity of FCM is O(n(C ma )s c=c ma c= t C ). Although the comleity of the MSCM(K ) rocedure is high when the data set is large, it can directly solve the validity roblem in a simle way. In order to reduce the comleity of the

12 6 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) = Fig.. (a) The data set. (b) The grah of correlation comarisons where = 8 is a suitable estimate. (c) The data locations after using the MSCM(K ) rocedure. a Height b Fig.. (a) The hierarchical tree of the data set in Fig. (c). (b) Identified three clusters with different symbols. MSCM(K ) rocedure for the large data set, we will roose a technique to deal with it later... Cluster validity and identified cluster for the MSCM(K ) rocedure Note that, after we have found the MSCM(K ) clustering results, we had simly identified clusters and the cluster number by merging the data oints which are centralized to the same location. However, if the data set contains different shae clusters such as a line or a circle structure, the data oints may not be centralized to the same small region. On the other hand, the estimated density shae may have flat curves on the modes. In this case, the method of identifying clusters by merging data oints may not be an effective means of finding the clusters. If we use the Agglomerative Hierarchical Clustering (AHC) algorithm with some linkage methods such as single linkage, comlete linkage and Ward s method, etc., we can identify different shae clusters from the MSCM(K ) clustering results. We demonstrate this identified method with AHC as follows. amle. Fig. (a) shows a three-cluster data set with different cluster shaes. The grahical lot of the correlation comarisons is shown in Fig. (b) where = 8 is a suitable stabilization arameter estimate. The locations of all data oints after rocessing the MSCM(K ) are shown in Fig. (c). In this situation, the identified method of merging data oints may cause difficulties. Since the mean shift rocedure can be seen as the method that shifts the similar data oints to the same location, the AHC with single linkage can hel us to find clusters so that similar data oints can be easily linked into the same cluster. We rocess the AHC with single linkage for the MSCM(K ) clustering results shown in Fig. (c). We obtain the hierarchical clustering tree as shown in Fig. (a). The hierarchical tree shows that the data set contains three wellsearated clusters. These clusters are shown in Fig. (b) with different symbols. In fact, most clustering algorithms are likely to fail when they are alied to the data set shown in Fig. (a). By combing MSCM(K ) with AHC, these methods can detect different shae clusters for most data sets. On the other hand, it may also decrease the convergence time for the MSCM(K ) rocedure. Note that, the estimated density shae may have flat curves on the modes for the data set so that the MSCM(K ) rocedure is too time consuming in this case, esecially for a large data set. Since the AHC method can connect the close data oints into the same cluster, we can then sto the MSCM(K ) rocedure when the data oints shift to the location near the mode. Therefore, we can set a larger stoing threshold for the MSCM(K ) rocedure or set a smaller maimum iteration... Imlementation for large continuous data sets We know that one imortant roerty of the MSCM(K ) rocedure is to take all data oints as the initial values so that these data oints can be centralized to the locations of the modes. The cluster validity roblems can be solved

13 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) 7 Fig. 6. (a) The Lenna image with size 8 8. (b) The results using the MSCM(K ) rocedure with = when T =. (c) The results with = when T =. (d) The results with = when T = 98. simultaneously in this way. However, this rocess wastes too much time for a large data set. One way to reduce the number of data oints for the MSCM(K ) rocedure is to first merge similar data oints from the original data set. Although this technique can reduce the number of data oints, the MSCM(K ) clustering results will differ as to the original nonblurring mean shift rocedure. Since the mode reresents the location of where most data are located, these data oints will shift to the mode at a few mean shift iterations. However, these data oints have to continue to rocess the mean shift rocedure until all data oints are centralized to these modes. This is because the mean shift rocedure is treated as a batch tye. Although all data oints are taken to be the initial values, there are no constrains on these initial values when we rocess the mean shift clustering algorithm using q. (). This means that we can treat the mean shift as a sequential clustering algorithm. The inut of the second data oint is erformed only when the first data oint shifts to the mode. The inut of the third data oint is erformed only when the second data oint shifts to the mode. The rest oints can be similarly induced. In this sequential-tye mean shift rocedure, only the data oints that are far away from modes require more iteration. However, most data oints (nearby the modes) require less iteration. The comutational comleity of this sequential-tye mean shift rocedure is O(ns n i= t i ) where t i denotes the iterative count of data oint i. However, the iteration of each data oint will be all equal to the largest iteration number in the batch-tye mean shift rocedure where the comutational comleity is O(n st K ) with t K = ma{t,...,t n }. Note that this technique is only feasible for the nonblurring mean shift rocedure, such as the roosed MSCM(K ) rocedure. Since the blurring mean shift algorithm udate the data oints at each iterative, the location of each data oint will be changed so that the sequential tye is not feasible for the blurring mean shift rocedure... Imlementation for large discrete data sets Suose that the data set {,..., n } only takes values on the set {y,...,y m } with corresonding counts {n,...,n m }. That is, there are n observations of {,..., n } with values y, n observations of {,..., n } with values y, etc. For eamle, a 8 8 gray level image will have 6 8 data oints (or iels). However, it only takes values on the gray level set {,,...,}. The mean shift rocedure has large comutational comleity in this n=6 8 case. The following theorem is a technique that can greatly reduce comutational comleity. Theorem. Suose that {,..., n }, y {y,...,y m } and = y. The mean shift of y is defined by mi= k( y y i )w(y i )n i y i y = m K (y) = mi= k( y y i. (9) )w(y i )n i Then we have m K () = m K (y). That is, the mean shift of using q. () can be relaced by the mean shift of y using q. (9) with the equivalent results. Proof. Since {,..., n } only take values on the set {y,...,y m } with corresonding counts {n,...,n m },wehave mi= k( y y i )w(y i )n i = n j= k( j )w( j ) and mi= k( y y i )w(y i )n i y i = n j= k( j )w( j ) i. Thus, m K () = m K (y). We can take {y,...,y m } to be initial values and then rocess the mean shift rocedure using q. (9) for the MSCM(K ) rocedure. The final locations of {y,...,y m } using q. (9) will be equivalent to the final locations of {,..., n } using q. (). In this discrete data case, the comutational comleity for the MSCM(K ) rocedure becomes O(m st) where t denotes the iterative counts and m<n. The reviously discussed sequential tye of continuous data case can also be used to greatly reduce comutational comleity in this discrete data case. The following is a simle alication in the image segmentation... Alication in image segmentation amle. Fig. 6(a) is the well-known Lenna image with the size 8 8. This image contains 6 8 iel values with a maimum value of 7 and minimum value of. This means that it only takes values on {,,...,7} and the count frequency of these iel values is shown in Fig. 7(a). The comutational comleity of the original nonblurring mean shift rocedure is O(n st) = O(6 8 st). After alying the described technique using q. (9), the comutational comleity of the nonblurring mean shift rocedure in this discrete case becomes O(m st)=o( st) which actually greatly reduces comutational comleity. The grah of correlation comarisons for the MSCM(K ) rocedure is shown in Fig. 7(b) where = is a suitable estimate. The estimated density shae with = is shown in Fig. 7(c) which quite closely matches

14 8 K.-L. Wu, M.-S. Yang / Pattern Recognition (7). count corr P= iel.997 P= P= T=98 iel iel iel Fig. 7. (a) The counts of the iel of the Lenna image. (b) The grah of correlation comarisons where = is a suitable estimate. (c) The density estimate with =. (d) All iel values are centralized to four locations after using the MSCM(K ) rocedure with iterative time T = 98. Frequency 6 data corr P= Height Fig. 8. (a) Histogram of the data set. (b) The grah of correlation comarisons. (c) The hierarchical tree of the data after imlementing the MSCM(K ) algorithm. the iel counts with four modes. The MSCM(K ) clustering results when t =, and 98 are shown in Figs. 6(b), (c) and (d), resectively. Following convergence, all iel values are centralized to four locations resented by the arrows in the -coordinate as shown in Fig. 7(d). Note that, for most c- means based clustering algorithms, c = 8 is mostly used as the cluster number of the Lenna data set. If we combine a validity inde to search for a good cluster number estimate for this data, the comutational comleity should become very large. In the MSCM(K ) rocedure, a suitable cluster number estimate is found automatically and the comutational comleity for this discrete large data set can be reduced by using the above-mentioned technique. For color image data, each iel contains a three-dimensional data oint that each dimension takes values on the set {,,...,} and hence we will have ossible iel values. However, for a 8 8 color image, the worst situation is to have 8 8 different iel values. In general, an image data will contain many overlaing iel values and hence the technique can also significantly reduce comutational comleity even for color images..6. Comarisons to other kernel-based clustering algorithms Comaniciu [] roosed the variable-bandwidth mean shift with data-driven bandwidth selection. To demonstrate their bandwidth selection method, Comaniciu [] used the data set drawn with equal robability from normal distributions N(8, ), N(, ), N(, 8) and N(, 6) with total n =. Fig. 8(a) shows the histogram of this data set. Comaniciu [] used analysis bandwidths in the range of

15 corr K.-L. Wu, M.-S. Yang / Pattern Recognition (7) P= Height data Fig. 9. (a) Data set. (b) The grah of correlation comarisons. (c) The hierarchical tree of the data after imlementing the MSCM(K ) algorithm. (d) Four identified clusters resented in different symbols P= Height.9 inde Fig.. (a) The grah of correlation comarisons and (b) the hierarchical trees after imlementing the MSCM(K ) algorithm for the Iris data P= Height 8 6 Fig.. (a) The grah of correlation comarisons and (b) the hierarchical trees after imlementing the MSCM(K ) algorithm for the Wine data.

16 K.-L. Wu, M.-S. Yang / Pattern Recognition (7)..999 P=9 Height Fig.. (a) The grah of correlation comarisons and (b) the hierarchical trees after imlementing the MSCM(K ) algorithm for the Crabs data.. 8 Height = Height Height Fig.. (a) stimated for Cmc data. (b), (c) and (d) are the single linkage, comlete linkage and Ward s method for the Cmc data after imlementing the MSCM(K ) algorithm where four classes of bandwidth were detected (see Ref. [,. 8-8]). Based on our roosed MSCM(K ), the grah of correlation comarisons for the data set of Fig. 8(a) gives = 8 and there are four clusters resented as shown in Figs. 8(b) and (c), resectively. Furthermore, we use the data set of nonlinear structures with multile scales in Comaniciu [] for our net comarisons as shown in Fig. 9(a). The grah of correlation comarisons, the hierarchical tree of the data after imlementing the MSCM(K ) algorithm and four identiﬁed clusters resented in different symbols are shown in Figs. 9(b) (d), resectively. These results with four identiﬁed clusters from MSCM(K ) are coincident to those of the variable bandwidth mean shift in Comaniciu []. Note that the variable-bandwidth mean shift in Comaniciu [] is a good technique to ﬁnd a suitable bandwidth for each data oint, but it requires imlementing the ﬁed bandwidth mean shift with many different selected bandwidths. Hence, it is time consuming. In our MSCM(K ) method, we are not to focus on estimating good bandwidth for each data oint, but on estimating the stabilization arameter so that the MSCM(K ) with this suitable can rovide suitable ﬁnal results. In this case, it requires imlementing the mean shift rocess only one time. The kernel k-means roosed by Girolami [] is another kernel-based clustering algorithm. The cluster numbers of the data sets Iris (c = ), Wine (c = ) and Crabs (c = ) are correctly detected by kernel k-means (see Ref. [, ]). We imlement the roosed method for the data sets Iris, Wine and Crabs where the results of these real data sets from MSCM(K ) are shown in Figs.,, and, resectively. The MSCM(K ) algorithm with = indicates that c = is suitable for the Iris data. However, there are some clues to see c = is also suitable for the Iris data according to the results of Fig. (b). Note that this is because one of the clusters

18 K.-L. Wu, M.-S. Yang / Pattern Recognition (7) the resentation of the aer. This work was suorted in art by the National Science Council of Taiwan, under Kuo-Lung Wu s Grant: NSC-9-8-M-68- and Miin-Shen Yang s Grant: NSC-9-8-M---MY. References [] V.N. Vanik, Statistical Learning Theory, Wiley, New York, 998. [] N. Cristianini, J. Shawe-Taylor, An Introduction to Suort Vector Machines, Cambridge University Press, Cambridge,. [] F. Camastra, A. Verri, A novel kernel method for clustering, I Trans. Pattern Anal. Mach. Intell. 7 () 8 8. [] M. Girolami, Mercer kernel based clustering in feature sace, I Trans. Neural Networks () [] K.R. Muller, S. Mika, G. Rätsch, K. Tsuda, B. Schölkof, An introduction to kernel-based learning algorithm, I Trans. Neural Networks () 8. [6] B.W. Silverman, Density stimation for Statistics and Data Analysis, Chaman & Hall, New York, 998. [7] K. Fukunaga, L.D. Hostetler, The estimation of the gradient of a density function with alications in attern recognition, I Trans. Inf. Theory (97). [8] Y. Cheng, Mean shift, mode seeking, and clustering, I Trans. Pattern Anal. Mach. Intell. 7 (99) [9] R.R. Yager, D.P. Filev, Aroimate clustering via the mountain method, I Trans. Syst. Man Cybern. (99) [] D. Comaniciu, An algorithm for data-driven bandwidth selection, I Trans. Pattern Anal. Mach. Intell. () [] D. Comaniciu, P. Meer, Mean shift: a robust aroach toward feature sace analysis, I Trans. Pattern Anal. Mach. Intell. () [] K.I. Kim, K. Jung, J.H. Kim, Teture-based aroach for tet detection in images using suort vector machines and continuously adative mean shift algorithm, I Trans. Pattern Anal. Mach. Intell. () [] X. Yang, J. Liu, Unsuervised teture segmentation with one-ste mean shift and boundary Markov random fields, Pattern Recognition Lett. () 7 8. [] D. Comaniciu, V. Ramesh, P. Meer, Kernel-based object tracking, I Trans. Pattern Anal. Mach. Intell. () [] N.S. Peng, J. Yang, Z. Liu, Mean shift blob tracking with kernel histogram filtering and hyothesis testing, Pattern Recognition Lett. 6 () 6 6. [6] O. Debeir, P.V. Ham, R. Kiss, C. Decaestecker, Tracking of migrating cells nuder hase-contrast video microscoy with combined mean-shift rocesses, I Trans. Med. Imaging () [7] H. Chen, P. Meer, Robust fusion of uncertain information, I Trans. Syst. Man Cybern. B Cybern. () [8] M. Fashing, C. Tomasi, Mean shift is a bound otimization, I Trans. Pattern Anal. Mach. Intell. 7 () 7 7. [9] S.L. Chiu, Fuzzy model identification based on cluster estimation, J. Intell. Fuzzy Syst. (99) [] M.S. Yang, K.L. Wu, A modified mountain clustering algorithm, Pattern Anal. Al. 8 () 8. [] N.R. Pal, D. Chakraborty, Mountain and subtractive clustering method: imrovements and generalizations, Int. J. Intell. Syst. () 9. [] B.P. Velthuizen, L.O. Hall, L.P. Clarke, M.L. Silbiger, An investigation of mountain method clustering for large data sets, Pattern Recognition (997). [] K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, New York, 99. [] G. Beni, X. Liu, A least biased fuzzy clustering method, I Trans. Pattern Anal. Mach. Intell. 6 (99) [] P.J. Huber, Robust Statistics, Wiley, New York, 98. [6] X. Zhuang, T. Wang, P. Zhang, A highly robust estimator through artially likelihood function modeling and its alication in comuter vision, I Trans. Pattern Anal. Mach. Intell. (99) 9. [7] K.L. Wu, M.S. Yang, Alternative c-means clustering algorithms, Pattern Recognition () [8] C.V. Stewart, Minran: a new robust estimator for comuter vision, I Trans. Pattern Anal. Mach. Intell. 7 (99) [9] R.N. Dave, R. Krishnauram, Robust clustering methods: a unified view, I Trans. Fuzzy Syst. (997) 7 9. [] H. Frigui, R. Krishnauram, A robust cometitive clustering algorithm with alications in comuter vision, I Trans. Pattern Anal. Mach. Intell. (999) 6. [] K. Rose,. Gurewitz, G.C. Fo, Statistical mechanics and hase transitions in clustering, Phys. Rev. Lett. 6 (99) [] K. Rose,. Gurewitz, G.C. Fo, A deterministic annealing aroach to clustering, Pattern Recognition Lett. (99) [] S. Chen, D. Zhang, Robust image segmentation using FCM with secial constraints based on new kernel-induced distance measure, I Trans. Syst. Man Cybern. B Cybern. () [] M.N. Ahmed, S.M. Yamany, N. Mohamed, A.A. Farag, T. Moriarty, A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data, I Trans. Med. Imaging () [] M.S. Yang, K.L. Wu, A similarity-based robust clustering method, I Trans. Pattern Anal. Mach. Intell. 6 () 8. [6] R. Krishnauram, J.M. Keller, A ossibilistic aroach to clustering, I Trans. Fuzzy Syst. (99) 98. [7] R. Krishnauram, H. Frigui, O. Nasraoui, Fuzzy and ossibilistic shell clustering algorithm and their alication to boundary detection and surface aroimation, I Trans. Fuzzy Syst. (99) 9 6. [8] H. Frigui, R. Krishnauram, Clustering by cometitive agglomeration, Pattern Recognition (997). [9] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithm, Plenum Press, New York, 98. [] J.C. Bezdek, Cluster validity with fuzzy sets, J. Cybern. (97) 8 7. [] Y. Fukuyama, M. Sugeno, A new method of choosing the number of clusters for fuzzy c-means method, in: Proceedings of the th Fuzzy System Symosium, 989,. 7, (in Jaanese). [] X.L. Xie, G. Beni, A validity measure for fuzzy clustering, I Trans. Pattern Anal. Mach. Intell. (99) [] K.L. Wu, M.S. Yang, A cluster validity inde for fuzzy clustering, Pattern Recognition Lett. 6 () 7 9. [] C.L. Blake, C.J. Merz, UCI reository of machine learning databases, a huge collection of artificial and real-world data sets, 998. Available from: htt://www.ics.uci.edu mlearn/mlreository.html. About the Author KUO-LUNG WU received the BS degree in mathematics in 997, the MS and PhD degrees in alied mathematics in and, all from the Chung Yuan Christian University, Chungli, Taiwan. Since, he has been an Assistant Professor in the Deartment of Information Management at Kun Shan University of Technology, Tainan, Taiwan. He is a member of the Phi Tau Phi Scholastic Honor Society of Taiwan. His research interests include fuzzy theorem, cluster analysis, attern recognition and neural networks. About the Author MIIN-SHN YANG received the BS degree in mathematics from the Chung Yuan Christian University, Chungli, Taiwan, in 977, the MS degree in alied mathematics from the National Chiao-Tung University, Hsinchu, Taiwan, in 98, and the PhD degree in statistics from the University of South Carolina, Columbia, USA, in 989. In989, he was an Associate Professor of the Deartment of Alied Mathematics at the Chung Yuan Christian University. Since 99, he has been a Professor at the same university, where, from to, he was the Chairman of the Deartment of Alied Mathematics. During , he was a Visiting Professor with the Deartment of Industrial ngineering, University of Washington, Seattle, USA. His current research interests include alications of statistics, fuzzy clustering, neural fuzzy systems, attern recognition and machine learning. Dr. Yang is an Associate ditor of the I Transactions on Fuzzy Systems.

### A MOST PROBABLE POINT-BASED METHOD FOR RELIABILITY ANALYSIS, SENSITIVITY ANALYSIS AND DESIGN OPTIMIZATION

9 th ASCE Secialty Conference on Probabilistic Mechanics and Structural Reliability PMC2004 Abstract A MOST PROBABLE POINT-BASED METHOD FOR RELIABILITY ANALYSIS, SENSITIVITY ANALYSIS AND DESIGN OPTIMIZATION

### Alpha Channel Estimation in High Resolution Images and Image Sequences

In IEEE Comuter Society Conference on Comuter Vision and Pattern Recognition (CVPR 2001), Volume I, ages 1063 68, auai Hawaii, 11th 13th Dec 2001 Alha Channel Estimation in High Resolution Images and Image

### Point Location. Preprocess a planar, polygonal subdivision for point location queries. p = (18, 11)

Point Location Prerocess a lanar, olygonal subdivision for oint location ueries. = (18, 11) Inut is a subdivision S of comlexity n, say, number of edges. uild a data structure on S so that for a uery oint

### CSI:FLORIDA. Section 4.4: Logistic Regression

SI:FLORIDA Section 4.4: Logistic Regression SI:FLORIDA Reisit Masked lass Problem.5.5 2 -.5 - -.5 -.5 - -.5.5.5 We can generalize this roblem to two class roblem as well! SI:FLORIDA Reisit Masked lass

### The Online Freeze-tag Problem

The Online Freeze-tag Problem Mikael Hammar, Bengt J. Nilsson, and Mia Persson Atus Technologies AB, IDEON, SE-3 70 Lund, Sweden mikael.hammar@atus.com School of Technology and Society, Malmö University,

### CABRS CELLULAR AUTOMATON BASED MRI BRAIN SEGMENTATION

XI Conference "Medical Informatics & Technologies" - 2006 Rafał Henryk KARTASZYŃSKI *, Paweł MIKOŁAJCZAK ** MRI brain segmentation, CT tissue segmentation, Cellular Automaton, image rocessing, medical

### Multistage Human Resource Allocation for Software Development by Multiobjective Genetic Algorithm

The Oen Alied Mathematics Journal, 2008, 2, 95-03 95 Oen Access Multistage Human Resource Allocation for Software Develoment by Multiobjective Genetic Algorithm Feng Wen a,b and Chi-Ming Lin*,a,c a Graduate

### The fast Fourier transform method for the valuation of European style options in-the-money (ITM), at-the-money (ATM) and out-of-the-money (OTM)

Comutational and Alied Mathematics Journal 15; 1(1: 1-6 Published online January, 15 (htt://www.aascit.org/ournal/cam he fast Fourier transform method for the valuation of Euroean style otions in-the-money

### 4 Perceptron Learning Rule

Percetron Learning Rule Objectives Objectives - Theory and Examles - Learning Rules - Percetron Architecture -3 Single-Neuron Percetron -5 Multile-Neuron Percetron -8 Percetron Learning Rule -8 Test Problem

### Softmax Model as Generalization upon Logistic Discrimination Suffers from Overfitting

Journal of Data Science 12(2014),563-574 Softmax Model as Generalization uon Logistic Discrimination Suffers from Overfitting F. Mohammadi Basatini 1 and Rahim Chiniardaz 2 1 Deartment of Statistics, Shoushtar

### A Virtual Machine Dynamic Migration Scheduling Model Based on MBFD Algorithm

International Journal of Comuter Theory and Engineering, Vol. 7, No. 4, August 2015 A Virtual Machine Dynamic Migration Scheduling Model Based on MBFD Algorithm Xin Lu and Zhuanzhuan Zhang Abstract This

### Moving Objects Tracking in Video by Graph Cuts and Parameter Motion Model

International Journal of Comuter Alications (0975 8887) Moving Objects Tracking in Video by Grah Cuts and Parameter Motion Model Khalid Housni, Driss Mammass IRF SIC laboratory, Faculty of sciences Agadir

### ENFORCING SAFETY PROPERTIES IN WEB APPLICATIONS USING PETRI NETS

ENFORCING SAFETY PROPERTIES IN WEB APPLICATIONS USING PETRI NETS Liviu Grigore Comuter Science Deartment University of Illinois at Chicago Chicago, IL, 60607 lgrigore@cs.uic.edu Ugo Buy Comuter Science

### Stochastic Derivation of an Integral Equation for Probability Generating Functions

Journal of Informatics and Mathematical Sciences Volume 5 (2013), Number 3,. 157 163 RGN Publications htt://www.rgnublications.com Stochastic Derivation of an Integral Equation for Probability Generating

### Universiteit-Utrecht. Department. of Mathematics. Optimal a priori error bounds for the. Rayleigh-Ritz method

Universiteit-Utrecht * Deartment of Mathematics Otimal a riori error bounds for the Rayleigh-Ritz method by Gerard L.G. Sleijen, Jaser van den Eshof, and Paul Smit Prerint nr. 1160 Setember, 2000 OPTIMAL

### Joint Distributions. Lecture 5. Probability & Statistics in Engineering. 0909.400.01 / 0909.400.02 Dr. P. s Clinic Consultant Module in.

-3σ -σ -σ +σ +σ +3σ Joint Distributions Lecture 5 0909.400.01 / 0909.400.0 Dr. P. s Clinic Consultant Module in Probabilit & Statistics in Engineering Toda in P&S -3σ -σ -σ +σ +σ +3σ Dealing with multile

### SQUARE GRID POINTS COVERAGED BY CONNECTED SOURCES WITH COVERAGE RADIUS OF ONE ON A TWO-DIMENSIONAL GRID

International Journal of Comuter Science & Information Technology (IJCSIT) Vol 6, No 4, August 014 SQUARE GRID POINTS COVERAGED BY CONNECTED SOURCES WITH COVERAGE RADIUS OF ONE ON A TWO-DIMENSIONAL GRID

### Minimizing the Communication Cost for Continuous Skyline Maintenance

Minimizing the Communication Cost for Continuous Skyline Maintenance Zhenjie Zhang, Reynold Cheng, Dimitris Paadias, Anthony K.H. Tung School of Comuting National University of Singaore {zhenjie,atung}@com.nus.edu.sg

### An Associative Memory Readout in ESN for Neural Action Potential Detection

g An Associative Memory Readout in ESN for Neural Action Potential Detection Nicolas J. Dedual, Mustafa C. Ozturk, Justin C. Sanchez and José C. Princie Abstract This aer describes how Echo State Networks

### Automatic Search for Correlated Alarms

Automatic Search for Correlated Alarms Klaus-Dieter Tuchs, Peter Tondl, Markus Radimirsch, Klaus Jobmann Institut für Allgemeine Nachrichtentechnik, Universität Hannover Aelstraße 9a, 0167 Hanover, Germany

### On Multicast Capacity and Delay in Cognitive Radio Mobile Ad-hoc Networks

On Multicast Caacity and Delay in Cognitive Radio Mobile Ad-hoc Networks Jinbei Zhang, Yixuan Li, Zhuotao Liu, Fan Wu, Feng Yang, Xinbing Wang Det of Electronic Engineering Det of Comuter Science and Engineering

### An important observation in supply chain management, known as the bullwhip effect,

Quantifying the Bullwhi Effect in a Simle Suly Chain: The Imact of Forecasting, Lead Times, and Information Frank Chen Zvi Drezner Jennifer K. Ryan David Simchi-Levi Decision Sciences Deartment, National

### Monitoring Frequency of Change By Li Qin

Monitoring Frequency of Change By Li Qin Abstract Control charts are widely used in rocess monitoring roblems. This aer gives a brief review of control charts for monitoring a roortion and some initial

### Project Management and. Scheduling CHAPTER CONTENTS

6 Proect Management and Scheduling HAPTER ONTENTS 6.1 Introduction 6.2 Planning the Proect 6.3 Executing the Proect 6.7.1 Monitor 6.7.2 ontrol 6.7.3 losing 6.4 Proect Scheduling 6.5 ritical Path Method

### An Analysis of Reliable Classifiers through ROC Isometrics

An Analysis of Reliable Classifiers through ROC Isometrics Stijn Vanderlooy s.vanderlooy@cs.unimaas.nl Ida G. Srinkhuizen-Kuyer kuyer@cs.unimaas.nl Evgueni N. Smirnov smirnov@cs.unimaas.nl MICC-IKAT, Universiteit

### DAY-AHEAD ELECTRICITY PRICE FORECASTING BASED ON TIME SERIES MODELS: A COMPARISON

DAY-AHEAD ELECTRICITY PRICE FORECASTING BASED ON TIME SERIES MODELS: A COMPARISON Rosario Esínola, Javier Contreras, Francisco J. Nogales and Antonio J. Conejo E.T.S. de Ingenieros Industriales, Universidad

### STUDIES ON DYNAMIC VISCOSITY CHANGES OF THE ENGINE S LUBRICATION OIL DEPENDING ON THE PRESSURE

Journal of KONES Powertrain and Transort, Vol. 20, No. 4 2013 STUDIES ON DYNAMIC VISCOSITY CHANGES OF THE ENGINE S LUBRICATION OIL DEPENDING ON THE PRESSURE Grzegorz Sikora Gdynia Maritime University Deartment

### Confidence Intervals for Capture-Recapture Data With Matching

Confidence Intervals for Cature-Recature Data With Matching Executive summary Cature-recature data is often used to estimate oulations The classical alication for animal oulations is to take two samles

### United Arab Emirates University College of Sciences Department of Mathematical Sciences HOMEWORK 1 SOLUTION. Section 10.1 Vectors in the Plane

United Arab Emirates University College of Sciences Deartment of Mathematical Sciences HOMEWORK 1 SOLUTION Section 10.1 Vectors in the Plane Calculus II for Engineering MATH 110 SECTION 0 CRN 510 :00 :00

### A Modified Measure of Covert Network Performance

A Modified Measure of Covert Network Performance LYNNE L DOTY Marist College Deartment of Mathematics Poughkeesie, NY UNITED STATES lynnedoty@maristedu Abstract: In a covert network the need for secrecy

### Forensic Science International

Forensic Science International 214 (2012) 33 43 Contents lists available at ScienceDirect Forensic Science International jou r nal h o me age: w ww.els evier.co m/lo c ate/fo r sc iin t A robust detection

### Re-Dispatch Approach for Congestion Relief in Deregulated Power Systems

Re-Disatch Aroach for Congestion Relief in Deregulated ower Systems Ch. Naga Raja Kumari #1, M. Anitha 2 #1, 2 Assistant rofessor, Det. of Electrical Engineering RVR & JC College of Engineering, Guntur-522019,

### On the predictive content of the PPI on CPI inflation: the case of Mexico

On the redictive content of the PPI on inflation: the case of Mexico José Sidaoui, Carlos Caistrán, Daniel Chiquiar and Manuel Ramos-Francia 1 1. Introduction It would be natural to exect that shocks to

### The Optimal Sequenced Route Query

The Otimal Sequenced Route Query Mehdi Sharifzadeh, Mohammad Kolahdouzan, Cyrus Shahabi Comuter Science Deartment University of Southern California Los Angeles, CA 90089-0781 [sharifza, kolahdoz, shahabi]@usc.edu

### Design of A Knowledge Based Trouble Call System with Colored Petri Net Models

2005 IEEE/PES Transmission and Distribution Conference & Exhibition: Asia and Pacific Dalian, China Design of A Knowledge Based Trouble Call System with Colored Petri Net Models Hui-Jen Chuang, Chia-Hung

### Loglikelihood and Confidence Intervals

Stat 504, Lecture 3 Stat 504, Lecture 3 2 Review (contd.): Loglikelihood and Confidence Intervals The likelihood of the samle is the joint PDF (or PMF) L(θ) = f(x,.., x n; θ) = ny f(x i; θ) i= Review:

### Machine Learning with Operational Costs

Journal of Machine Learning Research 14 (2013) 1989-2028 Submitted 12/11; Revised 8/12; Published 7/13 Machine Learning with Oerational Costs Theja Tulabandhula Deartment of Electrical Engineering and

### Large-Scale IP Traceback in High-Speed Internet: Practical Techniques and Theoretical Foundation

Large-Scale IP Traceback in High-Seed Internet: Practical Techniques and Theoretical Foundation Jun Li Minho Sung Jun (Jim) Xu College of Comuting Georgia Institute of Technology {junli,mhsung,jx}@cc.gatech.edu

### Fourier Analysis of Stochastic Processes

Fourier Analysis of Stochastic Processes. Time series Given a discrete time rocess ( n ) nz, with n :! R or n :! C 8n Z, we de ne time series a realization of the rocess, that is to say a series (x n )

### Synopsys RURAL ELECTRICATION PLANNING SOFTWARE (LAPER) Rainer Fronius Marc Gratton Electricité de France Research and Development FRANCE

RURAL ELECTRICATION PLANNING SOFTWARE (LAPER) Rainer Fronius Marc Gratton Electricité de France Research and Develoment FRANCE Synosys There is no doubt left about the benefit of electrication and subsequently

### A Complete Operational Amplifier Noise Model: Analysis and Measurement of Correlation Coefficient

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: FUNDAMENTAL THEORY AND APPLICATION, VOL. 47, NO. 3, MARCH 000 40 for ractical alication, oening the ath for widesread adotion of the clock-gating technique

### Incremental Clustering for Mining in a Data Warehousing Environment

Incremental Clustering for Mining in a Data Warehousing Environment Martin Ester, Hans-Peter Kriegel, Jörg Sander, Michael Wimmer, Xiaowei Xu Institute for Comuter Science, University of Munich Oettingenstr

### D.Sailaja, K.Nasaramma, M.Sumender Roy, Venkateswarlu Bondu

Predictive Modeling of Customers in Personalization Alications with Context D.Sailaja, K.Nasaramma, M.Sumender Roy, Venkateswarlu Bondu Nasaramma.K is currently ursuing her M.Tech in Godavari Institute

### A Simple Model of Pricing, Markups and Market. Power Under Demand Fluctuations

A Simle Model of Pricing, Markus and Market Power Under Demand Fluctuations Stanley S. Reynolds Deartment of Economics; University of Arizona; Tucson, AZ 85721 Bart J. Wilson Economic Science Laboratory;

### Comparing Dissimilarity Measures for Symbolic Data Analysis

Comaring Dissimilarity Measures for Symbolic Data Analysis Donato MALERBA, Floriana ESPOSITO, Vincenzo GIOVIALE and Valentina TAMMA Diartimento di Informatica, University of Bari Via Orabona 4 76 Bari,

### Multiperiod Portfolio Optimization with General Transaction Costs

Multieriod Portfolio Otimization with General Transaction Costs Victor DeMiguel Deartment of Management Science and Oerations, London Business School, London NW1 4SA, UK, avmiguel@london.edu Xiaoling Mei

### The Fundamental Incompatibility of Scalable Hamiltonian Monte Carlo and Naive Data Subsampling

The Fundamental Incomatibility of Scalable Hamiltonian Monte Carlo and Naive Data Subsamling Michael Betancourt Deartment of Statistics, University of Warwick, Coventry, UK CV4 7A BETANAPHA@GMAI.COM Abstract

NAVAL POSTGRADUATE SCHOOL MONTEREY CALIFORNIA THESIS SYMMETRICAL RESIDUE-TO-BINARY CONVERSION ALGORITHM PIPELINED FPGA IMPLEMENTATION AND TESTING LOGIC FOR USE IN HIGH-SPEED FOLDING DIGITIZERS by Ross

### Web Application Scalability: A Model-Based Approach

Coyright 24, Software Engineering Research and Performance Engineering Services. All rights reserved. Web Alication Scalability: A Model-Based Aroach Lloyd G. Williams, Ph.D. Software Engineering Research

### Time-Cost Trade-Offs in Resource-Constraint Project Scheduling Problems with Overlapping Modes

Time-Cost Trade-Offs in Resource-Constraint Proect Scheduling Problems with Overlaing Modes François Berthaut Robert Pellerin Nathalie Perrier Adnène Hai February 2011 CIRRELT-2011-10 Bureaux de Montréal

### A NONLINEAR SWITCHING CONTROL METHOD FOR MAGNETIC BEARING SYSTEMS MINIMIZING THE POWER CONSUMPTION

Coyright IFAC 5th Triennial World Congress, Barcelona, Sain A NONLINEAR SWITCHING CONTROL METHOD FOR MAGNETIC BEARING SYSTEMS MINIMIZING THE POWER CONSUMPTION Kang-Zhi Liu Λ; Akihiro Ikai Λ Akihiro Ogata

### Implementation of Statistic Process Control in a Painting Sector of a Automotive Manufacturer

4 th International Conference on Industrial Engineering and Industrial Management IV Congreso de Ingeniería de Organización Donostia- an ebastián, etember 8 th - th Imlementation of tatistic Process Control

### One-Chip Linear Control IPS, F5106H

One-Chi Linear Control IPS, F5106H NAKAGAWA Sho OE Takatoshi IWAMOTO Motomitsu ABSTRACT In the fi eld of vehicle electrical comonents, the increasing demands for miniaturization, reliability imrovement

### The impact of metadata implementation on webpage visibility in search engine results (Part II) q

Information Processing and Management 41 (2005) 691 715 www.elsevier.com/locate/inforoman The imact of metadata imlementation on webage visibility in search engine results (Part II) q Jin Zhang *, Alexandra

### lecture 25: Gaussian quadrature: nodes, weights; examples; extensions

38 lecture 25: Gaussian quadrature: nodes, weights; examles; extensions 3.5 Comuting Gaussian quadrature nodes and weights When first aroaching Gaussian quadrature, the comlicated characterization of the

### Discrete Stochastic Approximation with Application to Resource Allocation

Discrete Stochastic Aroximation with Alication to Resource Allocation Stacy D. Hill An otimization roblem involves fi nding the best value of an obective function or fi gure of merit the value that otimizes

### Migration to Object Oriented Platforms: A State Transformation Approach

Migration to Object Oriented Platforms: A State Transformation Aroach Ying Zou, Kostas Kontogiannis Det. of Electrical & Comuter Engineering University of Waterloo Waterloo, ON, N2L 3G1, Canada {yzou,

### Static and Dynamic Properties of Small-world Connection Topologies Based on Transit-stub Networks

Static and Dynamic Proerties of Small-world Connection Toologies Based on Transit-stub Networks Carlos Aguirre Fernando Corbacho Ramón Huerta Comuter Engineering Deartment, Universidad Autónoma de Madrid,

### where a, b, c, and d are constants with a 0, and x is measured in radians. (π radians =

Introduction to Modeling 3.6-1 3.6 Sine and Cosine Functions The general form of a sine or cosine function is given by: f (x) = asin (bx + c) + d and f(x) = acos(bx + c) + d where a, b, c, and d are constants

### Efficient Training of Kalman Algorithm for MIMO Channel Tracking

Efficient Training of Kalman Algorithm for MIMO Channel Tracking Emna Eitel and Joachim Seidel Institute of Telecommunications, University of Stuttgart Stuttgart, Germany Abstract In this aer, a Kalman

### The Lognormal Distribution Engr 323 Geppert page 1of 6 The Lognormal Distribution

Engr 33 Geert age 1of 6 The Lognormal Distribution In general, the most imortant roerty of the lognormal rocess is that it reresents a roduct of indeendent random variables. (Class Handout on Lognormal

### Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information

Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information Heiko Hirschmüller Institute of Robotics and Mechatronics Oberfaffenhofen German Aerosace Center (DLR) P.O. Box 1116,

### Design and Development of Decision Making System Using Fuzzy Analytic Hierarchy Process

American Journal of Alied Sciences 5 (7): 783787, 2008 ISSN 15469239 2008 Science Publications Design and Develoment of Decision Making System Using Fuzzy Analytic Hierarchy Process Chin Wen Cheong, Lee

### ChE 120B Lumped Parameter Models for Heat Transfer and the Blot Number

ChE 0B Lumed Parameter Models for Heat Transfer and the Blot Number Imagine a slab that has one dimension, of thickness d, that is much smaller than the other two dimensions; we also assume that the slab

### Approximate Guarding of Monotone and Rectilinear Polygons

Aroximate Guarding of Monotone and Rectilinear Polygons Erik Krohn Bengt J. Nilsson Abstract We show that vertex guarding a monotone olygon is NP-hard and construct a constant factor aroximation algorithm

### Piracy and Network Externality An Analysis for the Monopolized Software Industry

Piracy and Network Externality An Analysis for the Monoolized Software Industry Ming Chung Chang Deartment of Economics and Graduate Institute of Industrial Economics mcchang@mgt.ncu.edu.tw Chiu Fen Lin

### Classical Fourier Series Introduction: Real Fourier Series

Math 344 May 1, 1 Classical Fourier Series Introduction: Real Fourier Series The orthogonality roerties of the sine and cosine functions make them good candidates as basis functions when orthogonal eansions

### Load Balancing Mechanism in Agent-based Grid

Communications on Advanced Comutational Science with Alications 2016 No. 1 (2016) 57-62 Available online at www.isacs.com/cacsa Volume 2016, Issue 1, Year 2016 Article ID cacsa-00042, 6 Pages doi:10.5899/2016/cacsa-00042

### Principles of Hydrology. Hydrograph components include rising limb, recession limb, peak, direct runoff, and baseflow.

Princiles of Hydrology Unit Hydrograh Runoff hydrograh usually consists of a fairly regular lower ortion that changes slowly throughout the year and a raidly fluctuating comonent that reresents the immediate

### Local Connectivity Tests to Identify Wormholes in Wireless Networks

Local Connectivity Tests to Identify Wormholes in Wireless Networks Xiaomeng Ban Comuter Science Stony Brook University xban@cs.sunysb.edu Rik Sarkar Comuter Science Freie Universität Berlin sarkar@inf.fu-berlin.de

### High Quality Offset Printing An Evolutionary Approach

High Quality Offset Printing An Evolutionary Aroach Ralf Joost Institute of Alied Microelectronics and omuter Engineering University of Rostock Rostock, 18051, Germany +49 381 498 7272 ralf.joost@uni-rostock.de

### Risk in Revenue Management and Dynamic Pricing

OPERATIONS RESEARCH Vol. 56, No. 2, March Aril 2008,. 326 343 issn 0030-364X eissn 1526-5463 08 5602 0326 informs doi 10.1287/ore.1070.0438 2008 INFORMS Risk in Revenue Management and Dynamic Pricing Yuri

### Stability Improvements of Robot Control by Periodic Variation of the Gain Parameters

Proceedings of the th World Congress in Mechanism and Machine Science ril ~4, 4, ianin, China China Machinery Press, edited by ian Huang. 86-8 Stability Imrovements of Robot Control by Periodic Variation

### X How to Schedule a Cascade in an Arbitrary Graph

X How to Schedule a Cascade in an Arbitrary Grah Flavio Chierichetti, Cornell University Jon Kleinberg, Cornell University Alessandro Panconesi, Saienza University When individuals in a social network

### Buffer Capacity Allocation: A method to QoS support on MPLS networks**

Buffer Caacity Allocation: A method to QoS suort on MPLS networks** M. K. Huerta * J. J. Padilla X. Hesselbach ϒ R. Fabregat O. Ravelo Abstract This aer describes an otimized model to suort QoS by mean

### Concurrent Program Synthesis Based on Supervisory Control

010 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 30-July 0, 010 ThB07.5 Concurrent Program Synthesis Based on Suervisory Control Marian V. Iordache and Panos J. Antsaklis Abstract

### An Introduction to Risk Parity Hossein Kazemi

An Introduction to Risk Parity Hossein Kazemi In the aftermath of the financial crisis, investors and asset allocators have started the usual ritual of rethinking the way they aroached asset allocation

### The risk of using the Q heterogeneity estimator for software engineering experiments

Dieste, O., Fernández, E., García-Martínez, R., Juristo, N. 11. The risk of using the Q heterogeneity estimator for software engineering exeriments. The risk of using the Q heterogeneity estimator for

### Jena Research Papers in Business and Economics

Jena Research Paers in Business and Economics A newsvendor model with service and loss constraints Werner Jammernegg und Peter Kischka 21/2008 Jenaer Schriften zur Wirtschaftswissenschaft Working and Discussion

### HYPOTHESIS TESTING FOR THE PROCESS CAPABILITY RATIO. A thesis presented to. the faculty of

HYPOTHESIS TESTING FOR THE PROESS APABILITY RATIO A thesis resented to the faculty of the Russ ollege of Engineering and Technology of Ohio University In artial fulfillment of the requirement for the degree

### Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006

More on Mean-shift R.Collins, CSE, PSU Spring 2006 Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent

### A Brief Introduction to Design of Experiments

J. K. TELFORD D A Brief Introduction to Design of Exeriments Jacqueline K. Telford esign of exeriments is a series of tests in which uroseful changes are made to the inut variables of a system or rocess

### The Graphical Method. Lecture 1

References: Anderson, Sweeney, Williams: An Introduction to Management Science - quantitative aroaches to decision maing 7 th ed Hamdy A Taha: Oerations Research, An Introduction 5 th ed Daellenbach, George,

### Memory management. Chapter 4: Memory Management. Memory hierarchy. In an ideal world. Basic memory management. Fixed partitions: multiple programs

Memory management Chater : Memory Management Part : Mechanisms for Managing Memory asic management Swaing Virtual Page relacement algorithms Modeling age relacement algorithms Design issues for aging systems

### Lecture 21 and 22: The Prime Number Theorem

Lecture and : The Prime Number Theorem (New lecture, not in Tet) The location of rime numbers is a central question in number theory. Around 88, Legendre offered eerimental evidence that the number π()

### Improving convergence of quasi dynamic assignment models

Imroving convergence of quasi dynamic assignment mels Luuk Bredere,*,,2, Adam Pel, Luc Wismans 2,, Erik de Romh Deartment of Transort & Planning, Delft University of Technology 2 DAT.mobility a Goudael

### Int. J. Advanced Networking and Applications Volume: 6 Issue: 4 Pages: 2386-2392 (2015) ISSN: 0975-0290

2386 Survey: Biological Insired Comuting in the Network Security V Venkata Ramana Associate Professor, Deartment of CSE, CBIT, Proddatur, Y.S.R (dist), A.P-516360 Email: ramanacsecbit@gmail.com Y.Subba

### Simulink Implementation of a CDMA Smart Antenna System

Simulink Imlementation of a CDMA Smart Antenna System MOSTAFA HEFNAWI Deartment of Electrical and Comuter Engineering Royal Military College of Canada Kingston, Ontario, K7K 7B4 CANADA Abstract: - The

### Probabilistic models for mechanical properties of prestressing strands

Probabilistic models for mechanical roerties of restressing strands Luciano Jacinto a, Manuel Pia b, Luís Neves c, Luís Oliveira Santos b a Instituto Suerior de Engenharia de Lisboa, Rua Conselheiro Emídio

### THE RELATIONSHIP BETWEEN EMPLOYEE PERFORMANCE AND THEIR EFFICIENCY EVALUATION SYSTEM IN THE YOTH AND SPORT OFFICES IN NORTH WEST OF IRAN

THE RELATIONSHIP BETWEEN EMPLOYEE PERFORMANCE AND THEIR EFFICIENCY EVALUATION SYSTEM IN THE YOTH AND SPORT OFFICES IN NORTH WEST OF IRAN *Akbar Abdolhosenzadeh 1, Laya Mokhtari 2, Amineh Sahranavard Gargari

### Improved Algorithms for Data Visualization in Forensic DNA Analysis

Imroved Algorithms for Data Visualization in Forensic DNA Analysis Noor Maizura Mohamad Noor, Senior Member IACSIT, Mohd Iqbal akim arun, and Ahmad Faiz Ghazali Abstract DNA rofiles from forensic evidence

### An Efficient Method for Improving Backfill Job Scheduling Algorithm in Cluster Computing Systems

The International ournal of Soft Comuting and Software Engineering [SCSE], Vol., No., Secial Issue: The Proceeding of International Conference on Soft Comuting and Software Engineering 0 [SCSE ], San Francisco

### The Magnus-Derek Game

The Magnus-Derek Game Z. Nedev S. Muthukrishnan Abstract We introduce a new combinatorial game between two layers: Magnus and Derek. Initially, a token is laced at osition 0 on a round table with n ositions.

### More Properties of Limits: Order of Operations

math 30 day 5: calculating its 6 More Proerties of Limits: Order of Oerations THEOREM 45 (Order of Oerations, Continued) Assume that!a f () L and that m and n are ositive integers Then 5 (Power)!a [ f

### Chapter 9, Part B Hypothesis Tests. Learning objectives

Chater 9, Part B Hyothesis Tests Slide 1 Learning objectives 1. Able to do hyothesis test about Poulation Proortion 2. Calculatethe Probability of Tye II Errors 3. Understand ower of the test 4. Determinethe

### An inventory control system for spare parts at a refinery: An empirical comparison of different reorder point methods

An inventory control system for sare arts at a refinery: An emirical comarison of different reorder oint methods Eric Porras a*, Rommert Dekker b a Instituto Tecnológico y de Estudios Sueriores de Monterrey,

### Journal of Computational and Applied Mathematics

Journal of Comutational and Alied Mathematics 230 2009 69 82 Contents lists available at ScienceDirect Journal of Comutational and Alied Mathematics journal homeage: www.elsevier.com/locate/cam Proerties

### Rummage Web Server Tuning Evaluation through Benchmark

IJCSNS International Journal of Comuter Science and Network Security, VOL.7 No.9, Setember 27 13 Rummage Web Server Tuning Evaluation through Benchmark (Case study: CLICK, and TIME Parameter) Hiyam S.