Uncertain Data Mining: A New Research Direction

Transcription

1 Uncertan Data Mnng: A New Research Drecton Mchael Chau 1, Reynold Cheng, and Ben Kao 3 1: School of Busness, The Unversty of Hong Kong, Pokfulam, Hong Kong : Department of Computng, Hong Kong Polytechnc Unversty Kowloon, Hong Kong 3: Department of Computer Scence, The Unversty of Hong Kong, Pokfulam, Hong Kong Emals: mchau@busness.hku.hk, csckcheng@comp.polyu.edu.hk, kao@cs.hku.hk Abstract Data uncertanty s often found n real-world applcatons due to reasons such as mprecse measurement, outdated sources, or samplng errors. Recently, much research has been publshed n the area of managng data uncertanty n databases. We propose that when data mnng s performed on uncertan data, data uncertanty has to be consdered n order to obtan hgh qualty data mnng results. We call ths the "Uncertan Data Mnng" problem. In ths paper, we present a framework for possble research drectons n ths area. We also present the UK-means clusterng algorthm as an example to llustrate how the tradtonal K-means algorthm can be modfed to handle data uncertanty n data mnng. 1. Introducton Data s often assocated wth uncertanty because of measurement naccuracy, samplng dscrepancy, outdated data sources, or other errors. Ths s especally true for applcatons that requre nteracton wth the physcal world, such as locaton-based servces [15] and sensor montorng [3]. For example, n the scenaro of movng obects (such as vehcles or people), t s mpossble for the database to track the exact locatons of all obects at all tme nstants. Therefore, the locaton of each obect s assocated wth uncertanty between updates [4]. These varous sources of uncertanty have to be consdered n order to produce accurate query and mnng results. In recent years, there has been much research on the management of uncertan data n databases, such as the representaton of uncertanty n databases and queryng data wth uncertanty. However, lttle research work has addressed the ssue of mnng uncertan data. We note that wth uncertanty, data values are no longer atomc. To apply tradtonal data mnng technques, uncertan data has to be summarzed nto atomc values. Takng movng-obect applcatons as an example agan, the locaton of an obect can be summarzed ether by ts last recorded locaton, or by an expected locaton (f the probablty dstrbuton of an obect s locaton s taken nto account). Unfortunately, dscrepancy n the summarzed recorded values and the actual values could serously affect the qualty of the mnng results. Fgure 1 llustrates ths problem when a clusterng algorthm s appled to movng obects wth locaton uncertanty. Fgure 1(a) shows the actual locatons of a set of obects, and Fgure 1(b) shows the recorded locaton of these obects, whch are already outdated. The clusters obtaned from these outdated values could be sgnfcantly dfferent from those obtaned as f the actual locatons were avalable (Fgure 1(b)). If we solely rely on the recorded values, many obects could possbly be put nto wrong clusters. Even worse, each member of a cluster would change the cluster centrods, thus resultng n more errors. Chau, M., Cheng, R., and Kao, B., "Uncertan Data Mnng: A New Research Drecton," n Proceedngs of the Workshop on the Scences of the Artfcal, Hualen, Tawan, December 7-8, 005.

2 Fgure 1. (a) The real-world data are parttoned nto three clusters (a, b, c). (b) The recorded locatons of some obects (shaded) are not the same as ther true locaton, thus creatng clusters a, b, c and c. Note that a has one fewer obect than a, and b has one more obect than b. Also, c s mstakenly splt nto c and c. (c) Lne uncertanty s consdered to produce clusters a, b and c. The clusterng result s closer to that of (a) than (b). We suggest ncorporatng uncertanty nformaton, such as the probablty densty functons (pdf) of uncertan data, nto exstng data mnng methods so that the mnng results could resemble closer to the results obtaned as f actual data were avalable and used n the mnng process (Fgure (c)). In ths paper we study how uncertanty can be ncorporated n data mnng by usng data clusterng as a motvatng example. We call ths the Uncertan Data Mnng problem. In ths paper, we present a framework for possble research drectons n ths area. The rest of the paper s structured as follows. Related work s revewed n Secton. In Secton 3 we defne the problem of clusterng on data wth uncertanty and present our proposed algorthm. Secton 4 presents the applcaton of our algorthm to a movng-obect database. Detaled experment results are shown n Secton 5. We conclude our paper and suggest possble research drectons n Secton 6.. Research Background In recent years, there s sgnfcant research nterest n data uncertanty management. Data uncertanty can be categorzed nto two types, namely exstental uncertanty and value uncertanty. In the frst type t s uncertan whether the obect or data tuple exsts or not. For example, a tuple n a relatonal database could be assocated wth a probablty value that ndcates the confdence of ts presence [1,11]. In value uncertanty, a data tem s modelled as a closed regon whch bounds ts possble values, together wth a probablty densty functon (pdf) of ts value [3,4,1,15]. Ths model can be used to quantfy the mprecson of locaton and sensor data n a constantly-evolvng envronment. Most works n ths area have been devoted to mprecse queres, whch provde probablstc guarantees over correctness of answers. For example, n [5], ndexng solutons for range queres over uncertan data have been proposed. The same authors also proposed solutons for aggregate queres such as nearest-neghbor queres n [4]. Notce that all these works have appled the study of uncertan data management to smple database queres, nstead of to the relatvely more complcated data analyss and mnng problems. The clusterng problem has been well studed n data mnng research. A standard clusterng process conssts of fve maor steps: pattern representaton, defnton of a pattern smlarty metrc, clusterng or groupng, data abstracton, and output assessment [10]. Only a few studes on data mnng or data clusterng for uncertan data have been reported. Hamdan and Govaert have addressed the problem of fttng mxture denstes to uncertan data for clusterng usng the EM algorthm [8]. However, the model cannot be readly appled to other clusterng algorthms and s rather customzed for EM. Clusterng on nterval data also has been studed. Dfferent dstance measures, lke cty-block dstance or Mnkowsk dstance, have been used n measurng the smlarty between two ntervals [6,9]. The pdf of the nterval s not taken nto account n most of these metrcs. Another related area of research s fuzzy clusterng. Fuzzy clusterng has been long studed n fuzzy logc [13]. In fuzzy clusterng, a cluster s represented by a fuzzy subset of a set of obects. Each obect has a degree of belongngness for each cluster. In other words, an obect can belong to more than one cluster, each

3 wth a dfferent degree. The fuzzy c-means algorthm was one of the most wdely used fuzzy clusterng method [,7]. Dfferent fuzzy clusterng methods have been appled on normal data or fuzzy data to produce fuzzy clusters [14]. Whle ther work s based on a fuzzy data model, our work s developed based on the uncertanty model of movng obects. 3. Taxonomy of Uncertan Data Mnng In Fgure, we propose a taxonomy to llustrate how data mnng methods can be classfed based on whether data mprecson s consdered. There are a number of common data mnng technques, e.g., assocaton rule mnng, data classfcaton, data clusterng, that need to be modfed n order to handle uncertan data. Moreover, we dstngush two types of data clusterng: hard clusterng and fuzzy clusterng. Hard clusterng ams at mprovng the accuracy of clusterng by consderng expected data values after data uncertanty s consdered. On the other hand, fuzzy clusterng presents the clusterng result n a fuzzy form. An example of a fuzzy clusterng result s that each data tem s gven a probablty of beng assgned to each member n a set of clusters [14]. Fgure. A taxonomy of data mnng on data wth uncertanty For example, when uncertanty s consdered, there s an nterestng problem on how each tuple and the uncertanty assocated should be represented n the dataset. Moreover, the noton of support and other metrcs would need to be redefned. Well-known assocaton rule mnng algorthm (such as Apror) has to be revsed n order to take ths nto account. Smlarly, n data classfcaton and data clusterng, tradtonal algorthms may not work any more because uncertanty was not taken nto account. Important metrcs, lke cluster centrods, dstance between two obects, or dstance between an obect and a centrod, have to be redefned and further studed. 4. Example on Uncertan Data Clusterng In ths secton, we present our work on uncertan data clusterng as an example of uncertan data mnng. Ths llustrates our dea of adaptng tradtonal data mnng algorthm for uncertan data. 4.1 Problem Defnton Let S be a set of V-dmensonal vectors x, where = 1 to n, representng the attrbute values of all the records to be consdered n the clusterng applcaton. Each record o s assocated wth a probablty densty functon (pdf), f (x), whch s the probablty densty functon of o s attrbute values x at tme t. We do not lmt how the uncertanty functon evolves over tme, or what the probablty densty functon of a record s. An example pdf s the unform densty functon, whch depcts the worst-case or most uncertan scenaro [3]. Another pdf commonly used s the Gaussan dstrbuton, whch can be used to descrbe measurement errors [1,15].

4 The clusterng problem s to fnd a set C of clusters C, where = 1 to K, wth cluster means c based on smlarty. Dfferent clusterng algorthms have dfferent obectve functons, but the general dea s to mnmze the dstance between obects n the same cluster whle maxmzng the dstance between obects n dfferent clusters. Mnmzaton of ntra-cluster dstance can also be vewed as the mnmzaton of the dstance between each data pont x and the cluster means c of the cluster C that x s assgned to. In ths paper, we only consder hard clusterng,.e., every obect s assgned to one and only one cluster. 4. K-means Clusterng for Precse Data The classcal K-means clusterng algorthm whch ams at fndng a set C of K clusters C wth cluster mean c to mnmze the sum of squared errors (SSE). The SSE s usually calculated as follows: K = 1 x C c x (1) where. s a dstance metrc between a data pont x and a cluster mean c. For example, the Eucldean dstance s defned as: V x y = x y () =1 The mean (centrod) of a cluster C s defned by the followng vector: 1 c = x (3) C C The K-means algorthm s as follows: 1. Assgn ntal values for cluster means c 1 to c K. repeat 3. for = 1 to n do 4. Assgn each data pont x to cluster C where c - x s the mnmum. 5. end for 6. for = 1 to K do 7. Recalculate cluster mean c of cluster C 8. end for 9. untl convergence 10. return C Convergence can be defned based on dfferent crtera. Some example convergence crtera nclude: (1) when the change n the sum of squared errors s smaller than a certan user-specfed threshold, () when no obects are reassgned to a dfferent cluster n an teraton and (3) when the number of teratons has reached a pre-defned maxmum number. 4.3 K-means Clusterng for Uncertan Data In order to take nto account data uncertanty n the clusterng process, we propose a clusterng algorthm wth the goal of mnmzng the expected sum of squared errors E(SSE). Notce that a data obect x s specfed by an uncertanty regon wth an uncertanty pdf f(x ). Gven a set of clusters, C s the expected SSE can be calculated as follow: k E c x = 1 C k (4) = E c x = 1 C ( ) = k = 1 C c x f ( x ) dx

5 Cluster means are gven by: 1 c = E x C C 1 = C 1 = C C C E( x ) x f ( x ) dx (5) We now propose a new K-means algorthm, called UK-means, for clusterng uncertan data. 1. Assgn ntal values for cluster means c 1 to c K. repeat 3. for = 1 to n do 4. Assgn each data pont x to cluster C where E( c - x ) s the mnmum. 5. end for 6. for = 1 to K do 7. Recalculate cluster mean c of cluster C 8. end for 9. untl convergence 10. return C The man dfference between UK-mean clusterng and K-means clusterng les n the computaton of dstance and clusters. In partcular, UK-means compute the expected dstance and cluster centrods based on the data uncertanty model. Agan, convergence can be defned based on dfferent crtera. Note that f the convergence s based on squared error, E(SSE) as n Equaton (4) should be used nstead of SSE. In Step 4, t s often dffcult to determne E( c - x ) algebracally. In partcular, the varety of geometrc shapes of uncertanty regons (e.g., lne, crcle) and dfferent uncertanty pdf mply that numercal ntegraton methods are necessary. In vew of ths, E( c - x ), whch s easer to obtan, s used nstead. Ths allows us to determne the cluster assgnment (.e., Step 4) usng a smple algebrac expresson. 5. A Case Study and Evaluaton 5.1 Clusterng Data wth Lne-movng Uncertanty The UK-means algorthm presented n the last secton s applcable to any uncertanty regon and pdf. To demonstrate the feasblty of the approach, we descrbe how the proposed algorthm can be appled to uncertanty models specfc to movng obects that are movng n a two-dmensonal space. We also present the evaluaton results of the algorthm. The algorthm was appled to a model wth the undrectonal lne-movng uncertanty, whch requres that each obect s locaton s unformly dstrbuted n a lne segment along the lne of movement n one drecton. Suppose we have a centrod c = (p, q) and a data obect x specfed by a lne uncertanty regon wth a unform dstrbuton. Let the end ponts of the lne segment uncertanty be (a,b) and (c,d). The lne equaton can be parametrzed by (a + t (c - a), b + t (d - b)), where t s between [0,1]. Let the uncertanty pdf be f(t). Also, let the dstance of the lne segment uncertanty be D = ( c a) + ( d b). We have: E( c - x ) f ( t)( D t + Bt + C) dt (6) = 1 0

6 where B = [(c - a) (a - p) + (d - b) (b - q)] C = (p - a) + (q - b) If f(t) s unform, then f(t) = 1, and the above becomes: E(dstance of lne uncertanty from centrod )= D B + + C (7) 3 We are thus able to compute the expected squared dstance easly for lne-movng uncertanty for unform dstrbuton. These formulae can be readly used by the UK-means algorthm to decde the assgnment of clusters. Nonetheless, the use of unform dstrbuton s only a specfc example here. When the pdf s are not unform (e.g., Gaussan), samplng technques can be used to estmate E( c - x ). 5. Experments Experments were conducted to evaluate the performance of UK-means. The goal s to study whether the ncluson of data uncertanty mproves clusterng qualty. We smulate the followng scenaro: a system that tracks the locatons of a set of movng obects has taken a snapshot of the whereabouts of the obects. Ths locaton data s stored n a set called recorded. Each obect assumes an uncertanty model. Let uncertanty captures such uncertanty nformaton. We compare two clusterng approaches: (1) apply K-means to recorded and () apply UK-means to recorded + uncertanty. More specfcally, we frst generated a set of random data ponts n a 100 x 100 D space as recorded. For each data pont, we then randomly generated ts uncertanty accordng to the undrectonal lneuncertanty model. The uncertanty specfcaton (uncertanty) of an obect contans the type of the uncertanty (bdrectonal lne), the maxmum dstance d that the obect can move, and the drecton that the obect can move. The actual locatons of the obects were then generated based on recorded and uncertanty, smulatng the scenaro that the obects have moved away from ther orgnal locatons as regstered n recorded. Specfcally, for each data pont, we took ts poston n recorded and then generated a random number to decde the dstance that the obect should have moved. If t s free-movng (crcle) uncertanty or bdrectonal uncertanty, we generated another random number to see whch drecton the obect should move. We use actual to denote the set of actual locatons of the obects. Ideally, a system should know actual and apply K-means on the actual locatons. Although not practcal, such clusterng result serves as a good reference for the qualty of clusterng result. Hence, we compute and compare the cluster output of the followng data sets: (1) recorded (usng classcal K-means) () recorded + uncertanty (usng UK-means) (3) actual (usng classcal K-means) In order to verfy the ablty of the UK-means algorthm n generatng a set of clusters close to the ones generated from actual, we apply a wdely-used metrc known as the Adusted Rand Index (ARI), whch measures the smlarty between clusterng results [16]. A hgher ARI value ndcates a hgher degree of smlarty between two clusters. We wll compare the ARI between the sets of clusters created n () and (3) and the ARI between those created n (1) and (3). Three parameters, namely number of obects (n), number of clusters (K), and the maxmum dstance an obect can move (d), were vared durng the experment. Table 1 shows the dfferent experment results by varyng d whle keepng n = 1000 and K = 0. Under each set of dfferent parameter settngs, 500 rounds were run. In each round, the sets of recorded, uncertanty, and actual were frst generated and the same set of data was used for the three clusterng processes. The same set of ntal centrods were also used n each of the three processes n order to avod any bas resultng from the ntal condtons of the K-means and UK-means algorthms. In each round, both K-means (n (1) and

7 (3)) and UK-means (n ()) were allowed to run untl there was no change n cluster membershp for all obects n two consecutve teratons, or when the number of teratons reached The ARI values and tme elapsed were averaged across the 500 runs for UK-means and K-means, respectvely. As can be seen from Table 1, the UK-means algorthm consstently showed a hgher ARI than the tradtonal K-means algorthm appled on the recorded data. Parwse t-tests were conducted and the results showed that the dfference n the ARI values of the two methods was sgnfcant for all settng (p < for every case). The results demonstrated that the clusters produced by the UK-means algorthm were more smlar to those clusters obtaned from the real-world data. In other words, the UK-means algorthm can gve a set of clusters that could be a better predcton of the clusters that would be produced f the real-world data were avalable. Table 1. Experment results D ARI (UK-means) ARI (K-means) Improvement % of mprovement % 10.03% 13.84% 0.8% 44.34% % In terms of effcency, we found that the UK-means algorthm requres more computatonal tme than K-means, but t often only requred a reasonable amount of extra tme, whch s well ustfed snce the clusterng qualty s hgher when uncertanty s consdered. We further conducted experments by varyng n, K, and d for dfferent values, whle keepng the other varables constant. In all cases, we found that the UK-means algorthm showed mprovement over the tradtonal K-means algorthm, and the dfferences were all statstcally sgnfcant (as shown by the t- test result n each case). Our prelmnary results showed that the mprovement of the UK-means algorthm s hgher when the degree of uncertanty (as represented by d) ncreases. On the other hand, the number of obects and number of clusters do not have a large effect on the performance of the UK-means algorthm, except when the number of clusters s very small. 6. Concluson and Future Work Tradtonal data mnng algorthms do not consder uncertanty nherent n a data tem and can produce ncorrect mnng results that do not correspond to the real-world data. In ths paper we propose a taxonomy of research n the area of uncertan data mnng. We also present the UK-means algorthm as a case study and llustrate how the proposed algorthm can be appled. Wth the ncreasng complexty of real-world data brought by advanced sensor devces, we beleve that uncertan data mnng s an mportant and sgnfcant research area. Acknowledgement We would lke to thank Jackey Ng (Unversty of Hong Kong), Davd Cheung (Unversty of Hong Kong), Edward Hung (Hong Kong Polytechnc Unversty), and Kevn Yp (Yale Unversty) for ther valuable suggestons.

8 References 1. Barbara, D., Garca-Molna, H. and Porter, D. The Management of Probablstc Data, IEEE Transactons on Knowledge and Data Engneerng, 4(5), Bezdek, J. C. Pattern Recognton wth Fuzzy Obectve Functon Algorthms. Plenum Press, New York (1981). 3. Cheng, R., Kalashnkov, D., and Prabhakar, S. Evaluatng Probablstc Queres over Imprecse Data, Proceedngs of the ACM SIGMOD Internatonal Conference on Management of Data, June Cheng, R., Kalashnkov, D., and Prabhakar, S. Queryng Imprecse Data n Movng Obect Envronments, IEEE Transactons on Knowledge and Data Engneerng, 16(9) (004) Cheng, R., Xa, X., Prabhakar, S., Shah, R. and Vtter, J. Effcent Indexng Methods for Probablstc Threshold Queres over Uncertan Data, Proceedngs of VLDB, de Souza, R. M. C. R. and de Carvalho, F. de A. T. Clusterng of Interval Data Based on Cty Block Dstances, Pattern Recognton Letters, 5 (004) Dunn, J. C. A Fuzzy Relatve of the ISODATA Process and Its Use n Detectng Compact Well-Separated Clusters, Journal of Cybernetcs, 3 (1973) Hamdan, H. and Govaert, G. Mxture Model Clusterng of Uncertan Data, IEEE Internatonal Conference on Fuzzy Systems (005) Ichno, M., Yaguch, H. Generalzed Mnkowsk Metrcs for Mxed Feature Type Data Analyss, IEEE Transactons on Systems, Man and Cybernetcs, 4(4) (1994) Jan, A. and Dubes, R. Algorthms for Clusterng Data. Prentce Hall, New Jersey (1988). 11. Nlesh N. D. and Sucu, D. Effcent Query Evaluaton on Probablstc Databases, VLDB (004) Pfoser D. and Jensen, C. Capturng the Uncertanty of Movng-obects Representatons, Proceedngs of the SSDBM Conference, 13 13, Ruspn, E. H. A New Approach to Clusterng, Informaton Control, 15(1) (1969) Sato, M., Sato, Y., and Jan, L. Fuzzy Clusterng Models and Applcatons. Physca-Verlag, Hedelberg (1997). 15. Wolfson, O., Sstla, P., Chamberlan, S. and Yesha, Y. Updatng and Queryng Databases that Track Moble Unts, Dstrbuted and Parallel Databases, 7(3), Yeung, K. and Ruzzo, W. An Emprcal Study on Prncpal Component Analyss for Clusterng Gene Expresson Data, Bonformatcs, 17(9) (001)