Subspace clustering with gravitation

Transcription

1 Subsace clusering wih graviaion Jiwu Zhao Heinrich-Heine Universiy Insiue of Comuer Science Daabases and Informaion Sysems Universiaessr. D-4225 Duesseldorf, Germany ABSTRACT Daa mining is a rocess of discovering and exloiing hidden aerns from daa. Clusering as an imoran ask of daa mining divides he observaions ino grous (clusers), which is according o he rincile ha he observaions in he same cluser are similar, and he ones from differen clusers are dissimilar o each oher. Subsace clusering enables clusering in subsaces wihin a daa se, which means he clusers could be found no only in he whole sace bu also in subsaces. The well-known subsace clusering mehods have a common roblem, he arameers are hard o be decided. To face his issue, a new subsace clusering mehod based on Boom-U mehod is inroduced in his aricle. I akes a graviaion funcion o selec daa and dimensions by using self-comarison echnique. The arameer decision is easy, and does no deend on amoun of he daa, which makes he subsace clusering more racical. Keywords daa mining, subsace clusering, graviaion, high dimensional daa. INTRODUCTION Because of he modern echniques, daa collecion is nowadays efficien and cos-effecive. The daa s amoun is huge and mos daa is sored in a raw form, which is no analyzed ye. Usually we need o find ou unknown or hidden informaion from raw daa. Daa mining is such a rocess of discovering and exloiing hidden aerns from daa. I involves clusering, classificaion, regression, associaion, ec. Clusering divides he observaions ino grous (clusers), so ha he observaions in he same cluser are similar, meanwhile, he ones from differen clusers are dissimilar. Clusering is imoran for daa analysis, such as marke baske Coyrigh is held by he auhor/owner(s). GvD Worksho, , Bad Helmsed, Germany. analysis, bio science, fraud deecion and so on. Subsace clusering enables clusering in subsaces wihin a daa se, which means ha he clusers could be found in subsaces raher han only in he whole sace.. Relaed works In a review of subsace clusering [2], he subsace clusering algorihms are caegorized ino wo grous: o-down search and boom-u search mehods. To-down mehods like PROCLUS[], ORCLUS[2], FINDIT[5], σ-clusers[6], COSA[5] use mulile ieraions for imroving he clusering resuls. By conras, boom-u mehods find firsly clusers in lower subsaces, and hen exand he searching by adding more dimensions. Some examles are CLIQUE [3], ENCLUS [4], MAFIA [6], CBF [], CLTree [], DOC [3]. No maer in which grou, almos all subsace clusering algorihms have a common roblem wih finding aroriae values for heir arameers. For insance, mos of o-down mehods have o esimae arameers like number of clusers and subsaces, he clusering resuls are imroved by he ieraions ha are based on hese arameers, which are absoluely no easy o esimae. In boom-u mehods, he key arameers such as densiy, grid inerval, size of clusers, ec. are also hard o be deermined. I is necessary o find a mehod ha deermines arameers easily, in order o make he clusering job more racical. DENCLUE [9] is a densiy-based clusering algorihm by using Gaussian kernel funcion as is absrac densiy funcion and hill climbing mehod o find cluser ceners. DENCLUE 2. [8] is an imrovemen on DENCLUE. The algorihms differ from oher densiy-based aroaches in ha hey calculae densiy o each daa oin insead of an area in he aribue sace. DENCLUE has no o esimae he number or he osiion of clusers, because clusering is based on he densiy informaion of each oin. However i is sill necessary o esimae he arameers in hese wo algorihms, such as σ, ξ in DENCLUE and ɛ, in DENCLUE 2.. Besides, hey are no designed for subsace clusering. Alying he Newon s universal law of graviaion in clusering is no a novel idea. A graviaional clusering algorihm [7] simulaes he movemen of objecs by alying he graviaional force, and deecs clusers from merged objecs. A shrinking-based aroach [4] insired by gravi-

2 aion is a grid-based clusering mehod, which shrinks he objecs in a grid cell owards he daa cenroid and finds he clusers. However, for each algorihm we have o find aroriae values for is arameers..2 Conribuions of he aer In his aer, we inroduce a new densiy-based boom-u subsace clusering mehod called SUGRA (SUbsace clusering mehod by using GRAviaion s funcion). The basic idea is similar o DENCLUE, insead of using he Gaussian kernel funcion, we use graviaion s funcion wih scaled disance o reresen he densiy funcion, bu he objecs don no have o move owards he cenroid. From his simle idea we have found an ineresing roery ha in one dimensional subsace almos all cluser objecs and noncluser objecs (noise) are searaed by a consan. Wih his roery, we can deec clusers very disincly, meanwhile, SUGRA realizes he reducion of arameers by subsace clusering. The remainder of his aer is organized as follows. The idea of SUGRA is inroduced in secion 2, where secion 2. and 2.2 are definiions abou cluser and graviaion funcion resecively, secion 2.3 resens he algorihm of SUGRA. The las secion conains conclusions and areas of fuure work. 2. SUBSPACE CLUSTERING WITH GRAV- ITATION 2. Definiion of daa se and subsace cluser A daa se consiss of objecs and heir aribues. Usually, all objecs have common aribues in a daa se, such as color, rice, lengh ec., and every objec has a value for an aribue. The values ha are relaed o an aribue are in he same domain and conform o he same consrains. A daa se could be described as a able, where he objecs are jus rows, meanwhile he aribues are columns. The aribues could also be considered as dimensions, so ha each aribue reresens one dimension, and hen he objecs are oins in hese dimensions. In order o describe he aribues and objecs clearly, hey are defined as follows: Definiion. (Daa se) Generally, a daa se D could be considered as a air, which is he combinaion of A and O: D (A, O) () where A is a se of all aribues (dimensions), and O is a se of all objecs: A {a, a 2,, a i, }, O {o, o 2,, o, } (2) where o is an objec wih values on A: o {o a, o a 2,, o a i, } (3) We denoe he values of all objecs on aribue a i wih: o a i {o a i, oa i 2,, oa i, } (4) Definiion 2. (Subsace cluser) A subsace cluser S is also a daa se and defined as follows: S e D ( e A, e O) (5) where e A A and e O O, and S mus saisfy a aricular condiion C, which is defined differenly in every subsace clusering algorihm. A subsace cluser S could hen be wrien like his: S ( e A, e O) ({a, a 2, a 6, }, {o, o 5, o 3, }) The cardinaliy regarding he objecs and he dimensions in S are defined resecively: S O e O, S A e A (6) Remark. Suose S, S 2 are wo subsace clusers, where S (A, O ) and S 2 (A 2, O 2), hen If A A 2 O O 2 S S 2, he subsace clusers wih differen dimensions or objecs are considered as differen ones. A A, S (A, O ) is also a subsace cluser. If A A 2 O O 2 or A A 2 O O 2 S < S 2. Only he larges subsace cluser will be aken in he clusering resul. Definiion 3. The inersecion of wo subsace clusers is defined as follows: S S 2 (A A 2, O O 2) (7) Definiion 4. S a i is he se of all subsace clusers found in dimension a i, S D is he se of all subsace clusers found in D, finding S D is he ask of subsace clusering. 2.2 Graviaion Graviaion is a naural henomenon, which describes he force of aracion beween objecs wih mass. The graviaion is imoran, because i influences our normal lives. The Newon s law of universal graviaion is defined as follows: G G mm2 r 2 (8) where G is he graviy beween wo oin masses, G is he graviaional consan, m and m 2 are he masses of wo oins resecively, r is he disance beween he wo oin masses. SUGRA ries o use he graviaion s funcion for he measuremen beween he objecs. In order o make he calculaion easier, a simle graviy funcion is used here: Definiion 5. (Simle graviy funcion) q mmq r 2 q Suose ha a single objec o has a mass m, r q is he l q disance beween o and o q is defined as r q L/(N ), (9)

3 3.5 3 (a) Graviaion Aver. Grav. Objecs 7 6 (b) Graviaion Aver. Grav. Objecs Figure : Proeries of graviaion in one dimension where L max(o a i ) min(o a i ) is he lengh of his dimension and N O is he number of he objecs. r q indicaes a roorion of he real lengh l q o he average inerval L/(N ), so ha q l q L/(N ) 2 l 2 q(n ) 2 () Remark 2. If r q, o and o q sand a a same lace in a i. In order o le q calculable and o ge a logical resul, we should se r q greaer han bu smaller han any oher disances. An idea is seing r q o a half of he minimum disance in a i. For examle, o m, o n has he minimum disance r mn > in a i, hen seing r q r mn/2 make sure ha q > mn, which is execed. Remark 3. The r q is such defined as a roorion disance bu no a real disance because ha i enables he daa wih differen ranges of values o be calculaed ino a same range. For examle, he aribues age and salary are obviously in wo ranges, bu by using such a roorion disance he wo aribues could be calculaed and comared ogeher. An objec ha lies near o ohers (cluser objecs) has a greaer graviaion han ha of objecs far from ohers (non-cluser objecs) (see Figure (b)). Definiion 7. (Average graviaion) The average graviaion of dimension ai is he graviaion of he middle objec of uniformly disribued objecs in a i. is resened wih a doed line in Figure. Suose o m is he objec in he middle. could be calculaed as follows: X lm(n 2 ) 2 (N X 2 A, m (N ) B B X n N 2 X n N 2 C n 2 C A 2 N n)2 ( L A N 2 π2 6 l 2, m m π There are furher definiions, which are imoran for he SUGRA algorihm. Definiion 6. (Graviaion of an objec) The graviaion of an objec o in dimension a i is defined as he sum of graviaion from o wih oher objecs. X q, q q () 3.29 Remark 4. The graviaion of an objec defined in () has following roeries in one dimension: 3.29 An objec in he middle has a greaer value of graviaion han one a he edge, which could be clearly seen, if he objecs are disribued uniformly (see Figure (a)). Figure 2: SUGRA on wo dimensional daa Remark 5. The graviaion values of cluser objecs and

4 non-cluser objecs have grea differences. The non-cluser objecs have usually very small values of graviaion, meanwhile he cluser objecs have larger values. This roery is very imoran for he clusering rocess. If a daa se has many objecs, which means N is a big number, hen 3.29, from exerimens we found ou ha using he average graviaion as a hreshold o searae cluser and non-cluser objecs reurns good resuls. This is no a silver bulle, bu can be hough as a saring oin, he hreshold could be regulaed near his value. Figure 2 reresens an examle of SUGRA on wo dimensional daa. The value 3.29 have searaed he graviaion on he wo dimensional sace resecively, where he red oins illusraes he graviaion of rand oins. 2.3 Algorihm of SUGRA This algorihm consiss of wo ses:. Daa selecion (Clusering in one dimensional saces) 2. Dimension selecion (Clusering in high dimensional saces) As a Boom-U algorihm, SUGRA handles daa firsly in one dimensional sace, because one dimensional daa can be deal wih easily. Finding clusers in high dimensional sace is based on he clusers found in one dimension Daa selecion Algorihm : Daa selecion Inu: D (A, O) Ouu: S a i foreach a i A do Sor o a i 2 3 iniialize foreach o a i 4 o a i do 5 if > hen 6 if o a i Sa i add o a i 7 8 else 9 ++ add o a i end 2 end 3 end 4 end and o a i o S a i o S a i o a i < L hen As discussed above, he clusers are firsly seleced on each dimension hrough he graviaion. Firs of all, o a i are sored in ascending order. For examle, if has a greaer value han he hreshold, o a i is hen chosen as a clusercandidae. If is neighbor o a i is also a cluser-candidae and heir disance is smaller han he average disance, hey will be considered in one cluser, oherwise o a i is se ino a new cluser. The rocess will so when here is no more new cluser found in a i. The rocesses are he same for oher dimensions. Algorihm shows more deails abou he daa selecion. Afer he daa selecion rocess, we ge subsace clusering resuls in every one dimensional sace S a i, a subsace cluser S S a i could have he form like Dimension selecion S ({a i}, {o, o 5, o 9, }) (2) Algorihm 2: Dimension selecion Inu: S a i Ouu: S D add all S S a i o S D 2 foreach S S D do 3 while find S S D and S S do 4 if S S O 2 hen 5 add S S o S D 6 end 7 end 8 end 9 reurn In daa-selecion, he one dimensional clusers were found wih he forms like (2), he finding of subsace cluser in high dimension is jus based on he inersecion defined in (7). For subsace clusers S and S 2, if S S 2 O 2, hen S S 2 is a new subsace cluser. Every combinaion of clusers should be checked hrough he inersecion, his rocess will so when no more new cluser is found. The final resul will kee only he larges subsace clusers. The deailed algorihm is shown in Algorihm Furher discussions The choosing of arameers is usually difficul for a subsace clusering algorihm, a lile deviaion may cause a differen resul. The boundaries beween cluser objecs and noncluser objecs are esecially indisinc and hey could be recognized hardly. SUGRA uses he graviaion funcion ha marks he cluser and non-cluser objecs wih grea differences, which makes he arameer decision easily. Daa selecion. The exerimenal exerience shows ha could searae he cluser objecs and non-cluser objecs very well by daa selecion, as defined in Definiion 7, 3.29 does no deend on O and could be used as hreshold for almos all siuaions. If he resuls are no saisfying, he hreshold could be se a lile smaller or greaer. Dimension selecion. The condiion S S 2 O 2 is used in dimension selecion o decide, wheher S S 2 is a subsace cluser. The condiion indicaes ha an objec-grou wih more han wo objecs will be aken as a new cluser. This seing has a high recision, because no only big clusers bu also small clusers could be found, bu naurally i akes much ime. In conras, choosing a greaer number may gain ime bu lose some ineresing small clusers.

5 2.4. Run ime The run ime of he daa selecion in a dimension a i is O 2, and for D (A, O) is A O 2. In dimension selecion, every ossible combinaion of subsace clusers could be examined, so he maximum run ime of dimension selecion is 2 m, where m is he number of one dimensional subsace clusers found in daa selecion. 3. CONCLUSIONS Subsace clusering is able o discover clusers and exrac heir feaures from he subsace of high dimensional daa, which is commonly gahered in many fields. Mos familiar subsace clusering aroaches have he roblems wih deermining he arameers. We aem o aly he graviaion funcion in subsace clusering in order o find ou a new mehod make he deerminaion of he arameers easier. The mehod is named SUGRA, which belongs o Boom-U algorihms. Firsly, i finds ou clusers by using graviaion funcion in one dimensional sace, hen i combines he clusers in higher dimensions for searching high dimensional subsace clusers. In SUGRA, he non-cluser objecs have always low graviaion values (<3.29), meanwhile he cluser objecs have very large values, which deend on he clusers densiy and number of objecs. The value 3.29 does no deend on he number of objecs, so i enables searaing he non-cluser objecs in order o ge cluser objecs. We don have o choose arameers like oher algorihms, SUGRA can ge almos a saisfying resul for a sar by using his hreshold. The fuure research will be focused on oimizing he graviaion funcion and he algorihm in order o imrove he subsace clusering resuls. The graviaion echnique should be used no only in one dimensional daa bu also direcly in mulile dimensions. Anoher work is o le SUGRA ada various daa yes, such as caegorical daa. 4. REFERENCES [] C. C. Aggarwal, J. L. Wolf, P. S. Yu, C. Procoiuc, and J. S. Park. Fas algorihms for rojeced clusering. In Proceedings of he 999 ACM SIGMOD inernaional conference on Managemen of daa, ages 6 72, Philadelhia, Pennsylvania, Unied Saes, May 3-June [2] C. C. Aggarwal and P. S. Yu. Finding generalized rojeced clusers in high dimensional saces. In Proceedings of he 2 ACM SIGMOD inernaional conference on Managemen of daa, ages 7 8, Dallas, Texas, Unied Saes, May [3] R. Agrawal, J. Gehrke, D. Gunoulos, and P. Raghavan. Auomaic subsace clusering of high dimensional daa for daa mining alicaions. In Proceedings of he 998 ACM SIGMOD inernaional conference on Managemen of daa, ages 94 5, Seale, Washingon, Unied Saes, June [4] C.-H. Cheng, A. W. Fu, and Y. Zhang. Enroy-based subsace clusering for mining numerical daa. In Proceedings of he fifh ACM SIGKDD inernaional conference on Knowledge discovery and daa mining, ages 84 93, San Diego, California, Unied Saes, Augus [5] J. H. Friedman and J. J. Meulman. Clusering objecs on subses of aribues. Journal of he Royal Saisical Sociey: Series B (Saisical Mehodology), 22. [6] S. Goil, H. Nagesh, and A. Choudhary. Mafia: Efficien and scalable subsace clusering for very large daa ses. Technical Reor CPDC-TR-996-, Norhwesern Universiy, June 999. [7] J. Gomez, D. Dasgua, and O. Nasraoui. A new graviaional clusering algorihm. In In Proc. of he SIAM In. Conf. on Daa Mining (SDM, 23. [8] A. Hinneburg and H.-H. Gabriel. Denclue 2.: Fas clusering based on kernel densiy esimaion. In Proc. of Inernaional Symosium on Inelligen Daa Analysis 27 (IDA 7), Ljubljana, Slowenien, 27. LNAI Sringer. [9] A. Hinneburg and D. A. Keim. An efficien aroach o clusering in mulimedia daabases wih noise. In Proc. 4rd In. Conf. on Knowledge Discovery and Daa Mining, New York, 998. AAAI Press. [] D.-S. J. Jae-Woo Chang. A new cell-based clusering mehod for large, high-dimensional daa in daa mining alicaions. In Proceedings of he 22 ACM symosium on Alied comuing, ages 4, Madrid, Sain, March 22. [] B. Liu, Y. Xia, and P. S. Yu. Clusering hrough decision ree consrucion. In Proceedings of he ninh inernaional conference on Informaion and knowledge managemen, ages 2 29, McLean, Virginia, Unied Saes, November 6-2. [2] L. Parsons, E. Haque, and H. Liu. Subsace clusering for high dimensional daa: A review. Sigkdd Exloraions, 6:9 5, June 24. [3] C. M. Procoiuc, M. Jones, P. K. Agarwal, and T. M. Murali. A mone carlo algorihm for fas rojecive clusering. In Proceedings of he 22 ACM SIGMOD inernaional conference on Managemen of daa, Madison, Wisconsin, June [4] Y. Shi, Y. Song, and A. Zhang. A shrinking-based aroach for muli-dimensional daa analysis. In VLDB 23: Proceedings of he 29h inernaional conference on Very large daa bases, ages VLDB Endowmen, 23. [5] K.-G. Woo and J.-H. Lee. FINDIT: a Fas and Inelligen Subsace Clusering Algorihm using Dimension Voing. PhD hesis, Korea Advanced Insiue of Science and Technology, Taejon, Korea, 22. [6] J. Yang, W. Wang, H. Wang, and P. Yu. δ-clusers: Cauring subsace correlaion in a large daa se. In Proceedings of he 8h Inernaional Conference on Daa Engineering, age 57, February 26-March 22. [7] J. Zhao. Auomaische Parameerbesimmung durch Graviaion in Subsace Clusering. 2. Worksho Grundlagen von Daenbanken, Rosock, 29.