Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery

Transcription

1 Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery Nicolas Durand (1), Bruno Crémilleux (1) and Einoshin Suzuki (2) (1) GREYC, University of Caen, France {nicolas.durand, (2) Department of Informatics, Kyushu University, Japan Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.1/27

2 Introduction Data visualization: great importance in data mining To understand the data, To discover knowledge, Tools (information retrieval),... 80% of information: obtained from eyesight [Card et al. 1999][Fayyad et al. 2002] Visualization based on clustering SOM [Kohonen 1982][Vesanto 1999], CLUSION [Strehl et al. 2003], CDCS [Chang et al. 2005], gcluto [Rasmussen et al. 2004],... Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.2/27

3 Motivations Clustering = to form groups of similar objects Hard-clustering (distinct groups) Soft-clustering (overlapping) Problems: a lot of algorithms, goodness of a clustering (subjective) 1 clustering single view of the data Our idea: use of multiple clusterings/views of the data on the same display result Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.3/27

4 Contributions VISUMCLUST (Visualization of Multiple Clusterings): New approach of transactional data visualization How to represent multiple clustering results? clustering 1 data clustering 2 linking clusterings using colors image generation clustering n VisuMClust Algorithm of color attribution to clusters, based on hypergraph minimal transversals Help for the user to discover knowledge Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.4/27

5 Transactional data set Id Items t 1 A J 9 transactions t 2 A B C (consumers, patients, users,... ) t 3 A B C t 4 A B C 10 items t 5 D E (products, medical examinations, t 6 D E H web pages,... ) t 7 A D E F G H t 8 A F G I J t 9 H I Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.5/27

6 Clustering results Clustering algorithm: ECCLAT (Extraction of Clusters from Concept LATtice) [Durand et al. 2002] Soft-clustering (overlapping groups) Categorical data Based on frequent closed itemsets 3 clusterings obtained using different parameters: Id Clusters C 1 (A; t 1, t 2, t 3, t 4, t 7, t 8 ) (DE; t 5, t 6, t 7 ) (I; t 8, t 9 ) C 2 (AF G; t 7, t 8 ) (ABC; t 2, t 3, t 4 ) (DEH; t 6, t 7 ) (I; t 8, t 9 ) (J; t 1, t 8 ) C 3 (ABC; t 2, t 3, t 4 ) (DE; t 5, t 6, t 7 ) (I; t 8, t 9 ) (J; t 1, t 8 ) 1 cluster: (DE; t 5, t 6, t 7 ) Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.6/27

7 VISUMCLUST result Display result on the running example: C 1 line = clustering Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.7/27

8 VISUMCLUST result Display result on the running example: line = clustering column = transaction t 5 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.7/27

9 VISUMCLUST result Display result on the running example: Presence line = clustering column = transaction Absence rod = presence of transactions in clusterings t 5 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.7/27

10 VISUMCLUST result Display result on the running example: line = clustering column = transaction rod = presence of transactions in clusterings color = membership of transactions to clusters t 5 t 8 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.7/27

11 Allocated colors Allocated colors in the example: 1 (blue) 2 (green) 3 (yellow) 4 (orange) 5 (red) Clusters (I; t 8, t 9 ) (DE; t 5, t 6, t 7 ) (A; t 2, t 3, t 4, t 7, t 8 ) from C 1 Clusters (I; t 8, t 9 ) (DEH; t 6, t 7 ) (ABC; t 2, t 3, t 4 ) (J; t 1, t 8 ) (AF G; t 7, t 8 ) from C 2 Clusters (I; t 8, t 9 ) (DE; t 5, t 6, t 7 ) (ABC; t 2, t 3, t 4 ) (J; t 1, t 8 ) from C 3 1 color = 1 cluster of each clustering The clusters (same color) have to be the more similar Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.8/27

12 Color attribution algorithm: idea Related to clustering of clusters, but... Enumerate all possibilities: not feasible (k n where n=number of clusterings results, k=maximal number of cluster in a clustering result)... 1 cluster of each clustering... related to hypergraph transversals candidate groups, evaluated and selected according to a similarity measure Modelling of the clustering results with a hypergraph Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.9/27

13 Hypergraph C 1 A ; 1,2,3,4,7,8 DE ; 5,6,7 C 3 ABC ; 2,3,4 I ; 8,9 J ; 1,8 AFG ; 7,8 DEH ; 6,7 C 2 Vertices = clusters Hyperedges = clusterings Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.10/27

14 Hypergraph minimal transversals C 1 A ; 1,2,3,4,7,8 DE ; 5,6,7 ABC ; 2,3,4 I ; 8,9 J ; 1,8 AFG ; 7,8 DEH ; 6,7 C 3 Transversal = set of vertices meeting all hyperedges ({A,AFG,ABC}) (The transversals are also called hitting sets) Minimal transversal = tranversal with a minimal number of vertices ({A,ABC}) (T is a minimal transversal if T T, T is not a transversal) C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.11/27

15 Minimal transversals as candidate groups Set of all minimal transversals = set of combinations of clusters through the clustering results gathering the identical clusters Minimal transversals good candidate groups to form colors In the example: 7 minimal transversals (whereas 60 candidate groups in all) CMT prototype [Hébert 2005]: to compute the minimal transversals Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.12/27

16 Minimal transversals evaluation Similarity between two clusters = Jaccard coefficient (on the itemset) Jaccard(I 1, I 2 ) = I 1 I 2 I 1 I 2 Jaccard(ABC, AF G) = 0.2 Similarity of a group (set of clusters): intra-color similarity (noted SimC) SimC = average of the similarity values of all pairs of clusters of the group SimC({A, AF G, ABC}) = 0.29 SimC({A, ABC}) = 0.55 (ABC present both for C 2 and C 3 ) Goodness measure S D of the allocation of colors = average of the intra-color similarity values of all the selected groups Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.13/27

17 Color attribution algorithm Input: C 1,..., C n where C i = {c i,1,..., c i,m }. Output: D = (color 1,..., color p ) where color i = {c 1,j,..., c n,j }. Start 0. Color = 0 1. Transform C i into a hypergraph H Repeat step 2-6 until there are no clusters (c i,j ) without color: 2. Color += 1 3. Compute the minimal transversals of H 4. Select the best candidate Cand (the higher intra-color similarity) 5. Attribute the current Color to each c i,j Cand 6. Add Cand to D and remove the colored clusters (c i,j ) from H End Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.14/27

18 Example Color 1 (blue) minimal transversal: {I} <I, I, I> SimC = 1.00 C 1 A ; 1,2,3,4,7,8 DE ; 5,6,7 C 3 ABC ; 2,3,4 I ; 8,9 J ; 1,8 AFG ; 7,8 DEH ; 6,7 C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.15/27

19 Example Color 2 (green) minimal transversal: {DE, DEH} <DE, DEH, DE> SimC = 0.77 C 1 A ; 1,2,3,4,7,8 ABC ; 2,3,4 AFG ; 7,8 DE ; 5,6,7 DEH ; 6,7 J ; 1,8 C 3 C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.16/27

20 Example Color 3 (yellow) minimal transversal: {A, ABC} <A, ABC, ABC> SimC = 0.55 C1 ABC ; 2,3,4 A ; 1,2,3,4,7,8 AFG ; 7,8 J ; 1,8 C 3 C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.17/27

21 Example Color 4 (orange) minimal transversal: {J} <, J, J> SimC = 0.33 AFG ; 7,8 J ; 1,8 C 3 C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.18/27

22 Example Color 5 (red) The remaining cluster <,AF G, > AFG ; 7,8 C 2 Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.19/27

23 Experiments Data sets: Benchmarks: Mushroom, Votes, Hepatitis, Ionosphere, Titanic Geographical real-world database: Cross channel atlas (called Geo ) For each data set: ECCLAT to obtain several clusterings VISUMCLUST Evaluation of the color attribution using the goodness measure S D Comparison with a baseline method of color attribution (greedy strategy + intra-color similarity) Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.20/27

24 Results Mushroom Votes Hepatitis Ionosphere Titanic Geo Baseline method VISUMCLUST Gain +18.7% +35.9% +19.5% +13.9% +30.1% +44.9% Average gain = +27.1% Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.21/27

25 Cross channel atlas (Geo) 41 English counties (south) and 28 French regions (north, west) Described by 99 items (demographical and economic indicators) Aim: detect links between the geographical units Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.22/27

26 Result of VISUMCLUST on Geo 1 Finistère ; 2 Côtes d'armor ; 3 Ille et Vilaine ; 4 Morbihan ; 5 Eure ; 6 Seine-Maritime ; 7 Loire-Atlantique ; 8 Maine-et-Loire ; 9 Mayenne ; 10 Sarthe ; 11 Vendée ; 12 Calvados ; 13 Manche ; 14 Orne ; 15 Eure-et-Loir ; 16 Aisne ; 17 Oise ; 18 Somme ; 19 Nord ; 20 Pas-de-Calais ; 21 Essonne ; 22 Hauts-de-Seine ; 23 Ville de Paris ; 24 Seine Saint-Denis ; 25 Seine-et-Marne ; 26 Val d'oise ; 27 Val-de-Marne ; 28 Yvelines ; 29 Swindon ; 30 London ; 31 Isle of Wight ; 32 Poole ; 33 Bournemouth ; 34 Torbay ; 35 Plymouth ; 36 South Gloucestershire ; 37 North Somerset ; 38 City of Bristol ; 39 Bath and Noth Somerset ; 40 Kent ; 41 East Sussex ; 42 West Sussex ; 43 Buckinghamshire ; 44 Surrey ; 45 Windsor ; 46 Reading ; 47 Hampshire ; 48 West ; 49 Oxfordshire ; 50 Medway ; 51 Southampton ; 52 Milton Keynes ; 53 Bracknell Forest ; 54 Portsmouth ; 55 Brighton and Hove ; 56 Wockingham ; 57 Slough ; 58 Wiltshire ; 59 Gloucestershire ; 60 Dorset ; 61 Devon ; 62 Somerset ; 63 Cornwall and Isle of Scilly ; 64 Hertfordshire ; 65 Thurrock ; 66 Southend-on-Sea ; 67 Bedfordshire ; 68 Essex ; 69 Luton ; Display result and description of each cluster conclusions about links between the units Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.23/27

27 Result of VISUMCLUST on Geo 1 Finistère ; 2 Côtes d'armor ; 3 Ille et Vilaine ; 4 Morbihan ; 5 Eure ; 6 Seine-Maritime ; 7 Loire-Atlantique ; 8 Maine-et-Loire ; 9 Mayenne ; 10 Sarthe ; 11 Vendée ; 12 Calvados ; 13 Manche ; 14 Orne ; 15 Eure-et-Loir ; 16 Aisne ; 17 Oise ; 18 Somme ; 19 Nord ; 20 Pas-de-Calais ; 21 Essonne ; 22 Hauts-de-Seine ; 23 Ville de Paris ; 24 Seine Saint-Denis ; 25 Seine-et-Marne ; 26 Val d'oise ; 27 Val-de-Marne ; 28 Yvelines ; 29 Swindon ; 30 London ; 31 Isle of Wight ; 32 Poole ; 33 Bournemouth ; 34 Torbay ; 35 Plymouth ; 36 South Gloucestershire ; 37 North Somerset ; 38 City of Bristol ; 39 Bath and Noth Somerset ; 40 Kent ; 41 East Sussex ; 42 West Sussex ; 43 Buckinghamshire ; 44 Surrey ; 45 Windsor ; 46 Reading ; 47 Hampshire ; 48 West ; 49 Oxfordshire ; 50 Medway ; 51 Southampton ; 52 Milton Keynes ; 53 Bracknell Forest ; 54 Portsmouth ; 55 Brighton and Hove ; 56 Wockingham ; 57 Slough ; 58 Wiltshire ; 59 Gloucestershire ; 60 Dorset ; 61 Devon ; 62 Somerset ; 63 Cornwall and Isle of Scilly ; 64 Hertfordshire ; 65 Thurrock ; 66 Southend-on-Sea ; 67 Bedfordshire ; 68 Essex ; 69 Luton ; 1) Blue and green = English and French units far from London and Paris (respectively) population getting aged,... Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.24/27

28 Result of VISUMCLUST on Geo 1 Finistère ; 2 Côtes d'armor ; 3 Ille et Vilaine ; 4 Morbihan ; 5 Eure ; 6 Seine-Maritime ; 7 Loire-Atlantique ; 8 Maine-et-Loire ; 9 Mayenne ; 10 Sarthe ; 11 Vendée ; 12 Calvados ; 13 Manche ; 14 Orne ; 15 Eure-et-Loir ; 16 Aisne ; 17 Oise ; 18 Somme ; 19 Nord ; 20 Pas-de-Calais ; 21 Essonne ; 22 Hauts-de-Seine ; 23 Ville de Paris ; 24 Seine Saint-Denis ; 25 Seine-et-Marne ; 26 Val d'oise ; 27 Val-de-Marne ; 28 Yvelines ; 29 Swindon ; 30 London ; 31 Isle of Wight ; 32 Poole ; 33 Bournemouth ; 34 Torbay ; 35 Plymouth ; 36 South Gloucestershire ; 37 North Somerset ; 38 City of Bristol ; 39 Bath and Noth Somerset ; 40 Kent ; 41 East Sussex ; 42 West Sussex ; 43 Buckinghamshire ; 44 Surrey ; 45 Windsor ; 46 Reading ; 47 Hampshire ; 48 West ; 49 Oxfordshire ; 50 Medway ; 51 Southampton ; 52 Milton Keynes ; 53 Bracknell Forest ; 54 Portsmouth ; 55 Brighton and Hove ; 56 Wockingham ; 57 Slough ; 58 Wiltshire ; 59 Gloucestershire ; 60 Dorset ; 61 Devon ; 62 Somerset ; 63 Cornwall and Isle of Scilly ; 64 Hertfordshire ; 65 Thurrock ; 66 Southend-on-Sea ; 67 Bedfordshire ; 68 Essex ; 69 Luton ; 2) Yellow = capitals and their surrounding dynamic units, large agglomerations with good demographic indicators (birth rate,... ) 3)... Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.24/27

29 Result of VISUMCLUST on Geo 1 Finistère ; 2 Côtes d'armor ; 3 Ille et Vilaine ; 4 Morbihan ; 5 Eure ; 6 Seine-Maritime ; 7 Loire-Atlantique ; 8 Maine-et-Loire ; 9 Mayenne ; 10 Sarthe ; 11 Vendée ; 12 Calvados ; 13 Manche ; 14 Orne ; 15 Eure-et-Loir ; 16 Aisne ; 17 Oise ; 18 Somme ; 19 Nord ; 20 Pas-de-Calais ; 21 Essonne ; 22 Hauts-de-Seine ; 23 Ville de Paris ; 24 Seine Saint-Denis ; 25 Seine-et-Marne ; 26 Val d'oise ; 27 Val-de-Marne ; 28 Yvelines ; 29 Swindon ; 30 London ; 31 Isle of Wight ; 32 Poole ; 33 Bournemouth ; 34 Torbay ; 35 Plymouth ; 36 South Gloucestershire ; 37 North Somerset ; 38 City of Bristol ; 39 Bath and Noth Somerset ; 40 Kent ; 41 East Sussex ; 42 West Sussex ; 43 Buckinghamshire ; 44 Surrey ; 45 Windsor ; 46 Reading ; 47 Hampshire ; 48 West ; 49 Oxfordshire ; 50 Medway ; 51 Southampton ; 52 Milton Keynes ; 53 Bracknell Forest ; 54 Portsmouth ; 55 Brighton and Hove ; 56 Wockingham ; 57 Slough ; 58 Wiltshire ; 59 Gloucestershire ; 60 Dorset ; 61 Devon ; 62 Somerset ; 63 Cornwall and Isle of Scilly ; 64 Hertfordshire ; 65 Thurrock ; 66 Southend-on-Sea ; 67 Bedfordshire ; 68 Essex ; 69 Luton ; Observations validated by the geographers Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.24/27

30 Conclusion New approach for visualizing transactional data VISUMCLUST Display multiple clustering results at the same time Consistent view obtained with the color allocation algorithm based on the minimal transversals of a hypergraph Importance of multiple clusterings/views of the data Very useful for knowledge discovery Evaluation on benchmarks, experiment on a geographical database Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.25/27

31 Future Work More experiments (clustering algorithms, data sets) Interactive interface (with clickable objects to access to some details/descriptions,... ) Applications: comparison of clusterings results, help to choose a clustering algorithm, the parameters automatic detection of interesting transactions. Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.26/27

32 References N. Durand and B. Crémilleux. ECCLAT: a New Approach of Clusters Discovery in Categorical Data. In ES 2002, Cambridge, UK, December C. Hébert. Enumerating the Minimal Transversals of a Hypergraph Using Galois Connections. Technical report, University of Caen Basse-Normandie, France, mlearn/. UCI Machine Learning Repository. (Mushroom, Votes, Hepatitis, Ionosphere) (Titanic) transmanche.certic.unicaen.f r/index.gb.html. Cross Channel Atlas (Atlas Transmanche). Nicolas Durand - Visualizing Transactional Data with Multiple Clusterings for Knowledge Discovery - ISMIS september 2006 p.27/27