Lluis Belanche + Alfredo Vellido. Intelligent Data Analysis and Data Mining

Transcription

1 Lluis Belanche + Alfredo Vellido Intelligent Data Analysis and Data Mining a.k.a. Data Mining II

2 IDA DM 2012/2013. Alfredo Vellido Visual Data Mining (3)

3 visual DM RECAP: visualization in data exploration Data exploration is one of the basic building blocks, or constituting stages, of most knowledge discovery methodologies, and the task of data visualization is central to data exploration. Artificial pattern recognition (APR): Through the definition of visualization oriented techniques. Natural pattern recognition (NPR): Through the understanding of visualization as the cognitive processing of visual stimuli conducted by the human brain. Out of a deductive model of research (general proposi ons examples) to reap the benefits of a more inductive one (examples general propositions).

4 RECAP: Visual revolution William Playfair These days, visualization typically employs computers to process the information and computer screens. Standardized computer based information visualization has been around for barely a couple of decades.

5 RECAP: PRINCIPLES: the data mining visual cycle, or Visual Exploratory Data Mining Data gathering Data manipulation Hipothesis of reality DATA MODEL Pre-processing & transformation Graphic engine Data Exploration Control & navegation Visual-Spatial model Cognitive-Logic Model Model manipulation

6 Contents (for parts 1 & 2) A brief introduction to info visualization Visualization & history Perception: seeing with the brain Visual exploratory DM The good, the bad & the ugly

7 What type of visualization are we looking for? Descriptive? explicit Exploratory? implicit

8 Type DESCRIPTIVE: remember event logs

9 PRINCIPLES: A good exploratory visualization should......show data and/or results......at different levels of detail, from the overall landscape to the fine detail.... in a coherent manner, even if we are dealing with large collections.... avoiding, as much as possible, distortion in their representation...focus attention in the most relevantes features......minimizing the impact of uninformative and misleading data...integrating statistical results and linguistic descriptions (if possible and relevant: multimodality).

10 DATA EXPLORATION: Some dimensions

11 DATA EXPLORATION: The CURSE of dimensionality Most data available to us are stored in different kinds of databases and in numeric format, mostly organized in table structures (remember survey!) An extension of these are the data cubes generated by OLAP processes. How to display multiple dimensions in a visually intuitive manner? A simplified taxonomy of cases: Low dimensionality (1 3D) Moderate dimensionality (4 10D) High dimensionality (>10D)

12 DATA EXPLORATION: low-moderate dim < 10D Spatial coordinates 3D requires interactivity Further pre cognitive visual elements allow us to add extra dimensions: color, movement, shape, Exotic solutions Glyph*: Chernoff faces, stickfigures, whiskers... * A glyph is a graphical representation of one or more characters, or of part of a character. A character is a textual entity whereas a glypg is a graphical entity. ideogram, pictogram

13 some of those alternatives Chernoff faces Herman Chernoff (1973). "Using faces to represent points in k dimensional space graphically". Journal of the American Statistical Association 68 (342):

14 some of those alternatives

15 DATA EXPLORATION: high dimensionality data How do we visualize data of high (or even very high) dimensionality? Some of the alternatives are rather straightforward some others are not Eliminate dimensions (data variables): those which are redundant and / or uninformative (at least you manage to alleviate part of the problem ) Feature selection Divide & conquer: a classic: create multiple visualizations of low dimensionality. Latent and projection models

16 DATA EXPLORATION: The Grand Tour: multiple visualization of Iris data

17 DATA EXPLORATION: Too Grand a Tour?

18 TECHNIQUES: Latency and projection Projection Dimensionality compression Similitude information coding Grouping / Clustering Finding grouping structure in data Similitude information coding Self Organizing Maps (SOM) & their variants: manifold learning Examples of combined latent representation and clustering

19 TECHNIQUES: projection Representation in <4 D, so that the distance neighborhood relations between multi dimensional points are faithfully preserved It is impossible to preserve information integrally Some scale normalization is required Linear vs. non linear projections

20 TECHNIQUES: projection: methods Methods based oninter point distances, where: dx = distance in the original space dy = distance in the projection space h = neighborhood function E = (dx dy) 2 E = (dx dy) 2 / dx E = (dx dy) 2 e dy E = dx 2 h(dy) MDS, PCA Sammon s projection CCA SOM... and in which we aim to minimize an inherent projection distorsion (E)

21 TECHNIQUES: projection: methods in a nutshell MDS: technique used in data visualization for exploring similarities or dissimilarities in data. An MDS algorithm starts with a matrix of item item similarities, then assigns a location of each item in a low dimensional space, suitable for visualisation. Taxonomy: Metric multidimensional scaling assumes the input matrix is just an item item distance matrix. Analogous to PCA, an eigenvector problem is solved to find the locations that minimize distortions to the distance matrix. Its goal is to find a Euclidean distance approximating a given distance. Generalized multidimensional scaling (GMDS) A superset of metric MDS that allows for the target distances to be non Euclidean. Non metric multidimensional scaling It finds a non parametric monotonic relationship between the dissimilarities in the item item matrix and the Euclidean distance between items, and the location of each item in the low dimensional space Biblio: Abdi, H. (2007). Metric multidimensional scaling. In N.J. Salkind (Ed.): Encyclopedia of Measurement and Statistics. Thousand Oaks (CA): Sage. Kruskal, J. B., and Wish, M. (1978), Multidimensional Scaling, Sage University Paper series on Quantitative Application in the Social Sciences, Beverly Hills and London: Sage Publications.

22 TECHNIQUES: projection: methods in a nutshell PCA: It is a linear transformation that represents the data in a new coordinate system such that the greatest variance explained by the data lies on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on. PCA can be used for dimensionality reduction in a dataset by retaining only those characteristics of the dataset that contribute most to its variance. Taxonomy: Kernel PCA PPCA, CCA (when unfolding a nonlinear structure, Sammon's mapping cannot reproduce all distances. One way to face this problem consists in favouring local topology: CCA tries to reproduce short distances first, while long distances remain secondary. Some source code: Dimensionality Reduction Toolbox: Some papers: PPCA: M.E. Tipping, C.M. Bishop, Probabilistic Principal Component Analysis, 1999, J. of the Royal Statistical Society: Series B, 61(3): KPCA: B. Schölkopf, A. Smola and K.R. Müller, Kernel Principal Component Analysis, 1997, Lecture Notes in Computer Science, Vol. 1327,

23 TECHNIQUES: projection: example PCA Sammon s projection CCA

24 TECHNIQUES: projection: discussion, pros & cons Projection techniques code proximity / similarity information in spacial coordinates (sometimes, with extra precognitive elements such as colour...) They allow But... Finding natural data groupings (clusters) on the basis of some sort of similarity Finding the shapes of these groupings Projection is always limited by error and information loss. New projection coordinates are not always readily interpretable (latency by definition) The original relations between data dimensions are lost. Quite often, the computacional effort is to be taken into account, as most of these methods are based on distances between multivariate points.

25 TECHNIQUES: multiple visualizations How to get some of the info conveyed by observable variables back into the projections? One possibility: Using multiple visualizations. Parallel coordinates and pre cognitive stimuli (colour, position...)

26 TECHNIQUES: SOM & GTM Self Organizing Feature Map (or Kohonen Maps) k means is an special case of SOM Discretization (in the form of network grids) and projection are simultaneously performed Set of prototypes» model Cooperative learning (through neighbourhood function) Competitive learning (winner takes most if not all ) GTM is a probabilistic alternative to SOM (i.e., a form of statistical machine learning) GTM is a generative model and, therefore, aims to reproduce data density distributions It defines a proper error function It is a non linear latent model that can be interpreted as a mixture model, as well. All the learning parameters can be adaptively optimized.

27 TECHNIQUES: SOM & GTM: training / fitting The learning process for both models can be illustrated by the fisherman network simile.

28 TECHNIQUES: SOM & GTM: clustering The SOM and GTM units can be interpreted as microclusters U matrix (distance in local neighbourhood) or Magnification Factor (distorsion levels) Discrete or fuzzy clusters, from local density or probability maxima Hierarchical clustering and dendrograms

29 TECHNIQUES: SOM & GTM: multiple visualization

30 TECHNIQUES: SOM & GTM: Visualization of class membership

31 Visualization: text, hierarchies, graphs and other exotisms

32 hierarchies: Conic trees

33 ThemeRivers

34 Mapscapes

35 Visualization: software

36 Visualizing data: Simple but useful: Panopticon: Heatmaps

37 Complex and off the shelf: ( TheBrain Elevator pitch Simply type in your ideas. Drag and drop files and web pages. Any idea can be linked to anything else. Using your digital Brain is like cruising through a Web of your thinking. See new relationships. Discover connections. Go from the big picture of everything to a specific detail in seconds

38 Woven and off the shelf: Ixacta s Ixsite Web Analyzer Neighborhood sitemap diagram: Ixsite creates this diagram to help you visualize the relationship between the files on your site.

39 Woven and free:

40 SOM off the selve: Visumap ( Ellipse esom (

41 SOM fishing: REEFSOM Applied Neuroinformatics Group, Bielefeld University, Germany

42 Visualization: in summary

43 In summary... Which are the features of a good, successful visualization? Show the data (exploratory element) Focus the attention ( in the most relevant aspects) Never forget the human factor in visual perception The science of vision is the necessary framework for the visualization techniques You have to be careful with pre cognitive elements (position, movement, colour, shape) in visual coding of dimensions. How to use visualization in exploratory data mining? Visualization allows especulation and model validation. Visualization of high dimensional data sets can be accomplished through: projections and clustering methods multiple simultaneous visualizations.

44 Plan A brief introduction to data visualization Visualization & history Perception Visual exploratory DM The good, the bad & the ugly

45 The good... According to Michael Friendly s Gallery of Data Visualization (Psych./York Univ.) NY weather in NYT, Jan.1981: 2200 data pieces!!!

46 The good... According to Michael Friendly s Gallery of Data Visualization (Psych./York Univ.) Gapminder Trendalyzer Google Mo on Chart & Data Explorer

47 ... And the bad and ugly According to Michael Friendly s Gallery of Data Visualization (Psych./York Univ.)

48 Resources

49 InfoVis Wiki

50 Visualising data