ECS289- Data Mining Lecture Outline

Transcription

1 ECS289- Data Mining Lecture Outline What is Data Mining? Absurdly Simple Student Grade Example Fundamental Tasks in Data Mining Course Structure Overview Emphasis on social networks and graphs Our aim is to learn from mainly me but also each other.

2 Data Mining Definition Data Mining (DM) is: the process of finding novel and actionable patterns in data, stereotypically data bases... This is a course on algorithms for specific DM tasks Each task corresponds to a certain type of pattern We will formulate the basic problem first. Create basic and then more advanced algorithms. Try to formally understand their behavior. Important to understand algorithms whether your researching new algorithms or trying to apply them as each makes strong assumptions (some simple examples later on). Example data mining algorithm applications What items are bought together (Amazon, Netflix) Categorize images, video (Google) Group together similar documents (Yahoo!)

3 A Simple Dataset Student Grades M Sex 25 Age Semes ter F A 500 Grade A 503 Grade A 509 Grade A 531 Grade GPA 3.9 F 29 S A A B B 3.2 F 23 F B B A B 3.3 M 25 F B A B A 3.2 F 33 F D A A D 3 What types of patterns can we find in this dataset?

4 Some Terminology and Notation Fields, Attributes, Records, Instances, Points x i is the i th instance (record) in the dataset x i = {x i1 x im } x i1 x in : the independent fields Sometimes each record will have a label y i

5 Five Fundamental Tasks in Mining Associations (Patterns within records) What fields co-occur frequently P(503Grade=A, 500Grade=A) > s P(503Grade=A 500Grade=A) > α 500Grade=A => 503Grade=A Classification (Predictive patterns across records) P(Sex=Male 500Grade=A, 503Grade>B) > 2/5 If 500Grade=A and 503Grade>B then Male Segmenting/Clustering (Groups of common records) Three sub-populations of students, group #1 (Systems smart: 500Grade > A-, 503Grade <= C, 518Grade > A-) Outlier Detection (Unusual records) P(Some combination of field values) < ε Identifying if a particular record is rare. Feature Selection (Mapping the points to a lower dimensional space and then performing one of the above)

6 Course Structure For each Module on Association Rules, Classification, Clustering, Outlier Detection and Feature Selection I'll present the basics from the text (Tan, Steinberger and Kumar). Supplemental readings on more advanced topics. You'll present one of the advanced topics of interest to you to the class in a specific form. Some basic homeworks in MATLAB to better understand the algorithms. Term group project NETFLIX or your own research project (if your a Ph.D. student)

7 Get Started Thinking About NetFlix If you have not used Netflix, you need to know that after you view a movie, you are asked to rate it from 1 to 5 stars. The training data set consists of: one million transactions/records of features/attributes: <movie id, customer id, rating, date recommended> as your training set as well as another data set of <movie id, date released movie name> You will then be given a test data set of the form: <customer id, movie id,?, date> and your aim Your aim is to predict what star rating the person would have given this movie.

8 NetFlix Basic Ideas Data sets <movie-id>,<customer-id>,<rating>,<date of rating> ,11765,5, ,5858,4, ,5202,3, ,18818,4, <movie-id>,<year-made>,<title> 1,2003,Dinosaur Planet 2,2004,Isle of Man TT 2004 Review 3,1997,Character 4,1994,Paula Abdul's Get Up & Dance 8,2004,What the #$*! Do We Know!? Before you start creating an approach to predict rating, understand a little about the population. a) Team names, team members and a 1 page outline of your proposed approach. (01/30/08). What the approach is, why you think it will work

9 Single Attribute Analysis

10 Syllabus Get Started on Thinking About What Topic To Present On 1) Clustering Basics Chapter 8 Tan, Steinbacher, Kumar (TSK) K-means, hierarhical algorithms, density-based approaches Advanced topics Clustering in graphs, spectral clustering, sub-space clustering, subset clustering 2) Classification/Prediction Basics - Chapter 2 Mitchell, Chapter 4 TSK. Theories: PAC-model, VC-dimension, Maximum margin, Discrimantory versus generative frameworks Trees, boosting, Support Vector Machines, Kernels Advanced topics Edge prediction in graphs, Sample bias issues, Adversarial environments

11 Syllabus Get Started on Thinking About What Topic To Present On 3) Feature Selection Basics Appendix A,B TSK Principal component analysis, singular value decomposition, factor analysis Advanced topics Embedding vector spaces into graphs, Random projections, Local-linear embedding and non-linear methods. 4) Anomaly Detection Basics Chapter 10 TSK Advanced topics Outlier detection in graphs, ACM Surveys Outlier Detection 5) Association Rules (time permitting) Basics - Chapter 6,7 TSK. Apriori, Sequential rules, Sub-graph mining Advanced topics emerging patterns

12 Module A): Clustering

13 What is Clustering Identifying groups of similar instances Typically find a set partition of D Three classes of algorithms we'll study Non-hierarchical $k$-means style Problem, algorithm derivation, speeding up using KD-Trees. Hierarchical agglomerative Density based

14 Why Do Clustering Typical problems Find groups of similar stars, people or transactions. Typical uses Compression/Summarizing Information retrieval (cluster after querying) Outlier detection Pre-processing for other pattern recognition tasks (nearest neighbor algorithms)

15 Clustering Applications - 1 Lots of examples of clustering traditional vector data Customer demographic data base, Transactions Image Segmentation Spatial and Feature Space Clustering: Applications in Image Analysis, 6th Int. Conf. on Computer Analysis and Patterns, Prague, Czech Republic

16 Clustering Applications - 2 More recently applications to non-vector data Sequences (Web-usage) Curves/Trajectories (Web-usage) Trajectory clustering using mixtures of regression models S. Gaffney and P. Smyth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 1999 Visualization of navigation patterns on a Web site using model-based clustering I. Cadez, et al. Technical Report MSR-TR-00-18, Microsoft Research, March 2000

17 Non-Hierarchical Versus Hierarchical

18 K-Means Clustering Problem - Why we need to analyze the algorithms Input Set of observations x 1..n,1..j and k (the number of clusters to form). Processing Find the best set partition that minimizes the VQ error. Output A description or exemplar for each cluster For every instance A distance to each cluster center The cluster it is assigned to. Deriving the K-Means algorithm K-Means versus K-Median

19 Unconstrained Clustering Example (Number of Clusters=4,2) Height Weight

20 Limitations of K-means: Original Points K-means (3 Clusters)

21 Limitations of K-means: Differing Sizes Original Points K-means (3 Clusters)

22 Limitations of K-means: Original Points K-means (3 Clusters)

23 Limitations of K-means: Differing Density Original Points K-means (3 Clusters)

24 Limitations of K-means: Original Points K-means (2 Clusters)

25 Limitations of K-means: Non-globular Shapes Original Points K-means (2 Clusters)

26 Overcoming K-means Limitations Original Points K-means Clusters One solution is to use many clusters. Find parts of clusters, but need to put together.

27 Overcoming K-means Limitations Original Points K-means Clusters

28 Overcoming K-means Limitations Original Points K-means Clusters