Sutee Sujitparapitaya, Ph.D. Institutional Effectiveness and Analytics San José State University

Transcription

1 Sutee Sujitparapitaya, Ph.D. Associate Vice President for Institutional Effectiveness and Analytics San José State University Copyright Sutee Sujitparapitaya,

2 Data mining techniques are widely used for data analysis. While data mining may be viewed as expensive, time consuming, and too technical to understand dand apply, it is an institutional i i research tool used for efficiently i managing and extracting data from large databases and for expediting reporting through the use of statistical algorithms. This workshop will introduce the basic foundations of data mining and identify types of data typically found in large institutional databases, research questions to consider bf before mining i dt data, and issues of dt data quality. It will also address on how to mix traditional institutional research tools with data mining, and field additional questions typically posed by novices. Emphasis will be from a beginners (novice) perspective with an emphasis on institutional research data applications. 2

3 Describe the basic foundations of data mining from an institutional research (IR) )perspective. p Explain the principle components of IR data and research questions Describe why data mining process (CRISP DM Methodology) and primary techniques are valuable for IR Describe how the data quality and data selection works Explain the primary features of data mining tools Describe the relevant resources that are available to help the data mining projects 3

4

5 Strategic Decision Making Wealth Generation Analyzing trends Security 5

6 Data Mining is a process of finding hidden trends, patterns, and relationships in data that is not immediately apparent from summarizing the data. By examining data in largedatabases andinfers rules to a) obtain an insight; b) predict future behavior For example: Finding patterns in student data for student attrition or to identify student at risk and potential drop out from school. Motivation of Data Mining : 1. Important need for turning data into useful information 2. Fast growing amount of data, collected and stored in large and numerous databases exceeded the human ability for comprehension without powerful tools. 3. We are drowning in data, but starving for knowledge! 6

7 Traditional Statistics (Distributions, mathematics, etc.) Machine Learning: the discipline concerned with the design and development of algorithms that gives computers the ability to learn without being explicitly programmed. (Computer science, heuristics and induction algorithms). Artificial Intelligence: the study and design of intelligent agents to emulate human intelligence. Neural Networks: a mathematical model that uses an interconnected group of artificial neurons processes information between inputs and outputs or to find patterns in data. It is an adaptive model that changes its structure during a learning phase. (Biological models, psychology and engineering) 7

8 Evolutionary Step Business Question Enabling Technologies Characteristics Data Collection (1960s) What was # new applications for the last five years? Computers, tapes, disks Retrospective, static data delivery Data Access (1980s) Whatwas # new applications for College of Business last March? Relational databases, SQL, ODBC Retrospective, dynamic data delivery at record level Data Warehousing & Decision Support (1990s) Whatwas # new applications for College of Business last March? Drill down to Accounting Majors OLAP, multidimensional databases, data warehouses Retrospective, dynamic data delivery at multiple levels Data Mining (At Present Time) What s likely to happen to # new Accounting applications next month? Why? Advanced algorithms, multiprocessor computers, massive databases Prospective, proactive information delivery 8

9 Statistics Conceptual + Statistical = Proof Model Reasoning (Validation of (Hypothesis) Hypothesis) Data Mining Data Data Mining + Algorithm = based on Interestingness Pattern Discovery (Model, Rule) 9

10 Association Rules describes a method for discovering i interesting ti relations lti between variables in large databases. It produces dependency rules which will predict occurrence of an item based on occurrences of other items. Example 1: Which products are frequently bought together by customers? (Basket Analysis) DataTable = Receipts x Products Results could be used to change the placements of products Example 2: Which courses tend to be attended together? DataTable = Students x Courses Results could be used to avoid scheduling conflicts... 10

11 Market basket analysis identifies customers purchasing habits. It provides insight into the combination of products within a customers 'basket'. Ultimately, the purchasing insights provide the potential to create cross sell propositions: Which product combinations are bought When they are purchased; and in What sequence Observation Items 1 Break, Coke, Milk 2 Beer, Bread 3 Beer, Coke, Diapers, Milk 4 Beer, Bread, Diapers, Milk 5 Coke, Diapers, Milk Rules Discovered: {Milk} {Coke} {Diapers, Milk} {Beer} 11

12 The government's data mining projects fall into two broad categories: 1. Subject based Data Mining that retrieve data that could help an analyst follow a lead, and 2. Pattern based Data Mining that look for suspicious behaviors across a spread of activities. Most data mining experts consider the former a version of traditional police work chasing down leads but instead of a police officer examining a list of phone numbers of suspect calls, a computer does it. One subject based based data mining technique gaining traction among government practitioners and academics is called link analysis. Link analysis uses data to make connections between seemingly unconnected people or events. 12

13 Data Visualization is the study of the visual representation of data, meaning "information that has been abstracted in some schematic form. Itrefers to technique to communicate information clearly and effectively through graphical means (e.g., creating images, diagrams, or animations). Source: Bradbury Science Museum, Los Alamos, NM 13

14 14

15

16 + = Data Interestingness or Criteria Hidden Patterns Slice 16

17

18 Interaction data -Offers - Results - Context - Click streams - Notes Attitudinal data - Opinions - Preferences - Needs - Desires Descriptive data - Attributes - Characteristics - Self-declared info - (Geo)demographics Behavioral data - Orders - Transactions P hi - Usage history S lf d l d i f - Payment history Source: SPSS BI

19 Too many records Too manyvariables Complex non linear relationships Multi variable combination Proactive and prospective approach Source: Abbot, Data Mining: Level II 19

20 Traditional IR Work: Data file => Descriptive/Regression Analysis => Tabulations/Reports Historical Predictive Data Mining Driven IR Work: Database => Data Mining (Visualization, Association, Clustering, Predicative Modeling) => Immediate Actions Historical Predictive 20

21

22 Frequency Correlation Type of Interestingness Length of Occurrence (for sequence) Consistency Repeating/Periodicity Abnormal Behaviors Other patterns of Interestingness 22

23 Typical DBMS Approach What are total applications during the last 3 years? What is the first year retention of the fall 2006 first timefreshmen time freshmen from under representative minority? How many freshmen had attended the freshman orientation in November for the last 5 years? What is the total pledges for California alumni donation last year? How many agree and strongly agree responses did we received from the 2008 student/faculty satisfaction surveys? Data Mining Approach Which inquiries are most likely to turn into actual applications? What are the most important parameters to predict the first year attrition for next year s entering freshmen? Who are likely to enroll in the freshman orientation during the month of November? Who are likely to make pledges for alumni donation? What are the main clusters found in student/faculty satisfaction surveys? 23

24 What do we know about our students? DBMS Approach: List of students who passed English Proficiency Exam in the spring Summary of student s profile for those who failed, and dropped out last semester How many students enrolled the Business Policy course last fall semester? Data Mining Approach: What factors are contributive to learning? Who is likely to fail or drop out at the end of their 6 th year? What courses provide high FTES, better use of space? What are the course taking patterns? 24

25 DBMS Approach: List of all items that were sold in the last month? List all the items purchased by Sandy Smith? The total sales of the last month grouped by branch? How many sales transactions occurred in the month of December? Data Mining Approach: Which h items are sold together th? What items to stock k? How to place items? What discounts to offer? How best to target customers to increase sales? Which clients are most likely to respond to my next promotional mailing, and why? 25

26

27 Supervised Data Mining refers to the prior knowledge of what the outcomes exist in the data. Classification and Prediction describe and distinguish data classes or concepts, for the purpose of being able to use the model to predict the class of objects whose class label is unknown. Unsupervised Data Mining used when the researcher has no idea what hidden patterns there are in the vast database. Clustering involve in accurate identification of group membership based on maximizing the infraclass similarity and minimizing the interclass similarity. Associations and Sequences identify relationships between events that occur at one time, determines which things go together or sequential patterns in data. 27

28 Categorize your students Clustering Cafeteria meal planning Student housing planning Predict students retention/alumni donations Neural Nets/Regression Identify high risk students Estimate/predict alumni contributions Predict new student application rate Group similar students Segmentation Course planning Academic scheduling Identify student preferences for clubs and social organizations Identify courses that are taken together Association Faculty teaching load estimation Course C planning Academic scheduling Find patterns and trends over time Sequence Predict alumni donations Predict potential demand for library resources Source: Thulasi Kumar, 2004

29 Classification and Prediction Decision Trees (C&RT, C5.0, CHAID, and QUEST) Neural Networks Regressions (Linear and Logistic) Clustering K Means, TwoStep, and Kohonen SOM Association Rule/Affinity Analysis Generalized Rule Induction (GRI) CARMA (Continuous Association Rule Mining Algorithm) APRIORI 29

30

31 It is tree shaped structures that represent sets of decisions. These decisions generate rules for the classification of a dataset. The model predicts the value of a target variable based on several input variables. Two primary types of Decision trees: 1. Classification tree analysis is used when the predicted outcome is the class to which the data belongs. 2. Regression tree analysis is used when the predicted outcome can be considered a real number (e.g. the price of a house, or a patient s length of stay in a hospital). Advantages: Disadvantages: Fast Inherently unstable Simple to understand and interpret Can become large and complex Validation using statistical tests 31

32 Dependent Variable: Target classification is "should we play baseball?" which can be yes or no. Input Variables: Weather attributes are outlook, temperature, humidity, andwindspeed speed. They can have the following values: o outlook = { sunny, overcast, rain } o temperature = {hot, mild, cool } o humidity = { high, normal } o wind = {weak, strong } Day Outlook Temperature Humidity Wind Play ball D1 Sunny Hot High Weak No D2 Sunny Hot High Strong No D3 Overcast Hot High Weak Yes D4 Rain Mild High Weak Yes D5 Rain Cool Normal Weak Yes D6 Rain Cool Normal Strong No D7 Overcast Cool Normal Strong Yes D8 Sunny Mild High Weak No D9 Sunny Cool Normal Weak Yes D10 Rain Mild Normal Weak Yes D11 Sunny Mild Normal Strong Yes D12 Overcast Mild High Strong Yes D13 Overcast Hot Normal Weak Yes D14 Rain Mild High Strong No 32

33 C5.0 (Multiple split, no continuous targets) uses the C5.0 algorithm to build either a decision tree or a rule set. A C5.0 model works by splitting the sample based on the field that provides the maximum information gain. The Classification and Regression (C&R) Tree node is a tree-based classification and prediction method. Similar to C5.0, this method uses recursive e partitioning to split the training records into segments e s with similar output field values. (Binary split, continuous target) QUEST or Quick, Unbiased, Efficient Statistical Tree is a binary classification method for building decision trees. A major motivation in its development was to reduce the processing time required for large C&RT analyses with either many variables or many cases CHAID, or Chi-squared Automatic Interaction Detection, is a classification method for building decision trees by using chi-square statistics to identify optimal splits. CHAID first examines the cross tabulations between each of the predictor variables and the outcome and tests for significance using a chisquare independence test. 33

34 Neural network is a model that emulates human biological neural system to solve the prediction and classification problems. solutions for linear and non linear relationships between input and output variables. Does not assume any particular data distribution. 34

35 Advantages Has a mathematical foundation Robustwith noisy data Detects relationships and trends in data that traditional methods overlook Can fit complex non linear models Ability to detect all possible interactions between predictor variables Disadvantages Black Box" nature that does not easily analyze and interpret Greater computational tti lburden Virtually impossible to "interpret" the solution in traditional, analytic terms, such as those used to build theories that explain phenomena 35

36 Linear regression is an approach to modeling the relationship between a scalar dependent variable (y) and one or more predictor variables (X). The case of one predictor variable is called simple regression. More than one predictor variable ibl is multiple li l regression. The regression equation represents a straight line or plane that minimizes the squared differences between predicted and actual output values. This is a very common statistical technique for summarizing data and making predictions y= f(x) Advantages: Available in most software Widely accepted statistical technique Disadvantages: Not appropriate for many non linear problems Must meet underlying assumptions 36

37 Logistic regression is a type of regression analysis used for predicting the outcome of a categorical dependent variable based on one or more predictor variables that may be either continuous or categorical data. 1. Binomial orbinary logistic regression refers to the instance in which the observed outcome can have only two possible types (e.g., "dead" vs. "alive", "success" vs. "failure", or "yes" vs. "no"). 2. Multinomial logistic regression ess refers eesto cases where eethe outcome can have ae three or more possible types (e.g., "better' vs. "no change" vs. "worse"). For example, logistic regression might be used to predict whether a new student will graduate within 6 years, based on observed characteristics ti of the student (test score, age, gender, pre school preparation, etc). Advantages: Well established statistical procedure Simple and easy to interpret Very fast to train and build Can be used with small sample sizes Disadvantages: Strong sensitivity to outliers Multicollinearity 37

38 Cluster analysis is an exploratory data analysis tool (unsupervised) for solving classification problems. Its object is to sort cases (people, things, events, etc) into groups, or clusters, so that the degree of association is strong between members of the same cluster and weak between members of different clusters. It is not an automatic task, but an iterative process ofknowledge discovery (interactive multi objective optimization) that involves trial and failure until the result achieves the desired properties. The result of a cluster analysis shown as the coloring of the squares into three clusters. Types of Clustering K Means Two Step Kohonen Advantages: Make up of groups in attitudinal or behavioral tests Disadvantages: Individual group members may still differ 38

39

40 K Means clustering is an algorithm to classify or to group your objects based on attributes/features into K number of group. K is positive integer number. The grouping is done by minimizing the sum of squares of distances between data and the corresponding cluster centroid. Thus the purpose is to classify the data by partitioning n observations into k clusters in which each observation belongs to the cluster with the nearest mean. 40

41 Two step cluster analysis is a technique that groups cases into pre clusters that are treated as single cases. Standard hierarchical clustering is then applied to thepre pre clusters in the secondstep step. It appropriate for large datasets or datasets that have a mixture of continuous and categorical variables (not interval or dichotomous). It processes dt data with a one pass through the dataset h dt tmethod. Therefore, it does not require a proximity table (like hierarchical classification) or an iterative process (like K means clustering) 41

42 com 42

43 Kohonen networks are a type of neural network that perform clustering, also known as a knet or a self organizing map. It seeks to describe dataset in terms of natural clusters of cases. This type of network can be used to cluster the data set into distinct groups when you don't know what those groups are at the beginning. Don't even need to know the number of groups to look for. Kohonen networks start with a large number of units, and as training progresses, theunits gravitate toward the natural clusters in the data. Source: SPSS BI 43

44

45 Association or affinity analysis is a data mining technique that discovers co occurrence relationships among activities performed by specific individuals or groups. These relationships are then expressed as a collection of association rules. Association rules are statements in the form if antecedent(s) then consequent(s) Used to perform market basket analysis, in which retailers seek to understand the purchase behavior of customers. Types of Association GRI Apriori CARMA 45

46 Customer Purchase 1 jam 2 milk 3 jam 3 bread 4 jam 4 bread 4 milk Customer Jam Bread Milk 1 T F F 2 F F T 3 T T F 4 T T T 46

47

48 Business Understanding di Data Understanding Data Preparation Modeling Evaluation Deployment Source: dm.org 48

49 Business Understanding Data Understanding Data Preparation Modeling Evaluation Deployment Determine Collect tinitial ldt Data Dt Data Set Select tmodeling Evaluate Results Plan Deployment Business Objectives Initial Data Collection Data Set Description Technique Assessment of Data Deployment Plan Background Report Modeling Technique Mining Results w.r.t. Business Objectives Select Data Modeling Assumptions Business Success Plan Monitoring and Business Success Describe Data Rationale for Inclusion / Criteria Maintenance Criteria Data Description Report Exclusion Generate Test Design Approved Models Monitoring and Situation Assessment Explore Data Clean Data Inventory of Resources Data Exploration Report Data Cleaning Report Requirements, Assumptions, and Constraints Verify Data Quality Data Quality Report Construct Data Derived Attributes Risks and Contingencies Generated Records Terminology Costs and Benefits Integrate Data Merged Data Determine Data Mining Goal Data Mining Goals Data Mining Success Criteria Format Data Reformatted Data Test Design Build Model Parameter Settings Models Model Description Assess Model Model Assessment Revised Parameter Settings Review Process Review of Process Determine Next Steps List of Possible Actions Decision Mi Maintenance Plan Produce Final Report Final Report Final Presentation Review Project Experience Documentation Produce Project Plan Project Plan Initial Asessment of Tools and Techniques Source: SPSS BI

50

51 Good data= better decisions = more profit Bad data= risky decisions = potential disaster: Bad data= Errors = losses We cannot offer enough courses = angry students, drop out or transfer out to another institution You re not admitted dto your intended dmajor = angry students and parents, lost revenue We have more rooms in the dorm for new students = bad decisions if the number of students is inflated by bad data. 51

52 52

53 53

54 Scalar refer to a quantity consisting of a single real number used to measured magnitude (size). Interval = Scale with a fixed and defined interval e.g. temperature or time. Ordinal = Scale for ordering observations from low to high with any ties attributed to lack of measurement sensitivity e.g. score from a questionnaire. Nominal with order = Scale for grouping into categories with order e.g. mild, moderate or severe. This can be difficult to separate from ordinal. Nominal without order = Scale for grouping into unique categories eg e.g. eye color. Dichotomous = As for nominal but two categories only e.g. male/female. Non Scalar contains more than one value (e.g., lists, arrays, records) 54

55 Case or likewise deletion Pairwise deletion Single value substitution (by mean, median or mode of variable) Regression substitution (using values of other variables in the same row or using the overall relationships of variables into account) Marking with a dummy variable 55

56 Identify outliers (Anomaly Detection Node) Verify distributions (Data Audit Node) Relationship of variables Predictive power of variables (Auto Data Prep Node) Data reduction 56

57 Data Audit/Data Distribution Charts Number of variables Number of records Information content/predictive power 57

58

59 59

60 Successful data mining strategy involves: 1. Make data mining models comprehensible to business users 2. Translate user s questions into a data mining i problem Well defined goals, project objectives, and questions 3. Ensure to use sufficient and relevant data 4. Close the loop: identify causality, suggest actions, and measure their effect. Need domain expertise in institutional research to build, test, validate, anddeploydeploy models. 5. Careful consideration and selection of software and analysts (tech and domain expert) 6. Support from senior administrators (VPs and the President) 7. Cope with privacy and security issues 8. Misuse of information/inaccurate information 60

61

62 Free Open source Data Mining Software and Applications: R RapidMiner WEKA Commercial Data Mining Software and Applications: PASW Modeler (IBM) STATISTICA Data Miner (StatSoft) Enterpriser Miner (SAS) Oracle Data Mining CART/MARS (Salford Systems) Low Price XLMiner ($199) 62

63 63

64 64

65

66 Information www 01.ibm.com/software/analytics/spss/products/modeler d l Training modeling agency.com dli canada.html 66

67 67

68 p// /