Data Mining. Knowledge Discovery, Data Warehousing and Machine Learning Final remarks. Lecturer: JERZY STEFANOWSKI

Transcription

1 Data Mining Knowledge Discovery, Data Warehousing and Machine Learning Final remarks Lecturer: JERZY STEFANOWSKI

2 Data Mining a step in A KDD Process Data mining: the core step of knowledge discovery process. Data Mining Pattern Evaluation Task-relevant Data Data Warehouse Selection Data Cleaning Data Integration Databases

3 Steps of a KDD Process Learning the application domain: relevant prior knowledge and goals of application Creating a target data set: data selection Data cleaning and preprocessing Data reduction and projection: Find useful features, dimensionality/variable reduction, invariant representation. Choosing the mining algorithm(s) Data mining: search for patterns of interest Interpretation: analysis of results. visualization, transformation, removing redundant patterns, etc. Use of discovered knowledge

4 Interacting with a user / expert in KDD KDD is not a fully automatically way of analysis. The user is an important element in KDD process. Should decide about, e.g. Choosing task and algorithms, selection in preprocessing. Interpretation and evaluation of patterns Objective interestingness measures, Subjective, By definition, KDD may have several iterations.

5 Data Preparation for Knowledge Discovery A crucial issue: The majority of time / effort is put there.

6 Data Understanding: Relevance What data is available for the task? Is this data relevant? Is additional relevant data available? How much historical data is available? Who is the data expert?

7 Data Understanding: Quantity Number of instances (records) Rule of thumb: 5,000 or more desired if less, results are less reliable; use special methods (boosting, ) Number of attributes (fields) Rule of thumb: for each field, 10 or more instances If more fields, use feature reduction and selection Number of targets Rule of thumb: >100 for each class if very unbalanced, use stratified sampling

8 Data Cleaning Steps Data acquisition and metadata Missing values Unified date format Converting nominal to numeric Discretization of numeric data Data validation and statistics

9 Data Cleaning: Metadata Field types: binary, nominal (categorical), ordinal, numeric, For nominal fields: tables translating codes to full descriptions Field role: input : inputs for modeling target : output id/auxiliary : keep, but not use for modeling ignore : don t use for modeling weight : instance weight Field descriptions

10 Data Cleaning: Unified Date Format We want to transform all dates to the same format internally Some systems accept dates in many formats e.g. Sep 24, 2003, 9/24/03, , etc dates are transformed internally to a standard value Frequently, just the year (YYYY) is sufficient For more details, we may need the month, the day, the hour, etc Representing date as YYYYMM or YYYYMMDD can be OK, but has problems Q: What are the problems with YYYYMMDD dates? A: Ignoring for now the Looming Y10K (year 10,000 crisis ) YYYYMMDD does not preserve intervals: /= This can introduce bias into models

11 Data Cleaning: Missing Values Missing data can appear in several forms: <empty field> NA Standardize missing value code(s) Dealing with missing values: ignore records with missing values (only if you have enough data) treat missing value as a separate value Not-recommended approach Imputation / Substitution: Fill in with mean or mode values Several options (all examples vs. class) Regression or dependency from other fields

12 Data Cleaning: Discretization Some methods require discrete values, e.g. most versions of Naïve Bayes, CHAID Discretization is very useful for generating a summary of data Also called binning Many approaches have been proposed: Supervised vs. unsupervised, Global vs. local (attribute point of view), Dynamic vs. Statitic choice of paramteres

13 Discretization: Equal-Width Temperature values: Count [64,67) [67,70) [70,73) [73,76) [76,79) [79,82) [82,85] Equal Width, bins Low <= value < High

14 Discretization: Equal-Height Temperature values: Count [ ] [ ] [ ] [ ] Equal Height = 4, except for the last bin

15 Supervised discretization Use information about attribute value distribution + class assignment. 12 frequency Class 1 Class values of the attribute Minimal entropy based approaches; Chi-Merge, others

16 Data Cleaning: Attribute Selection First: Remove fields with no or little variability Examine the number of distinct field values Rule of thumb: remove a field where almost all values are the same (e.g. null), except possibly in minp % or less of all records. minp could be 0.5% or more generally less than 5% of the number of targets of the smallest class More sophisticated (statistical or ML) techniques specific for data mining tasks In WEKA see attribute selection

17 A few remarks on selecting attributes Irrelevant attributes (features) in the input data may decrease the classification performance Attribute (feature) selection: Find the smallest subset of attributes leading to a higher classification accuracy than all attributes Search problem in the space of attribute subsets Three components: Search algorithm Evaluation function Classifier

18 Wrapper approach Filter vs. Wrapper approach (Kohavi, and ) Search Algorithm Data set input attributes Attribute Evaluation selected attributes Classifier Classifier The classifier is used by the evaluation function Search algorithms: Forward selection Backward elimination Random search

19 Different attribute selection methods Random selection. Correlation-based measure. Contextual-merit. Info-Gain. Gain ratio Chi-squared statistic Liu Consistency measure and Relief method Wrapper model

20 Conclusion Good data preparation is key to producing valid and reliable models!

21 Examples of Systems for Data Mining IBM: QUEST and Intelligent Miner Silicon Graphics: MineSet SAS Institute: Enterprise Miner SPSS / Integral Solutions Ltd.: Clementine Oracle Miner Rapid Miner (YALE) Orange Other systems Information Discovery Inc.: Data Mining Suite SFU: DBMiner, GeoMiner, MultiMediaMiner

22 RapidMiner (YALE)

23 Orange (Slovenia)

24 IBM Intelligent Miner: Major Features Highly scalable, large database-oriented data mining algorithms Multiple data mining functions: Association Classification Sequencing analysis Clustering. Visual graphical display Influential in database and data mining research communities.

25 IBM Miner example of visualisation

26 Statistica Statsoft ( / *.com) User friendly for MS Windows; mainly based on statistical approaches. It contains numerous data analysis methods. Efficient calculations, good managing results and reports. Excellent graphical visualisation. Comprehensive help, documentations, supporting books and teaching materials. Drivers to data bases and other data sources Main systems: Statistica 6.0 mainly statistical software Statistica Data Miner specific for DM / user friendly Specialized systems Statistica Neural Networks. Quality and Control Cards Corporation Tools

27 DataMiner main panel

28 Data Miner loading data and selecting attributes

29 Data Miner choosing methods

30 Extra tools for defining projects

31 Using several methods on the same data

32 SAS Enterprise Miner data organized in libraries database-specific parameters supported databases

33 Enterprise miner project SEMMA nodes to choose from data table form training, test, and validation sets impute missing values linear or logistic statistical exploration and analysis various algorithms assess and compare models

34 Data Mining and Business Intelligence Increasing potential to support business decisions Making Decisions End User Data Presentation Visualization Techniques Data Mining Information Discovery Data Exploration Statistical Analysis, Querying and Reporting Business Analyst Data Analyst Data Warehouses / Data Marts OLAP, MDA Data Sources Paper, Files, Information Providers, Database Systems, OLTP DBA

35 Industries/fields where you currently apply data mining [KDD Pool votes total] Banking (29) 13% Bioinformatics/Biotech (18) 8% Direct Marketing/Fundraising (19) 9% ecommerce/web (12) 6% Entertainment/News (1) 0% Fraud Detection (19) 9% Insurance (15) 7% Investment/Stocks (9) 4% Manufacturing (9) 4% Medical/Pharma (15) 7% Retail (9) 4% Scientific data (20) 9% Security (8) 4% Telecommunications (12) 6% Travel (2) 1% Other (19) 9%

36 Controversial Issues: Society and Privacy Data mining (or simple analysis) on people may come with a profile that would raise controversial issues of Discrimination Privacy Security Examples: Should males between 18 and 35 from countries that produced terrorists be singled out for search before flight? Can people be denied mortgage based on age, sex, race? Women live longer. Should they pay less for life insurance? Can discrimination be based on features like sex, age, national origin? In some areas (e.g. mortgages, employment), some features cannot be used for decision making

37 Data Mining and Privacy Can information collected for one purpose be used for mining data for another purpose In Europe, generally no, without explicit consent! In US, generally yes, Companies routinely collect information about customers and use it for marketing, etc. People may be willing to give up some of their privacy in exchange for some benefits

38 Data Mining Future Directions Currently, most data mining is on flat tables Richer data sources text, links, web, images, multimedia, knowledge bases Advanced methods Link mining, Stream mining, Applications Web, Bioinformatics, Customer modeling,

39 Challenges for Data Mining Technical tera-bytes and peta-bytes complex, multi-media, structured data integration with domain knowledge Business finding good application areas Societal Privacy issues

40 Data Mining Central Quest Find true patterns and avoid overfitting (false patterns due to randomness). So, be lucky in using this course!

41 Background literature Witten Ian and Eibe Frank, Data Mining, Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann, Han Jiawei and Kamber M. Data mining: Concepts and techniques, Morgan Kaufmann, Hand D., Mannila H., Smyth P. Principles of Data Mining, MIT Press, Fayyad, Piatetsky-Shapiro, Smyth, and Uthurusamy, Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press Mitchell T.M., Machine Learning, McGrawHill, Krawiec K, Stefanowski J., Uczenie maszynowe i sieci neuronowe, PP Press, 2003.

42 Thank you!