Data Mining: Concepts and Techniques. Jiawei Han. Micheline Kamber. Simon Fräser University К MORGAN KAUFMANN PUBLISHERS. AN IMPRINT OF Elsevier

Transcription

1 Data Mining: Concepts and Techniques Jiawei Han Micheline Kamber Simon Fräser University К MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF Elsevier

2 Contents Foreword Preface xix vii Chapter I Introduction I I. I What Motivated Data Mining? Why Is It Important? I 1.2 So, What Is Data Mining? Data Mining On What Kind of Data? Relational Databases Data Warehouses Transactional Databases Advanced Database Systems and Advanced Database Applications Data Mining Functionalities What Kinds of Patterns Can Be Mined? Concept/Class Description: Characterization and Discrimination Association Analysis Classification and Prediction Cluster Analysis Outlier Analysis Evolution Analysis Are All of the Patterns Interesting? Classification of Data Mining Systems Major Issues in Data Mining Summary 33 Exercises 34 Bibliographic Notes 35 Chapter 2 Data Warehouse and OLAP Technology for Data Mining What Is a Data Warehouse? Differences between Operational Database Systems and Data Warehouses But, Why Have a Separate Data Warehouse? 44 ix

3 x Contents 2.2 A Multidimensional Data Model From Tables and Spreadsheets to Data Cubes Stars, Snowflakes, and Fact Constellations: Schemas for Multidimensional Databases Examples for Defining Stan Snowflake, and Fact Constellation Schemas Measures: Their Categorization and Computation Introducing Concept Hierarchies OLAP Operations in the Multidimensional Data Model A Starnet Query Model for Querying Multidimensional Databases Data Warehouse Architecture Steps for the Design and Construction of Data Warehouses A Three-Tier Data Warehouse Architecture Types of OLAP Servers: ROLAP versus MOLAP versus HOLAP Data Warehouse Implementation Efficient Computation of Data Cubes Indexing OLAP Data Efficient Processing of OLAP Queries Metadata Repository Data Warehouse Back-End Tools and Utilities Further Development of Data Cube Technology Discovery-Driven Exploration of Data Cubes Complex Aggregation at Multiple Granularities: Multifeature Cubes Other Developments From Data Warehousing to Data Mining Data Warehouse Usage From On-Line Analytical Processing to On-Line Analytical Mining Summary 98 Exercises 99 Bibliographic Notes 103 Chapter 3 Data Preprocessing Why Preprocess the Data? Data Cleaning Missing Values Noisy Data I Inconsistent Data I Data Integration and Transformation Data Integration I Data Transformation 114

4 Contents xi 3.4 Data Reduction Data Cube Aggregation I Dimensionality Reduction I Data Compression Numerosity Reduction Discretization and Concept Hierarchy Generation Discretization and Concept Hierarchy Generation for Numeric Data Concept Hierarchy Generation for Categorical Data Summary 140 Exercises 141 Bibliographic Notes 142 Chapter 4 Data Mining Primitives, Languages, and System Architectures Data Mining Primitives: What Defines a Data Mining Task? Task-Relevant Data The Kind of Knowledge to be Mined Background Knowledge: Concept Hierarchies Interestingness Measures Presentation and Visualization of Discovered Patterns A Data Mining Query Language Syntax for Task-Relevant Data Specification Syntax for Specifying the Kind of Knowledge to be Mined Syntax for Concept Hierarchy Specification Syntax for Interestingness Measure Specification Syntax for Pattern Presentation and Visualization Specification Putting It All Together An Example of a DMQL Query Other Data Mining Languages and the Standardization of Data Mining Primitives Designing Graphical User Interfaces Based on a Data Mining Query Language Architectures of Data Mining Systems Summary 174 Exercises 174 Bibliographic Notes 176 Chapter 5 Concept Description: Characterization and Comparison What Is Concept Description? Data Generalization and Summarization-Based Characterization 181

5 xii Contents Attribute-Oriented Induction Efficient Implementation of Attribute-Oriented Induction Presentation of the Derived Generalization Analytical Characterization: Analysis of Attribute Relevance Why Perform Attribute Relevance Analysis? Methods of Attribute Relevance Analysis Analytical Characterization: An Example Mining Class Comparisons: Discriminating between Different Classes Class Comparison Methods and Implementations Presentation of Class Comparison Descriptions Class Description: Presentation of Both Characterization and Comparison Mining Descriptive Statistical Measures in Large Databases Measuring the Central Tendency Measuring the Dispersion of Data Graph Displays of Basic Statistical Class Descriptions Discussion Concept Description: A Comparison with Typical Machine Learning Methods Incremental and Parallel Mining of Concept Description Summary 220 Exercises 222 Bibliographic Notes 223 Chapter 6 Mining Association Rules in Large Databases Association Rule Mining Market Basket Analysis: A Motivating Example for Association Rule Mining Basic Concepts Association Rule Mining: A Road Map Mining Single-Dimensional Boolean Association Rules from Transactional Databases The Apriori Algorithm: Finding Frequent Itemsets Using Candidate Generation Generating Association Rules from Frequent Itemsets Improving the Efficiency of Apriori Mining Frequent Itemsets without Candidate Generation Iceberg Queries Mining Multilevel Association Rules from Transaction Databases 244

6 Contents xiii Multilevel Association Rules Approaches to Mining Multilevel Association Rules Checking for Redundant Multilevel Association Rules Mining Multidimensional Association Rules from Relational Databases and Data Warehouses Multidimensional Association Rules Mining Multidimensional Association Rules Using Static Discretization of Quantitative Attributes Mining Quantitative Association Rules Mining Distance-Based Association Rules From Association Mining to Correlation Analysis Strong Rules Are Not Necessarily Interesting: An Example From Association Analysis to Correlation Analysis Constraint-Based Association Mining Metarule-Guided Mining of Association Rules Mining Guided by Additional Rule Constraints Summary 269 Exercises 271 Bibliographic Notes 276 Chapter 7 Classification and Prediction What Is Classification? What Is Prediction? Issues Regarding Classification and Prediction Preparing the Data for Classification and Prediction Comparing Classification Methods Classification by Decision Tree Induction Decision Tree Induction Tree Pruning Extracting Classification Rules from Decision Trees Enhancements to Basic Decision Tree Induction Scalability and Decision Tree Induction Integrating Data Warehousing Techniques and Decision Tree Induction Bayesian Classification Bayes Theorem Naive Bayesian Classification Bayesian Belief Networks Training Bayesian Belief Networks Classification by Backpropagation A Multilayer Feed-Forward Neural Network Defining a Network Topology 304

7 xiv Contents Backpropagation Backpropagation and Interpretability Classification Based on Concepts from Association Rule Mining Other Classification Methods k-nearest Neighbor Classifiers Case-Based Reasoning Genetic Algorithms Rough Set Approach Fuzzy Set Approaches Prediction Linear and Multiple Regression Nonlinear Regression Other Regression Models Classifier Accuracy Estimating Classifier Accuracy Increasing Classifier Accuracy Is Accuracy Enough to Judge a Classifier? Summary 326 Exercises 328 Bibliographic Notes 330 Chapter 8 Cluster Analysis What Is Cluster Analysis? Types of Data in Cluster Analysis Interval-Scaled Variables Binary Variables Nominal, Ordinal, and Ratio-Scaled Variables Variables of Mixed Types A Categorization of Major Clustering Methods Partitioning Methods Classical Partitioning Methods: k-means and k-medoids Partitioning Methods in Large Databases: From k-medoids to CLARANS Hierarchical Methods Agglomerative and Divisive Hierarchical Clustering BIRCH: Balanced Iterative Reducing and Clustering Using Hierarchies CURE: Clustering Using REpresentatives Chameleon: A Hierarchical Clustering Algorithm Using Dynamic Modeling 361

8 Contents xv 8.6 Density-Based Methods DBSCAN: A Density-Based Clustering Method Based on Connected Regions with Sufficiently High Density OPTICS: Ordering Points To Identify the Clustering Structure DENCLUE: Clustering Based on Density Distribution Functions Grid-Based Methods STING: STatistical INformation Grid WaveCluster: Clustering Using Wavelet Transformation CLIQUE: Clustering High-Dimensional Space Model-Based Clustering Methods Statistical Approach Neural Network Approach Outlier Analysis Statistical-Based Outlier Detection Distance-Based Outlier Detection Deviation-Based Outlier Detection Summary 388 Exercises 389 Bibliographic Notes 391 Chapter 9 Mining Complex Types of Data Multidimensional Analysis and Descriptive Mining of Complex Data Objects Generalization of Structured Data Aggregation and Approximation in Spatial and Multimedia Data Generalization Generalization of Object Identifiers and Class/Subclass Hierarchies Generalization of Class Composition Hierarchies Construction and Mining of Object Cubes Generalization-Based Mining of Plan Databases by Divide-and- Conquer Mining Spatial Databases Spatial Data Cube Construction and Spatial OLAP Spatial Association Analysis Spatial Clustering Methods 41 I Spatial Classification and Spatial Trend Analysis 41 I Mining Raster Databases Mining Multimedia Databases Similarity Search in Multimedia Data Multidimensional Analysis of Multimedia Data Classification and Prediction Analysis of Multimedia Data 416

9 xvi Contents Mining Associations in Multimedia Data Mining Time-Series and Sequence Data Trend Analysis Similarity Search in Time-Series Analysis Sequential Pattern Mining Periodicity Analysis Mining Text Databases Text Data Analysis and Information Retrieval Text Mining: Keyword-Based Association and Document Classification Mining the World Wide Web Mining the Web's Link Structures to Identify Authoritative Web Pages Automatic Classification of Web Documents Construction of a Multilayered Web Information Base Web Usage Mining Summary 443 Exercises 444 Bibliographic Notes 446 Chapter 10 Applications and Trends in Data Mining Data Mining Applications Data Mining for Biomedical and DNA Data Analysis Data Mining for Financial Data Analysis Data Mining for the Retail Industry Data Mining for the Telecommunication Industry Data Mining System Products and Research Prototypes How to Choose a Data Mining System Examples of Commercial Data Mining Systems Additional Themes on Data Mining Visual and Audio Data Mining Scientific and Statistical Data Mining Theoretical Foundations of Data Mining Data Mining and Intelligent Query Answering Social Impacts of Data Mining Is Data Mining a Hype or a Persistent, Steadily Growing Business? Is Data Mining Merely Managers' Business or Everyone's Business? Is Data Mining a Threat to Privacy and Data Security? Trends in Data Mining 478

10 Contents xvii 10.6 Summary 480 Exercises 481 Bibliographic Notes 483 Appendix A An Introduction to Microsoft's OLE DB for Data Mining 485 A. I Creating a DMM object 486 A.2 Inserting Training Data into the Model and Training the Model 488 A3 Using the Model 488 Appendix В An Introduction to DBMiner 493 B. I System Architecture 494 B.2 Input and Output 494 B.3 Data Mining Tasks Supported by the System 495 B.4 Support for Task and Method Selection 498 B.5 Support of the KDD Process 499 B.6 Main Applications 499 B.7 Current Status 499 Bibliography 501 Index 533