Data Mining for Scientific & Engineering Applications

Transcription

1 Data Mining for Scientific & Engineering Applications Robert Grossman, Laboratory for Advanced Computing, University of Illinois & Magnify Chandrika Kamath, Lawrence Livermore National Laboratory Vipin Kumar, Army High Performance Research Center, University of Minnesota

2 Chapter 10 Data Mining Systems Robert Grossman, Laboratory for Advanced Computing, University of Illinois & Magnify

3 Goals of Chapter 10 What are the four critical interfaces in a data mining system? Is data mining about rows or columns? What are the standards in data mining? What data mining systems are available? R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 3

4 Outline 10.1 Overview of Data Mining Systems 10.2 Case Study Using a System 10.3 Managing Data for Data Mining 10.4 Data Mining Standards 10.5 Commercial and Open Source Systems R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 4

5 10.1 Overview of Data Mining Systems Following R. L. Grossman, S. Bailey, A. Ramu and B. Malhi, P. Hallstrom, I. Pulleyn and X. Qin, The Management and Mining of Multiple Predictive Models Using the Predictive Modeling Markup Language (PMML), Information and Software Technology, R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 5

6 Four Generations of DM Systems First Generation Data mining algorithms Second Generation Data mining algorithms Fourth Generation Third Generation Predictive modeling Agents & & Internet Data management mobile data Predictive modeling Data mining algorithms Data management NGI Data mining algorithms Data management R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 6

7 Layered Systems for DM & PM Move results and metadata: agents Other protocols and Internet services pred. models data mining Move models: NGI with data Predictive management Model Markup QoS Language (PMML) agents pred. models data mining data management Agents can move metadata around via net Warehouse can move data around via NGI Move data: DSTP, distributed databases, etc. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 7

8 Phases in the Data Mining & Predictive Modeling Process Phase B, C: Warehousing Data Mining Mart Phase D: Data Mining Learning set Phase E: Predictive Modeling PM or Rule Set Data Mining Trans -formations (DXML) Data Mining Primitives (DMP) Predictive Model Markup Language (PMML) Operational data PM rule or Rule Set Phase F: Deployment Scores R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 8

9 Four Critical Interfaces Data Mining Transformation (DXML) Interface between operational data and data mining mart Data Mining Primitives (DMP) interface between data mining mart and data mining system Data Mining Application Interface (DM-API) interface between data mining applications and data mining system, DMQL, OLE DB for Data Mining, Predictive Model Markup Language (PMML) interface between data mining system and predictive modeling system R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 9

10 10.2 Building a Model R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 10

11 Some (Selected) Steps to Build Models 1. Define the data schema 2. Clean and load the data 3. Define the mining schema 4. Compute derived attributes 5. Build the model 6. Analyze the model 7. Deploy the model R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 11

12 1. Define the Data Schema Data Types: int, double, float, datetime, string, etc. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 12

13 2. Clean and Load the Data Select data schema. Select data source: text, database, etc. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 13

14 3. Define the Mining Schema Select mining role: dependent, independent, excluded, key, etc. Select mining type: continuous, ordinal, categorical, binary, etc. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 14

15 4. Compute Derived Attributes Define petal_length/sepal_length R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 15

16 5. Build the Model Select Mining Schema Select Data Store Select Parameters R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 16

17 5. Build the Model (cont d) Classification tree. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 17

18 5. Build the Model - Tuning Select Model Select Parameters R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 18

19 6. Analyze the Model Analyze how well the model predicted class labels. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 19

20 7. Deploy the Model Move PMML files to scoring engine. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 20

21 10.3 Physical Data Management Arranging data by record and by attribute; data mining primitives. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 21

22 B+ Trees The cost to access one record is exactly the same as to access a block of records Use variants of techniques from databases to lower the cost of accessing out of memory data There are a variety of tree-based methods for efficiently indexing blocks of data, such as B+ trees R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 22

23 Horizontal vs. Vertical terabye of complex objects Select all objects where is less than 10. Select all objects where is less + than 10. Horizontal Vertical R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 23

24 Thinking about Columns NC Mb/s GB Sec Events/s Horizontal Vertical R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 24

25 Data Mining Primitives For many algorithms, data infrastructure only needs to supply: (Attribute Id, Attribute Value, Class Value, Count) Specialized data structures can be created to do this. SQL databases can be extended to do this. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 25

26 10.4 Data Mining Standards See for more information. Following R. L. Grossman, S. Bailey, A. Ramu and B. Malhi, P. Hallstrom, I. Pulleyn and X. Qin, The Management and Mining of Multiple Predictive Models Using the Predictive Modeling Markup Language (PMML), Information and Software Technology, R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 26

27 Predictive Model Markup Language (PMML) Current Version 2.0 Products shipping with PMML Version 1.1 PMML Working Group Full Members IBM, Magnify, MineIt, NCR, Oracle, Salford Systems, SPSS, xchange, University of Illinois at Chicago PMML Working Group Supporting Members Angoss, Insightful, KXEN, Microsoft, SGI Part of xml.org Repository & Source Forge R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 27

28 Layered Systems for DM & PM agents pred. models Internet agents pred. models data mining Move models: NGI with data Predictive management Model Markup QoS Language (PMML) data mining data management Agents can move metadata around via net Warehouse can move data around via NGI R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 28

29 Point of View data mining algorithm <PMML version= 1.1 <TreeModel ModelName= response etc. <Node frequency= freq_12_month"> etc. </TreeModel> </PMML> View data mining: 1. Extract a learning set from a data warehouse 2. Apply a data mining algorithm 3. To produce a statistical model, data mining model or rule set. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 29

30 Problems with Current Techniques Models are deployed in proprietary formats Models are application dependent Models are system dependent Models are architecture dependant Time required to integrate models with other applications can be long. R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 30

31 High Performance Data Mining & PMML 1. Scatter the query. 2. Compute the classifiers independently. partition 1 partition PMML2 partition 3 3. Gather and merge the PMML files R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 31

32 Distributed DM & PMML Combine Data Mining System Predictive Modeling System Data Mining System Data Warehouse Data Warehouse PMML Data - Chicago Data - Amsterdam R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 32

33 Example: PMML <TreeModel modelname="golfing"> <MiningSchema> <MiningField name="temperature"/> <MiningField name="humidity"/> </MiningSchema> <Node score="play"> <Predicate field="outlook" operator="equal" value="sunny"/> <Node score="play"> <CompoundPredicate booleanoperator="and" > <Predicate field="temperature operator="lessthan" value="90f" /> R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 33

34 Predictive Model Markup Language (PMML) Based on XML Benefits of PMML Open standard for Data Mining & Statistical Models Not concerned with the process of creating a model Provides independence from application, platform, and operating system Simplifies use of data mining models by other applications (consumers of data mining models) R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 34

35 Philosophy Very important to understand what PMML is not concerned with PMML is a specification of a model, not an implementation of a model PMML allows a simple means of binding parameters to values for an agreed upon set of data mining models & transformations Also, PMML includes the metadata required to deploy models R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 35

36 PMML Document Structure PMML Documents Data dictionary Transformation dictionary One or more PMML models Support for taxonomies/hierarchies PMML Model Mining Schema Univariate statistics (ModelStats) Optional extensions R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 36

37 PMML Producers, Consumers, & Data Flow learning sets derived Fields PMML Producers Data Mining Warehouse Data Mining System miningfields datafields PMML models derived Fields PMML Consumers Operational Data miningfields Campaign Manager campaigns R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 37

38 Data Flow - Recap Data Dictionary defines data Mining Schema defines specific inputs (MiningFields) required for model Transformation Dictionary defines optional additional derived fields Two types of attributes: attributes defined by the mining schema derived attributes defined via transformations Models themselves can also support certain transformations R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 38

39 Models in PMML v2.0 polynomial regression general regression trees center based clusters density based clusters associations neural nets logistic regression naïve Bayes sequences R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 39

40 Conformance Producer conformance In case, an application can write valid PMML documents for at least one type of model Consumer conformance In case an application can read valid PMML documents for at least one type of model Core and non-core features For a given model, certain features are identified as core by the DTD and must be supported Others are identified as optional R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 40

41 Other Data Mining Standards OMG CWM DM Object model for representing data mining metadata: models, model results (UML/DTD/XML) SQL-like interface for data mining operations (OLE DB/SQL) OLE DB for DM DMG PMML Representation of data mining models for intervendor exchange (DTD/XML) JSR-073 JDMAPI SQL/MM Pt. 6 DM SQL objects for defining, creating, and applying data mining models, and obtaining their results (SQL) Java API for defining, creating, and applying data mining models, and obtaining their results (Java) R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 41

42 10.5 Commercial & Open Source Systems What do you do when you get home? R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 42

43 SAS SPSS Data Mining and Related Systems Splus (open source R) Matlab (open source Octave) Many other specialized systems R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 43

44 References Jiawei Han and Micheline Kamber, Data Mining: Concepts and Techniques, Morgan Kaufmann Publishers, San Francisco, a good introduction to data mining from the systems and database perspective. Ian H. Witten and Eibe Frank, Data Mining, Morgan Kaufmann Publishers, San Francisco, a good introduction which includes Java tools for the common algorithms. Ian H. Witten, Alistair Moffat and Timothy C. Bell, Managing Gigabytes, Second Edition, Morgan Kaufmann, San Diego, a good book describing the infrastructurre and theory required for working with large collections of text or images. J. R. Quinlan, C4.5 Programs for Machine Learning, Morgan Kauffmann, San Mateo, California, Predictive Model Markup Language (PMML), see R. Grossman, C. Kamath, V. Kumar Data Mining for Scientific and Engineering Applications Ch 10/ 44