Overview. Background. Data Mining Analytics for Business Intelligence and Decision Support
|
|
- Thomas Underwood
- 8 years ago
- Views:
Transcription
1 Mining Analytics for Business Intelligence and Decision Support Chid Apte, PhD Manager, Abstraction Research Group IBM TJ Watson Research Center Overview Knowledge discovery and data mining (KDD) techniques are used for analyzing and discovering actionable insights from data. The talk will Provide technical descriptions of the core algorithms that comprise data mining analytics Describe some business application scenarios for KDD Discuss issues in business intelligence systems Map trends in this area Background Widespread and explosive growth in use and size of databases Traditional use: query based report generation Size and volumes raise new issues: will data help business to achieve an advantage can data be used to model underlying processes and predict their behavior can we understand the data Providing capabilities to support exploration, summarization, and modeling of large databases is the goal of Business Intelligence systems 1
2 From Transactions to Warehouses Transactional databases: Reliable and accurate data capture; logging, book-keeping warehousing: Turning transactional data into a history repository Can be queried for summaries and aggregate reports First step in transforming transactional data (primary purpose: reliable storage) to one whose primary use is business intelligence May require integration of multiple sources of data Dealing with multiple formats; multiple database systems; integrating distributed databases; data cleaning; creating unified logical view of underlying non-homogeneous data On-Line Analytical Processing (OLAP) Supports query driven exploration of the data warehouse Utilizing pre-computed aggregates along data dimensions Deciding which aggregates to pre-compute and how to derive or reliably estimate from pre-computed projections Extends the Structured Query Language (SQL) framework to accommodate queries that would otherwise have been computationally impossible on a relational database management system Beyond OLAP Supporting queries at much more abstract level than SQL and OLAP Computer-driven exploration of data as opposed to human analyst-driven Facilitating data exploration of high dimensional data Providing solutions when user cannot describe goal in terms of a specific query e.g. discovering fraudulent cases in credit card or telephone uses Visualizing and understanding massive volumes of highdimensional data Rates of growth of data sets exceed by far any rates with which traditional human analyst techniques can cope 2
3 A Definition for Mining Automated search procedures for discovering credible and actionable insights from large volumes of high dimensional data emphasis upon symbolic learning and modeling methods (i.e. techniques that produce interpretable results) data management methods use of techniques from statistics, pattern recognition, and machine learning machine learning and statistical modeling also heavily used in vision, speech recognition, image processing, handwriting recognition, natural language understanding, etc. issues of scalability and automated business intelligence solutions drive much of and differentiate data mining from the other applications of machine learning and statistical modeling Machine Learning and Statistical Modeling serve as an important core Temporal Modeling Complex Pattern Detection Systems Performance Management Management Business Decision Support Systems Knowledge Discovery and Mining Machine Learning Statistical Modeling Feature Creation and Analysis Markov Modeling Speech Understanding Handwriting Recognition Computational Linguistics Statistical Text Processing Vision / Image Knowledge Management NLP/NLU Others (Agents, Education, etc..) Typical Business Intelligence Applications Risk Analysis Given a set of current customers and their finance/insurance history data, build a predictive model that can be used to classify a new customer into a risk category Targeted Marketing Given a set of current customers and history on their purchases and their responses to promotions, target new promotions to those most likely to respond Customer Retention Given a set of past customers and their behavior prior to leaving, predict who is most likely to leave and take proactive action Fraud Detection Detect fraudulent activities either proactively or on-line real-time Many other new applications keep surfacing 3
4 There s More to it Than Just Mining The process of identifying valid, novel, potentially useful, and understandable patterns in data requires one or more of: Selecting or sampling data from a data warehouse Cleaning or pre-processing it Transforming or reducing it Applying a data mining component to extract models or patterns Evaluating the derived structure The process is also known as KDD (Knowledge Discovery from Mining) mining is a key component concerned with the algorithmic means by which structures are extracted from data while meeting computational efficiency constraints Identify Business Opportunity The KDD Process Select Transform Mine Assimilate Warehouse Selected data mining n. The process of extracting valid, previously unknown, and ultimately comprehensible and actionable information from large databases and using it to make crucial business decisions. Visualization Mining Techniques Predictive Modeling Predict a specific attribute (database field) based upon the other attributes (fields) in the data Clustering (Segmentation) Group data records into subsets where items in subsets are more similar to each other than to items in other subsets Frequent Patterns Find interesting similarities between a few attributes in subsets of the data Change & Deviation Detect and account for interesting sequence of information in data records Dependencies Generate the joint probability density function that might have generated the data 4
5 Predictive Modeling Estimate a function? that maps points from an input space? to an output space???given only a finite sampling of the mapping Predict value of field (???in a database based on the other fields (?? Accurately construct an estimator ƒ of ƒfrom a finite sample known as the training set May be corrupted (i.e. noisy) If predicted quantity is numeric (i.e.??r, the real line) then the prediction problem is that of regression modeling If the predicted quantity is discrete (i.e.???????????) then the prediction problem is that of classification modeling Issues in Predictive Modeling Transformations on input space X to improve estimation capability Feature extraction / construction / selection Evaluating the estimate ƒ in terms of how well it performs on data not present in the training set Maximizing prediction accuracy by avoiding underfitting or over-fitting Trading off model complexity versus model accuracy Bias-variance tradeoff, penalized likelihood, minimum message length (MML) or minimum description length (MDL) Classification Predicting the most likely state of a categorical variable (the class) given the values of other variables Density estimation problem: deriving the value of Y given x?? from the joint density on Y and? Kernel density estimators Metric-space based methods (k-nearest neighbor) Projection into decision regions divide attribute space into decision regions and associate prediction with each region Linear classifiers, neural networks, decision trees, disjunctive normal form (DNF) rule-based classifiers Projection methods by far the most practical for data mining 5
6 Regression Predicting the most likely value of a numerical variable (the target column) given the values of other variables Numerical function approximation problem: deriving the value of Y given x?? from the joint probability distribution on Y and? Statistical probability models (e.g. linear regression) Projection into decision regions divide attribute space into decision regions and associate constant value with each region Neural networks, decision trees, disjunctive normal form (DNF) rule-based classifiers Hybrid Coupling projection methods with statistical models Projection and hybrid methods by far the most practical for data mining The Predictive Modeling Process Mine historical data to train patterns/models that can predict future behaviors Behaviors Response to Direct Mail Product Quality (Defects) Declining Activity Credit Risk Delinquency Likelihood to buy specific products Profitability etc. Score with models to reflect likelihood to exhibit the modeled behavior Act to optimize business objectives based on these scores Decision Trees for Predictive Modeling Tree generation algorithm Beginning with the training set at the root node, recursively split until a stopping criteria is met Split using best test among all possible tests on all attributes Prune tree (MDL, cross-validation, etc.) 6
7 Issues in Decision Tree Building Splitting at nodes greedy search: GINI (entropy minimizing), class probability profile difference, log-loss likelihood, etc. exhaustive search: ReliefF, Contextual Merit, etc. Testing of attributes Numerical attributes: inequality tests on cut-points Categorical attributes: subset tests Leaf models Piecewise constant Linear Probability functions Scalability A Typical Decision Tree Expected Sales Revenue Historical Sales per Mlg Less than 7 Historical Sales per Mlg 7-15 Historical Sales per Mlg Greater than 15 Segment 8 Hist Avg Sales per Order Less than 113 Segment 1 Hist Avg Sales per Order Greater or equal to 113 Credit Limit Less than 2200 Segment 6 Credit Limit Greater or equal to 2200 Segment 7 Climate Indicator 0 Segment 2 Climate Indicator 1 Risk Score Less than 687 Segment 3 Risk Score Greater or equal to 687 Outdoor Purchases Less than 3 Segment 4 Outdoor Purchases Greater or equal to 3 Segment 5 Training a Predictive Model Observations Predictive Model Predicted Outcomes Prediction Errors Actual Outcomes 7
8 Training Generalization Training Validation Too many segments Over fit Too few segments Under fit About right About right Prediction Error Optimal Training Validation Error Training Error Optimum Rule Set Number of Rules (degrees of freedom) Clustering Given a finite sampling of points, group them into sets of similar points Representing clusters of points with common characteristics In predictive modeling, class (or value) membership is known in the training data In clustering, this knowledge is not known a- priori, and is perhaps being discovered by clustering or segmentation 8
9 Techniques for Clustering/Segmentation Two-stage approach outer loop to determine cluster number k inner loop to fit points to clusters Metric distance-based methods; find best k-way partition so that points in a partition are closer to each other than to points in other partitions Model based methods: a best fit (very typically probabilistic) model is hypothesized for each cluster Partition based methods: iteratively enumerating and scoring various partition scenarios using heuristic scoring functions k-mean Clustering Algorithm Widely used in data mining Given k cluster centers c 1,j,c 2,j,,c k,j at iteration j, compute c 1,j+1,c 2,j+1,,c k,j+1 Cluster assignment: For each i=1,,m, assign x i to cluster l(i) such that c l(i),j is nearest to x i Cluster Center Update: For l=1,,k set c l,j+1 to be the mean of all x i assigned to c l,j Stop when c l,j = c l,j+1, l=1,,k Extensions include support for scalability, efficient placement of initial k means, and (harder problem) determining the number of clusters k Frequent Patterns Extracting compact patterns that describe subsets of data Row-wise patterns Column-wise patterns Association rules: detecting combinations of attribute values that occur with a minimum level of frequency (support) and certainty (confidence) Scalable algorithms can find all such rules in linear time under certain conditions of data sparseness Rules are not statements about causal effects amongst attributes, but can still provide useful insights 9
10 Change and Deviation Detecting sequence information, temporal or otherwise Ordering information of transactions (rows) is utilized Under certain conditions of data sparseness, sequences with desired levels of frequency and certainty can be computed in linear time Dependency Modeling Detecting causal structure within data Causal models Discovering probabilistic distributions governing the data Discovering functional dependencies between attributes in the data Techniques Density estimation methods Expectation maximization Explicit causal modeling Bayesian networks Applying Mining in Business Profit Customer Satisfaction Efficiency Who are the best customers to sell my products to? What are the most effective market segments for my business? How do I increase market share of my products? How do I reduce my costs and not impact production? How do I optimize my inventory? 10
11 Mapping Operations into Applications Predictive Modeling Assigning risk levels to new insurance and financial contracts Clustering / Segmentation Identifying distinct market groups in customer population Frequent Patterns Market basket analysis (what gets shopped together in a supermarket) Change and Deviation Fraud discovery in health claim data Discovering shopping patterns over time Business Application Opportunities Retail/Distribution Category management Merchandise planning Product management Production planning/tracking Insurance/Healthcare Claims analysis Provider analysis Managed care Outcomes analysis Manufacturing Product costing Manufacturing quality and efficiency Parts analysis Utilities Industrial customer profiles Financial analysis "Bulk Power" analysis Government Budgeting Financial reporting Demographics Telecommunications Customer profiles/ segmentation Product profitability Demand forecasting Usage analysis Cross Industry Market basket analysis Target marketing Customer segmentation Customer service Fraud and abuse Financial performance Transportation Yield management Pricing/rate analysis Logistics Financial Customer profitability and segmentation Products/portfolio profitability Risk management Cross-sale analysis Branch performance The Challenge Where Is It? What Does it Mean? How Can I Get It? What Format Is It In? No Single View of Many Interfaces Difficult to Access Multiple Sources Different Formats, Platforms Inconsistencies, Redundancies 11
12 An Efficient Environment for Mining Enterprise Model Operational Enterprise Warehouse Global mart Specialized Analysis marts External Mining Analysis Transaction Transformation (legacy system context removal) Meta-data Definition (e.g. consistent business terms) Analytical dimension definition (e.g. time, policyholder) Summarization Aggregation Identification / Collection Review Conversion / Reduction / Normalization Representation The Business Intelligence Process Access Transform Distribute Store Find & Understand Operational & External Enhancing Staging Relational Summarizing Aggregating Flow & Process Flow Joining From Multiple Sources Populating On-Demand Automate & Manage Multiple Platforms & Hardware Information Catalog Business Views Models Discover, Analyze, Visualize Query Interpretation Multi- Dimensional Analysis Mining Multi-Vendor Support Open Interfaces Mining Marketplace Status Enabler for business intelligence systems mining algorithm suites Loosely coupled with database technology Emphasis on data warehousing followed by exploratory data mining Typically conducted by consultants or in-house analytic teams Issues warehouse requirements Sophisticated analytics requirements 12
13 Key Challenges and Trends Infrastructure Enabling transparent and pervasive usage Algorithms Optimized and robust mining Solutions Vertically integrated for critical problems Enhanced emphasis on the Internet Infrastructure Making data mining transparent base extenders e.g. DB2/UDB User Defined Functions for model training/scoring Sufficient statistics (e.g. histograms, counts, samples, etc.) Parallel and distributed data mining Scalability (sampling and parallelization) XML based APIs for database coupling and application embedding Interoperability Training and scoring in different environments Intelligent or semi-automated data warehousing for mining Industry specific templates Meta-data mining Algorithms Robust and Automated Evaluation metrics Automated feature extraction / transformation / selection Discovering relational and hierarchical structures amongst attributes Incorporating prior knowledge to account for costs / benefits / uncertainty / missing values Incremental and on-line mining Privacy preserving data mining Heterogeneous data mining 13
14 Solutions Business Risk management Targeted marketing Portfolio management Systems Performance management Internet Site profiling and performance tuning User personalization Summary mining is being embedded in vertical solutions for business intelligence and decision support Management ecommerce Critical Large-Scale Solutions (CRM, etc.) Using data mining should eventually become as easy and pervasive as working with databases and spreadsheets today References Mathematical Programming for Mining: Formulations and Challenges, by Bradley et al., INFORMS Journal on Computing, Volume 11, No. 3, Summer 1999 KDD Nuggets IBM
Data Mining Analytics for Business Intelligence and Decision Support
Data Mining Analytics for Business Intelligence and Decision Support Chid Apte, T.J. Watson Research Center, IBM Research Division Knowledge Discovery and Data Mining (KDD) techniques are used for analyzing
More informationIntroduction to Data Mining
Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:
More informationOLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP
Data Warehousing and End-User Access Tools OLAP and Data Mining Accompanying growth in data warehouses is increasing demands for more powerful access tools providing advanced analytical capabilities. Key
More informationPrinciples of Data Mining by Hand&Mannila&Smyth
Principles of Data Mining by Hand&Mannila&Smyth Slides for Textbook Ari Visa,, Institute of Signal Processing Tampere University of Technology October 4, 2010 Data Mining: Concepts and Techniques 1 Differences
More informationData Mining and Knowledge Discovery in Databases (KDD) State of the Art. Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland
Data Mining and Knowledge Discovery in Databases (KDD) State of the Art Prof. Dr. T. Nouri Computer Science Department FHNW Switzerland 1 Conference overview 1. Overview of KDD and data mining 2. Data
More informationnot possible or was possible at a high cost for collecting the data.
Data Mining and Knowledge Discovery Generating knowledge from data Knowledge Discovery Data Mining White Paper Organizations collect a vast amount of data in the process of carrying out their day-to-day
More informationData Mining for Fun and Profit
Data Mining for Fun and Profit Data mining is the extraction of implicit, previously unknown, and potentially useful information from data. - Ian H. Witten, Data Mining: Practical Machine Learning Tools
More informationData Mining: Overview. What is Data Mining?
Data Mining: Overview What is Data Mining? Recently * coined term for confluence of ideas from statistics and computer science (machine learning and database methods) applied to large databases in science,
More informationCourse 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing
More informationInformation Management course
Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)
More informationData Mining Solutions for the Business Environment
Database Systems Journal vol. IV, no. 4/2013 21 Data Mining Solutions for the Business Environment Ruxandra PETRE University of Economic Studies, Bucharest, Romania ruxandra_stefania.petre@yahoo.com Over
More informationThe Data Mining Process
Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data
More informationSanjeev Kumar. contribute
RESEARCH ISSUES IN DATAA MINING Sanjeev Kumar I.A.S.R.I., Library Avenue, Pusa, New Delhi-110012 sanjeevk@iasri.res.in 1. Introduction The field of data mining and knowledgee discovery is emerging as a
More informationChapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
More informationDigging for Gold: Business Usage for Data Mining Kim Foster, CoreTech Consulting Group, Inc., King of Prussia, PA
Digging for Gold: Business Usage for Data Mining Kim Foster, CoreTech Consulting Group, Inc., King of Prussia, PA ABSTRACT Current trends in data mining allow the business community to take advantage of
More informationInternational Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014
RESEARCH ARTICLE OPEN ACCESS A Survey of Data Mining: Concepts with Applications and its Future Scope Dr. Zubair Khan 1, Ashish Kumar 2, Sunny Kumar 3 M.Tech Research Scholar 2. Department of Computer
More informationAn Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015
An Introduction to Data Mining for Wind Power Management Spring 2015 Big Data World Every minute: Google receives over 4 million search queries Facebook users share almost 2.5 million pieces of content
More informationChapter 5. Warehousing, Data Acquisition, Data. Visualization
Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization 5-1 Learning Objectives
More informationData Mining Algorithms Part 1. Dejan Sarka
Data Mining Algorithms Part 1 Dejan Sarka Join the conversation on Twitter: @DevWeek #DW2015 Instructor Bio Dejan Sarka (dsarka@solidq.com) 30 years of experience SQL Server MVP, MCT, 13 books 7+ courses
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationIntroduction to Data Mining
Introduction to Data Mining Jay Urbain Credits: Nazli Goharian & David Grossman @ IIT Outline Introduction Data Pre-processing Data Mining Algorithms Naïve Bayes Decision Tree Neural Network Association
More informationAn Introduction to Data Mining
An Introduction to Intel Beijing wei.heng@intel.com January 17, 2014 Outline 1 DW Overview What is Notable Application of Conference, Software and Applications Major Process in 2 Major Tasks in Detail
More informationWelcome. Data Mining: Updates in Technologies. Xindong Wu. Colorado School of Mines Golden, Colorado 80401, USA
Welcome Xindong Wu Data Mining: Updates in Technologies Dept of Math and Computer Science Colorado School of Mines Golden, Colorado 80401, USA Email: xwu@ mines.edu Home Page: http://kais.mines.edu/~xwu/
More informationIntroduction to Data Mining and Machine Learning Techniques. Iza Moise, Evangelos Pournaras, Dirk Helbing
Introduction to Data Mining and Machine Learning Techniques Iza Moise, Evangelos Pournaras, Dirk Helbing Iza Moise, Evangelos Pournaras, Dirk Helbing 1 Overview Main principles of data mining Definition
More informationFoundations of Business Intelligence: Databases and Information Management
Foundations of Business Intelligence: Databases and Information Management Problem: HP s numerous systems unable to deliver the information needed for a complete picture of business operations, lack of
More informationAn Overview of Knowledge Discovery Database and Data mining Techniques
An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,
More informationData Warehouse: Introduction
Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,
More informationData Mining: Concepts and Techniques. Jiawei Han. Micheline Kamber. Simon Fräser University К MORGAN KAUFMANN PUBLISHERS. AN IMPRINT OF Elsevier
Data Mining: Concepts and Techniques Jiawei Han Micheline Kamber Simon Fräser University К MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF Elsevier Contents Foreword Preface xix vii Chapter I Introduction I I.
More informationWHITEPAPER. Creating and Deploying Predictive Strategies that Drive Customer Value in Marketing, Sales and Risk
WHITEPAPER Creating and Deploying Predictive Strategies that Drive Customer Value in Marketing, Sales and Risk Overview Angoss is helping its clients achieve significant revenue growth and measurable return
More informationISSN: 2321-7782 (Online) Volume 3, Issue 4, April 2015 International Journal of Advance Research in Computer Science and Management Studies
ISSN: 2321-7782 (Online) Volume 3, Issue 4, April 2015 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online
More informationDecision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010
Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010 Ernst van Waning Senior Sales Engineer May 28, 2010 Agenda SPSS, an IBM Company SPSS Statistics User-driven product
More informationChapter 20: Data Analysis
Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification
More informationData Mining. 1 Introduction 2 Data Mining methods. Alfred Holl Data Mining 1
Data Mining 1 Introduction 2 Data Mining methods Alfred Holl Data Mining 1 1 Introduction 1.1 Motivation 1.2 Goals and problems 1.3 Definitions 1.4 Roots 1.5 Data Mining process 1.6 Epistemological constraints
More informationDATA MINING AND WAREHOUSING CONCEPTS
CHAPTER 1 DATA MINING AND WAREHOUSING CONCEPTS 1.1 INTRODUCTION The past couple of decades have seen a dramatic increase in the amount of information or data being stored in electronic format. This accumulation
More informationData Mining + Business Intelligence. Integration, Design and Implementation
Data Mining + Business Intelligence Integration, Design and Implementation ABOUT ME Vijay Kotu Data, Business, Technology, Statistics BUSINESS INTELLIGENCE - Result Making data accessible Wider distribution
More informationDATA MINING TECHNIQUES AND APPLICATIONS
DATA MINING TECHNIQUES AND APPLICATIONS Mrs. Bharati M. Ramageri, Lecturer Modern Institute of Information Technology and Research, Department of Computer Application, Yamunanagar, Nigdi Pune, Maharashtra,
More informationIntroduction to Data Mining and Business Intelligence Lecture 1/DMBI/IKI83403T/MTI/UI
Introduction to Data Mining and Business Intelligence Lecture 1/DMBI/IKI83403T/MTI/UI Yudho Giri Sucahyo, Ph.D, CISA (yudho@cs.ui.ac.id) Faculty of Computer Science, University of Indonesia Objectives
More informationData Mining System, Functionalities and Applications: A Radical Review
Data Mining System, Functionalities and Applications: A Radical Review Dr. Poonam Chaudhary System Programmer, Kurukshetra University, Kurukshetra Abstract: Data Mining is the process of locating potentially
More informationDATA MINING TECHNIQUES SUPPORT TO KNOWLEGDE OF BUSINESS INTELLIGENT SYSTEM
INTERNATIONAL JOURNAL OF RESEARCH IN COMPUTER APPLICATIONS AND ROBOTICS ISSN 2320-7345 DATA MINING TECHNIQUES SUPPORT TO KNOWLEGDE OF BUSINESS INTELLIGENT SYSTEM M. Mayilvaganan 1, S. Aparna 2 1 Associate
More informationExample application (1) Telecommunication. Lecture 1: Data Mining Overview and Process. Example application (2) Health
Lecture 1: Data Mining Overview and Process What is data mining? Example applications Definitions Multi disciplinary Techniques Major challenges The data mining process History of data mining Data mining
More informationDatabase Marketing, Business Intelligence and Knowledge Discovery
Database Marketing, Business Intelligence and Knowledge Discovery Note: Using material from Tan / Steinbach / Kumar (2005) Introduction to Data Mining,, Addison Wesley; and Cios / Pedrycz / Swiniarski
More informationClassification and Prediction
Classification and Prediction Slides for Data Mining: Concepts and Techniques Chapter 7 Jiawei Han and Micheline Kamber Intelligent Database Systems Research Lab School of Computing Science Simon Fraser
More informationSPATIAL DATA CLASSIFICATION AND DATA MINING
, pp.-40-44. Available online at http://www. bioinfo. in/contents. php?id=42 SPATIAL DATA CLASSIFICATION AND DATA MINING RATHI J.B. * AND PATIL A.D. Department of Computer Science & Engineering, Jawaharlal
More informationData Mining is sometimes referred to as KDD and DM and KDD tend to be used as synonyms
Data Mining Techniques forcrm Data Mining The non-trivial extraction of novel, implicit, and actionable knowledge from large datasets. Extremely large datasets Discovery of the non-obvious Useful knowledge
More informationData Warehousing and Data Mining in Business Applications
133 Data Warehousing and Data Mining in Business Applications Eesha Goel CSE Deptt. GZS-PTU Campus, Bathinda. Abstract Information technology is now required in all aspect of our lives that helps in business
More information1. What are the uses of statistics in data mining? Statistics is used to Estimate the complexity of a data mining problem. Suggest which data mining
1. What are the uses of statistics in data mining? Statistics is used to Estimate the complexity of a data mining problem. Suggest which data mining techniques are most likely to be successful, and Identify
More informationANALYTICS CENTER LEARNING PROGRAM
Overview of Curriculum ANALYTICS CENTER LEARNING PROGRAM The following courses are offered by Analytics Center as part of its learning program: Course Duration Prerequisites 1- Math and Theory 101 - Fundamentals
More informationA Survey on Web Research for Data Mining
A Survey on Web Research for Data Mining Gaurav Saini 1 gauravhpror@gmail.com 1 Abstract Web mining is the application of data mining techniques to extract knowledge from web data, including web documents,
More informationAdvanced In-Database Analytics
Advanced In-Database Analytics Tallinn, Sept. 25th, 2012 Mikko-Pekka Bertling, BDM Greenplum EMEA 1 That sounds complicated? 2 Who can tell me how best to solve this 3 What are the main mathematical functions??
More informationIntroduction. A. Bellaachia Page: 1
Introduction 1. Objectives... 3 2. What is Data Mining?... 4 3. Knowledge Discovery Process... 5 4. KD Process Example... 7 5. Typical Data Mining Architecture... 8 6. Database vs. Data Mining... 9 7.
More informationAn Overview of Database management System, Data warehousing and Data Mining
An Overview of Database management System, Data warehousing and Data Mining Ramandeep Kaur 1, Amanpreet Kaur 2, Sarabjeet Kaur 3, Amandeep Kaur 4, Ranbir Kaur 5 Assistant Prof., Deptt. Of Computer Science,
More informationMDM and Data Warehousing Complement Each Other
Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There
More informationIII JORNADAS DE DATA MINING
III JORNADAS DE DATA MINING EN EL MARCO DE LA MAESTRÍA EN DATA MINING DE LA UNIVERSIDAD AUSTRAL PRESENTACIÓN TECNOLÓGICA IBM Alan Schcolnik, Cognos Technical Sales Team Leader, IBM Software Group. IAE
More informationA STUDY OF DATA MINING ACTIVITIES FOR MARKET RESEARCH
205 A STUDY OF DATA MINING ACTIVITIES FOR MARKET RESEARCH ABSTRACT MR. HEMANT KUMAR*; DR. SARMISTHA SARMA** *Assistant Professor, Department of Information Technology (IT), Institute of Innovation in Technology
More informationCOMP3420: Advanced Databases and Data Mining. Classification and prediction: Introduction and Decision Tree Induction
COMP3420: Advanced Databases and Data Mining Classification and prediction: Introduction and Decision Tree Induction Lecture outline Classification versus prediction Classification A two step process Supervised
More informationData Mining Techniques Chapter 6: Decision Trees
Data Mining Techniques Chapter 6: Decision Trees What is a classification decision tree?.......................................... 2 Visualizing decision trees...................................................
More informationHexaware E-book on Predictive Analytics
Hexaware E-book on Predictive Analytics Business Intelligence & Analytics Actionable Intelligence Enabled Published on : Feb 7, 2012 Hexaware E-book on Predictive Analytics What is Data mining? Data mining,
More informationImportance or the Role of Data Warehousing and Data Mining in Business Applications
Journal of The International Association of Advanced Technology and Science Importance or the Role of Data Warehousing and Data Mining in Business Applications ATUL ARORA ANKIT MALIK Abstract Information
More informationData Mining. Vera Goebel. Department of Informatics, University of Oslo
Data Mining Vera Goebel Department of Informatics, University of Oslo 2011 1 Lecture Contents Knowledge Discovery in Databases (KDD) Definition and Applications OLAP Architectures for OLAP and KDD KDD
More informationData Mining for Knowledge Management. Classification
1 Data Mining for Knowledge Management Classification Themis Palpanas University of Trento http://disi.unitn.eu/~themis Data Mining for Knowledge Management 1 Thanks for slides to: Jiawei Han Eamonn Keogh
More informationFluency With Information Technology CSE100/IMT100
Fluency With Information Technology CSE100/IMT100 ),7 Larry Snyder & Mel Oyler, Instructors Ariel Kemp, Isaac Kunen, Gerome Miklau & Sean Squires, Teaching Assistants University of Washington, Autumn 1999
More informationUsing reporting and data mining techniques to improve knowledge of subscribers; applications to customer profiling and fraud management
Using reporting and data mining techniques to improve knowledge of subscribers; applications to customer profiling and fraud management Paper Jean-Louis Amat Abstract One of the main issues of operators
More informationA Review of Data Mining Techniques
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 4, April 2014,
More informationTOWARDS SIMPLE, EASY TO UNDERSTAND, AN INTERACTIVE DECISION TREE ALGORITHM
TOWARDS SIMPLE, EASY TO UNDERSTAND, AN INTERACTIVE DECISION TREE ALGORITHM Thanh-Nghi Do College of Information Technology, Cantho University 1 Ly Tu Trong Street, Ninh Kieu District Cantho City, Vietnam
More informationSQL Server 2005 Features Comparison
Page 1 of 10 Quick Links Home Worldwide Search Microsoft.com for: Go : Home Product Information How to Buy Editions Learning Downloads Support Partners Technologies Solutions Community Previous Versions
More informationData Mining Techniques
15.564 Information Technology I Business Intelligence Outline Operational vs. Decision Support Systems What is Data Mining? Overview of Data Mining Techniques Overview of Data Mining Process Data Warehouses
More informationData Mining: An Introduction
Data Mining: An Introduction Michael J. A. Berry and Gordon A. Linoff. Data Mining Techniques for Marketing, Sales and Customer Support, 2nd Edition, 2004 Data mining What promotions should be targeted
More informationThe basic data mining algorithms introduced may be enhanced in a number of ways.
DATA MINING TECHNOLOGIES AND IMPLEMENTATIONS The basic data mining algorithms introduced may be enhanced in a number of ways. Data mining algorithms have traditionally assumed data is memory resident,
More informationWhat is Customer Relationship Management? Customer Relationship Management Analytics. Customer Life Cycle. Objectives of CRM. Three Types of CRM
Relationship Management Analytics What is Relationship Management? CRM is a strategy which utilises a combination of Week 13: Summary information technology policies processes, employees to develop profitable
More informationData Mining for Successful Healthcare Organizations
Data Mining for Successful Healthcare Organizations For successful healthcare organizations, it is important to empower the management and staff with data warehousing-based critical thinking and knowledge
More informationData Mining with SAS. Mathias Lanner mathias.lanner@swe.sas.com. Copyright 2010 SAS Institute Inc. All rights reserved.
Data Mining with SAS Mathias Lanner mathias.lanner@swe.sas.com Copyright 2010 SAS Institute Inc. All rights reserved. Agenda Data mining Introduction Data mining applications Data mining techniques SEMMA
More informationChapter ML:XI. XI. Cluster Analysis
Chapter ML:XI XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster
More information2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist
2015 Analyst and Advisor Summit Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist Agenda Key Facts Offerings and Capabilities Case Studies When to Engage
More informationOracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc.
Oracle9i Data Warehouse Review Robert F. Edwards Dulcian, Inc. Agenda Oracle9i Server OLAP Server Analytical SQL Data Mining ETL Warehouse Builder 3i Oracle 9i Server Overview 9i Server = Data Warehouse
More informationDATA MINING TECHNOLOGY. Keywords: data mining, data warehouse, knowledge discovery, OLAP, OLAM.
DATA MINING TECHNOLOGY Georgiana Marin 1 Abstract In terms of data processing, classical statistical models are restrictive; it requires hypotheses, the knowledge and experience of specialists, equations,
More informationHealthcare Measurement Analysis Using Data mining Techniques
www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 03 Issue 07 July, 2014 Page No. 7058-7064 Healthcare Measurement Analysis Using Data mining Techniques 1 Dr.A.Shaik
More informationData Mining Classification: Decision Trees
Data Mining Classification: Decision Trees Classification Decision Trees: what they are and how they work Hunt s (TDIDT) algorithm How to select the best split How to handle Inconsistent data Continuous
More informationNew Approach of Computing Data Cubes in Data Warehousing
International Journal of Information & Computation Technology. ISSN 0974-2239 Volume 4, Number 14 (2014), pp. 1411-1417 International Research Publications House http://www. irphouse.com New Approach of
More informationThe Scientific Data Mining Process
Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In
More informationNine Common Types of Data Mining Techniques Used in Predictive Analytics
1 Nine Common Types of Data Mining Techniques Used in Predictive Analytics By Laura Patterson, President, VisionEdge Marketing Predictive analytics enable you to develop mathematical models to help better
More informationClass 10. Data Mining and Artificial Intelligence. Data Mining. We are in the 21 st century So where are the robots?
Class 1 Data Mining Data Mining and Artificial Intelligence We are in the 21 st century So where are the robots? Data mining is the one really successful application of artificial intelligence technology.
More informationHarnessing the power of advanced analytics with IBM Netezza
IBM Software Information Management White Paper Harnessing the power of advanced analytics with IBM Netezza How an appliance approach simplifies the use of advanced analytics Harnessing the power of advanced
More information131-1. Adding New Level in KDD to Make the Web Usage Mining More Efficient. Abstract. 1. Introduction [1]. 1/10
1/10 131-1 Adding New Level in KDD to Make the Web Usage Mining More Efficient Mohammad Ala a AL_Hamami PHD Student, Lecturer m_ah_1@yahoocom Soukaena Hassan Hashem PHD Student, Lecturer soukaena_hassan@yahoocom
More informationHow To Perform An Ensemble Analysis
Charu C. Aggarwal IBM T J Watson Research Center Yorktown, NY 10598 Outlier Ensembles Keynote, Outlier Detection and Description Workshop, 2013 Based on the ACM SIGKDD Explorations Position Paper: Outlier
More informationChapter 12 Discovering New Knowledge Data Mining
Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to
More informationECLT 5810 E-Commerce Data Mining Techniques - Introduction. Prof. Wai Lam
ECLT 5810 E-Commerce Data Mining Techniques - Introduction Prof. Wai Lam Data Opportunities Business infrastructure have improved the ability to collect data Virtually every aspect of business is now open
More informationMobile Phone APP Software Browsing Behavior using Clustering Analysis
Proceedings of the 2014 International Conference on Industrial Engineering and Operations Management Bali, Indonesia, January 7 9, 2014 Mobile Phone APP Software Browsing Behavior using Clustering Analysis
More informationOracle Real Time Decisions
A Product Review James Taylor CEO CONTENTS Introducing Decision Management Systems Oracle Real Time Decisions Product Architecture Key Features Availability Conclusion Oracle Real Time Decisions (RTD)
More informationIntegrated Data Mining and Knowledge Discovery Techniques in ERP
Integrated Data Mining and Knowledge Discovery Techniques in ERP I Gandhimathi Amirthalingam, II Rabia Shaheen, III Mohammad Kousar, IV Syeda Meraj Bilfaqih I,III,IV Dept. of Computer Science, King Khalid
More informationDATA MINING METHODS WITH TREES
DATA MINING METHODS WITH TREES Marta Žambochová 1. Introduction The contemporary world is characterized by the explosion of an enormous volume of data deposited into databases. Sharp competition contributes
More informationData Mining as Part of Knowledge Discovery in Databases (KDD)
Mining as Part of Knowledge Discovery in bases (KDD) Presented by Naci Akkøk as part of INF4180/3180, Advanced base Systems, fall 2003 (based on slightly modified foils of Dr. Denise Ecklund from 6 November
More informationData are everywhere. IBM projects that every day we generate 2.5 quintillion bytes of data. In relative terms, this means 90
FREE echapter C H A P T E R1 Big Data and Analytics Data are everywhere. IBM projects that every day we generate 2.5 quintillion bytes of data. In relative terms, this means 90 percent of the data in the
More informationData Refinery with Big Data Aspects
International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data
More informationData Science & Big Data Practice
INSIGHTS ANALYTICS INNOVATIONS Data Science & Big Data Practice Customer Intelligence - 360 Insight Amplify customer insight by integrating enterprise data with external data Customer Intelligence 360
More informationA Near Real-Time Personalization for ecommerce Platform Amit Rustagi arustagi@ebay.com
A Near Real-Time Personalization for ecommerce Platform Amit Rustagi arustagi@ebay.com Abstract. In today's competitive environment, you only have a few seconds to help site visitors understand that you
More informationData Mining: Motivations and Concepts
POLYTECHNIC UNIVERSITY Department of Computer Science / Finance and Risk Engineering Data Mining: Motivations and Concepts K. Ming Leung Abstract: We discuss here the need, the goals, and the primary tasks
More informationHow Organisations Are Using Data Mining Techniques To Gain a Competitive Advantage John Spooner SAS UK
How Organisations Are Using Data Mining Techniques To Gain a Competitive Advantage John Spooner SAS UK Agenda Analytics why now? The process around data and text mining Case Studies The Value of Information
More informationWhite Paper. Data Mining for Business
White Paper Data Mining for Business January 2010 Contents 1. INTRODUCTION... 3 2. WHY IS DATA MINING IMPORTANT?... 3 FUNDAMENTALS... 3 Example 1...3 Example 2...3 3. OPERATIONAL CONSIDERATIONS... 4 ORGANISATIONAL
More informationData Mart/Warehouse: Progress and Vision
Data Mart/Warehouse: Progress and Vision Institutional Research and Planning University Information Systems What is data warehousing? A data warehouse: is a single place that contains complete, accurate
More informationDiscovering, Not Finding. Practical Data Mining for Practitioners: Level II. Advanced Data Mining for Researchers : Level III
www.cognitro.com/training Predicitve DATA EMPOWERING DECISIONS Data Mining & Predicitve Training (DMPA) is a set of multi-level intensive courses and workshops developed by Cognitro team. it is designed
More information