Roadmap DB Sys. Design & Impl. Citation. Detailed Outline. Data Ware-housing. Problem. Data Cubes. Christos Faloutsos
|
|
- Damian Oswin Daniels
- 7 years ago
- Views:
Transcription
1 572 DB Sys. Design & mpl. Data Cubes Christos Faloutsos Roadmap ) Roots: System R and ngres 2) mplementation: buffering, indexing, qopt 3) Transactions: locking, recovery 4) Distributed DBMSs 5) Parallel DBMSs: Gamma, Alphasort 6) OO/OR DBMS 7) Data Analysis data mining data cubes association rules 8) Benchmarks 9) vision statements extras (streams/sensors, graphs, multimedia, web, fractals) 572 C. Faloutsos 2 Detailed Outline Problem Getting the data: Data Warehouses,, OLAP Supervised learning: decision trees Unsupervised learning association rules (clustering) Citation Gray, et al.: "Data Cube: A Relational Aggregation Operator Generalizing Groupby, CrossTab, and Sub Totals." Data Mining and Knowledge Discovery (): 2953 (997) 572 C. Faloutsos C. Faloutsos 4 Problem Given: multiple data sources Find: patterns (classifiers, rules, clusters, outliers...) PGH NY SF sales(pid, cid, date, $price) customers( cid, age, income,...)??? 572 C. Faloutsos 5 Data Warehousing First step: collect the data, in a single place (= Data Warehouse) How? How often? How about discrepancies / nonhomegeneities? 572 C. Faloutsos 6
2 Data Warehousing First step: collect the data, in a single place (= Data Warehouse) How? A: Triggers/Materialized views How often? A: [Art!] How about discrepancies / nonhomegeneities? A: Wrappers/Mediators Data Warehousing Step 2: collect counts. (/OLAP) Eg.: 572 C. Faloutsos C. Faloutsos 8 sales OLAP Problem: is it true that shirts in large s sell better in dark s? cid pid Size Color $ C0 Shirt L Blue 30 C0 Pants XL Red 50 C20 Shirt XL White C. Faloutsos 9, : DMENSONS count : MEASURE 572 C. Faloutsos 0, : DMENSONS count : MEASURE, : DMENSONS count : MEASURE 572 C. Faloutsos 572 C. Faloutsos 2 2
3 , : DMENSONS count : MEASURE, : DMENSONS count : MEASURE 572 C. Faloutsos C. Faloutsos 4, : DMENSONS count : MEASURE DataCube 572 C. Faloutsos 5 SQL query to generate DataCube: Naively (and painfully:) select,, count(*) from sales where pid = shirt group by, select, count(*) from sales where pid = shirt group by C. Faloutsos 6 SQL query to generate DataCube: with cube by keyword: select,, count(*) from sales where pid = shirt cube by, DataCube issues: Q: How to store them (and/or materialize portions on demand) Q2: How to index them Q3: Which operations to allow 572 C. Faloutsos C. Faloutsos 8 3
4 DataCube issues: Q: How to store them (and/or materialize portions on demand) A: ROLAP/MOLAP Q2: How to index them A: bitmap indices Q3: Which operations to allow A: rollup, drill down, slice, dice [More details: book by HanKamber] Q: How to store a datacube? 572 C. Faloutsos C. Faloutsos 20 Q: How to store a datacube? A: Relational (ROLAP) Color Size count all all 47 Blue all 4 Blue M 3 Q: How to store a datacube? A2: Multidimensional (MOLAP) A3: Hybrid (HOLAP) 572 C. Faloutsos C. Faloutsos 22 Pros/Cons: ROLAP strong points: (DSS, Metacube) Pros/Cons: ROLAP strong points: (DSS, Metacube) use existing RDBMS technology scale up better with dimensionality 572 C. Faloutsos C. Faloutsos 24 4
5 Pros/Cons: MOLAP strong points: (EssBase/hyperion.com) faster indexing (careful with: highdimensionality; sparseness) Q: How to store a datacube Q3: How to index a datacube? HOLAP: (MS SQL server OLAP services) detail data in ROLAP; summaries in MOLAP 572 C. Faloutsos C. Faloutsos 26 Rollup 572 C. Faloutsos C. Faloutsos 28 Drilldown Slice 572 C. Faloutsos C. Faloutsos 30 5
6 Dice Rollup Drilldown Slice Dice 572 C. Faloutsos C. Faloutsos 32 Q: How to store a datacube Q3: How to index a datacube? Q3: How to index a datacube? 572 C. Faloutsos C. Faloutsos 34 Q3: How to index a datacube? A: Bitmaps S M L Red Blue Gray Q3: How to index a datacube? A2: Join indices (see [HanKamber]) 572 C. Faloutsos C. Faloutsos 36 6
7 D/W OLAP Conclusions D/W: copy (summarized) data analyze OLAP concepts: DataCube R/M/HOLAP servers dimensions ; measures Outline Problem Getting the data: Data Warehouses,, OLAP Supervised learning: decision trees Unsupervised learning association rules (clustering) 572 C. Faloutsos C. Faloutsos 38 Decision trees Problem Age Chollevel Gender CLASSD M?? 572 C. Faloutsos 39 Pictorially, we have num. attr#2 (eg., chollevel) Decision trees num. attr# (eg., age ) 572 C. Faloutsos 40 Decision trees and we want to label? Decision trees so we build a decision tree: num. attr#2 (eg., chollevel)? num. attr#2 (eg., chollevel) 40? num. attr# (eg., age ) 572 C. Faloutsos 4 50 num. attr# (eg., age ) 572 C. Faloutsos 42 7
8 Decision trees so we build a decision tree: age<50 Y N chol. <40 Y N C. Faloutsos 43 Outline Problem Getting the data: Data Warehouses,, OLAP Supervised learning: decision trees problem approach scalability enhancements Unsupervised learning association rules (clustering) 572 C. Faloutsos 44 Decision trees Typically, two steps: tree building tree pruning (for overtraining/overfitting) How? num. attr#2 (eg., chollevel) num. attr# (eg., age ) 572 C. Faloutsos C. Faloutsos 46 How? A: Partition, recursively pseudocode: Partition ( Dataset S) if all points in S have same label then return evaluate splits along each attribute A pick best split, to divide S into S and S2 Partition(S); Partition(S2) 572 C. Faloutsos 47 Q: how to introduce splits along attribute A i Q2: how to evaluate a split? 572 C. Faloutsos 48 8
9 Q: how to introduce splits along attribute A i A: for num. attributes: binary split, or multiple split for categorical attributes: compute all subsets (expensive!), or use a greedy algo Q: how to introduce splits along attribute A i Q2: how to evaluate a split? 572 C. Faloutsos C. Faloutsos 50 Q: how to introduce splits along attribute A i Q2: how to evaluate a split? A: by how close to uniform each subset is ie., we need a measure of uniformity: entropy: H(p, p) Any other measure? p 572 C. Faloutsos C. Faloutsos 52 entropy: H(p, p ) gini index: p 2 p 2 entropy: H(p, p ) gini index: p 2 p p p (How about multiple labels?) 572 C. Faloutsos C. Faloutsos 54 9
10 ntuition: entropy: #bits to encode the class label gini: classification error, if we randomly guess with prob. p 572 C. Faloutsos 55 Thus, we choose the split that reduces entropy/classificationerror the most: Eg.: num. attr#2 (eg., chollevel) num. attr# (eg., age ) 572 C. Faloutsos 56 Before split: we need (n n ) * H( p, p ) = (76) * H(7/3, 6/3) bits total, to encode all the class labels After the split we need: 0 bits for the first half and (26) * H(2/8, 6/8) bits for the second half What for? num. attr#2 (eg., chollevel) Tree pruning num. attr# (eg., age ) C. Faloutsos C. Faloutsos 58 Tree pruning Shortcut for scalability: DYNAMC pruning: stop expanding the tree, if a node is reasonably homogeneous ad hoc threshold [Agrawal, vldb92] Minimum Description Language (MDL) criterion (SLQ) [Mehta, edbt96] Tree pruning Q: How to do it? A: use a training and a testing set prune nodes that improve classification in the testing set. (Drawbacks?) A2: or, rely on MDL (= Minimum Description Language) in detail: 572 C. Faloutsos C. Faloutsos 60 0
11 Tree pruning envision the problem as compression (of what?) Tree pruning envision the problem as compression (of what?) and try to min. the # bits to compress (a) the class labels AND (b) the representation of the decision tree 572 C. Faloutsos C. Faloutsos 62 (MDL) a brilliant idea eg.: best ndegree polynomial to compress these points: the one that minimizes (sum of errors n ) 572 C. Faloutsos 63 Outline Problem Getting the data: Data Warehouses,, OLAP Supervised learning: decision trees problem approach scalability enhancements Unsupervised learning association rules (clustering) 572 C. Faloutsos 64 Scalability enhancements nterval Classifier [Agrawal,vldb92]: dynamic pruning SLQ: dynamic pruning with MDL; vertical partitioning of the file (but label column has to fit in core) SPRNT: even more clever partitioning Conclusions for classifiers Classification through trees Building phase splitting policies Pruning phase (to avoid overfitting) For scalability: dynamic pruning clever data partitioning 572 C. Faloutsos C. Faloutsos 66
12 Overall Conclusions Data Mining: of high commercial interest DM = DB ML Stat Data warehousing / OLAP: to get the data Tree classifiers (SLQ, SPRNT) Association Rules apriori algorithm (clustering: BRCH, CURE, OPTCS) Reading material Agrawal, R., T. mielinski, A. Swami, Mining Association Rules between Sets of tems in Large Databases, SGMOD M. Mehta, R. Agrawal and J. Rissanen, `SLQ: A Fast Scalable Classifier for Data Mining, Proc. of the Fifth nt'l Conference on Extending Database Technology (EDBT), Avignon, France, March C. Faloutsos C. Faloutsos 68 Additional references Agrawal, R., S. Ghosh, et al. (Aug. 2327, 992). An nterval Classifier for Database Mining Applications. VLDB Conf. Proc., Vancouver, BC, Canada. Jiawei Han and Micheline Kamber, Data Mining, Morgan Kaufman, 200, chapters , 6.6.2, C. Faloutsos 69 2
A Technical Review on On-Line Analytical Processing (OLAP)
A Technical Review on On-Line Analytical Processing (OLAP) K. Jayapriya 1., E. Girija 2,III-M.C.A., R.Uma. 3,M.C.A.,M.Phil., Department of computer applications, Assit.Prof,Dept of M.C.A, Dhanalakshmi
More informationOLAP & DATA MINING CS561-SPRING 2012 WPI, MOHAMED ELTABAKH
OLAP & DATA MINING CS561-SPRING 2012 WPI, MOHAMED ELTABAKH 1 Online Analytic Processing OLAP 2 OLAP OLAP: Online Analytic Processing OLAP queries are complex queries that Touch large amounts of data Discover
More informationDATA WAREHOUSING - OLAP
http://www.tutorialspoint.com/dwh/dwh_olap.htm DATA WAREHOUSING - OLAP Copyright tutorialspoint.com Online Analytical Processing Server OLAP is based on the multidimensional data model. It allows managers,
More informationClassification and Prediction
Classification and Prediction Slides for Data Mining: Concepts and Techniques Chapter 7 Jiawei Han and Micheline Kamber Intelligent Database Systems Research Lab School of Computing Science Simon Fraser
More informationDATA WAREHOUSING AND OLAP TECHNOLOGY
DATA WAREHOUSING AND OLAP TECHNOLOGY Manya Sethi MCA Final Year Amity University, Uttar Pradesh Under Guidance of Ms. Shruti Nagpal Abstract DATA WAREHOUSING and Online Analytical Processing (OLAP) are
More informationBuilding Data Cubes and Mining Them. Jelena Jovanovic Email: jeljov@fon.bg.ac.yu
Building Data Cubes and Mining Them Jelena Jovanovic Email: jeljov@fon.bg.ac.yu KDD Process KDD is an overall process of discovering useful knowledge from data. Data mining is a particular step in the
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Fall 2007 Lecture 16 - Data Warehousing
CSE 544 Principles of Database Management Systems Magdalena Balazinska Fall 2007 Lecture 16 - Data Warehousing Class Projects Class projects are going very well! Project presentations: 15 minutes On Wednesday
More informationData W a Ware r house house and and OLAP II Week 6 1
Data Warehouse and OLAP II Week 6 1 Team Homework Assignment #8 Using a data warehousing tool and a data set, play four OLAP operations (Roll up (drill up), Drill down (roll down), Slice and dice, Pivot
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Winter 2009 Lecture 15 - Data Warehousing: Cubes
CSE 544 Principles of Database Management Systems Magdalena Balazinska Winter 2009 Lecture 15 - Data Warehousing: Cubes Final Exam Overview Open books and open notes No laptops and no other mobile devices
More informationII. OLAP(ONLINE ANALYTICAL PROCESSING)
Association Rule Mining Method On OLAP Cube Jigna J. Jadav*, Mahesh Panchal** *( PG-CSE Student, Department of Computer Engineering, Kalol Institute of Technology & Research Centre, Gujarat, India) **
More informationLearning Objectives. Definition of OLAP Data cubes OLAP operations MDX OLAP servers
OLAP Learning Objectives Definition of OLAP Data cubes OLAP operations MDX OLAP servers 2 What is OLAP? OLAP has two immediate consequences: online part requires the answers of queries to be fast, the
More informationMulti-dimensional index structures Part I: motivation
Multi-dimensional index structures Part I: motivation 144 Motivation: Data Warehouse A definition A data warehouse is a repository of integrated enterprise data. A data warehouse is used specifically for
More informationOLAP and Data Mining. Data Warehousing and End-User Access Tools. Introducing OLAP. Introducing OLAP
Data Warehousing and End-User Access Tools OLAP and Data Mining Accompanying growth in data warehouses is increasing demands for more powerful access tools providing advanced analytical capabilities. Key
More informationData Mining: Concepts and Techniques. Jiawei Han. Micheline Kamber. Simon Fräser University К MORGAN KAUFMANN PUBLISHERS. AN IMPRINT OF Elsevier
Data Mining: Concepts and Techniques Jiawei Han Micheline Kamber Simon Fräser University К MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF Elsevier Contents Foreword Preface xix vii Chapter I Introduction I I.
More informationCHAPTER 4 Data Warehouse Architecture
CHAPTER 4 Data Warehouse Architecture 4.1 Data Warehouse Architecture 4.2 Three-tier data warehouse architecture 4.3 Types of OLAP servers: ROLAP versus MOLAP versus HOLAP 4.4 Further development of Data
More informationMauro Sousa Marta Mattoso Nelson Ebecken. and these techniques often repeatedly scan the. entire set. A solution that has been used for a
Data Mining on Parallel Database Systems Mauro Sousa Marta Mattoso Nelson Ebecken COPPEèUFRJ - Federal University of Rio de Janeiro P.O. Box 68511, Rio de Janeiro, RJ, Brazil, 21945-970 Fax: +55 21 2906626
More informationOLAP. Business Intelligence OLAP definition & application Multidimensional data representation
OLAP Business Intelligence OLAP definition & application Multidimensional data representation 1 Business Intelligence Accompanying the growth in data warehousing is an ever-increasing demand by users for
More informationCOMP3420: Advanced Databases and Data Mining. Classification and prediction: Introduction and Decision Tree Induction
COMP3420: Advanced Databases and Data Mining Classification and prediction: Introduction and Decision Tree Induction Lecture outline Classification versus prediction Classification A two step process Supervised
More informationAnalyzing Polls and News Headlines Using Business Intelligence Techniques
Analyzing Polls and News Headlines Using Business Intelligence Techniques Eleni Fanara, Gerasimos Marketos, Nikos Pelekis and Yannis Theodoridis Department of Informatics, University of Piraeus, 80 Karaoli-Dimitriou
More informationApplication of Data Warehouse and Data Mining. in Construction Management
Application of Data Warehouse and Data Mining in Construction Management Jianping ZHANG 1 (zhangjp@tsinghua.edu.cn) Tianyi MA 1 (matianyi97@mails.tsinghua.edu.cn) Qiping SHEN 2 (bsqpshen@inet.polyu.edu.hk)
More informationData Mining for Knowledge Management. Classification
1 Data Mining for Knowledge Management Classification Themis Palpanas University of Trento http://disi.unitn.eu/~themis Data Mining for Knowledge Management 1 Thanks for slides to: Jiawei Han Eamonn Keogh
More informationDBTech Pro Workshop. Knowledge Discovery from Databases (KDD) Including Data Warehousing and Data Mining. Georgios Evangelidis
DBTechNet DBTech Pro Workshop Knowledge Discovery from Databases (KDD) Including Data Warehousing and Data Mining Dimitris A. Dervos dad@it.teithe.gr http://aetos.it.teithe.gr/~dad Georgios Evangelidis
More informationOLAP Systems and Multidimensional Expressions I
OLAP Systems and Multidimensional Expressions I Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master
More informationDATA WAREHOUSE E KNOWLEDGE DISCOVERY
DATA WAREHOUSE E KNOWLEDGE DISCOVERY Prof. Fabio A. Schreiber Dipartimento di Elettronica e Informazione Politecnico di Milano DATA WAREHOUSE (DW) A TECHNIQUE FOR CORRECTLY ASSEMBLING AND MANAGING DATA
More informationCourse 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing
More informationWeb Log Data Sparsity Analysis and Performance Evaluation for OLAP
Web Log Data Sparsity Analysis and Performance Evaluation for OLAP Ji-Hyun Kim, Hwan-Seung Yong Department of Computer Science and Engineering Ewha Womans University 11-1 Daehyun-dong, Seodaemun-gu, Seoul,
More informationMain Memory & Near Main Memory OLAP Databases. Wo Shun Luk Professor of Computing Science Simon Fraser University
Main Memory & Near Main Memory OLAP Databases Wo Shun Luk Professor of Computing Science Simon Fraser University 1 Outline What is OLAP DB? How does it work? MOLAP, ROLAP Near Main Memory DB Partial Pre
More informationAnwendersoftware Anwendungssoftwares a. Data-Warehouse-, Data-Mining- and OLAP-Technologies. Online Analytic Processing
Anwendungssoftwares a Data-Warehouse-, Data-Mining- and OLAP-Technologies Online Analytic Processing Online Analytic Processing OLAP Online Analytic Processing Technologies and tools that support (ad-hoc)
More informationDatabase Applications. Advanced Querying. Transaction Processing. Transaction Processing. Data Warehouse. Decision Support. Transaction processing
Database Applications Advanced Querying Transaction processing Online setting Supports day-to-day operation of business OLAP Data Warehousing Decision support Offline setting Strategic planning (statistics)
More informationChapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
More informationPart 22. Data Warehousing
Part 22 Data Warehousing The Decision Support System (DSS) Tools to assist decision-making Used at all levels in the organization Sometimes focused on a single area Sometimes focused on a single problem
More informationData Warehouse design
Data Warehouse design Design of Enterprise Systems University of Pavia 21/11/2013-1- Data Warehouse design DATA PRESENTATION - 2- BI Reporting Success Factors BI platform success factors include: Performance
More informationReview. Data Warehousing. Today. Star schema. Star join indexes. Dimension hierarchies
Review Data Warehousing CPS 216 Advanced Database Systems Data warehousing: integrating data for OLAP OLAP versus OLTP Warehousing versus mediation Warehouse maintenance Warehouse data as materialized
More informationData Mining and Database Systems: Where is the Intersection?
Data Mining and Database Systems: Where is the Intersection? Surajit Chaudhuri Microsoft Research Email: surajitc@microsoft.com 1 Introduction The promise of decision support systems is to exploit enterprise
More informationCopyright 2007 Ramez Elmasri and Shamkant B. Navathe. Slide 29-1
Slide 29-1 Chapter 29 Overview of Data Warehousing and OLAP Chapter 29 Outline Purpose of Data Warehousing Introduction, Definitions, and Terminology Comparison with Traditional Databases Characteristics
More informationApplication Tool for Experiments on SQL Server 2005 Transactions
Proceedings of the 5th WSEAS Int. Conf. on DATA NETWORKS, COMMUNICATIONS & COMPUTERS, Bucharest, Romania, October 16-17, 2006 30 Application Tool for Experiments on SQL Server 2005 Transactions ŞERBAN
More informationData Mining as Part of Knowledge Discovery in Databases (KDD)
Mining as Part of Knowledge Discovery in bases (KDD) Presented by Naci Akkøk as part of INF4180/3180, Advanced base Systems, fall 2003 (based on slightly modified foils of Dr. Denise Ecklund from 6 November
More informationData Warehouse: Introduction
Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,
More information2074 : Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000
2074 : Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000 Introduction This course provides students with the knowledge and skills necessary to design, implement, and deploy OLAP
More informationDATA CUBES E0 261. Jayant Haritsa Computer Science and Automation Indian Institute of Science. JAN 2014 Slide 1 DATA CUBES
E0 261 Jayant Haritsa Computer Science and Automation Indian Institute of Science JAN 2014 Slide 1 Introduction Increasingly, organizations are analyzing historical data to identify useful patterns and
More information1. What are the uses of statistics in data mining? Statistics is used to Estimate the complexity of a data mining problem. Suggest which data mining
1. What are the uses of statistics in data mining? Statistics is used to Estimate the complexity of a data mining problem. Suggest which data mining techniques are most likely to be successful, and Identify
More informationWeek 3 lecture slides
Week 3 lecture slides Topics Data Warehouses Online Analytical Processing Introduction to Data Cubes Textbook reference: Chapter 3 Data Warehouses A data warehouse is a collection of data specifically
More informationDecision Trees from large Databases: SLIQ
Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values
More informationData Mining Jargon. Bob Muenchen The Statistical Consulting Center
Data Mining Jargon Bob Muenchen The Statistical Consulting Center Data mining is the automated search for useful patterns in data. It uses tools from many different disciplines, each of which uses its
More information(b) How data mining is different from knowledge discovery in databases (KDD)? Explain.
Q2. (a) List and describe the five primitives for specifying a data mining task. Data Mining Task Primitives (b) How data mining is different from knowledge discovery in databases (KDD)? Explain. IETE
More informationData Mining Algorithms Part 1. Dejan Sarka
Data Mining Algorithms Part 1 Dejan Sarka Join the conversation on Twitter: @DevWeek #DW2015 Instructor Bio Dejan Sarka (dsarka@solidq.com) 30 years of experience SQL Server MVP, MCT, 13 books 7+ courses
More informationData Warehousing, OLAP, and Data Mining
Data Warehousing, OLAP, and Marek Rychly mrychly@strathmore.edu Strathmore University, @ilabafrica & Brno University of Technology, Faculty of Information Technology Advanced Databases and Enterprise Systems
More informationOutline. Data Warehousing. What is a Warehouse? What is a Warehouse?
Outline Data Warehousing What is a data warehouse? Why a warehouse? Models & operations Implementing a warehouse 2 What is a Warehouse? Collection of diverse data subject oriented aimed at executive, decision
More informationChapter 5. Warehousing, Data Acquisition, Data. Visualization
Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization 5-1 Learning Objectives
More informationDecision Support. Chapter 23. Database Management Systems, 2 nd Edition. R. Ramakrishnan and J. Gehrke 1
Decision Support Chapter 23 Database Management Systems, 2 nd Edition. R. Ramakrishnan and J. Gehrke 1 Introduction Increasingly, organizations are analyzing current and historical data to identify useful
More informationData Mining. Vera Goebel. Department of Informatics, University of Oslo
Data Mining Vera Goebel Department of Informatics, University of Oslo 2011 1 Lecture Contents Knowledge Discovery in Databases (KDD) Definition and Applications OLAP Architectures for OLAP and KDD KDD
More information1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing
1. OLAP is an acronym for a. Online Analytical Processing b. Online Analysis Process c. Online Arithmetic Processing d. Object Linking and Processing 2. What is a Data warehouse a. A database application
More informationThe Cubetree Storage Organization
The Cubetree Storage Organization Nick Roussopoulos & Yannis Kotidis Advanced Communication Technology, Inc. Silver Spring, MD 20905 Tel: 301-384-3759 Fax: 301-384-3679 {nick,kotidis}@act-us.com 1. Introduction
More informationA DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT
A DATA WAREHOUSE SOLUTION FOR E-GOVERNMENT Xiufeng Liu 1 & Xiaofeng Luo 2 1 Department of Computer Science Aalborg University, Selma Lagerlofs Vej 300, DK-9220 Aalborg, Denmark 2 Telecommunication Engineering
More informationOn-Line Application Processing. Warehousing Data Cubes Data Mining
On-Line Application Processing Warehousing Data Cubes Data Mining 1 Overview Traditional database systems are tuned to many, small, simple queries. Some new applications use fewer, more time-consuming,
More informationIntroduction to Data Mining
Introduction to Data Mining Jay Urbain Credits: Nazli Goharian & David Grossman @ IIT Outline Introduction Data Pre-processing Data Mining Algorithms Naïve Bayes Decision Tree Neural Network Association
More informationData Warehousing: Data Models and OLAP operations. By Kishore Jaladi kishorejaladi@yahoo.com
Data Warehousing: Data Models and OLAP operations By Kishore Jaladi kishorejaladi@yahoo.com Topics Covered 1. Understanding the term Data Warehousing 2. Three-tier Decision Support Systems 3. Approaches
More informationM2074 - Designing and Implementing OLAP Solutions Using Microsoft SQL Server 2000 5 Day Course
Module 1: Introduction to Data Warehousing and OLAP Introducing Data Warehousing Defining OLAP Solutions Understanding Data Warehouse Design Understanding OLAP Models Applying OLAP Cubes At the end of
More informationA Critical Review of Data Warehouse
Global Journal of Business Management and Information Technology. Volume 1, Number 2 (2011), pp. 95-103 Research India Publications http://www.ripublication.com A Critical Review of Data Warehouse Sachin
More informationData Warehousing and Decision Support. Introduction. Three Complementary Trends. Chapter 23, Part A
Data Warehousing and Decision Support Chapter 23, Part A Database Management Systems, 2 nd Edition. R. Ramakrishnan and J. Gehrke 1 Introduction Increasingly, organizations are analyzing current and historical
More informationWeek 13: Data Warehousing. Warehousing
1 Week 13: Data Warehousing Warehousing Growing industry: $8 billion in 1998 Range from desktop to huge: Walmart: 900-CPU, 2,700 disk, 23TB Teradata system Lots of buzzwords, hype slice & dice, rollup,
More informationWhat is OLAP - On-line analytical processing
What is OLAP - On-line analytical processing Vladimir Estivill-Castro School of Computing and Information Technology With contributions for J. Han 1 Introduction When a company has received/accumulated
More informationUnit -3. Learning Objective. Demand for Online analytical processing Major features and functions OLAP models and implementation considerations
Unit -3 Learning Objective Demand for Online analytical processing Major features and functions OLAP models and implementation considerations Demand of On Line Analytical Processing Need for multidimensional
More informationUNIT-3 OLAP in Data Warehouse
UNIT-3 OLAP in Data Warehouse Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Dr.Deepali Kamthania U2.1 OLAP Demand for Online analytical processing Major features
More informationLearning Example. Machine learning and our focus. Another Example. An example: data (loan application) The data and the goal
Learning Example Chapter 18: Learning from Examples 22c:145 An emergency room in a hospital measures 17 variables (e.g., blood pressure, age, etc) of newly admitted patients. A decision is needed: whether
More informationBusiness Intelligence Solutions. Cognos BI 8. by Adis Terzić
Business Intelligence Solutions Cognos BI 8 by Adis Terzić Fairfax, Virginia August, 2008 Table of Content Table of Content... 2 Introduction... 3 Cognos BI 8 Solutions... 3 Cognos 8 Components... 3 Cognos
More informationData Warehousing. Outline. From OLTP to the Data Warehouse. Overview of data warehousing Dimensional Modeling Online Analytical Processing
Data Warehousing Outline Overview of data warehousing Dimensional Modeling Online Analytical Processing From OLTP to the Data Warehouse Traditionally, database systems stored data relevant to current business
More informationCS2032 Data warehousing and Data Mining Unit II Page 1
UNIT II BUSINESS ANALYSIS Reporting Query tools and Applications The data warehouse is accessed using an end-user query and reporting tool from Business Objects. Business Objects provides several tools
More informationDecision Tree Induction in High Dimensional, Hierarchically Distributed Databases
Decision Tree Induction in High Dimensional, Hierarchically Distributed Databases Amir Bar-Or, Assaf Schuster, Ran Wolff Faculty of Computer Science Technion, Israel {abaror, assaf, ranw}@cs.technion.ac.il
More informationPaintingClass: Interactive Construction, Visualization and Exploration of Decision Trees
PaintingClass: Interactive Construction, Visualization and Exploration of Decision Trees Soon Tee Teoh Department of Computer Science University of California, Davis teoh@cs.ucdavis.edu Kwan-Liu Ma Department
More informationA Dynamic Load Balancing Strategy for Parallel Datacube Computation
A Dynamic Load Balancing Strategy for Parallel Datacube Computation Seigo Muto Institute of Industrial Science, University of Tokyo 7-22-1 Roppongi, Minato-ku, Tokyo, 106-8558 Japan +81-3-3402-6231 ext.
More informationData Warehouse and OLAP. Methodologies, Algorithms, Trends
58 Data Warehouse and OLAP. Methodologies, Algorithms, Trends Radu LOVIN IT Consultant Tata Consultancy Services GE European Equipment Finance - Data Warehouse Project radu.lovin@ge.com On-Line Analytical
More informationScalable Classification over SQL Databases
Scalable Classification over SQL Databases Surajit Chaudhuri Usama Fayyad Jeff Bernhardt Microsoft Research Redmond, WA 98052, USA Email: {surajitc,fayyad, jeffbern}@microsoft.com Abstract We identify
More informationImproving Analysis Of Data Mining By Creating Dataset Using Sql Aggregations
International Refereed Journal of Engineering and Science (IRJES) ISSN (Online) 2319-183X, (Print) 2319-1821 Volume 1, Issue 3 (November 2012), PP.28-33 Improving Analysis Of Data Mining By Creating Dataset
More informationData Warehousing & OLAP
Data Warehousing & OLAP What is Data Warehouse? A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of management s decisionmaking process. W.
More informationHorizontal Aggregations in SQL to Prepare Data Sets for Data Mining Analysis
IOSR Journal of Computer Engineering (IOSRJCE) ISSN: 2278-0661, ISBN: 2278-8727 Volume 6, Issue 5 (Nov. - Dec. 2012), PP 36-41 Horizontal Aggregations in SQL to Prepare Data Sets for Data Mining Analysis
More informationData Warehousing and OLAP Technology for Knowledge Discovery
542 Data Warehousing and OLAP Technology for Knowledge Discovery Aparajita Suman Abstract Since time immemorial, libraries have been generating services using the knowledge stored in various repositories
More informationBussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University
Bussiness Intelligence and Data Warehouse Schedule Bussiness Intelligence (BI) BI tools Oracle vs. Microsoft Data warehouse History Tools Oracle vs. Others Discussion Business Intelligence (BI) Products
More informationDataset Preparation and Indexing for Data Mining Analysis Using Horizontal Aggregations
Dataset Preparation and Indexing for Data Mining Analysis Using Horizontal Aggregations Binomol George, Ambily Balaram Abstract To analyze data efficiently, data mining systems are widely using datasets
More informationData Warehousing. Paper 133-25
Paper 133-25 The Power of Hybrid OLAP in a Multidimensional World Ann Weinberger, SAS Institute Inc., Cary, NC Matthias Ender, SAS Institute Inc., Cary, NC ABSTRACT Version 8 of the SAS System brings powerful
More informationData Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining
Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary
More informationData Warehousing. Read chapter 13 of Riguzzi et al Sistemi Informativi. Slides derived from those by Hector Garcia-Molina
Data Warehousing Read chapter 13 of Riguzzi et al Sistemi Informativi Slides derived from those by Hector Garcia-Molina What is a Warehouse? Collection of diverse data subject oriented aimed at executive,
More informationNew Approach of Computing Data Cubes in Data Warehousing
International Journal of Information & Computation Technology. ISSN 0974-2239 Volume 4, Number 14 (2014), pp. 1411-1417 International Research Publications House http://www. irphouse.com New Approach of
More informationIntroduction to Data Mining
Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:
More informationData Warehousing & OLAP
Data Warehousing & OLAP Motivation: Business Intelligence Customer information (customer-id, gender, age, homeaddress, occupation, income, family-size, ) Product information (Product-id, category, manufacturer,
More informationLecture 10: Regression Trees
Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,
More informationPrinciples of Data Mining by Hand&Mannila&Smyth
Principles of Data Mining by Hand&Mannila&Smyth Slides for Textbook Ari Visa,, Institute of Signal Processing Tampere University of Technology October 4, 2010 Data Mining: Concepts and Techniques 1 Differences
More informationA Data Mining Tutorial
A Data Mining Tutorial Presented at the Second IASTED International Conference on Parallel and Distributed Computing and Networks (PDCN 98) 14 December 1998 Graham Williams, Markus Hegland and Stephen
More informationMonitoring Genebanks using Datamarts based in an Open Source Tool
Monitoring Genebanks using Datamarts based in an Open Source Tool April 10 th, 2008 Edwin Rojas Research Informatics Unit (RIU) International Potato Center (CIP) GPG2 Workshop 2008 Datamarts Motivation
More informationFEATURES TO CONSIDER IN A DATA WAREHOUSING SYSTEM
By Narasimhaiah Gorla FEATURES TO CONSIDER IN A DATA WAREHOUSING SYSTEM Evaluating and assessing the important distinctions between data processing capability and data currency. In order for an organization
More informationDATA WAREHOUSING AND DATA MINING - A CASE STUDY
Yugoslav Journal of Operations Research 15 (2005), Number 1, 125-145 DATA WAREHOUSING AND DATA MINING - A CASE STUDY Milija SUKNOVIĆ, Milutin ČUPIĆ, Milan MARTIĆ Faculty of Organizational Sciences, University
More informationOverview. Background. Data Mining Analytics for Business Intelligence and Decision Support
Mining Analytics for Business Intelligence and Decision Support Chid Apte, PhD Manager, Abstraction Research Group IBM TJ Watson Research Center apte@us.ibm.com http://www.research.ibm.com/dar Overview
More information(Week 10) A04. Information System for CRM. Electronic Commerce Marketing
(Week 10) A04. Information System for CRM Electronic Commerce Marketing Course Code: 166186-01 Course Name: Electronic Commerce Marketing Period: Autumn 2015 Lecturer: Prof. Dr. Sync Sangwon Lee Department:
More informationA Brief Tutorial on Database Queries, Data Mining, and OLAP
A Brief Tutorial on Database Queries, Data Mining, and OLAP Lutz Hamel Department of Computer Science and Statistics University of Rhode Island Tyler Hall Kingston, RI 02881 Tel: (401) 480-9499 Fax: (401)
More informationOracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc.
Oracle9i Data Warehouse Review Robert F. Edwards Dulcian, Inc. Agenda Oracle9i Server OLAP Server Analytical SQL Data Mining ETL Warehouse Builder 3i Oracle 9i Server Overview 9i Server = Data Warehouse
More informationOLAP Systems and Multidimensional Queries II
OLAP Systems and Multidimensional Queries II Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master
More informationWhile people are often a corporation s true intellectual property, data is what
While people are often a corporation s true intellectual property, data is what feeds the people, enabling employees to see where the company stands and where it will go. Quick access to quality data helps
More informationIndexing Techniques for Data Warehouses Queries. Abstract
Indexing Techniques for Data Warehouses Queries Sirirut Vanichayobon Le Gruenwald The University of Oklahoma School of Computer Science Norman, OK, 739 sirirut@cs.ou.edu gruenwal@cs.ou.edu Abstract Recently,
More informationSpatial Data Warehouse and Mining. Rajiv Gandhi
Spatial Data Warehouse and Mining Rajiv Gandhi Roll Number 05331002 Centre of Studies in Resource Engineering Indian Institute of Technology Bombay Powai, Mumbai -400076 India. As part of the first stage
More informationVisual Data Mining in Indian Election System
Visual Data Mining in Indian Election System Prof. T. M. Kodinariya Asst. Professor, Department of Computer Engineering, Atmiya Institute of Technology & Science, Rajkot Gujarat, India trupti.kodinariya@gmail.com
More information