Linköpings Universitet  ITN TNM DBSCAN. A DensityBased Spatial Clustering of Application with Noise


 Daniela Townsend
 3 years ago
 Views:
Transcription
1 DBSCAN A DensityBased Spatial Clustering of Application with Noise Henrik Bäcklund (henba892), Anders Hedblom (andh893), Niklas Neijman (nikne866) 1
2 1. Introduction Today data is received automatically from many different kinds of equipments. Satellites, xrays and traffic cameras are just a few of them. To make this information/data understandable for us, it has to be processed. When working with large data sets it is in most scenarios useful to be able to separate information by dividing the data into smaller categories, and eventually, to do class identification. Not least is this important when treating large spatial databases. A satellite, for example, gathers images as it travels around our earth. It is desired to classify what parts of the images are houses, cars, roads, lakes, forests, etc. Since the image database is big, a good classification algorithm is needed. Classification can, for instance, be done with the help of clustering algorithms, which clumps similar data together into different clusters. However, using clustering algorithms involves some problems: It can often be difficult to know which input parameters that should be used for a specific database, if the user does not have enough knowledge of the domain. Furthermore, spatial data sets can contain huge amounts of data, and trying to find cluster patterns in several dimensions is very computationally costly. Short computing time is always favorable. Last, the shapes of the clusters can be arbitrary and in bad cases very complex. Finding these shapes can be very cumbersome. There are some wellused clustering algorithms out there; one of them is the famous CLARANS. Other methods are Kmeans, Kmedoid, Hierarchical Clustering and SelfOrganized Maps. Nevertheless, none of these algorithms can handle all these three mentioned problems in a good way. This report will not discuss these methods but focus on the DBSCAN [1] (Density Based Spatial Clustering of Applications with Noise) algorithm, which introduces solutions to these problems. The following structure will be used in this paper. Section 2 will discuss how the DBSCAN algorithm works in. Section 3 will present various possible applications for the DBSCAN and some comparisons with CLARANS, in terms of efficiency. Finally, section 4 contains the conclusion of this paper and sums up the positive and negative aspects of the DBSCAN algorithm. 2
3 2. The DBSCAN algorithm The DBSCAN algorithm can identify clusters in large spatial data sets by looking at the local density of database elements, using only one input parameter. Furthermore, the user gets a suggestion on which parameter value that would be suitable. Therefore, minimal knowledge of the domain is required. The DBSCAN can also determine what information should be classified as noise or outliers. In spite of this, its working process is quick and scales very well with the size of the database almost linearly. By using the density distribution of nodes in the database, DBSCAN can categorize these nodes into separate clusters that define the different classes. DBSCAN can find clusters of arbitrary shape, as can be seen in figure 1 [1]. However, clusters that lie close to each other tend to belong to the same class. Figure 1. The node distribution of three different databases, taken from SEQUOIA 2000 benchmark database. The following section will describe further how the DBSCAN algorithm works. Its computing process is based on six rules or definitions, creating two lemmas. Definition 1: (The Epsneighborhood of a point) N Eps (p) = {q ϵ D dist(p,q)<eps} For a point to belong to a cluster it needs to have at least one other point that lies closer to it than the distance Eps. Definition 2: (Directly densityreachable) There are two kinds of points belonging to a cluster; there are border points and core points, as can be seen in figure 2 [1]. Figure 2. Border and core points The Epsneighborhood of a border point tends to have significantly less points than the Epsneighborhood of a core point. [1]. The border points will still be a part of the cluster and in order to include these points, they must belong to the Epsneighborhood of a core point q as seen in figure 3 [1]. 3
4 1) p ϵ N Eps (q) In order for point q to be a core point it needs to have a minimum number of points within its Epsneighborhood. 2) N Eps (q) MinPts (core point condition) Figure 3. Point p is directly densityreachable from point q but not vice versa. Definition 3: (Densityreachable) A point p is densityreachable from a point q with respect to Eps and MinPts if there is a chain of points p 1,p n, p 1 =q, p n =p such that p i+1 is directly densityreachable from p i. [1] Figure 4 [1] shows an illustration of a densityreachable point. Figure 4. Point p is densityreachable from point q and not vice versa. Definition 4: (Densityconnected) There are cases when two border points will belong to the same cluster but where the two border points don t share a specific core point. In these situations the points will not be densityreachable from each other. There must however be a core point q from which they are both densityreachable. Figure 5 [1] shows how density connectivity works. A point p is densityconnected to a point q with respect to Eps and MinPts if there is a point o such that both, p and q are densityreachable from o with respect to Eps and MinPts. Figure 5. Density connectivity. 4
5 Definition 5: (cluster) If point p is a part of a cluster C and point q is densityreachable from point p with respect to a given distance and a minimum number of points within that distance, then q is also a part of cluster C. 1) " p, q: if p ϵ C and q is densityreachable from p with respect to Eps and MinPts, then q ϵ C. Two points belongs to the same cluster C, is the same as saying that p is densityconnected to q with respect to the given distance and the number of points within that given distance. 2) " p, q ϵ C: p is densityconnected to q with respect to Eps and MinPts. Definition 6: (noise) Noise is the set of points, in the database, that don t belong to any of the clusters. Lemma 1: A cluster can be formed from any of its core points and will always have the same shape. Lemma 2: Let p be a core point in cluster C with a given minimum distance (Eps) and a minimum number of points within that distance (MinPts). If the set O is densityreachable from p with respect to the same Eps and MinPts, then C is equal to the set O. To find a cluster, DBSCAN starts with an arbitrary point p and retrieves all points densityreachable from p with respect to Eps and MinPts. If p is a core point, this procedure yields a cluster with respect to Eps and MinPts (see Lemma 2). If p is a border point then no points are densityreachable from p and DBSCAN visits the next point of the database. [1] 3. Applications An example of software program that has the DBSCAN algorithm implemented is WEKA. The following of this section gives some examples of practical application of the DBSCAN algorithm. Satellites images A lot of data is received from satellites all around the world and this data have to be translated into comprehensible information, for instance, classifying areas of the satellitetaken images according to forest, water and mountains. Before the DBSCAN algorithm can classify these three elements in the database, some work have to be done with image processing. Once the image processing is done, the data appears as spatial data where the DBSCAN can classify the clusters as desired. Xray crystallography Xray crystallography is another practical application that locates all atoms within a crystal, which results in a large amount of data. The DBSCAN algorithm can be used to find and classify the atoms in the data. 5
6 Anomaly Detection in Temperature Data This kind of application focuses on pattern anomalies in data, which is important in several cases, e.g. credit fraud, health condition etc. This application measures anomalies in temperatures [3], which is relevant due to the environmental changes (global warming). It can also discover equipment errors and so forth. These unusual patterns need to be detected and examined to get control over the situation. The DBSCAN algorithm has the capability to discover such patterns in the data. 4. Comparisons (DBSCAN vs. CLARANS) Through the original report [1], the DBSCAN algorithm is compared to another clustering algorithm. This one is called CLARANS (Clustering Large Applications based on RANdomized Search). It is an improvement of the kmedoid algorithms (one object of the cluster located near the center of the cluster, instead of the gravity point of the cluster, i.e. kmeans). The good properties compared to kmedoid are that CLARANS works efficient for databases with about a thousand objects. When the database grows larger, CLARANS will fall behind because the algorithm temporarily stores all the objects in the main memory, i.e. the run time will increase. The following part shows that the difference in performance between DBSCAN and CLARANS is vast. The used databases are the ones shown in figure 1 and the result from using CLARANS is shown in figure 6 [1]. Figure 6. The classification of the CLARANS algorithm. As seen in the figure 6, CLARANS did not manage to classify all the clusters correctly for the three different databases, in contrast to DBSCAN that does, as shown in figure 7 [1]. Figure 7. The classification of the DBSCAN algorithm. 6
7 So far the DBSCAN algorithm have done a good job classifying all the clusters. Nevertheless, it is also superior when comparing the run time between the two algorithms. Graph 1 shows the comparisons, and the resulting differences in time are large. Data taken from paper [1]. Graph 1. Run time in seconds. To summarize the table, the DBSCAN has an almost linear increase in computing time, relative to the number of points in the database. The CLARANS gain, however, is exponential and gets outperformed with a factor of 250 to 1900 in this test. The factor will continue to get bigger as the size of the database increases. 5. Conclusion Classification on large spatial databases is a difficult and computational heavy task. Using clustering algorithms is one way of doing it, and we have seen that there are several methods out there that get the job done. However, we have also seen the problems included when using these methods; finding the right input parameters, localizing clusters of arbitrary shapes and, last but not least, doing the whole process in a reasonable time. The DBSCAN algorithm has a solution to all these problems as it can find those complicated cluster shapes in a very quickly, with only one assigned input parameter. A value for this parameter is also suggested to the user. We have discussed the efficiency, in terms of accuracy, of the DBSCAN algorithm compared to the famous CLARANS algorithm. The results were extremely onesided. In some cases with huge databases, the DBSCAN algorithm was over 1900 times faster and still ended up with the better final result. Its only real weakness is that it has some difficulties in distinguishing separated clusters if they are located too close to each other, even though they have different densities. 7
8 Bibliography [1]  A DensityBased Algorithm for Discovering Clusters in Large Spatial Databases with Noise Martin Ester, HansPeter Kriegel, Jörg Sander, Xiaowei Xu [2]  Advances in knowledge discovery and data mining: 9th PacificAsia conference, PAKDD 2005, Hanoi, Vietnam, May 1820, 2005; proceedings Tu Bao Ho, David Cheung, Huan Liu [3]  Anomaly Detection in Temperature Data Using DBSCAN Algorithm: Erciyes Univeristy, Mete Celic, Filiz DadaserCelic, Ahmet Sakir DOKUZ 8
An Analysis on Density Based Clustering of Multi Dimensional Spatial Data
An Analysis on Density Based Clustering of Multi Dimensional Spatial Data K. Mumtaz 1 Assistant Professor, Department of MCA Vivekanandha Institute of Information and Management Studies, Tiruchengode,
More informationComparative Analysis of EM Clustering Algorithm and Density Based Clustering Algorithm Using WEKA tool.
International Journal of Engineering Research and Development eissn: 2278067X, pissn: 2278800X, www.ijerd.com Volume 9, Issue 8 (January 2014), PP. 1924 Comparative Analysis of EM Clustering Algorithm
More informationPublic Transportation BigData Clustering
Public Transportation BigData Clustering Preliminary Communication Tomislav Galba J.J. Strossmayer University of Osijek Faculty of Electrical Engineering Cara Hadriana 10b, 31000 Osijek, Croatia tomislav.galba@etfos.hr
More informationA Novel Density based improved kmeans Clustering Algorithm Dbkmeans
A Novel Density based improved kmeans Clustering Algorithm Dbkmeans K. Mumtaz 1 and Dr. K. Duraiswamy 2, 1 Vivekanandha Institute of Information and Management Studies, Tiruchengode, India 2 KS Rangasamy
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/2004 Hierarchical
More informationA Comparative Analysis of Various Clustering Techniques used for Very Large Datasets
A Comparative Analysis of Various Clustering Techniques used for Very Large Datasets Preeti Baser, Assistant Professor, SJPIBMCA, Gandhinagar, Gujarat, India 382 007 Research Scholar, R. K. University,
More informationA DistributionBased Clustering Algorithm for Mining in Large Spatial Databases
Published in the Proceedings of 14th International Conference on Data Engineering (ICDE 98) A DistributionBased Clustering Algorithm for Mining in Large Spatial Databases Xiaowei Xu, Martin Ester, HansPeter
More informationA Comparative Study of clustering algorithms Using weka tools
A Comparative Study of clustering algorithms Using weka tools Bharat Chaudhari 1, Manan Parikh 2 1,2 MECSE, KITRC KALOL ABSTRACT Data clustering is a process of putting similar data into groups. A clustering
More informationRobust Outlier Detection Technique in Data Mining: A Univariate Approach
Robust Outlier Detection Technique in Data Mining: A Univariate Approach Singh Vijendra and Pathak Shivani Faculty of Engineering and Technology Mody Institute of Technology and Science Lakshmangarh, Sikar,
More informationDATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS
DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDDLAB ISTI CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 by Tan, Steinbach, Kumar 1 What is Cluster Analysis? Finding groups of objects such that the objects in a group will
More informationClustering methods for Big data analysis
Clustering methods for Big data analysis Keshav Sanse, Meena Sharma Abstract Today s age is the age of data. Nowadays the data is being produced at a tremendous rate. In order to make use of this largescale
More informationData Mining Clustering (2) Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining
Data Mining Clustering (2) Toon Calders Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining Outline Partitional Clustering Distancebased Kmeans, Kmedoids,
More informationClustering: Techniques & Applications. Nguyen Sinh Hoa, Nguyen Hung Son. 15 lutego 2006 Clustering 1
Clustering: Techniques & Applications Nguyen Sinh Hoa, Nguyen Hung Son 15 lutego 2006 Clustering 1 Agenda Introduction Clustering Methods Applications: Outlier Analysis Gene clustering Summary and Conclusions
More informationData Mining: Foundation, Techniques and Applications
Data Mining: Foundation, Techniques and Applications Lesson 1b :A Quick Overview of Data Mining Li Cuiping( 李 翠 平 ) School of Information Renmin University of China Anthony Tung( 鄧 锦 浩 ) School of Computing
More informationThe Role of Visualization in Effective Data Cleaning
The Role of Visualization in Effective Data Cleaning Yu Qian Dept. of Computer Science The University of Texas at Dallas Richardson, TX 750830688, USA qianyu@student.utdallas.edu Kang Zhang Dept. of Computer
More informationCluster Analysis: Advanced Concepts
Cluster Analysis: Advanced Concepts and dalgorithms Dr. Hui Xiong Rutgers University Introduction to Data Mining 08/06/2006 1 Introduction to Data Mining 08/06/2006 1 Outline Prototypebased Fuzzy cmeans
More informationData Mining. Cluster Analysis: Advanced Concepts and Algorithms
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 More Clustering Methods Prototypebased clustering Densitybased clustering Graphbased
More informationOUTLIER ANALYSIS. Data Mining 1
OUTLIER ANALYSIS Data Mining 1 What Are Outliers? Outlier: A data object that deviates significantly from the normal objects as if it were generated by a different mechanism Ex.: Unusual credit card purchase,
More informationChapter 7. Cluster Analysis
Chapter 7. Cluster Analysis. What is Cluster Analysis?. A Categorization of Major Clustering Methods. Partitioning Methods. Hierarchical Methods 5. DensityBased Methods 6. GridBased Methods 7. ModelBased
More informationCluster Analysis. Alison Merikangas Data Analysis Seminar 18 November 2009
Cluster Analysis Alison Merikangas Data Analysis Seminar 18 November 2009 Overview What is cluster analysis? Types of cluster Distance functions Clustering methods Agglomerative Kmeans Densitybased Interpretation
More informationClustering. Adrian Groza. Department of Computer Science Technical University of ClujNapoca
Clustering Adrian Groza Department of Computer Science Technical University of ClujNapoca Outline 1 Cluster Analysis What is Datamining? Cluster Analysis 2 Kmeans 3 Hierarchical Clustering What is Datamining?
More informationClustering. Data Mining. Abraham Otero. Data Mining. Agenda
Clustering 1/46 Agenda Introduction Distance Knearest neighbors Hierarchical clustering Quick reference 2/46 1 Introduction It seems logical that in a new situation we should act in a similar way as in
More informationChapter ML:XI (continued)
Chapter ML:XI (continued) XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis DensityBased Cluster Analysis Cluster Evaluation Constrained
More informationClustering Techniques: A Brief Survey of Different Clustering Algorithms
Clustering Techniques: A Brief Survey of Different Clustering Algorithms Deepti Sisodia Technocrates Institute of Technology, Bhopal, India Lokesh Singh Technocrates Institute of Technology, Bhopal, India
More informationANALYSIS OF VARIOUS CLUSTERING ALGORITHMS OF DATA MINING ON HEALTH INFORMATICS
ANALYSIS OF VARIOUS CLUSTERING ALGORITHMS OF DATA MINING ON HEALTH INFORMATICS 1 PANKAJ SAXENA & 2 SUSHMA LEHRI 1 Deptt. Of Computer Applications, RBS Management Techanical Campus, Agra 2 Institute of
More informationScalable Cluster Analysis of Spatial Events
International Workshop on Visual Analytics (2012) K. Matkovic and G. Santucci (Editors) Scalable Cluster Analysis of Spatial Events I. Peca 1, G. Fuchs 1, K. Vrotsou 1,2, N. Andrienko 1 & G. Andrienko
More informationIntroduction to Data Mining
Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationPreAnalysis and Clustering of Uncertain Data from Manufacturing Processes
PreAnalysis and Clustering of Uncertain Data from Manufacturing Processes Peter Benjamin Volk, Martin Hahmann, Dirk Habich, Wolfgang Lehner Dresden University of Technology Database Technology Group 01187
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Clustering Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Clustering Algorithms Kmeans and its variants Hierarchical clustering
More informationA comparison of various clustering methods and algorithms in data mining
Volume :2, Issue :5, 3236 May 2015 www.allsubjectjournal.com eissn: 23494182 pissn: 23495979 Impact Factor: 3.762 R.Tamilselvi B.Sivasakthi R.Kavitha Assistant Professor A comparison of various clustering
More informationData Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analsis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining b Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining /8/ What is Cluster
More informationA Review of Clustering Methods forming NonConvex clusters with, Missing and Noisy Data
International Journal of Computer Sciences and Engineering Open Access Review Paper Volume4, Issue3 EISSN: 23472693 A Review of Clustering Methods forming NonConvex clusters with, Missing and Noisy
More informationCLASSIFICATION AND CLUSTERING. Anveshi Charuvaka
CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training
More information[EN024]Clustering radar tracks to evaluate efficiency indicators
ENRI Int. Workshop on ATM/CNS. Tokyo, Japan (EIWAC2010) [EN024]Clustering radar tracks to evaluate efficiency indicators + R. Winkler A. Temme C. Bösel R. Kruse German Aerospace Center (DLR) Braunschweig
More informationBuilding Data Cubes and Mining Them. Jelena Jovanovic Email: jeljov@fon.bg.ac.yu
Building Data Cubes and Mining Them Jelena Jovanovic Email: jeljov@fon.bg.ac.yu KDD Process KDD is an overall process of discovering useful knowledge from data. Data mining is a particular step in the
More informationData Mining Project Report. Document Clustering. Meryem UzunPer
Data Mining Project Report Document Clustering Meryem UzunPer 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. Kmeans algorithm...
More informationComparision of kmeans and kmedoids Clustering Algorithms for Big Data Using MapReduce Techniques
Comparision of kmeans and kmedoids Clustering Algorithms for Big Data Using MapReduce Techniques Subhashree K 1, Prakash P S 2 1 Student, Kongu Engineering College, Perundurai, Erode 2 Assistant Professor,
More informationThe Data Mining Process
Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data
More informationIdentifying erroneous data using outlier detection techniques
Identifying erroneous data using outlier detection techniques Wei Zhuang 1, Yunqing Zhang 2 and J. Fred Grassle 2 1 Department of Computer Science, Rutgers, the State University of New Jersey, Piscataway,
More informationAlgorithms and Applications for Spatial Data Mining
Published in Geographic Data Mining and Knowledge Discovery, Research Monographs in GIS, Taylor and Francis, 2001. Algorithms and Applications for Spatial Data Mining Martin Ester, HansPeter Kriegel,
More informationA Method for Decentralized Clustering in Large MultiAgent Systems
A Method for Decentralized Clustering in Large MultiAgent Systems Elth Ogston, Benno Overeinder, Maarten van Steen, and Frances Brazier Department of Computer Science, Vrije Universiteit Amsterdam {elth,bjo,steen,frances}@cs.vu.nl
More informationData Mining Cluster Analysis: Advanced Concepts and Algorithms. Lecture Notes for Chapter 9. Introduction to Data Mining
Data Mining Cluster Analysis: Advanced Concepts and Algorithms Lecture Notes for Chapter 9 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
More informationThe Scientific Data Mining Process
Chapter 4 The Scientific Data Mining Process When I use a word, Humpty Dumpty said, in rather a scornful tone, it means just what I choose it to mean neither more nor less. Lewis Carroll [87, p. 214] In
More informationInternational Journal of Enterprise Computing and Business Systems. ISSN (Online) : 2230
Analysis and Study of Incremental DBSCAN Clustering Algorithm SANJAY CHAKRABORTY National Institute of Technology (NIT) Raipur, CG, India. Prof. N.K.NAGWANI National Institute of Technology (NIT) Raipur,
More informationUsing Data Mining for Mobile Communication Clustering and Characterization
Using Data Mining for Mobile Communication Clustering and Characterization A. Bascacov *, C. Cernazanu ** and M. Marcu ** * Lasting Software, Timisoara, Romania ** Politehnica University of Timisoara/Computer
More informationComparison the various clustering algorithms of weka tools
Comparison the various clustering algorithms of weka tools Narendra Sharma 1, Aman Bajpai 2, Mr. Ratnesh Litoriya 3 1,2,3 Department of computer science, Jaypee University of Engg. & Technology 1 narendra_sharma88@yahoo.com
More informationSynflood Spoof Source DDOS Attack Defence Based on Packet ID Anomaly Detection PIDAD
Synflood Spoof Source DDOS Attack Defence Based on Packet ID Anomaly Detection PIDAD Tran Manh Thang and Van K. Nguyen Dept of Software Engineering, School of Information Technology and Communication,
More informationInformation Management course
Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)
More informationCOPYRIGHTED MATERIAL. Contents. List of Figures. Acknowledgments
Contents List of Figures Foreword Preface xxv xxiii xv Acknowledgments xxix Chapter 1 Fraud: Detection, Prevention, and Analytics! 1 Introduction 2 Fraud! 2 Fraud Detection and Prevention 10 Big Data for
More informationUsing an Ontologybased Approach for Geospatial Clustering Analysis
Using an Ontologybased Approach for Geospatial Clustering Analysis Xin Wang Department of Geomatics Engineering University of Calgary Calgary, AB, Canada T2N 1N4 xcwang@ucalgary.ca Abstract. Geospatial
More informationContextaware taxi demand hotspots prediction
Int. J. Business Intelligence and Data Mining, Vol. 5, No. 1, 2010 3 Contextaware taxi demand hotspots prediction Hanwen Chang, Yuchin Tai and Jane Yungjen Hsu* Department of Computer Science and Information
More informationCluster Analysis: Basic Concepts and Algorithms
Cluster Analsis: Basic Concepts and Algorithms What does it mean clustering? Applications Tpes of clustering Kmeans Intuition Algorithm Choosing initial centroids Bisecting Kmeans Postprocessing Strengths
More informationData Mining Process Using Clustering: A Survey
Data Mining Process Using Clustering: A Survey Mohamad Saraee Department of Electrical and Computer Engineering Isfahan University of Techno1ogy, Isfahan, 8415683111 saraee@cc.iut.ac.ir Najmeh Ahmadian
More informationData Mining Clustering. Sheets are based on the those provided by Tan, Steinbach, and Kumar. Introduction to Data Mining
Data Mining Clustering Toon Calders Sheets are based on the those provided b Tan, Steinbach, and Kumar. Introduction to Data Mining What is Cluster Analsis? Finding groups of objects such that the objects
More informationGraphZip: A Fast and Automatic Compression Method for Spatial Data Clustering
GraphZip: A Fast and Automatic Compression Method for Spatial Data Clustering Yu Qian Kang Zhang Department of Computer Science, The University of Texas at Dallas, Richardson, TX 750830688, USA {yxq012100,
More informationFig. 1 A typical Knowledge Discovery process [2]
Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering
More informationFor supervised classification we have a variety of measures to evaluate how good our model is Accuracy, precision, recall
Cluster Validation Cluster Validit For supervised classification we have a variet of measures to evaluate how good our model is Accurac, precision, recall For cluster analsis, the analogous question is
More informationSCAN: A Structural Clustering Algorithm for Networks
SCAN: A Structural Clustering Algorithm for Networks Xiaowei Xu, Nurcan Yuruk, Zhidan Feng (University of Arkansas at Little Rock) Thomas A. J. Schweiger (Acxiom Corporation) Networks scaling: #edges connected
More informationVisually driven analysis of movement data by progressive clustering
Visually driven analysis of movement data by progressive clustering Salvatore Rinzivillo, Dino Pedreschi, Mirco Nanni, Fosca Giannotti, Natalia Andrienko, Gennady Andrienko Information Visualization 2008
More informationIndex Contents Page No. Introduction . Data Mining & Knowledge Discovery
Index Contents Page No. 1. Introduction 1 1.1 Related Research 2 1.2 Objective of Research Work 3 1.3 Why Data Mining is Important 3 1.4 Research Methodology 4 1.5 Research Hypothesis 4 1.6 Scope 5 2.
More informationUnsupervised Learning: Clustering with DBSCAN Mat Kallada
Unsupervised Learning: Clustering with DBSCAN Mat Kallada STAT 2450  Introduction to Data Mining Supervised Data Mining: Predicting a column called the label The domain of data mining focused on prediction:
More informationClustering & Association
Clustering  Overview What is cluster analysis? Grouping data objects based only on information found in the data describing these objects and their relationships Maximize the similarity within objects
More informationDensityBased Clustering over an Evolving Data Stream with Noise
DensityBased Clustering over an Evolving Data Stream with Noise Feng Cao Martin Ester Weining Qian Aoying Zhou Abstract Clustering is an important task in mining evolving data streams. Beside the limited
More informationOn Clustering Validation Techniques
Journal of Intelligent Information Systems, 17:2/3, 107 145, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques MARIA HALKIDI mhalk@aueb.gr YANNIS
More informationData Mining: Concepts and Techniques. Jiawei Han. Micheline Kamber. Simon Fräser University К MORGAN KAUFMANN PUBLISHERS. AN IMPRINT OF Elsevier
Data Mining: Concepts and Techniques Jiawei Han Micheline Kamber Simon Fräser University К MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF Elsevier Contents Foreword Preface xix vii Chapter I Introduction I I.
More informationOn Data Clustering Analysis: Scalability, Constraints and Validation
On Data Clustering Analysis: Scalability, Constraints and Validation Osmar R. Zaïane, Andrew Foss, ChiHoon Lee, and Weinan Wang University of Alberta, Edmonton, Alberta, Canada Summary. Clustering is
More informationTianjun Fu and Hsinchun Chen, Artificial Intelligence Lab, University of Arizona Christopher C. Yang 1,2 and Tobun D. Ng 2
Analyzing Analysis Content of Cyberactivism: Development and Visualizing A Case Study Social of Online Interactions Free Tibet in Web Activities Forum Tianjun Fu and Hsinchun Chen, Artificial Intelligence
More informationMultirepresented knnclassification for Large Class Sets
Multirepresented knnclassification for Large Class Sets HansPeter Kriegel, Alexey Pryakhin, and Matthias Schubert Institute for Computer Science University of Munich Oettingenstr. 67, 80538 Munich,
More informationForschungskolleg Data Analytics Methods and Techniques
Forschungskolleg Data Analytics Methods and Techniques Martin Hahmann, Gunnar Schröder, Phillip Grosse Prof. Dr.Ing. Wolfgang Lehner Why do we need it? We are drowning in data, but starving for knowledge!
More informationCar Insurance. Jan Tomášek Štěpán Havránek Michal Pokorný
Car Insurance Jan Tomášek Štěpán Havránek Michal Pokorný Competition details Jan Tomášek Official text As a customer shops an insurance policy, he/she will receive a number of quotes with different coverage
More informationAn Evaluation of Machine Learning Method for Intrusion Detection System Using LOF on Jubatus
An Evaluation of Machine Learning Method for Intrusion Detection System Using LOF on Jubatus Tadashi Ogino* Okinawa National College of Technology, Okinawa, Japan. * Corresponding author. Email: ogino@okinawact.ac.jp
More informationExample application (1) Telecommunication. Lecture 1: Data Mining Overview and Process. Example application (2) Health
Lecture 1: Data Mining Overview and Process What is data mining? Example applications Definitions Multi disciplinary Techniques Major challenges The data mining process History of data mining Data mining
More informationIntroduction. A. Bellaachia Page: 1
Introduction 1. Objectives... 3 2. What is Data Mining?... 4 3. Knowledge Discovery Process... 5 4. KD Process Example... 7 5. Typical Data Mining Architecture... 8 6. Database vs. Data Mining... 9 7.
More informationAn Introduction to Cluster Analysis for Data Mining
An Introduction to Cluster Analysis for Data Mining 10/02/2000 11:42 AM 1. INTRODUCTION... 4 1.1. Scope of This Paper... 4 1.2. What Cluster Analysis Is... 4 1.3. What Cluster Analysis Is Not... 5 2. OVERVIEW...
More informationClustering & Visualization
Chapter 5 Clustering & Visualization Clustering in highdimensional databases is an important problem and there are a number of different clustering paradigms which are applicable to highdimensional data.
More informationUsing multiple models: Bagging, Boosting, Ensembles, Forests
Using multiple models: Bagging, Boosting, Ensembles, Forests Bagging Combining predictions from multiple models Different models obtained from bootstrap samples of training data Average predictions or
More informationCategorical Data Visualization and Clustering Using Subjective Factors
Categorical Data Visualization and Clustering Using Subjective Factors ChiaHui Chang and ZhiKai Ding Department of Computer Science and Information Engineering, National Central University, ChungLi,
More informationWhy is Internal Audit so Hard?
Why is Internal Audit so Hard? 2 2014 Why is Internal Audit so Hard? 3 2014 Why is Internal Audit so Hard? Waste Abuse Fraud 4 2014 Waves of Change 1 st Wave Personal Computers Electronic Spreadsheets
More informationNetwork Intrusion Detection using Data Mining Technique
Network Intrusion Detection using Data Mining Technique Abstract  In recent years, most of the research has been done in the field of Intrusion Detection System (IDS) to detect attacks in network traffic
More informationCluster Analysis: Basic Concepts and Algorithms
8 Cluster Analysis: Basic Concepts and Algorithms Cluster analysis divides data into groups (clusters) that are meaningful, useful, or both. If meaningful groups are the goal, then the clusters should
More informationData Mining + Business Intelligence. Integration, Design and Implementation
Data Mining + Business Intelligence Integration, Design and Implementation ABOUT ME Vijay Kotu Data, Business, Technology, Statistics BUSINESS INTELLIGENCE  Result Making data accessible Wider distribution
More informationThe Predictive Data Mining Revolution in Scorecards:
January 13, 2013 StatSoft White Paper The Predictive Data Mining Revolution in Scorecards: Accurate Risk Scoring via Ensemble Models Summary Predictive modeling methods, based on machine learning algorithms
More informationFacebook Friend Suggestion Eytan Daniyalzade and Tim Lipus
Facebook Friend Suggestion Eytan Daniyalzade and Tim Lipus 1. Introduction Facebook is a social networking website with an open platform that enables developers to extract and utilize user information
More informationOPTICS: Ordering Points To Identify the Clustering Structure
Proc. ACM SIGMOD 99 Int. Conf. on Management of Data, Philadelphia PA, 1999. OPTICS: Ordering Points To Identify the Clustering Structure Mihael Ankerst, Markus M. Breunig, HansPeter Kriegel, Jörg Sander
More informationCluster Analysis: Basic Concepts and Methods
10 Cluster Analysis: Basic Concepts and Methods Imagine that you are the Director of Customer Relationships at AllElectronics, and you have five managers working for you. You would like to organize all
More informationSPATIAL DATA CLASSIFICATION AND DATA MINING
, pp.4044. Available online at http://www. bioinfo. in/contents. php?id=42 SPATIAL DATA CLASSIFICATION AND DATA MINING RATHI J.B. * AND PATIL A.D. Department of Computer Science & Engineering, Jawaharlal
More informationConclusions and Future Directions
Chapter 9 This chapter summarizes the thesis with discussion of (a) the findings and the contributions to the stateoftheart in the disciplines covered by this work, and (b) future work, those directions
More information3 HOW WERE STARS FORMED?
3 HOW WERE STARS FORMED? David Christian explains how the first stars were formed. This twopart lecture begins by focusing on what the Universe was like in its first 200 million years of existence, a
More informationKnowledge Discovery in Data with FITMiner
Knowledge Discovery in Data with FITMiner Michal Šebek, Martin Hlosta and Jaroslav Zendulka Faculty of Information Technology, Brno University of Technology, Božetěchova 2, Brno {isebek,ihlosta,zendulka}@fit.vutbr.cz
More informationPhoCA: An extensible serviceoriented tool for Photo Clustering Analysis
paper:5 PhoCA: An extensible serviceoriented tool for Photo Clustering Analysis Yuri A. Lacerda 1,2, Johny M. da Silva 2, Leandro B. Marinho 1, Cláudio de S. Baptista 1 1 Laboratório de Sistemas de Informação
More informationOriginal Article Survey of Recent Clustering Techniques in Data Mining
International Archive of Applied Sciences and Technology Volume 3 [2] June 2012: 6875 ISSN: 09764828 Society of Education, India Website: www.soeagra.com/iaast/iaast.htm Original Article Survey of Recent
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationCrime Hotspots Analysis in South Korea: A UserOriented Approach
, pp.8185 http://dx.doi.org/10.14257/astl.2014.52.14 Crime Hotspots Analysis in South Korea: A UserOriented Approach Aziz Nasridinov 1 and YoungHo Park 2 * 1 School of Computer Engineering, Dongguk
More informationAdaptive Anomaly Detection for Network Security
International Journal of Computer and Internet Security. ISSN 09742247 Volume 5, Number 1 (2013), pp. 19 International Research Publication House http://www.irphouse.com Adaptive Anomaly Detection for
More informationUnsupervised Data Mining (Clustering)
Unsupervised Data Mining (Clustering) Javier Béjar KEMLG December 01 Javier Béjar (KEMLG) Unsupervised Data Mining (Clustering) December 01 1 / 51 Introduction Clustering in KDD One of the main tasks in
More informationLocation Analytics for Financial Services. An Esri White Paper October 2013
Location Analytics for Financial Services An Esri White Paper October 2013 Copyright 2013 Esri All rights reserved. Printed in the United States of America. The information contained in this document is
More informationInternational Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, MayJun 2014
RESEARCH ARTICLE OPEN ACCESS A Survey of Data Mining: Concepts with Applications and its Future Scope Dr. Zubair Khan 1, Ashish Kumar 2, Sunny Kumar 3 M.Tech Research Scholar 2. Department of Computer
More informationUsing MapReduce for Large Scale Analysis of GraphBased Data
Using MapReduce for Large Scale Analysis of GraphBased Data NAN GONG KTH Information and Communication Technology Master of Science Thesis Stockholm, Sweden 2011 TRITAICTEX2011:218 Using MapReduce
More information