Data Quality Mining: Employing Classifiers for Assuring consistent Datasets Fabian Grüning Carl von Ossietzky Universität Oldenburg, Germany, fabian.gruening@informatik.uni-oldenburg.de Abstract: Independent from the concrete definition of the term data quality consistency always plays a major role. There are two main points when dealing with the data quality of a database: Firstly, the data quality has to be measured, and secondly, if is necessary, it must be improved. A classifier can be used for both purposes regarding consistency demands by calculating the distance of the classified value to the stored value for measuring and using the classified value for correction. Keywords: data mining, data quality, classifiers, ontology, utilities 1 Introduction A good introduction of the main topics of the field of data quality can be found in (Scannapieco et al. 2005) where a motivation is given and relevant data quality dimensions are highlighted. Having discussed an ontology that describes such a definition and the semantical integration of data quality aspects into given data schemas using an ontological approach in Grüning (2006) we now come to the appliance of data quality mining algorithms to estimate the consistency of a given data set and suggest correct values where necessary. This is one of the four identified algorithms needed for the holistic data quality management approach to be developed in a projected funded by a major German utility. One of its goals is to provide an ICT infrastructure for managing the upcoming power plant mix consisting of more decentralized, probably regenerative, and sustainable power plants, e.g. wind power and biogas plants, and combined heat and power generation together with the conventional power plants. As many decisions for controlling relevant parts of the system are made automatically, good data quality is vital for the health of the overall system, as false data leads to wrong decisions that may worsen the system s overall performance. The
2 Fabian Grüning system is used for both day-to-day business and strategical decisions. Examples for those decisions are the regulation of conventional power plants with the wind forecast in mind to provide an optimal integration of the sustainable power plants like wind power plants into the distribution grid. A strategical decision might be the decision where another wind park is built by taking statistical series of wind measurements into account. The system contains costumer data as well as technical data about the distribution grid and power plants. The data is critical for the company as well as the state as it contains information about vital distribution systems so that concrete information about the data cannot be given in this paper. Therefore the example given later in this paper will only contain a simple list of dates. The paper focuses more on the concepts of the approach discussed beforehand. The paper is structured as follows: First we are going to give a short summary of the term data quality mining and the dimensions belonging to it with focus on consistency. We than are going to reasonably chose a concrete classification algorithm that fits our needs in the examined field. The process of using a classifier for checking consistency in data sets is going to be described in the following section giving an example of the algorithm s performance. We are going to touch the subject of using domain experts knowledge through employing ontologies and eventually getting to conclusions and further work to do. 2 Excursus: About Data Quality Mining The definition of data quality by Redman (1996) defines four different data quality dimensions: accuracy, consistency, currency as a specialization of timeliness constraints and correctness. After having discussed the semantics of those dimensions in the previous paper we now concentrate on the realization of the algorithms for data quality mining, namely for checking and improving consistency. The term data quality mining is meant in the way that algorithms of the data mining domain are utilized for the purpose of data quality management (see Hipp et all. 2002). In this paper we are discussing the consistency aspect of data quality. We will explain that classification algorithms are reasonable applicable for this purpose.
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets 3 2.1 Consistency as a Data Quality Dimension Whenever there is redundancy in a data set, inconsistencies might occur. A good example is the topology of a distribution grid that consists of power supply lines and connections. An inconsistency in a relational orientated data store leads to non realizable topologies where e.g. a power supply line only has one connection or is connected more than twice. Such a datacentric problem leads to real world problems in the sense that power flow algorithms cannot be applied to the data so that management systems and the power grid get unstable or short circuit cannot be detected or are registered all the time. This example also shows that a consistency check can only be done by considering a real world entity, here the distribution grid, on the whole and that the verification of consistency works better all the more the semantical correlation between real world entities and data schemas is realized so that relationships between the single data properties can be utilized (see (Noy and Guinness 2001) and Grüning 2006). A particular good approach for assuring this correlation is using ontologies for modeling a real world extract as they explicitly keep the relationships inside of and between the examined real world s concepts in contrast to for example normalized relational data schemas. 2.2 Employing Classifiers for Consistency Checking Classification algorithms are used to classify one data item of a data record by using the information of the remaining data items. E.g. a triple of two power supply lines and one connection implies that those two lines are connected by the very connector. This is only true if the value of the connector is in fact a valid identifier for a connector. If the connector s name is different from a certain pattern that identifies such a connector, a different resource is addressed and an invalid topology is represented. Such dependence can be learned by a classifier. If the classified value and the stored value differ from one another, an inconsistency in the dataset might have been identified which even can be corrected by using the classified value as a clue. Classifiers can therefore be found basically usable for finding and correcting inconsistencies in datasets and a prototype will confirm this assumption as shown in the following sections. To check every possible consistency violation every data item has to be classified with the rest of the data record respectively. It is therefore necessary to train n classifiers for a data record consisting of n data items.
4 Fabian Grüning 3 Using Support Vector Machines as a concrete Classification Algorithm There are several different algorithms for classification tasks like decision trees (C4.5), rule sets (CN2), neural networks, Bayes classifiers, evolutionary algorithms, and support vector machines. A decision has to be made which algorithm fits the needs for the classification task in the field of checking consistency in data sets. The classifiers have in common that their implementation consists of two consecutively phases: In the first phase the algorithm learns through the usage of a representative data set the characteristics of the data. This phase is called the training phase. In the second phase the algorithm classifies not known data records utilizing the knowledge gained from phase one (see (Witten and Frank 2005)). There are two main points a classification algorithm has to fulfill in this kind of application: The dataset for the learning task in which the algorithms adapts to the characteristics of the data is in comparison to the whole data set relatively small. This is related to the fact that the data set for the learning task has to be constructed out of error-free data so that the classifier will detect and complain about data that differs from these. The labeling, i.e. the task of deciding whether a data record is correct or not, has to be done by domain experts and therefore is a complex and expensive task. The classification approach has to be quite general because not much is known about the data to be classified beforehand. A well qualified classification algorithm therefore needs only few parameters to be configured to be adjusted to the classification task. Both demands are fulfilled by support vector machines (SVM) as they scale well for even small data sets and the configuration efforts are restricted to the choice and configuration of the kernel function that is used to map the training set s samples into the high dimensional feature space and the adaptation of the coefficient weighting the costs for misclassification (see (Russell and Norvig 2003) for an introduction to SVMs). A classifier s output can also be understood as a recommendation in the case where the classified value differs from the stored value. The SVM can both be used as a classification or regression algorithm making it possible to not only give recommendations for discrete but also for continuous values. The algorithm for the regression version of SVM does not differ much from the classifier version so that it is easy to be used either way. Classification and regression can be used nearly synonymously when it comes to SVM because the learning phases do not differ much from one another.
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets 5 4 Prototype for Consistency Checking The considerations made so far have to be verified by an appliance to realworld data. For this reason a prototype was developed employing YALE (see Mierswa 2006), a learning environment that allows to orchestrate processes that are necessary in the field of learning algorithms. As the whole approach for data quality mining is encouraged by a German utility real data was available for testing purposes. We will show promising results from a prototype utilizing SVMs as a classification algorithm for checking consistency in a given data set. 4.1 Phase I: Selection To compile the training set for the first phase of the learning algorithm, a choice out of the existing data records has to be made (see figure 4.1). On the one hand all relevant equivalent classes for the classification task have to be covered which is addressed by the stratified sampling, on the other hand the cardinal number of the training set has to be restricted because of the expensive labeling task for the training set (see section 3). Therefore the absolute sampling assures that only a certain amount of data records are in the training set at most. Fig. 4.1. Selection phase The data itself is converted to interval scale (see (Bortz 2005)) by one of the following algorithms: If the data originally is in nominal scale the data is mapped to [0, 1] equidistantly. Ordinal data gets normalized and therefore also mapped to [0, 1] where the sequence of the data gets conserved. Strings are addressed separately: They are mapped to interval scale under a given string distance function in a way that similar strings have less distance to one another than less similar strings. The results are clusters of
6 Fabian Grüning similar strings that get normalized to [0, 1], having obtained a certain amount of semantics. This preprocessing phase produces data sets that only consist of interval scaled values that are therefore suitable for getting processed via the regression version of the SVM algorithm. We now can use the distance between the outcome of the regression algorithm and the mapped value as a degree of quality. The outcome of the regression algorithm can directly be used as a recommendation for the correct value. Mentioned as a side note we do not lose any practicability by the data s preprocessing as it is still possible to establish arbitrary bounds to use the classification version of the algorithm. 4.2 Phase II: Learning In the learning phase the classifier adapts to the characteristics of the data set. This mainly means to adjust the SVM parameter set so that it adapts optimally to the training set. As (Russel and Norvig 2003) describe, this means to choose the kernel function that adapts the best to the training set and to choose the correct values for the kernel s parameters for optimal results. The learning phase consists of several steps (see figure 4.2): 1. In the preprocessing phase the data sets are completed where necessary because the classification algorithm cannot handle empty data items. This is no problem as the values filled in are uniform so that they cannot be taken into account for classification because they are not characteristic for any data set. 2. The next steps are repeatedly executed to find the optimal parameter setting for the SVM: The training set is split into a learning and a classification subset as the procedure of cross validation plans. The first set is used for training the classifier and the second set is used for validating the trained classifier. Cross validation avoids a too strict adaptation to the training set so that the classifier only adapts to the characteristics of the training set and does not mimic it. Having done that with a defined number of combinations the overall performance of the classifier is evaluated and associated with the parameter configuration. The more parameter combinations of the classification algorithms are tested the better the classifier is as the result of this process. This is one of the strengths of the SVMs as only three variables are used to configure a certain SVM in the case when using the radial basis function as kernel function. The parameter space can therefore be mined quite in
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets 7 great detail for finding the optimal parameter configuration so that the out coming classifier is of high quality. 3. Finally, the optimal parameter configuration is used to eventually train a classifier with the whole training set which gets stored for the last step of the process of finding inconsistent data, namely to apply the classifier to not known data records. Fig. 4.2. Learning phase 4.3 Phase III: Appliance In the last phase (see figure 4.3) the classifier is applied to the data records of the whole data set searching for discrepancies between classified and
8 Fabian Grüning stored values. The more discrepancies are found the lower the data quality is regarding the consistency aspect. As SVMs can also be used for regression, a concrete recommendation for a correct value can be made for the cases where inconsistencies occur. Such a recommendation is not only a range but a concrete value in contrast to other classification algorithms only capable of classifications, like decision trees, again showing the adequate choice of the classification algorithm. Fig. 4.3. Appliance phase 4.4 Results A typical result is shown in Table 1. It was generated out of a training set consisting of 128 examples that were proved to be valid. The classifier was then used to find inconsistencies between the classified and the stored values. In the examples given there are two major discrepancies between the stored and the classified values (marked by italics). The first one is a result of a deliberate falsification to show the approach s functionality. The correct value had been 1995 so that the distance relative to the remaining distances between stored and classified values is large and implies an error in the specific data set. The classified value can be used as a correction and meets the non-falsified value quite well.
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets 9 The second one also shows a huge distance between the classified and the stored value although no falsification has taken place. This is an example that shows that the training set missed a relevant equivalent class so that the algorithm wrongly detects an inconsistency. The user has to mark this wrong classification. Those data sets are then included in the training set so that in the next learning phase the classifier better adapts to the data s characteristics. This procedure may be executed until the classifier has adapted well enough to the relevant data set or regularly to adapt to changes in the underlying structure of the data. Classified Value Stored Value 1994.66 2000.0 1994.66 1995.0 1994.66 1995.0 1992.17 1995.0 1990.26 1990.0 1991.68 1990.0 1990.26 1990.0 1990.26 1990.0 1992.35 2003.0 [ ] [ ] Table 1: Prototype's results sample (classified and stored values are shown) 5 Using Ontologies for further Improvements As already pointed out in section 2.1 the usage of ontologies for modeling the examined real world extract is beneficial for the sake of building a classifier for the discovery of inconsistencies in data sets. But not only the semantical coherence of the modeled concepts is useful but also further information the modeling domain expert can annotate to the identified concepts. This information is made explicit and can therefore considered to be directly usable knowledge. We gave examples in chapter 4.1 where the information about the values scale was given by domain experts and annotated to the data scheme. These annotations, can be used to configure the data quality mining s algorithms for further improvements of the achieved results by adjusting them to the needs induced by the underlying data schema and the domain expert s knowledge that would otherwise not be available or would difficulty be utilizable.
10 Fabian Grüning 6 Conclusions and further Work In this paper it was shown that classifiers can be employed to find inconsistencies in data sets and to give concrete recommendations for correct values. This approach was first made plausible through a discussion together with the decision to employ support vector machines as the classification algorithm and later through the results of a prototype. For a holistic approach for data quality mining there are still the data quality dimensions accuracy, correctness, and currency open for further research. The solutions for these dimensions will be discussed in upcoming papers. The positive influence of ontologies for the data quality mining approach in particular and checking for consistency problems in general by employing the additional semantical knowledge in contrast to other modeling techniques was highlighted. The results presented in this paper were achieved in a project funded by EWE AG (see http://www.ewe.de/), which is a major German utility. Bibliography Bortz J (2005) Statistik. Springer Medizin Verlag, Heidelberg. Grüning F (2006) Data Quality Mining in Ontologies for Utilities. In: Managing Environmental Knowledge, 20 th International Conference of Informatics in Environmental Protection Hipp J, Güntzer U, Nakhaeizadeh G (2002) Data Mining of Association Rules and the Process of Knowledge Discovery in Databases. In: Lecture Notes of Computer Science: Advances in Data Mining: Applications in E- Commerce, Medicine, and Knowledge Management, Springer Berlin/Heidelberg, Volume 2394/2002. Noy F N, McGuinness D L (2001) Ontology Development 101: A Guide to Creating Your First Ontology. Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical Informatics Technical Report SMI-2001-0880. Redman TC (1996) Data Quality for the Information Age. Artech House, Inc. Russell S, Norvig P (2003) Artificial Intelligence: A Modern Approach. Prentice Hall. Scannapieco M, Missier P, Batini C (2005) Data Quality at a Glance. In: Datenbank-Spektrum, Volume 14, Pages 6-14.
Data Quality Mining: Employing Classifiers for Assuring consistent Datasets 11 Witten I H, Frank E (2005) Data Mining: Practical machine learning tools and techniques. 2 nd Edition, Morgan Kaufmann, San Francisco. Mierswa I (2007) YALE Yet Another Learning Environment. http://yale.sf.net/ (last access 31.1.2007)