Towards a Practical Approach to Discover Internal Dependencies in Rule-Based Knowledge Bases

Transcription

1 Towards a Practical Approach to Discover Internal Dependencies in Rule-Based Knowledge Bases Roman Simiński, Agnieszka Nowak-Brzezińska, Tomasz Jach, and Tomasz Xiȩski University of Silesia, Institute of Computer Science ul. Bȩdzińska 29, Sosnowiec, Poland {roman.siminski,agnieszka.nowak, tomasz.jach,tomasz.xieski}@us.edu.pl Abstract. In this paper, we intend to introduce the conception of discovering the knowledge about rules saved in large rule-based knowledge bases, both generated automatically and acquired from human experts in the classical way. This paper presents a preliminary study of a new project in which we are going to join the two approaches: the hierarchical decomposition of large rule bases using cluster analysis and the decision units conception. Our goal is to discover useful, potentially implicit and directly unreadable information from large rule sets. Keywords: rule knowledge bases, inference, decision unit, cluster analysis, data mining. 1 Introduction The last decade resulted in significant development of data mining methods, tools and applications. Data mining and knowledge-discovery in databases is the process of automatically searching large volumes of data for useful patterns (typically rules) [1, 2]. We will assume that the result of data mining is a set of rules, herein after referred as a knowledge base. From our point of view, the resulting knowledge base is the most interesting feature, rather than the process of acquiring rules from data, but usually we will have in mind the automatic rules generation method based on the rough set theory [3 5]. When the rule sets have been induced from data they can be applied in several ways, for example to classify the unseen cases in classifiers or to build the knowledge base of decision support systems. However, in many applications the process of rule induction is in fact discovering previously unknown knowledge, discovered rules are previously unknown and potentially surprising. Several hundred and sometimes thousands of rules is a frequent result of a data mining process applied on the real-world data sets. The dependencies between rules from different large rule sets are typically not simple and clear enough. An interesting paradox arises: data mining, which was to bring out knowledge hidden in large databases, often discovers and provides large sets of rules, J.T. Yao et al. (Eds): RSKT 2011, LNCS 6954, pp , c Springer-Verlag Berlin Heidelberg 2011

2 Towards a Practical Approach to Discover Internal Dependencies 233 which contain knowledge which is hard to understand and are directly useless for domain experts. Thus, in many cases the methods of the nontrivial extraction of implicit... and potentially useful information from......large data sets... produce large rule sets with nontrivial, probably useful, but unreadable knowledge for human experts who attempt to verify the discovered knowledge. The main goal of this paper is to introduce the conception of discovering the knowledge about rules saved in large rule-based knowledge bases, both generated automatically and acquired from human experts in the classical way. This paper presents a preliminary study of a new project in which we are going to join the two approaches: the hierarchical decomposition of large rule bases using cluster analysis and the decision units conception. The proposed approach assumes that we reorganize any attributive rule knowledge base from the set of not related rules to groups of similar rules (using cluster analysis) [8] or decision units [6]. These structurs will be a source for extracting relevant information about the rules, this information is useful for the understanding of the internal structure of the knowledge base, as it describes the knowledge about knowledge saved in the rule sets. We can use this information to extract the current decision model implicitly stored in the rule sets, as well as for formal verifying and validating the rules against the expert knowledge. We plan to build a modular, hierarchically organized rule base using the cluster analysis method and decision units. These methods have been successfully used in the optimization of the inference task [6] due to the analysis of the internal properties discovered in the rule sets [8, 12]. In this work we want to introduce an extension of the clustered rules and the decision units oriented for extraction of additional knowledge from rule-based knowledge bases. The practical goal of the project is the implementation of visual-oriented software tool for knowledge engineering: a new, second version of kbbuilder system [12] and HKB_Builder system [8]. 2 Preliminaries and Basic Notation The proposed approach is dedicated to the rule knowledge bases containing the Horn clause rules, where literals are coded using attribute-value pairs. Such form of rules representation is very popular and widely used in data mining. Let R be the rule knowledge base containing m rules: R = {r 1,r 2,...,r m },where r : l 1 l 2... l n c and n is the number of conditional literals in the rule r, l i denotes the i-th conditional literal of r rule and c denotes the conclusion literal of r rule. Let A = {a 1,a 2,...a n } is a non-empty finite set of conditional and decision attributes and for every a A the set V a is called the value set of a: A = {a 1,a 2,...a n }, V a = {v1 a,va 2,...,va k }. Each attribute in a A may be a conditional and/or a decision attribute. Let literals of the rules from R become (a, vi a)wherea A and va i V a and let notations (a, vi a)anda = va i be equivalent. We make no assumption about rules in the set R. This set can be a result of a data mining process as well as it may be created by human expert or knowledge engineer, still it is possible

3 234 R. Simiński et al. that R contains errors and anomalies, we will use such a rule set for forward and backward inference, which probably will create a deep inference path. 3 Decision Units as a Decision Model for Rule Base The concept of decision units originally came into existence as a tool of rule base verification, the decision units can be also considered as a simple tool for rule knowledge base modeling. The concept of a decision unit is described in the [10 12] and due to the limited space in this paper can not be presented in detail. The idea of decision units allows us to divide a set of rules into subsets according to a simple criterion: dividing by the elementary decision (the rules contained in the subset have exactly this same attribute-value pair (a, v) in the conclusion), dividing by the decision (the rules contained in the subset have exactly this same attribute a in the each attribute-value pair (a, v) in the conclusion) or dividing by the general decision (like above, but we do not take into account the information about values appearing in the conclusions of the rules). The decision unit is the triple (I,O,R)whereR is the subset of rules (R R) created according to one of the three criteria described above. O denotes the set of the output entries of the decision unit, output entries are conclusions of rules from R, I denotes the set of the input entries of decision unit, input entries are the conditions of rules from R [12]. According to the selected rules dividing criterion, we obtain three kinds of decision units [11]: elementary decision units denoted as triple U e =(I e,o e,r e ), ordinal decision units denoted as triple U =(I,O,R), general decision units, related to ordinal decision units, but only information about attributes are considered, denoted as triple U g =(I g,o g,r g ). In the case of the two kinds of decision units elementary and ordinal sets I e, I and O e, O contain attribute-value pairs (a, v). For the general decision units we consider only the attributes, therefore I g and O g are the sets of attributes. An attribute usually represents in knowledge base particular concept or property from the real word. The value of an attribute express the determined meaning of the concept or state of the property. If we consider a specified (a, v) pair,we think about a particular kind of concept or about a particular property state. Because in the considered case the pair (a, v) is a conclusion literal, attribute a is a decision attribute. On the high level of abstraction, an elementary decision units D e is similar to i-th decision class XA i tn the decision table [4]. O e represent concrete, ther e describes reasoning about such concrete: we can confirm (or not) hypothesis about concrete using goal-driven backward inference or we can check the availability of concrete based on currently know fact using data-driven forward inference. We can say that an elementary decision units is a model of elementary decision about concrete from real-word concepts. An ordinary decision can be considered as the model of decision making about particular concept or property from the real word. We consider decision about concepts to be on higher level of abstraction than decision about meaning of concepts. Thus therulesetr can be constructed as the composition of elementary decision units D e. A decision unit is similar to decision table DT =(U, A {d}) with single decision attribute d.

4 Towards a Practical Approach to Discover Internal Dependencies 235 Sometimes it is interesting which relations the particular attribute a depends on. It is especially useful when we attempt to discover hidden decision model in an extensive data set which is not known at the early stage. Connections between attributes can express the relations between concepts. For this reason, we introduce general decision units U g. The connections between conclusion and conditional literals are usually hidden in rule bases and only during an inference we discover how sometimes deep the inference chains are. Such knowledge bases generate a few decision units with connections between input and output entries in this way we obtain a decision units net. An example of the usage of the properties of decision units in the inference optimization task is presented in [6, 7], same issues concerning rule base modeling contains [12]. 4 Rules Clusters as a Decision Model for Rule Base The clustering techniques have to be used whenever the size of input data exceeds capabilities of the traditional methods of data analysis. This is the case especially when we deal with a problem of large number of rules in knowledge bases in modern typical decision support system, we should cluster rules in order to increase the overall efficiency. Effective inference in such structures can be a tremendous task usually overwhelming classical systems. This is why the authors try to find some previously unknown connections in the rules which can aid to create more effective systems. At first, the rules are organized into groups of similar rules. Clustering is considered optimal if each cluster consists of very similar rules and if different clusters are easily distinguished with every another cluster. There is a vast number of methods to achieve this goal, among them there are the hierarchical and the non-hierarchical clustering algorithms. Both try to divide the datasets into the clusters, but the first one produces the hierarchical structure of the rules. By organizing data into the clusters, several additional benefits are achieved. Primo, additional information about analyzed data is acquired. The groups of rules can be further analyzed in order to observe the additional patterns which can be used to simplify the structure of the knowledge base either by simplifying the rules or by reducing the number of them. This data would not be found if the division into groups was not established. Secundo, clustering rules is the faster method of finding relevant rules in the knowledge base. It is an undeniable fact that the cluster analysis methods allow to look for the relevant rule much faster than browsing through the entire knowledge base. In this case, the hierarchical methods are better because of the possibility of finding the relevant rule by traversing through the tree of rules made by the hierarchical algorithm: AHC as well as mahc [6 8]. Every level of a tree produces more accurate answer and, additionally, the number of comparisons needed is much less than in the classical systems. The concept of new hierarchical model of the knowledge base is described in [6 8] and due to the limits of this paper can not be presented in detail. Let us just remind that it is represented by Tree = {w 1,,w 2n 1 } or Tree = {w 1,,w k } where

5 236 R. Simiński et al. k 2n 1 (if we apply the mahc algorithm which creates a smaller tree). It is essential to note that each node in that tree: w i = {d i,c i,f,i,j} (where i, j are numbers of clustered groups) is the representative of the conditional parts c i as well as the decisions d i of the rules belonging to it. Also the similarity values f : R R R [0 1] of rules forming the cluster w i to each other is stored. Clustering the rules from the knowledge base allows us to agglomerate a set of the rules into the subsets according to a simple criterion of their inner similarity. Apparently, each representant c i consists of all (or the most frequent) literals c i : l 1 l 2... l k d j of the conditional parts of the rules clustered in a cluster i (k is the number of conditional literals). At the first step of the algorithm, having the set of rules from knowledge base (R), each rule r i (r j R) represents a node w i in created Tree structure. In the second step, two most similar rules (on the basis of comparison the set of conditional attributes and their values) are found and linked in a cluster, which creates a new node at the higher level of the Tree. We may say that all clusters from nodes that are not leaves represent some kind of concepts. That is why the description of the rules forming a cluster is just a connection of all (or the most frequent) pairs (a, v) from conditional part of those rules. The process ends when there is only one cluster with every rule from the system in it (AHC) or at the desired, previously set level (mahc)[6, 8]. Having rules organized into clusters shortens the time needed to find relevant ones. Because the created structure is a kind of binary tree, we can state that the time efficiency of searching such trees is O(log 2 n). It means that the rule interpreter in a given decision support system does not have to search the whole knowledge base in the time efficiency of O(n) (like it is for the classical knowledge bases). However, in order to have hierarchical structure computed (needed for fast lookup of queries), firstly the clustering algorithm has to be executed. Time complexity of typical hierarchical algorithms varies, but is often a member of O(n 2 ) class (noticeable fact: one must execute this algorithm only once). 5 Conclusions The main purpose of this study was to introduce a foundations of practical approach to the internal dependencies discovery in rule-based knowledge bases. We have presented the motivation and objectives of the project, next we have introduced the concept of the representation of knowledge using the clusters analysis and the decision unit. We plan to build a modular, hierarchically organized rule base using these methods. We paraphrase the classical definition of data mining and we shall to introduce the the term knowledge mining the clusters of rules can be analyzed in order to discover the hidden patterns in the knowledge base, the decision units allow us to discover, verify and improve the current decision model. These methods have been successfully used in the optimization of the inference task. We believe that the techniques described in this paper will allow us to extract some kind of the new, metaknowledge from the large knowledge bases. This information will be useful both in knowledge engineering and in the

6 Towards a Practical Approach to Discover Internal Dependencies 237 utilisation of rule bases within the decision support systems. In the first case we expect that the proposed approach will result in improved the quality of knowledge base, in the second case the groving of efficiency of inference is expected. The practical goal of the project is the implementation of visual-oriented software tool for knowledge engineering: a new, second version of kbbuilder system and HKB_Builder system. References 1. Frawley, W., Piatetsky-Shapiro, G., Matheus, C.: Knowledge Discovery in Databases: An Overview. AI Magazine, (Fall 1992) 2. Hand, D., Mannila, H., Smyth, P.: Principles of Data Mining. MIT Press, Cambridge (2001) 3. Nguyen, H.S., Skowron, A.: Rough Set Approach to KDD (Extended Abstract). In: Wang, G., Li, T., Grzymala-Busse, J.W., Miao, D., Skowron, A., Yao, Y. (eds.) RSKT LNCS (LNAI), vol. 5009, pp Springer, Heidelberg (2008) 4. Skowron, A., Komorowski, H.J., Pawlak, Z., Polkowski, L.T.: A rough set perspective on data and knowledge. In: Handbook of Data Mining and Knowledge Discovery, pp Oxford University Press, Oxford (2002) 5. Moshkov, M., Skowron, A., Suraj, Z.: On Minimal Rule Sets for Almost All Binary Information Systems. Fundamenta Informaticae 80 (2007) 6. Nowak, A., Simiński, R., Wakulicz-Deja, A.: Towards modular representation of knowledge base. In: Advances in Soft Computing, pp Physica-Verlag, Springer Verlag Company, Heidelberg (2006) 7. Nowak, A., Simiński, R., Wakulicz-Deja, A.: Two-way optimizations of inference for rule knowledge bases. In: Proceedings of International Conference CSP 2008, Concurrency, Specification And Programming, September 29-October 1, vol. 3, pp Humboldt-Universität, Berlin (2008) 8. Nowak, A., Wakulicz-Deja, A.: The way of rules representation in composited knowledge bases. In: Advanced In Intelligent and Soft Computing, Man - Machine Interactions, pp Springer, Heidelberg (2009) 9. Nowak-Brzezińska, A., Jach, T., Xiȩski, T.: Wybór algorytmu grupowania a efektywność wyszukiwania dokumentów. Studia Informatica, Zeszyty Naukowe Politechniki Śląskiej 31(2A(89)), (2010) 10. Simiński, R., Wakulicz-Deja, A.: Verification of Rule Knowledge Bases Using Decision Units. In: Advances in Soft Computing, Intelligent Information Systems, pp Physica Verlag, Springer Verlag Company, Heidelberg (2000) 11. Simiński, R., Wakulicz-Deja, A.: Decision units as a tool for rule base modeling and verification. In: Advances in Soft Computing, Information Processing and Web Mining, pp Springer Verlag Company, Heidelberg (2003) 12. Simiński, R.: Decision units approach in knowledge base modeling. In: Recent Advances in Intelligent Information Systems, pp Academic Publishing House EXIT, New Jersey (2009)