the beginning of a run of a GA a large population of random chromosomes is created. Chromosome models depend on the case.

Transcription

1 Comparison of Document Similarity in Information Retrieval System by Different Formulation Poltak Sihombing 1, Abdullah Embong 2, Putra Sumari 3 1, 2,3 School of Computer Science, Universiti Sains Malaysia, Penang, Malaysia 1 poltakhombing@yahoo.com ; 2 ae@cs.usm.my; 3 putras@cs.usm.my ABSTRACT In this paper we are going to implement Horng & Yeh s formulation in Information Retrieval System, (IRS) and to compare it with the Jaccard s formulation and Dice s formulation. In the previous research, we have developed the Jaccard and Dice s formulation in a prototype called the Journal Browser. Each technique has been implemented in IRS using Genetic Algorithm (GA). The objective of GA was to find a set of documents which best fit the searcher s needs. In this study, an evaluation function for the fitness of each chromosome was selected based on Horng & Yeh s score. This score is formulated to measure the relationship of the query with some documents in a database. To initialize a population of the queries, we need first to decide the number of genes for each individual and the total number of chromosomes (popsize) in the initial population. GA is basically based on natural biological evolution theory. The parent solution (chromosome) with the higher level of fitness has a bigger similarity percentage of documents, while those with lower level of fitness have less similarity percentage of documents. By the similarity percentage of documents, the user can choose the most relevant document from the database. KEYWORDS Database, information, retrieval, document, similarity, genetic algorithm. 1. Introduction The goal of any IRS is to help a user to locate the most similar documents that have the potential to satisfy the user information needs. To solve this problem, researchers have implemented several methods such as inverted index, Boolean querying, knowledge-based, neural network, probabilistic retrieval and machine learning approach [1]. A newer paradigm, generally considered to be the machine learning approach has attracted the attention of researchers in artificial intelligence, computer science, and other functional disciplines such as engineering, medicine, and business. In contrast to performance systems which acquire knowledge from human experts, machine learning systems acquire knowledge automatically from examples, i.e., from data source. The most frequently used techniques include symbolic, inductive learning algorithms, multiple-layered, feed-forward neural networks such as back propagation networks, and evolution-based GA[2] [3]. The GA was used because of its ability to determine document similarity by keywords competition. A keyword represents a gene (a bit pattern), a document's list of keywords represents individuals (a bit string), and a collection of documents initially judged relevant by a user represents the initial population. [4]. In this research we are interested to investigate the use of GA to determine a document similarity by using Horng and Yeh s fitness measure and to implement this technique in a prototype of Journal Browser. We have also implemented the formulation of Jaccard and formulation of Dice in the previous research. 2. Using GA for IRS The literature study revealed several implementations of genetic algorithms in information retrieval. For example, Gordon[5] proposed genetic algorithms based on approach of document indexing. Competing document descriptions (keywords) are associated with a document and altered over time by using genetic mutation and crossover operators. In his design, a keyword represents a gene (a bit pattern), a document's list of keywords represents individuals (a bit string), and a collection of documents initially judged relevant by a user represents the initial population. Based on a Jaccard's formulation (fitness measure), the initial population evolved through generations and eventually 1

2 converged to an optimal (improved) population - a set of keywords which best described the documents. Genetic Algorithm (GA) is a part of evolutionary computing, which is rapidly growing area of artificial intelligence. GA is inspired by Darwin s theory of evolution. Simply said, problems are solved by an evolutionary process resulting in a best (fittest) solution [6]. A GA maintains a population of individuals, P (t) = x 1, x n at iteration t. Each individual represents a potential solution to the problem at hand and is implemented as some data structure S. Each solution x i is evaluated to give some measure of fitness. Then a new population at iteration t+1 is formed by selecting the fitter individuals. Some members of the new population undergo transformation by means of genetic operators to form new solutions. There are unary transformations m i (mutation type), which create new individuals by a small change in a single individual and higher order transformations c j (crossover type), which create new individuals by combining parts from several (two or more) individuals. The crossover and mutation are the most important parts of the genetic algorithm [6]. The performance is influenced mainly by these two operators. The genetic algorithm was executed in the following steps: a. Encoding of a Chromosome Algorithm begins with a set of solutions (represented by chromosomes) called population. Solutions from one population are taken and used to form a new population. This is motivated by the expectation that the new population will be better than the old one. Solutions which are then selected to form new solutions (offspring) are selected according to their fitness [8]. In IRS the keywords used in the set of userselected documents were first identified to represent the underlying bit strings for the initial population. A chromosome is formed by gene which represents bit (0 and 1). Each bit represents the same unique keyword throughout the complete GA process. When a keyword is present in a document, the bit is set to 1; otherwise it is set to 0. Each document could then be represented in terms of a sequence of 0s and 1s which is called a chromosome model. At the beginning of a run of a GA a large population of random chromosomes is created. Chromosome models depend on the case. b. Crossover. This is simply the chance that two chromosomes will swap their bits. Crossover is performed by selecting a random gene along the length of the chromosomes and swapping all the genes after that point [7]. E.g. Given two chromosomes A: B: Choose a random bit along the length, say at position 9, and swap all the bits after that point, so the above become: c. Mutation A : B : After a crossover is performed, mutation takes place. Mutation is intended to prevent falling of all solutions in the population into a local optimum of the solved problem. Mutation operation randomly changes the offspring resulted from crossover. In case of binary encoding we can switch a few randomly chosen bits from 1 to 0 or from 0 to 1. Mutation can be illustrated as follows [7]: Original offspring Original offspring Mutated offspring Mutated offspring The technique of mutation (as well as crossover) depends mainly on the encoding of chromosomes. For example when we are encoding permutations, mutation could be performed as an exchange of two genes. d. Determination Of Population Determination of population on next generation is based on the fitness score. The higher level of fitness has a bigger probability to reproduce, while those with lower level of fitness have less probability to reproduce. Usually, this probability is selected by Roulette wheel selection method. In the next generation, all population is evaluated to determine whether they have reached the expected solution [7]. 2

3 3. Implementation In the previous research before we have implemented the Jaccard formulation and also the Dice formulation. In Jaccard s formulation, for 2 queries, the similarity value was found to be 25.00%. In Dice s formulation, for 2 queries, the similarity value was found to be 27.27%. For 3 queries, in Jaccard s formulation the similarity value was found to be 20.00%. In Dice s formulation, for 3 queries, the similarity value was found to be 27.39% [16]. 3.1 Jaccard Formulation Jaccard s score is formulated below: #( X Y ) = #( X Y ) Where #(S) showing number of element in S. For example [16]: S = {a, b, c, d, e, f, g, h, i, j} If X = {a, b, e, g, h, i, j} and Y = {b, c, d, f, g, j} then X Y X Y therefore = { b, g, j} #( X Y ) = #( X Y ) 3.2 Dice Formulation = { a, b, c, d, e, f, g, h, i, j} 3 10 = 0.3 Dice s score is formulated below: From this formula it can be seen that if X is equal to Y then Dice s score is equal to 1, meaning that document X is precisely the same as document Y, although this case is very difficult to be found in the database[17]. 3.3 Horng & Yeh Formulation In this research, Horng &Yeh formulation has been implemented in a form of a prototype of Journal Browser as shown in Fig.1, menu detail in Fig. 2 and Copy Files in Fig. 3. Horng & Yeh formulation as shown below: Where D is total of document retrieval and r (d) is a function of relevant document, d is set to 1 if a keyword present in document, otherwise it is to 0. The formulation of Horng and Yeh represented the value of similarity measure in Genetic Algorithm. The fitness with a higher score reflects a higher probability similarity of document. Application of this technique will facilitate searching and retrieval of the required document from one or more databases based on the similarity level [13] [14]. 4. The Experimental Result and its Analysis Some of the example results of document similarity are shown in appendix 1. Several queries have been given with different number of document request. The result shows that documents similarity levels are consistent even though the percentage of similarity may change. This means that the order of the document resulted from the queries may changes but most of the documents resulted in the top choice remain. When more documents were used as the source of the queries, the document similarity level may decrease because the fitness value decreases. For 2 queries, 3 queries and 4 queries, the similarity value can be seen as shown in Table 1, Table 2, Table 3, and Fig. 4. Table 1. Comparison of Percentage Similarity (2 Queries) Method Percentage Jaccard Dice Horng & Yeh Doc Doc Doc Doc Doc

4 Fig.1. Journal Browser Fig. 2. Menu Detail Fig. 3. Menu Copy Files 4

5 Table 2. Comparison of Percentage Similarity (3 Queries) Method Percentage Jaccard Dice Horng & Yeh Doc Doc Doc Doc Doc Table 3. Comparison of Percentage Similarity (4 Queries) Method Percentage Jaccard Dice Horng & Yeh Doc Doc Doc Doc Doc The experimental result, it is observed that if the sum of document as a query is increased, not only the sum of generation is increase but also the sum of mutation is increase as shown in Table 4 and Table 5. Table 4. Comparison of Generation Sum of Generation Sum of Jaccard Dice Horng & Yeh Query 2 Queries Queries Queries Table 5. Comparison of Mutation Sum of Mutation Sum of Jaccard Dice Horng & Yeh Query 2 Queries Queries Queries Percentage Comparison Of Document Similarity Jaccard Dice Horng & Yeh Formulation Percentage Doc. 1 Doc. 2 Doc. 3 Doc. 4 Doc. 5 Fig. 4. Comparison of Percentage Similarity The experimental result of document similarity by different formulation shows that the higher percentage of document similarity is a Dice formulation. In Dice s formulation was found to be 27.39%, Horng &Yeh s formulation was found to be 25.00%, and Jaccard s formulation was found to be 20.00% as shown in Table 2 and Fig. 4. S um O f G eneration Comparison Of Generation Jaccard Dice Horng & Yeh Formulation Fig. 5. Comparison of Generation 2 Queries 3 Queries 4 Queries It is observed that by increasing the sum generation, it can not be guaranteed that the similarity level of the retrieved document is also increased as shown in Fig. 5. 5

6 5. Conclusion and Future Work What we have learned from this is that we can draw some conclusions about overall trends in the technique of probability in document similarity by different formulation. It is observed that if the sum of query is increased, the percentage of similarity of the document retrieved may increase or decrease as shown above. But if the sum of the queries are increased the sum of crossover, mutation and generation may also increase. We can conclude that by increasing the sum of crossover, mutation and generation, it can not be guaranteed that the similarity level of the retrieved document is also increased. In a future study, we are going to use other method, and compare the result with the other method which has been developed. References [1]. Chen, Hinchey, Machine Learning for Information Retrieval: Neural Networks, Symbolic Learning, and Genetic Algorithms, Artificial Intelligence Lab, Eller College of Management, The University of Arizona, tml [2]. J. R. Quinlan. Discovering rules by induction from large collections of examples. In Expert Systems in the Micro-electronic Age, Pages , Michie, D., Editor, Edinburgh University Press, Edinburgh, Scotland, [3]. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In Parallel Distributed Processing, pages , D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Editors, The MIT Press, Cambridge, MA, [4]. R.K.Belew.Adaptive information retrieval. In Proceedings of the Twelfth Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, pages 11-20, Cambridge, MA, June 25-28, [5]. M. Gordon. Probabilistic and genetic algorithms for document retrieval. Communications of the ACM, 31(10): , October [6]. Hsiung, Sam, An Introduction to Genetic Algorithms, generation5, 2000, p, [7]. Anonym, Genetic Algorithm Tutorial, ai-junkie, [8]. Marek Obitko, Genetic Algorithms, [9]. D. E. Goldberg. Genetic and evolutionary algorithms come of age. Communications of the ACM, 37(3): , March [10]. H. Chen and K. J. Lynch. Automatic construction of networks of concepts characterizing document databases.ieee Transactions on Systems, Man and Cybernetics, 22(5): , September/October [11]. H. Chen, K. J. Lynch, K. Basu, and D. T. Ng. Generating, integrating, and activating thesauri for concept-based document retrieval. IEEE EXPERT, Special Series on Artificial Intelligence in Text-based Information Systems, 8(2):25-34, April [12]. H. Chen and J. Kim. GANNET: a machine learning approach to document retrieval. Journal of Management Information Systems, 11(3):7-41, Winter [13]. Cristina Lopez-Pujalte, Vicente P. Guerrero- Bote. Order-Based Fitness Functions for Genetic Algorithms Applied to Relevance Feedback. Journal of the American Society for Information Science and Technology- January 15, [14]. M. D. Gordon. User-based document clustering by redescribing subject descriptions with a genetic algorithm. Journal of the American Society for Information Science, 42(5): , June [15]. V. V. Raghavan and B. Agarwal. Optimal determination of user-oriented clusters: An application for the reproductive plan. In Proceedings of the Second International Conference on Genetic Algorithms and Their Applications, pages , Cambridge, MA, July [16]. Poltak Sihombing, Abdullah Embong and Putra Sumari. A Technique Of Probability In Document Similarity Comparison In Information Retrieval System. In Proceedings of the IMT-GT Conference. Parapat, Indonesia. June [17]. Poltak Sihombing, Abdullah Embong and Putra Sumari. Application of Genetic Algorithm to Determine a Document Similarity Level in IRS. In Proceedings of the Malaysian Software Engineering Conference 05 (MySec05). Penang, Malaysia. Des

7 APPENDIX 1. HORNG & YEH METHOD FOR 2 QUERIES ================= Information Retrieval Report 3/12/ :03:47 AM ================= Dokumen Yang Dipilih 1. Relevant Rule Discovery by Language Bias for Semantic Query Optimization Keywords: Database, Language, Bias, Generator, Knowledge, Discovery, Rule, Semantic, Query, Optimization 2. Using Rough Set Theory for Automatic Data Analysis Keywords: Database, Analysis, Rough Set, data, automatic ========== Himpunan Kata Kunci Keseluruhan ========== Database, Language, Bias, Generator, Knowledge, Discovery, Rule, Semantic, Query, Optimization, Analysis, Rough Set, data, automatic Generasi ke-1 Populasi Awal : [ ] [ ] Populasi Sesudah Seleksi : Populasi Sesudah Persilangan : Proses Mutasi : Populasi Sesudah Mutasi : ===== Generasi Akhir ===== Solusi : Kata kunci : Database, Analysis, Rough Set, data, automatic Metode yang digunakan: Horng and Yeh Lama Proses: 0 detik THE RETRIEVAL RESULT FOR 2 QUERIES: (22.22%)A New Approach to Active Rule Analysis and Reorganization (12.50%)Data Partitioning for Incremental Data Mining (12.50%)New Institutions for Doing Science From Databases to Open Source Biology (12.50%)Bias Generator for View Discovery in Deductive Databases (11.11%)Data Classification Techniques for Cancer Dataset HORNG & YEH METHOD FOR 3 QUERIES ================= Information Retrieval Report 3/12/ :03:47 AM ================= Dokumen Yang Dipilih 1. Relevant Rule Discovery by Language Bias for Semantic Query Optimization Keywords : Database, Language, Bias, Generator, Knowledge, Discovery, Rule, Semantic, Query, Optimization 2. Using Rough Set Theory for Automatic Data Analysis Keywords : Database, Analysis, Rough Set, data, automatic 3. A Comparative Study of Techniques to Handle Missing Values in the Classification Task of Data Mining Keywords : Data Mining, value, missing, attribute ========== Himpunan Kata Kunci Keseluruhan ========== Database, Language, Bias, Generator, Knowledge, Discovery, Rule, Semantic, Query, Optimization, Analysis, Rough Set, data, automatic, Data Mining, value, missing, attribute Generasi ke-1 Populasi Awal : [ ] [ ] [ ] 7

8 Populasi Sesudah Seleksi : Populasi Sesudah Persilangan : Proses Mutasi : Populasi Sesudah Mutasi : Generasi ke-2 Populasi Awal : [ ] [ ] [ ] Populasi Sesudah Seleksi : Proses Persilangan : Kromosom 1 dengan kromosom 3, posisi 15 Populasi Sesudah Persilangan : Proses Mutasi : Populasi Sesudah Mutasi : Generasi ke-3 Populasi Awal : [ ] [ ] [ ] Populasi Sesudah Seleksi : Proses Persilangan : Kromosom 3 dengan kromosom 1, posisi 13 Populasi Sesudah Persilangan : Proses Mutasi : Populasi Sesudah Mutasi : ===== Generasi Akhir ===== Solusi : Kata kunci : Database, Language, Bias, Generator, Knowledge, Discovery, Rule, Semantic, Query, Optimization Metode yang digunakan: Horng and Yeh Lama Proses: 1 detik THE RETRIEVAL RESULT FOR 3 QUERIES: (27.27%) Bias Generator for View Discovery in Deductive Databases (23.08%) Knowledge Acquisition for Viewbased Query Processing (23.08%) A New Approach to Active Rule Analysis and Reorganization (23.08%) Knowledge Discovery for Trigger Conflict Resolution (16.67%) SUT Filter: A System for Data Preparation to Support Knowledge Discovery 8