A Review- Text Mining Method for Classification and Clustering

Transcription

1 A Review- Text Mining Method for Classification and Clustering Gaurav Gulhane Prashant Khobragade Ashish Golghate Dept. of Computer Sci. & Engi. Dept. of Computer Sci. & Engi. Dept. of Computer Sci. & Engi. IBSSCOE, Amravati RGCER, Nagpur RGCER, Nagpur ABSTRACT: Research project selection is an important task for government organization and private research funding agencies also it is an important and frequent activity in many organizations. When a large number of research project proposals are received to government or private agencies, it is common to group them according to their similarities in research disciplines. The current methods for grouping project proposals are based on manual matching of similar research discipline areas or frequency of keywords. However, the exact research discipline areas of the proposals cannot be accurately designated by the applicants due to their subjective views and possible misinterpretations. This paper gives process for Text-mining which can solve the problem by automatically classifying text documents. Keywords: Classification, Clustering, Optimization, Text Mining. I. INTRODUCTION Text mining is a new field that attempts to glean meaningful information from natural language text. It may be loosely characterized as the process of analyzing text to extract information that is useful for particular purposes. Compared with the kind of data stored in databases, text is unstructured, amorphous, and difficult to deal with algorithmically. The field of text mining usually deals with texts whose function is the communication of factual information or opinions, and the motivation for trying to extract information from such text automatically is compelling even if success is only partial. Research project selection is an important task for many private agencies and in government sector. The number of research proposal are received in government sector and research funding agencies has more than doubled in the past four years, with over proposals submitted in one deadline in a year. Four to five reviewers are assigned to review each proposal so as to assure accurate and reliable opinions on proposals [1] [9]. To deal with the large volume, it is necessary to group proposals according to their similarities in research disciplines and then to assign the proposal groups to relevant reviewers. The scientific departments are the decision-making units responsible for funding recommendations and management of funded projects. The department is responsible for the selection tasks, and it dedicates the tasks to divisions or programs [3] [7]. Division managers or program directors then group the proposals and assign them to external reviewers for evaluation and commentary. However, they may not have adequate knowledge in all

2 research disciplines, and contents of many proposals were not fully understood when the proposals were grouped. Therefore, there was an urgent need for an effective and feasible approach to group the submitted research proposals with computer supports. This can be solved by Text Mining method used in research project selection. 3. Anne DePiante Henriksen and Ann Jensen Traynor (1999) decision model with knowledge rules, for the assignment of expert to review of R&D project proposal. This present a method for performing R&D project selection based on the relative value to the organization of the proposed research. Figure 1: Process of Research Project Proposal The figure 1 shows process of project proposal in system; after the project proposal is submitted in government and research funding agencies it may categories it on the basis of research discipline and for grouping of this project proposal is based on the number of keywords occurring in paper [5][6]. The categorized proposal is then assigned to expert for the review. Text mining method is effectively solving the discipline area of project paper proposal to expertise to gives there review on their respective areas. II. LITERATURE REVIEW Evaluation Approach It present a hybrid knowledge and model approach with integrate mathematical 4 Cook et al.(2006) Sr References No. 1. Young- Houng Sun, Jian Ma. (2007) 5 Jian Ma, Wei Xu, Yonghong Sun, fraim Turban, Shouyang Wang.(2012) It presented a method of optimal allocation of proposals to reviewers in order to facilitate the selection process. This presents a novel ontology-based textmining approach to cluster research proposals based on their similarities in research areas. III. METHOD FOR CLASSIFICATION After proposals are submitted, the next important task is to group proposals and assign them to reviewers. The proposals in each group should have similar research characteristics. For instance, if the proposals in a group fall into the same primary research discipline (e.g., supply chain management) and the number of proposals is small, manual grouping based on keywords

3 listed in proposals can be used. However, if the number of proposals is large, it is very difficult to group proposals manually. Figure 2. Selection of Project for Expertise In figure 2 shows classification of project proposal in to their disciplines, in which expertise gives there reviews at their level, area of discipline having more than one reviewer. Classification of proposal is categories on the basis of frequencies of keyword occurring in project proposal. The first step is to classify the experts into different disciplines according to their research areas. The level setting rule described above is one of the many alternative methods to give respective expertise levels to reviewers, and is open to discussion for possible deficiencies. In order to obtain objective and fair evaluation of the proposed projects, the conflicts of interests between applicants and reviewers should be avoided. With above three step reviewer make decision for project to classify on respective groups [4]. IV. TEXT MINNIG PROCESS Although there are several text-mining approaches that can be used to cluster and classify documents. First, a research project containing the projects funded in latest five years is constructed according to keywords, and it is updated annually [3]. Then, new research proposals are classified according to discipline areas using a simple sorting algorithm. The new proposals in each discipline are clustered using a selforganized mapping (SOM) algorithm [9]. If the number of proposals in each cluster is still very large, they will be further decomposed into subgroups [1]. Phase 1: Funding agencies maintain a directory of discipline areas that form a tree structure in domain of different research areas. Research project is a public concept set of the research project management domain. The research topics of different disciplines can be clearly expressed by a research methodology of domain. Suppose that there are K discipline areas, and A k denotes discipline area k (k = 1, 2,..., K). A research proposal can be constructed in the following three steps to represent the topics of the disciplines. Phase 2: Classifying new research proposals in to disciplines areas to which they belong. A simple sorting algorithm is used next for proposals classification. Phase 3: The main clustering process consists of five steps, as shown in Fig. 3 text document collection, text document preprocessing, text document encoding,

4 vector dimension reduction, and text vector clustering [3]. Figure 3. Text Mining Process Step1) Text document collection. After the research proposals are classified according to the discipline areas, the proposal documents in each discipline Ak (k = 1, 2,..., K) are collected for text document preprocessing. Step2) Text document preprocessing. The content of research proposal contains unstructured data on this data preprocessing is performed. Step3) Text document encoding. After text documents are segmented, they are converted into a feature vector representation. Step4) Vector dimension reduction. The dimension of feature vectors is often too large; thus, it is necessary to reduce the vectors size by automatically selecting a subset containing frequency of keyword. Step5) Text vector clustering. This step uses an SOM algorithm to cluster the feature vectors based on similarities of research areas. 5. METHOD FOR CLUSTERING The optimization technique is designed to perform clustering process based on the conceptual optimal weight. This technique can transform a feature-represented document into a concept represented one. Therefore, the target document corpus will be clustered in accordance with the concepts representing individual document, and thus, achieve the proceeding of document clustering at the conceptual level. The system uses the text documents for the clustering process. Initially document preprocessing is done. Then the next step is to identify the featured words. The feature selection process is carried out using improved Niching memetic algorithm and improved GA algorithm. Figure 4. Optimization Technique System Then conceptual optimal weight is calculated and based on this optimal weight document clustering is performed. The concept weight is also called the Semantic weight. The figure 4 shows the overview of Optimization Technique system. Optimization technique is used in text mining for clustering of text document for optimal result. CONCLUSION

5 In this paper process of Text mining is discussed for the section of text paper of respective domain. The classification method is used to classify the text based on frequency of keywords. The clustering method is used to optimize the current search of keywords. REFERENCES: [1] Y. H. Sun, J. Ma, Z. P. Fan, and J. Wang, A group decision support approach to evaluate experts for R&D project selection, IEEE Trans. Eng. Manag., vol. 55, no. 1, pp , Feb [2] T. H. Cheng and C. P. Wei, A clustering-based approach for integrating document-category hierarchies, IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 38, no. 2, pp , Mar [3] Jian Ma, Wei Xu, Yong-hong Sun, Efraim Turban, Shouyang Wang, and Ou Liu, An Ontology-Based Text-Mining Method to Cluster Proposals for Research Project Selection, IEEE Trans. Syst., Man, Cybern. Part A, Syst & Humans, vol. 42, no. 3, May [4] Yong-Hong Sun, Jian Ma, Zhi-Ping Fan, Jun Wang A Hybrid Knowledge and Model Approach for Reviewer Assignment, 40th Hawaii International Conference on System Sciences [5] H. C. Yang and C. H. Lee, A text mining approach for automatic construction of hypertexts, Expert Syst. Appl., vol. 29, no. 4, pp , Nov [6] Z. P. Fan, Y. Chen, J. Ma, and Y. Zhu, Decision support for proposal grouping: A hybrid approach using knowledge rule and genetic algorithm, Expert Syst. Appl., vol. 36, no. 2, pp , Mar [7] O. Liu and J. Ma, A multilingual ontology framework for R&D project management systems, Expert Syst. Appl., vol. 37, no. 6, pp , Jun [8] C. Choi and Y. Park, R&D proposal screening system based on text mining approach, Int. J. Technol. Intell. Plan., vol. 2, no. 1, pp , [9] J. Vesanto and E. Alhoniemi, Clustering of the self-organizing map, IEEE Trans. Neural Network., vol. 11, no. 3, pp , May 2000.