# Anais do IX Congresso Brasileiro de Redes Neurais / Inteligência Computacional (IX CBRN) Ouro Preto de Outubro de 2009

Save this PDF as:

Size: px
Start display at page:

Download "Anais do IX Congresso Brasileiro de Redes Neurais / Inteligência Computacional (IX CBRN) Ouro Preto 25-28 de Outubro de 2009"

## Transcription

2 responders) is induced from the training set and from some assumptions about the working set. The basic assumption of our work is that the separating surface should have maximum margins in both the training and working sets. A new SSL method for artificial neural networks (ANN) learning, based on the geometrical computation of the separation margin is presented and applied to the problem. The separation region is associated to the lowest density region in the input space formed by both data sets. A method to geometrically identify the separation region and to compute the margins is also presented. The final solution is then obtained by addressing the problem as a multi-objective learning problem and by selecting the final model as the one which maximizes the separation margins in both datasets. In this paper we will review the basic concepts that are discussed or applied in our work, then we will describe the implemented semi-supervised method, we will present the results that we have obtained and compare them to those of other approaches for the same data. Finally we will discuss the results and some issues of this approach. This multiobjective algorithm minimizes two sliding modes surfaces ( that are defined by S v = (e e k ) and by S w = w 2 w k 2) (Costa et al., 2007). 2.3 Margin The margin is defined as the distance between the separation hyperplane and the closest points of the data set (Haykin, 2001). For support vector machines (SVM) these points are called support vectors. The optimal separation hyperplane is the one equidistant to both classes. Figure 1 is an example of the concept of optimal hyperplane and margin, denoted by ρ in the figure. The equations to compute the margin ρ are given by equation (1), (2), and (3) (Shawe-taylor and Cristianini, n.d.) : ρ = m y i d i (1) i=1 d i (w, b, x) = (w x i + b) w (2) 2 Review ρ (w, b) = 2 w (3) 2.1 Semi-Supervised Learning In semi-supervised learning we are concerned by using the information conveyed by the unlabeled data set, in addition to that of the labeled (training) set. The use of the unlabeled data together with a small amount of labeled data can improve the efficiency of the learning process. SSL can be of great value when labeled data is difficult to obtain (Zhu, 2008). In contrast, the acquisition of unlabeled data can be of lesser cost. In SSL, a supervisor can compare the outpouts of the network with the known labels of the learning dataset, and make the necessary adjustments of the network s weights. But the supervisor is of no use for the unlabeled data. where m is the number of selected patterns for computing the margin, and d is the distance between the pattern i and the separation hyperplane (computed according to equation 2.) In general, the margin is computed for the support vectors because they are in each class, the points which are the closest to the separation hyperplane. These points synthesize all the information about the classes that define the separation hyperplane. In other words, training the network with all the input patterns is equivalent to training it with the only support vectors. The optimal hyperplane is found by maximizing the margin ρ (w, b) subject to y i [(w x i ) + b] 1, (Gunn, 1997), which takes the form of equation 3. Maximizing the separation margin is equivalent to minimizing the weight vectors of the network (w) (Haykin, 2.2 Sliding Mode of Control for Multi-Objective Neural Networks (SMC-MOBJ). A multi-objective algorithm (Costa et al., 2007) aims at reaching a balance between the bias and variance of a neural network, by selecting Pareto solutions in the objectives space defined by the vector norm w of the network s weights, and the classification error, e, on the training set. The sliding mode algorithm is capable of generating arbitrary trajectories in the space of the objectives and reaching any solution (e k, w k ) in the space of the objectives defined by the sum of the quadratic error e k and the weights vector norm w k. Figure 1: Illustration of an optimal hyperplane for linearly separable patterns.

3 (a) The two moons problem (a) Slicing method applied to the two moons problem (b) Solution grid in the objectives space Figure 2: The two moons problem and its solution grid (b) Patterns selected in all simulation (100%) Figure 3: MILFAT method 2001). Given a set of solutions we can compute geometrically the distance between the separation hyperplane and a given data point, then select the solution of maximum margin. 3 Description of the new method Given a set of solutions (fig. 2(b)) obtained from a training dataset generated by the SMC-MOBJ method (Costa et al., 2007), we take into consideration the distribution of both patterns of the labeled and unlabeled sets in order to select the solution of best generalization. This solution is located between the classes, in the center of the area of patterns lower density, considering that such an area exists. This optimal separation solution has maximum margins, and we can compute it geometrically in relation to the patterns that deleneate the lower density area between the classes in all its extensions (Coelho, 2009). in this section we will illustrate the method on a well known two dimensional problem, the two moons problem (figure 2(a)). The figure 2(a) depicts the patterns belonging to the labeled (training) and unlabeled sets. 3.1 First proposition: Method for Identifying the Borders by Slicing - MILFAT The method for identifying the limits by slicing (MIL- FAT) is based on a grouping method (Fuzzy c-means clustering FCM) (Bezdec, 1981). We have applied the FCM method to the unlabeled data set for any final number of partitions c. In a second stage, for each axis of the input space, we have separated the data into bands and later we have determined the borders between the regions of the FCM partition, walking along the sorted data values of the others axis (fig. 3(a)). Most of these borders were real frontiers between classes. However, some meaningless borders (artifacts) were also present. This process was iterated for each axis. To discard the artifact borders, the FCM algorithm was run for an additional iteration on the unlabeled set, with a number of partitions different from the one used in the first execution of the algorithm. We have compared the borders found in the two simulations of the FCM and we have checked them against the one that had been selected in all the simulations. Those of the patterns that were next to the regions of lower density between classes tended to be selected in all the simulations with different partitions, while the artifacts (those which were on the borders between partitions but in the same class) tended to be differ from one execution to another (cf. fig. 3(b)).

4 3.2 Second proposition: Method for Identifying the Borders by Probabilities - MILP In this method, at the end of the FCM simulation, instead of slicing we have used the information of the FCM resultant probability matrix, taking as the borders the n patterns of lowest probability among all of those that had been clustered in one partition c (and this, for each partition). Iterating this process for different numbers of FCM partitions, we have marked as border patterns those chosen in all the simulations. 3.3 The Geometric Margin Computation For computing the margins, we need to go back to equation 1. Our proposition is to compute the distance between the separation hyperplane and the selected input patterns through all the borders between the classes in the hidden layer of the network. For reasons that are discussed in (Coelho, 2009) we have defined three different ways to compute the geometric margin (ρ S = ρ 1 + ρ 2, ρ M = ρ 1 ρ 2 and ρ D = min (ρ 1, ρ 2 ) /max (ρ 1, ρ 2 )) to evaluate which of them was the best one. In these expressions, ρ 1 is the sum of the distances of the limit patterns of class -1 to the hyperplane, weighed by the network output, and ρ 2 is the same for class +1, as in equation Selection of the Best Solution We have generated a solution set in the objectives space by training the network with the inductive set (fig. 2(b)) using a sliding mode MOBJ algorithm (Costa et al., 2007). For each solution we have summed the computed geometric margin for the inductive set with the one computed for the unlabeled set, then we have chosen the solution that had the largest total margin. We have chosen the grid solution G : (e, w ) that maximized the total geometric margin ρ tot explicited in (4). The values ρ labeled and ρ unlabeled are the results defined in section 3.3 for labeled and unlabeled sets respectively. ρtot = ρ labeled + ρ unlabeled (4) 4 Applying the Method to Genomic Datasets The data of this application of the method to the prediction of the response to preoperative chemotherapy for breast cancer are, on the one hand, the expression levels of a set of genes measured on tumor tissues for 133 patients. The patients with no residual disease at the time of surgery were responders to the treatment (pathological complete response PCR ), while those with residual disease were non responders (NOPCR patient cases). Our goal was to design a model relying on the gene expression levels for predicting the outcome of the treatment (for patient cases that were not member of the learning set). The 82 patient data from the MD Anderson Cancer Center (Texas, USA) were the training dataset, and the 51 patient data from the Institut Gustave Roussy (Villejuif, France) were the unlabeled dataset. The works of Horta (Horta, 2008), Hess (K.R. Hess and Pusztai., 2006), Natowicz (Rene Natowicz and Rouzier., march 2008) and Braga (R. Natowicz and Costa, 2008) (Braga et al., n.d.), have made use of these data to select relevant probes for prediction. Among them, we have selected three probes sets for our application of our semi-supervised learning method : the 30 probes set selected by Natowicz (Rene Natowicz and Rouzier., march 2008), the 18 probes set and the 11 probes set selected by Horta (Horta, 2008) The results (table 1) found by the SSL method based on geometric margin are very interesting, because the results obtained at works like Horta s one (Horta, 2008), (table 2), were achieved considering all training data as labeled ones. The SSL method consider labeled data and uses aditional information from unlabeled data distribution. In tables FP, FN, Ac, Se and Sp means, respectively, false positive results, false negative, acuracy (%), sensitivity (%) and specificity (%) for both training and validation sets. Model A refers to set of 18 probes with MILP/MILFAT methods for ρ S /ρ M /ρ D methods calculation, model B is 11 probes with MILP/MILFAT for ρ S /ρ M /ρ D, model C is 30 probes with MILP for ρ S /ρ M, model D is 30 probes with MILP for ρ D, model E is 30 probes with MILFAT for ρ S /ρ M and model F is 30 probes with MILFAT for ρ D. In table 2 model G is 18 probes with SVM RBF, model H is 32 probes with SVM RBF and model I is 18 probes with LASSO. Table 1: MSMG s results Training Validation Models FP FN Ac Se Sp FP FN Ac Se Sp A ,9 85,7 96, ,4 84,6 81,6 B ,7 85,7 95, ,4 84,6 78,9 C ,1 90,5 82, ,4 92,3 76,3 D ,4 85,7 85, ,4 92,3 73,7 E ,5 85,7 78, ,4 92,3 78,9 F ,7 85,7 80, ,4 92,3 78,9 Table 2: Horta s results Training Validation Modelo FP FN Ac Se Sp FP FN Ac Se Sp G ,1 85,7 98, ,3 84,6 86,8 H ,9 85,7 96, ,3 92,3 84,2 I ,7 81,0 96, ,3 76,9 89,5 The best SSL method results are comparable to best results achieved in (R. Natowicz, 2009) that used a SVM with linear kernels (See table 4 of (R. Natowicz, 2009)). Note that we achieved acuracy values for training and validation a little smaller than the best result on Horta s work, however, it is still a good result. We get the same sensibilities values and ours specificities were slightly worse compared to the best result in table 2.

5 5 Discussion The semi-supervised method based on geometric margin was applied to the problem of predicting the response to preoperative chemotherapy in the treatment of breast cancer. The results obtained with 18 and 11 probes are comparable to the best results achieved by Horta (Horta, 2008). The results are balanced, stable, with low error rate. The results obtained by computing the geometric margins in different ways were very close for the same probes set (defining the best way to compute it is an important issue for the SSL model (Coelho, 2009)). MILFAT method supposes to define a number of slices for searching and, as the algorithm has high computational cost, this factor is of extreme importance: more slices, higher will be processing time. In MILP, one has to set a cutoff point of the probability of a pattern belonging to the limit of the partition. The number of times the FCM has to be run in both methods and the number of partitions on each of them can also influence the outcome. However these parameters appeared to have a lesser influence on the performances of the predictor. References Ancona, N., Maglietta, R., Piepoli, A., D Addabbo, A., Cotugno, R., Savino, M., Liuni, S., Carella, M., Pesole, G. and Perri, F. (n.d.). On the statistical assessment of classifiers using dna microarray data. Bezdec, J. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York. Braga, A. P., Horta, E. G., Natowicz, R., Rouzier, R., Incitti, R., Rodrigues, T. S., Costa, M. A., Pataro, C. D. M. and Çela., A. (n.d.). Bayesian classifiers for predicting the outcome of breast cancer preoperative chemotherapy., In The Third International Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR08). Chapelle, O., Schölkopf, B. and Zien, A. (eds) (2006). Semi-Supervised Learning, MIT Press, Cambridge, MA. URL: 1 Coelho, F. G. F. (2009). Modelo semi-supervisionado aplicado à previsão da eficiência da quimioterapia neoadjuvante no tratamento de câncer de mama, Dissertação de mestrado, Universidade Federal de Minas Gerais. Cooper, C. (n.d.). Applications of microarray technology in breast cancer research. Costa, M. A., de Pádua Braga, A. and de Menezes, B. R. (2007). Improving generalization of mlps with sliding mode control and the levenbergmarquardt algorithm, Neurocomput. 70(7-9): Glas, A., Floore, A., Delahaye, L., Witteveen, A., Pover, R., Bakx, N., Lahti-Domenici, J., Bruinsma, T., Warmoes, M., Bernards, R., Wessels, L. and Veer, L. V. (n.d.). Converting a breast cancer microarray signature into a high-throughput diagnostic test. Gunn, S. (1997). Support vector machines for classification and regression, Image Speech & Intelligent Systems Group - University of southampton. Haykin, S. (2001). Redes Neurais: Princípios e Prática, Bookman. Horta, E. G. (2008). Previsores para a eficiência da quimioterapia neoadjuvante no câncer de mama, Dissertação de mestrado, Universidade Federal de Minas Gerais. K.R. Hess, K. Anderson, W. S. V. V. N. I. J. M. D. B. R. T. A. B. P. D. R. R. N. S. J. R. T. V. H. G. G. H. and Pusztai., L. (2006). Pharmacogenomic predictor of sensitivity to preoperative chemotherapy with paclitaxel and fluorouracil, doxorubicin, and cyclophosphamide in breast cancer., Journal of Clinical Oncology,24(26): Michiels, S., Koscielny, S. and Hill, C. (n.d.). Prediction of cancer outcome with microarrays: A multiple random validation strategy. R. Natowicz, A. P. Braga, R. I. E. G. H. R. R. T. S. R. and Costa, M. A. (2008). A new method of dna probes selection and its use with multi-objective neural networks for predicting the outcome of breast cancer preoperative chemotherapy., European Symposium on Artificial Neural Networks. R. Natowicz, R. Incitti, R. R. A. C. A. B. E. H. T. R. M. C. C. P. (2009). Downsizing multigenic predictors of the response to preoperative chemotherapy in breast cancer, Investigating Human Cancer with Computational Intelligence Techniques, KES International Recent Research Results Series pp Rene Natowicz, Roberto Incitti, E. G. H. B. C. P. G. K. Y. C. C. F. A. L. P. and Rouzier., R. (march 2008). Prediction of the outcome of preoperative chemotherapy in breast cancer by dna probes that convey information on both complete and non complete responses., BMC Bioinformatics, 9:149. Shawe-taylor, J. and Cristianini, N. (n.d.). Smola, bartlett, scholkopf, and schuurmans: Advances in large margin classifiers, introduction to large margin classifiers. Zhu, X. (2008). survey. Semi-supervised learning literature

### Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data

CMPE 59H Comparison of Non-linear Dimensionality Reduction Techniques for Classification with Gene Expression Microarray Data Term Project Report Fatma Güney, Kübra Kalkan 1/15/2013 Keywords: Non-linear

### Chapter 7. Diagnosis and Prognosis of Breast Cancer using Histopathological Data

Chapter 7 Diagnosis and Prognosis of Breast Cancer using Histopathological Data In the previous chapter, a method for classification of mammograms using wavelet analysis and adaptive neuro-fuzzy inference

### Advances in Natural and Applied Sciences. Breast Cancer Classification Using Various Machine Learning Techniques

AENSI Journals Advances in Natural and Applied Sciences ISSN:1995-0772 EISSN: 1998-1090 Journal home page: www.aensiweb.com/anas Breast Cancer Classification Using Various Machine Learning Techniques 1

### FRAUD DETECTION IN ELECTRIC POWER DISTRIBUTION NETWORKS USING AN ANN-BASED KNOWLEDGE-DISCOVERY PROCESS

FRAUD DETECTION IN ELECTRIC POWER DISTRIBUTION NETWORKS USING AN ANN-BASED KNOWLEDGE-DISCOVERY PROCESS Breno C. Costa, Bruno. L. A. Alberto, André M. Portela, W. Maduro, Esdras O. Eler PDITec, Belo Horizonte,

### Support Vector Machines with Clustering for Training with Very Large Datasets

Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano

### Support Vector Machines Explained

March 1, 2009 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),

### Review of Computer Engineering Research WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM. Geeta R.B.* Shobha R.B.

Review of Computer Engineering Research journal homepage: http://www.pakinsight.com/?ic=journal&journal=76 WEB PAGES CATEGORIZATION BASED ON CLASSIFICATION & OUTLIER ANALYSIS THROUGH FSVM Geeta R.B.* Department

### Predictive Data modeling for health care: Comparative performance study of different prediction models

Predictive Data modeling for health care: Comparative performance study of different prediction models Shivanand Hiremath hiremat.nitie@gmail.com National Institute of Industrial Engineering (NITIE) Vihar

### Supervised and unsupervised learning - 1

Chapter 3 Supervised and unsupervised learning - 1 3.1 Introduction The science of learning plays a key role in the field of statistics, data mining, artificial intelligence, intersecting with areas in

### Tree based ensemble models regularization by convex optimization

Tree based ensemble models regularization by convex optimization Bertrand Cornélusse, Pierre Geurts and Louis Wehenkel Department of Electrical Engineering and Computer Science University of Liège B-4000

### Data Mining Techniques for Prognosis in Pancreatic Cancer

Data Mining Techniques for Prognosis in Pancreatic Cancer by Stuart Floyd A Thesis Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUE In partial fulfillment of the requirements for the Degree

### Classification algorithm in Data mining: An Overview

Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department

### Machine Learning. 01 - Introduction

Machine Learning 01 - Introduction Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge

### Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

### BREAST CANCER DIAGNOSIS USING STATISTICAL NEURAL NETWORKS

ISTANBUL UNIVERSITY JOURNAL OF ELECTRICAL & ELECTRONICS ENGINEERING YEAR VOLUME NUMBER : 004 : 4 : (1149-1153) BREAST CANCER DIAGNOSIS USING STATISTICAL NEURAL NETWORKS Tüba KIYAN 1 Tülay YILDIRIM 1, Electronics

### TOWARD BIG DATA ANALYSIS WORKSHOP

TOWARD BIG DATA ANALYSIS WORKSHOP 邁 向 巨 量 資 料 分 析 研 討 會 摘 要 集 2015.06.05-06 巨 量 資 料 之 矩 陣 視 覺 化 陳 君 厚 中 央 研 究 院 統 計 科 學 研 究 所 摘 要 視 覺 化 (Visualization) 與 探 索 式 資 料 分 析 (Exploratory Data Analysis, EDA)

### Learning with Local and Global Consistency

Learning with Local and Global Consistency Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 7276 Tuebingen, Germany

### Learning with Local and Global Consistency

Learning with Local and Global Consistency Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 7276 Tuebingen, Germany

### Triangular Visualization

Triangular Visualization Tomasz Maszczyk and W lodzis law Duch Department of Informatics, Nicolaus Copernicus University, Toruń, Poland tmaszczyk@is.umk.pl;google:w.duch http://www.is.umk.pl Abstract.

### 203.4770: Introduction to Machine Learning Dr. Rita Osadchy

203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:

### Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms

Comparing the Results of Support Vector Machines with Traditional Data Mining Algorithms Scott Pion and Lutz Hamel Abstract This paper presents the results of a series of analyses performed on direct mail

### Maximum Margin Clustering

Maximum Margin Clustering Linli Xu James Neufeld Bryce Larson Dale Schuurmans University of Waterloo University of Alberta Abstract We propose a new method for clustering based on finding maximum margin

### Network Intrusion Detection using Semi Supervised Support Vector Machine

Network Intrusion Detection using Semi Supervised Support Vector Machine Jyoti Haweliya Department of Computer Engineering Institute of Engineering & Technology, Devi Ahilya University Indore, India ABSTRACT

### Semi-Supervised Support Vector Machines and Application to Spam Filtering

Semi-Supervised Support Vector Machines and Application to Spam Filtering Alexander Zien Empirical Inference Department, Bernhard Schölkopf Max Planck Institute for Biological Cybernetics ECML 2006 Discovery

### EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering Martin

### Data Mining and Machine Learning in Bioinformatics

Data Mining and Machine Learning in Bioinformatics PRINCIPAL METHODS AND SUCCESSFUL APPLICATIONS Ruben Armañanzas http://mason.gmu.edu/~rarmanan Adapted from Iñaki Inza slides http://www.sc.ehu.es/isg

### Knowledge Discovery and Data Mining

Knowledge Discovery and Data Mining Lecture 15 - ROC, AUC & Lift Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-17-AUC

### MS1b Statistical Data Mining

MS1b Statistical Data Mining Yee Whye Teh Department of Statistics Oxford http://www.stats.ox.ac.uk/~teh/datamining.html Outline Administrivia and Introduction Course Structure Syllabus Introduction to

### ANN-DEA Integrated Approach for Sensitivity Analysis in Efficiency Models

Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 14-24 ANN-DEA Integrated Approach for Sensitivity Analysis in Efficiency Models L. Karamali *,1, A. Memariani 2, G.R. Jahanshahloo 3 Here,

### Data Integration. Lectures 16 & 17. ECS289A, WQ03, Filkov

Data Integration Lectures 16 & 17 Lectures Outline Goals for Data Integration Homogeneous data integration time series data (Filkov et al. 2002) Heterogeneous data integration microarray + sequence microarray

### Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

### Novel Mining of Cancer via Mutation in Tumor Protein P53 using Quick Propagation Network

Novel Mining of Cancer via Mutation in Tumor Protein P53 using Quick Propagation Network Ayad. Ghany Ismaeel, and Raghad. Zuhair Yousif Abstract There is multiple databases contain datasets of TP53 gene

### Ensemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 2015-03-05

Ensemble Methods Knowledge Discovery and Data Mining 2 (VU) (707004) Roman Kern KTI, TU Graz 2015-03-05 Roman Kern (KTI, TU Graz) Ensemble Methods 2015-03-05 1 / 38 Outline 1 Introduction 2 Classification

### ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015

ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti

### Intrusion Detection via Machine Learning for SCADA System Protection

Intrusion Detection via Machine Learning for SCADA System Protection S.L.P. Yasakethu Department of Computing, University of Surrey, Guildford, GU2 7XH, UK. s.l.yasakethu@surrey.ac.uk J. Jiang Department

### HYBRID INTELLIGENT SUITE FOR DECISION SUPPORT IN SUGARCANE HARVEST

HYBRID INTELLIGENT SUITE FOR DECISION SUPPORT IN SUGARCANE HARVEST FLÁVIO ROSENDO DA SILVA OLIVEIRA 1 DIOGO FERREIRA PACHECO 2 FERNANDO BUARQUE DE LIMA NETO 3 ABSTRACT: This paper presents a hybrid approach

### Review of some concepts in predictive modeling

Review of some concepts in predictive modeling Brigham and Women s Hospital Harvard-MIT Division of Health Sciences and Technology HST.951J: Medical Decision Support A disjoint list of topics? Naïve Bayes

### Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons

Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion

### Artificial Neural Network, Decision Tree and Statistical Techniques Applied for Designing and Developing E-mail Classifier

International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-1, Issue-6, January 2013 Artificial Neural Network, Decision Tree and Statistical Techniques Applied for Designing

### Machine Learning CS 6830. Lecture 01. Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu

Machine Learning CS 6830 Razvan C. Bunescu School of Electrical Engineering and Computer Science bunescu@ohio.edu What is Learning? Merriam-Webster: learn = to acquire knowledge, understanding, or skill

### Data Clustering. Dec 2nd, 2013 Kyrylo Bessonov

Data Clustering Dec 2nd, 2013 Kyrylo Bessonov Talk outline Introduction to clustering Types of clustering Supervised Unsupervised Similarity measures Main clustering algorithms k-means Hierarchical Main

### Introduction to Support Vector Machines. Colin Campbell, Bristol University

Introduction to Support Vector Machines Colin Campbell, Bristol University 1 Outline of talk. Part 1. An Introduction to SVMs 1.1. SVMs for binary classification. 1.2. Soft margins and multi-class classification.

### Rulex s Logic Learning Machines successfully meet biomedical challenges.

Rulex s Logic Learning Machines successfully meet biomedical challenges. Rulex is a predictive analytics platform able to manage and to analyze big amounts of heterogeneous data. With Rulex, it is possible,

### Active Learning with Boosting for Spam Detection

Active Learning with Boosting for Spam Detection Nikhila Arkalgud Last update: March 22, 2008 Active Learning with Boosting for Spam Detection Last update: March 22, 2008 1 / 38 Outline 1 Spam Filters

### SURVIVABILITY ANALYSIS OF PEDIATRIC LEUKAEMIC PATIENTS USING NEURAL NETWORK APPROACH

330 SURVIVABILITY ANALYSIS OF PEDIATRIC LEUKAEMIC PATIENTS USING NEURAL NETWORK APPROACH T. M. D.Saumya 1, T. Rupasinghe 2 and P. Abeysinghe 3 1 Department of Industrial Management, University of Kelaniya,

### Towards better accuracy for Spam predictions

Towards better accuracy for Spam predictions Chengyan Zhao Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 2E4 czhao@cs.toronto.edu Abstract Spam identification is crucial

### Statistics for BIG data

Statistics for BIG data Statistics for Big Data: Are Statisticians Ready? Dennis Lin Department of Statistics The Pennsylvania State University John Jordan and Dennis K.J. Lin (ICSA-Bulletine 2014) Before

### DEA implementation and clustering analysis using the K-Means algorithm

Data Mining VI 321 DEA implementation and clustering analysis using the K-Means algorithm C. A. A. Lemos, M. P. E. Lins & N. F. F. Ebecken COPPE/Universidade Federal do Rio de Janeiro, Brazil Abstract

### DATA ANALYTICS USING R

DATA ANALYTICS USING R Duration: 90 Hours Intended audience and scope: The course is targeted at fresh engineers, practicing engineers and scientists who are interested in learning and understanding data

### Introduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu

Introduction to Machine Learning Lecture 1 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction Logistics Prerequisites: basics concepts needed in probability and statistics

### Mathematical Models of Supervised Learning and their Application to Medical Diagnosis

Genomic, Proteomic and Transcriptomic Lab High Performance Computing and Networking Institute National Research Council, Italy Mathematical Models of Supervised Learning and their Application to Medical

### Introduction to Machine Learning Using Python. Vikram Kamath

Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression

### Search Taxonomy. Web Search. Search Engine Optimization. Information Retrieval

Information Retrieval INFO 4300 / CS 4300! Retrieval models Older models» Boolean retrieval» Vector Space model Probabilistic Models» BM25» Language models Web search» Learning to Rank Search Taxonomy!

### Active Learning SVM for Blogs recommendation

Active Learning SVM for Blogs recommendation Xin Guan Computer Science, George Mason University Ⅰ.Introduction In the DH Now website, they try to review a big amount of blogs and articles and find the

### Chapter 12 Discovering New Knowledge Data Mining

Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to

### Knowledge Discovery in Stock Market Data

Knowledge Discovery in Stock Market Data Alfred Ultsch and Hermann Locarek-Junge Abstract This work presents the results of a Data Mining and Knowledge Discovery approach on data from the stock markets

### Novelty Detection in image recognition using IRF Neural Networks properties

Novelty Detection in image recognition using IRF Neural Networks properties Philippe Smagghe, Jean-Luc Buessler, Jean-Philippe Urban Université de Haute-Alsace MIPS 4, rue des Frères Lumière, 68093 Mulhouse,

### SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING

AAS 07-228 SPECIAL PERTURBATIONS UNCORRELATED TRACK PROCESSING INTRODUCTION James G. Miller * Two historical uncorrelated track (UCT) processing approaches have been employed using general perturbations

### K-Means Clustering Tutorial

K-Means Clustering Tutorial By Kardi Teknomo,PhD Preferable reference for this tutorial is Teknomo, Kardi. K-Means Clustering Tutorials. http:\\people.revoledu.com\kardi\ tutorial\kmean\ Last Update: July

### SURVIVABILITY OF COMPLEX SYSTEM SUPPORT VECTOR MACHINE BASED APPROACH

1 SURVIVABILITY OF COMPLEX SYSTEM SUPPORT VECTOR MACHINE BASED APPROACH Y, HONG, N. GAUTAM, S. R. T. KUMARA, A. SURANA, H. GUPTA, S. LEE, V. NARAYANAN, H. THADAKAMALLA The Dept. of Industrial Engineering,

### Notes on Support Vector Machines

Notes on Support Vector Machines Fernando Mira da Silva Fernando.Silva@inesc.pt Neural Network Group I N E S C November 1998 Abstract This report describes an empirical study of Support Vector Machines

### Data Quality Mining: Employing Classifiers for Assuring consistent Datasets

Data Quality Mining: Employing Classifiers for Assuring consistent Datasets Fabian Grüning Carl von Ossietzky Universität Oldenburg, Germany, fabian.gruening@informatik.uni-oldenburg.de Abstract: Independent

### Predicting the Risk of Heart Attacks using Neural Network and Decision Tree

Predicting the Risk of Heart Attacks using Neural Network and Decision Tree S.Florence 1, N.G.Bhuvaneswari Amma 2, G.Annapoorani 3, K.Malathi 4 PG Scholar, Indian Institute of Information Technology, Srirangam,

### Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones

Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 What is machine learning? Data description and interpretation

### A fast multi-class SVM learning method for huge databases

www.ijcsi.org 544 A fast multi-class SVM learning method for huge databases Djeffal Abdelhamid 1, Babahenini Mohamed Chaouki 2 and Taleb-Ahmed Abdelmalik 3 1,2 Computer science department, LESIA Laboratory,

### Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j

Analysis of kiva.com Microlending Service! Hoda Eydgahi Julia Ma Andy Bardagjy December 9, 2010 MAS.622j What is Kiva? An organization that allows people to lend small amounts of money via the Internet

### AUTOMATION OF ENERGY DEMAND FORECASTING. Sanzad Siddique, B.S.

AUTOMATION OF ENERGY DEMAND FORECASTING by Sanzad Siddique, B.S. A Thesis submitted to the Faculty of the Graduate School, Marquette University, in Partial Fulfillment of the Requirements for the Degree

### Knowledge Discovery from patents using KMX Text Analytics

Knowledge Discovery from patents using KMX Text Analytics Dr. Anton Heijs anton.heijs@treparel.com Treparel Abstract In this white paper we discuss how the KMX technology of Treparel can help searchers

### HYBRID PROBABILITY BASED ENSEMBLES FOR BANKRUPTCY PREDICTION

HYBRID PROBABILITY BASED ENSEMBLES FOR BANKRUPTCY PREDICTION Chihli Hung 1, Jing Hong Chen 2, Stefan Wermter 3, 1,2 Department of Management Information Systems, Chung Yuan Christian University, Taiwan

### A STUDY ON DATA MINING INVESTIGATING ITS METHODS, APPROACHES AND APPLICATIONS

A STUDY ON DATA MINING INVESTIGATING ITS METHODS, APPROACHES AND APPLICATIONS Mrs. Jyoti Nawade 1, Dr. Balaji D 2, Mr. Pravin Nawade 3 1 Lecturer, JSPM S Bhivrabai Sawant Polytechnic, Pune (India) 2 Assistant

### Comparison of K-means and Backpropagation Data Mining Algorithms

Comparison of K-means and Backpropagation Data Mining Algorithms Nitu Mathuriya, Dr. Ashish Bansal Abstract Data mining has got more and more mature as a field of basic research in computer science and

### International Journal of Software and Web Sciences (IJSWS) Web Log Mining Based on Improved FCM Algorithm using Multiobjective

International Association of Scientific Innovation and Research (IASIR) (An Association Unifying the Sciences, Engineering, and Applied Research) ISSN (Print): 2279-0063 ISSN (Online): 2279-0071 International

### Network Machine Learning Research Group. Intended status: Informational October 19, 2015 Expires: April 21, 2016

Network Machine Learning Research Group S. Jiang Internet-Draft Huawei Technologies Co., Ltd Intended status: Informational October 19, 2015 Expires: April 21, 2016 Abstract Network Machine Learning draft-jiang-nmlrg-network-machine-learning-00

### Fig. 1 A typical Knowledge Discovery process [2]

Volume 4, Issue 7, July 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Clustering

### International Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014

RESEARCH ARTICLE OPEN ACCESS A Survey of Data Mining: Concepts with Applications and its Future Scope Dr. Zubair Khan 1, Ashish Kumar 2, Sunny Kumar 3 M.Tech Research Scholar 2. Department of Computer

### KERNEL LOGISTIC REGRESSION-LINEAR FOR LEUKEMIA CLASSIFICATION USING HIGH DIMENSIONAL DATA

Rahayu, Kernel Logistic Regression-Linear for Leukemia Classification using High Dimensional Data KERNEL LOGISTIC REGRESSION-LINEAR FOR LEUKEMIA CLASSIFICATION USING HIGH DIMENSIONAL DATA S.P. Rahayu 1,2

### ENSEMBLE DECISION TREE CLASSIFIER FOR BREAST CANCER DATA

ENSEMBLE DECISION TREE CLASSIFIER FOR BREAST CANCER DATA D.Lavanya 1 and Dr.K.Usha Rani 2 1 Research Scholar, Department of Computer Science, Sree Padmavathi Mahila Visvavidyalayam, Tirupati, Andhra Pradesh,

### Making Sense of the Mayhem: Machine Learning and March Madness

Making Sense of the Mayhem: Machine Learning and March Madness Alex Tran and Adam Ginzberg Stanford University atran3@stanford.edu ginzberg@stanford.edu I. Introduction III. Model The goal of our research

### MapReduce Approach to Collective Classification for Networks

MapReduce Approach to Collective Classification for Networks Wojciech Indyk 1, Tomasz Kajdanowicz 1, Przemyslaw Kazienko 1, and Slawomir Plamowski 1 Wroclaw University of Technology, Wroclaw, Poland Faculty

### Bayesian sample size determination of vibration signals in machine learning approach to fault diagnosis of roller bearings

Bayesian sample size determination of vibration signals in machine learning approach to fault diagnosis of roller bearings Siddhant Sahu *, V. Sugumaran ** * School of Mechanical and Building Sciences,

### Support Vector Machine. Tutorial. (and Statistical Learning Theory)

Support Vector Machine (and Statistical Learning Theory) Tutorial Jason Weston NEC Labs America 4 Independence Way, Princeton, USA. jasonw@nec-labs.com 1 Support Vector Machines: history SVMs introduced

### Evaluation of Machine Learning Techniques for Green Energy Prediction

arxiv:1406.3726v1 [cs.lg] 14 Jun 2014 Evaluation of Machine Learning Techniques for Green Energy Prediction 1 Objective Ankur Sahai University of Mainz, Germany We evaluate Machine Learning techniques

### UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS

UNSUPERVISED MACHINE LEARNING TECHNIQUES IN GENOMICS Dwijesh C. Mishra I.A.S.R.I., Library Avenue, New Delhi-110 012 dcmishra@iasri.res.in What is Learning? "Learning denotes changes in a system that enable

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Robotics 2 Clustering & EM. Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard

Robotics 2 Clustering & EM Giorgio Grisetti, Cyrill Stachniss, Kai Arras, Maren Bennewitz, Wolfram Burgard 1 Clustering (1) Common technique for statistical data analysis to detect structure (machine learning,

### Handling attrition and non-response in longitudinal data

Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### Content-Based Recommendation

Content-Based Recommendation Content-based? Item descriptions to identify items that are of particular interest to the user Example Example Comparing with Noncontent based Items User-based CF Searches

### CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES

CHARACTERISTICS IN FLIGHT DATA ESTIMATION WITH LOGISTIC REGRESSION AND SUPPORT VECTOR MACHINES Claus Gwiggner, Ecole Polytechnique, LIX, Palaiseau, France Gert Lanckriet, University of Berkeley, EECS,

### Joint models for classification and comparison of mortality in different countries.

Joint models for classification and comparison of mortality in different countries. Viani D. Biatat 1 and Iain D. Currie 1 1 Department of Actuarial Mathematics and Statistics, and the Maxwell Institute

### Scalable Developments for Big Data Analytics in Remote Sensing

Scalable Developments for Big Data Analytics in Remote Sensing Federated Systems and Data Division Research Group High Productivity Data Processing Dr.-Ing. Morris Riedel et al. Research Group Leader,

### Question 2 Naïve Bayes (16 points)

Question 2 Naïve Bayes (16 points) About 2/3 of your email is spam so you downloaded an open source spam filter based on word occurrences that uses the Naive Bayes classifier. Assume you collected the

### Machine learning for algo trading

Machine learning for algo trading An introduction for nonmathematicians Dr. Aly Kassam Overview High level introduction to machine learning A machine learning bestiary What has all this got to do with

### FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT MINING SYSTEM

International Journal of Innovative Computing, Information and Control ICIC International c 0 ISSN 34-48 Volume 8, Number 8, August 0 pp. 4 FUZZY CLUSTERING ANALYSIS OF DATA MINING: APPLICATION TO AN ACCIDENT

### BACK CALCULATION PROCEDURE FOR THE STIFFNESS MODULUS OF CEMENT TREATED BASE LAYERS USING COMPUTATIONAL INTELLIGENCE BASED MODELS

BACK CALCULATION PROCEDURE FOR THE STIFFNESS MODULUS OF CEMENT TREATED BASE LAYERS USING COMPUTATIONAL INTELLIGENCE BASED MODELS Maryam Miradi m.miradi@tudelft.nl André.A. A. Molenaar * a.a.a.molenaar@tudelft.nl

### AUTO CLAIM FRAUD DETECTION USING MULTI CLASSIFIER SYSTEM

AUTO CLAIM FRAUD DETECTION USING MULTI CLASSIFIER SYSTEM ABSTRACT Luis Alexandre Rodrigues and Nizam Omar Department of Electrical Engineering, Mackenzie Presbiterian University, Brazil, São Paulo 71251911@mackenzie.br,nizam.omar@mackenzie.br

### FIoT: A Framework for Self-Adaptive and Self-Organizing Internet of Things Applications. Nathalia Moraes do Nascimento nnascimento@inf.puc-rio.

FIoT: A Framework for Self-Adaptive and Self-Organizing Internet of Things Applications Nathalia Moraes do Nascimento nnascimento@inf.puc-rio.br Roadmap Motivation FIoT Main Idea Application Scenario Decentralized

### Data Mining Project Report. Document Clustering. Meryem Uzun-Per

Data Mining Project Report Document Clustering Meryem Uzun-Per 504112506 Table of Content Table of Content... 2 1. Project Definition... 3 2. Literature Survey... 3 3. Methods... 4 3.1. K-means algorithm...