ARTIFICIAL NEURAL NETWORKS FOR DATA MINING

Size: px
Start display at page:

Download "ARTIFICIAL NEURAL NETWORKS FOR DATA MINING"

Transcription

1 ARTIFICIAL NEURAL NETWORKS FOR DATA MINING Amrender Kumar I.A.S.R.I., Library Avenue, Pusa, New Delhi Introduction Neural networks, more accurately called Artificial Neural Networks (ANNs), are computational models that consist of a number of simple processing units that communicate by sending signals to one another over a large number of weighted connections. They were originally developed from the inspiration of human brains. In human brains, a biological neuron collects signals from other neurons through a host of fine structures called dendrites. The neuron sends out spikes of electrical activity through a long, thin stand known as an axon, which splits into thousands of branches. At the end of each branch, a structure called a synapse converts the activity from the axon into electrical effects that inhibit or excite activity in the connected neurons. When a neuron receives excitatory input that is sufficiently large compared with its inhibitory input, it sends a spike of electrical activity down its axon. Learning occurs by changing the effectiveness of the synapses so that the influence of one neuron on another changes. Like human brains, neural networks also consist of processing units (artificial neurons) and connections (weights) between them. The processing units transport incoming information on their outgoing connections to other units. The "electrical" information is simulated with specific values stored in those weights that make these networks have the capacity to learn, memorize, and create relationships amongst data. A very important feature of these networks is their adaptive nature where "learning by example" replaces "programming" in solving problems. This feature makes such computational models very appealing in application domains where one has little or incomplete understanding of the problem to be solved but where training data is readily available. These networks are neural in the sense that they may have been inspired by neuroscience but not necessarily because they are faithful models of biological neural or cognitive phenomena. ANNs have powerful pattern classification and pattern recognition capabilities through learning and generalize from experience. ANNs are non-linear data driven self adaptive approach as opposed to the traditional model based methods. They are powerful tools for modelling, especially when the underlying data relationship is unknown. ANNs can identify and learn correlated patterns between input data sets and corresponding target values. After training, ANNs can be used to predict the outcome of new independent input data. ANNs imitate the learning process of the human brain and can process problems involving non-linear and complex data even if the data are imprecise and noisy. These techniques are being successfully applied across an extraordinary range of problem domains, in areas as diverse as finance, medicine, engineering, geology, physics, biology and agriculture. There are many different types of neural networks. Some of the most traditional applications include classification, noise reduction and prediction. 2. Review Genesis of ANN modeling and its applications appear to be a recent development. However, this field was established before the advent of computers. It started with the modeling the functions of a human brain by McCulloch and Pitts in 1943, proposed a model of computing element called Mc-Culloch Pitts neuron, which performs weighted sum of the inputs to the element followed by a threshold logic operation. Combinations of these computing elements were used to

2 realize several logical computations. The main drawback of this model of computation is that the weights are fixed and hence the model could not learn from examples. Hebb (1949) proposed a learning scheme for adusting a connection weight based on pre and post synaptic values of the variables. Hebb s law became a fundamental learning rule in neuron network literature. Rosenblatt (1958) proposed the perceptron models, which have weights adustable by the perceptron learning law. Widrows and Hoff (1960) proposed an ADALINE (Adaptive Linear Element) model for computing elements and LMS (Least Mean Square) learning algorithm to adust the weights of an ADALINE model. Hopfield (1982) gave energy analysis of feed back neural networks. The analysis has shown the existence of stable equilibrium states in a feed back network, provided the network has symmetrical weights. Rumelhart et al. (1986) showed that it is possible to adust the weights of a multilayer feed forward neural network in a systematic way to learn the implicit mapping in a set of input output patterns pairs. The learning law is called generalized delta rule or error back propagation. Cheng and Titterington (1994) made a detailed study of ANN models vis-a-vis traditional statistical models. They have shown that some statistical procedures including regression, principal component analysis, density function and statistical image analysis can be given neural network expressions. Warner and Misra (1996) reviewed the relevant literature on neural networks, explained the learning algorithm and made a comparison between regression and neural network models in terms of notations, terminologies and implementation. Kaastra and Boyd (1996) developed neural network model for forecasting financial and economic time series. Dewolf and Francl (1997, 2000) demonstrated the applicability of neural network technology for plant diseases forecasting. Zhang et al. (1998) provided the general summary of the work in ANN forecasting, providing the guidelines for neural network modeling, general paradigm of the ANNs especially those used for forecasting. They have reviewed the relative performance of ANNs with the traditional statistical methods, wherein in most of the studies ANNs were found to be better than the latter. Sanzogni and Kerr (2001) developed models for predicting milk production from farm inputs using standard feed forward ANN. Chakraborty et al. (2004) utilized the ANN technique for predicted severity of anthracnose diseases in legume crop. Gaudart et al. (2004) compared the performance of MLP and that of linear regression for epidemiological data with regard to quality of prediction and robustness to deviation from underlying assumptions of normality, homoscedasticity and independence of errors and it was found that MLP performed better than linear regression. More general books on neural networks, to cite a few, Hassoun (1995), Patterson (1996), Schalkoff (1997), Yegnanarayana (1999), Anderson (2003) etc. are available. Software on neural networks has also been made, to cite a few, Statistica, Matlab etc. Commercial Software:- Statistica Neural Network, TNs2Server,DataEngine, Know Man Basic Suite, Partek, Saxon, ECANSE - Environment for Computer Aided Neural Software Engineering, Neuroshell, Neurogen, Matlab:Neural Network Toolbar, Taran, FCM(Fuzzy Control manager) etc. Freeware Software:- NetII, Spider Nets Neural Network Library, NeuDC, Binary Hopfeild Net with free Java source, Neural shell, PlaNet, Valentino Computational Neuroscience Work bench, Neural Simulation language version-nsl, etc. 3. Characteristics of neural networks The following are the basic characteristics of neural network: Exhibit mapping capabilities, that is, they can map input patterns to their associated output patterns. Learn by examples. Thus, NN architectures can be trained with known examples of a 158

3 problem before they are tested for their inference capability on unknown instances of the problem. They can, therefore, identify new obects previously untrained. Possess the capability to generalize. Thus, they can predict new outcomes from past trends. Robust systems and are fault tolerant. They can, therefore, recall full patterns from incomplete, partial or noisy patterns. 4. Basics of artificial neural networks The terminology of artificial neural networks has developed from a biological model of the brain. A neural network consists of a set of connected cells: The neurons. The neurons receive impulses from either input cells or other neurons and perform some kind of transformation of the input and transmit the outcome to other neurons or to output cells. The neural networks are built from layers of neurons connected so that one layer receives input from the preceding layer of neurons and passes the output on to the subsequent layer. A neuron is a real function of the input vector (y 1,, y k ). The output is obtained as k f (x ) = f ( w y ) where f is a function, typically the sigmoid (logistic or tangent i 1 i i hyperbolic) function. A graphical presentation of neuron is given in figure 1. Mathematically a Multi-Layer Perceptron network is a function consisting of compositions of weighted sums of the functions corresponding to the neurons. Fig. 1: A single neuron 5. Neural networks architectures An ANNs is defined as a data processing system consisting of a large number of simple highly inter connected processing elements (artificial neurons) in an architecture inspired by the structure of the cerebral cortex of the brain. There are several types of architecture of ANNs. However, the two most widely used ANNs are discussed below: a. Feed forward networks In a feed forward network, information flows in one direction along connecting pathways, from the input layer via the hidden layers to the final output layer. There is no feedback (loops) i.e., the output of any layer does not affect that same or preceding layer. A graphical presentation of feed forward network is given in figure

4 Fig. 2: A multi-layer feed forward neural network b. Recurrent networks These networks differ from feed forward network architectures in the sense that there is at least one feedback loop. Thus, in these networks, for example, there could exist one layer with feedback connections as shown in figure below. There could also be neurons with selffeedback links, i.e. the output of a neuron is fed back into itself as input. A graphical presentation of feed forward network is given in figure 3. Input layer Hidden layer Output layer Fig. 3: Recurrent neural network 6. Types of neural networks There are wide variety of neural networks and their architectures. Types of neural networks range from simple Boolean networks (perceptions) to complex self-organizing networks (Kohonen networks). There are also many other types of networks like Hopefield networks, Pulse networks, Radial-Basis Function networks, Boltzmann machine. The most important class of neural networks for real world problems solving includes Multilayer Perceptron Radial Basis Function Networks Kohonen Self Organizing Feature Maps 6.1 Multilayer Perceptron The most popular form of neural network architecture is the multilayer perceptron (MLP). A multilayer perceptron: has any number of inputs. has one or more hidden layers with any number of units. 160

5 uses linear combination functions in the input layers. uses generally sigmoid activation functions in the hidden layers. has any number of outputs with any activation function. has connections between the input layer and the first hidden layer, between the hidden layers, and between the last hidden layer and the output layer. Given enough data, enough hidden units, and enough training time, an MLP with ust one hidden layer can learn to approximate virtually any function to any degree of accuracy. (A statistical analogy is approximating a function with nth order polynomials.) For this reason MLPs are known as universal approximators and can be used when you have little prior knowledge of the relationship between inputs and targets. Although one hidden layer is always sufficient provided you have enough data, there are situations where a network with two or more hidden layers may require fewer hidden units and weights than a network with one hidden layer, so using extra hidden layers sometimes can improve generalization. 6.2 Radial Basis Function Networks Radial basis functions (RBF) networks are also feedforward, but have only one hidden layer. A RBF network: has any number of inputs. typically has only one hidden layer with any number of units. uses radial combination functions in the hidden layer, based on the squared Euclidean distance between the input vector and the weight vector. typically uses exponential or softmax activation functions in the hidden layer, in which case the network is a Gaussian RBF network. has any number of outputs with any activation function. has connections between the input layer and the hidden layer, and between the hidden layer and the output layer. MLPs are said to be distributed-processing networks because the effect of a hidden unit can be distributed over the entire input space. On the other hand, Gaussian RBF networks are said to be local-processing networks because the effect of a hidden unit is usually concentrated in a local area centered at the weight vector. 6.3 Kohonen Neural Network Self Organizing Feature Map (SOFM, or Kohonen) networks are used quite differently to the other networks. Whereas all the other networks are designed for supervised learning tasks, SOFM networks are designed primarily for unsupervised learning (Patterson, 1996). At first glance this may seem strange. Without outputs, what can the network learn? The answer is that the SOFM network attempts to learn the structure of the data. One possible use is therefore in exploratory data analysis. A second possible use is in novelty detection. SOFM networks can learn to recognize clusters in the training data, and respond to it. If new data, unlike previous cases, is encountered, the network fails to recognize it and this indicates novelty. A SOFM network has only two layers: the input layer, and an output layer of radial units (also known as the topological map layer). Schematic representation of Kohonen network is given in Fig

6 Fig. 4: A Kohonen Neural Network Applications 7. Learning of ANNs The most significant property of a neural network is that it can learn from environment, and can improve its performance through learning. Learning is a process by which the free parameters of a neural network i.e. synaptic weights and thresholds are adapted through a continuous process of stimulation by the environment in which the network is embedded. The network becomes more knowledgeable about environment after each iteration of learning process. There are three types of learning paradigms namely, supervised learning, reinforced learning and self-organized or unsupervised learning. 7.1 Supervised learning In this, every input pattern that is used to train the network is associated with an output pattern, which is the target or the desired pattern. A teacher is assumed to be present during the learning process, when a comparison is made between the network s computed output and the correct expected output, to determine the error. The error can then be used to change network parameters, which result in an improvement in performance. Learning law describes the weight vector for the i th processing unit at time instant (t+1) in terms of the weight vector at time instant (t) as follows: wi ( t 1) wi ( t) wi ( t), where w i (t) is the change in the weight vector. The network adapts as follows: change the weight by an amount proportional to the difference between the desired output and the actual output. As an equation: Δ W i = η * (D-Y).I i where η is the learning rate, D is the desired output, Y is the actual output, and I i is the i th input. This is called the Perceptron Learning Rule. The weights in an ANN, similar to coefficients in a regression model, are adusted to solve the problem presented to ANN. Learning or training is term used to describe process of finding values of these weights. Supervised learning which 162

7 incorporates an external teacher, so that each output unit is told what its desired response to input signals ought to be. During the learning process global information may be required. An important issue concerning supervised learning is the problem of error convergence, i.e. the minimization of error between the desired and computed unit values. The aim is to determine a set of weights which minimizes the error. 7.2 Unsupervised learning With unsupervised learning, there is no feedback from the environment to indicate if the outputs of the network are correct. The network must discover features, regulations, correlations, or categories in the input data automatically. In fact, for most varieties of unsupervised learning, the targets are the same as inputs. In other words, unsupervised learning usually performs the same task as an auto-associative network, compressing information from the inputs. 7.3 Reinforced learning In supervised learning there is a target output value for each input value. However, in many situations, there is less detailed information available. In extreme situations, there is only a single bit of information after a long sequence of inputs telling whether the output is right or wrong. Reinforcement learning is one method developed to deal with such situations. Reinforcement learning is a kind of learning in that some feedback from the environment is given. However the feedback signal is only evaluative, not instructive. Reinforcement learning is often called learning with a critic as opposed to learning with a teacher. 8. Development of an ANN model The various steps in developing a neural network model are: 8.1 Variable selection The input variables important for modeling/ forecasting variable(s) under study are selected by suitable variable selection procedures. 8.2 Formation of training, testing and validation sets The data set is divided into three distinct sets called training, testing and validation sets. The training set is the largest set and is used by neural network to learn patterns present in the data. The testing set is used to evaluate the generalization ability of a supposedly trained network. A final check on the performance of the trained network is made using validation set. 8.3 Neural network structure Neural network architecture defines its structure including number of hidden layers, number of hidden nodes and number of output nodes etc. (a) Number of hidden layers: The hidden layer(s) provide the network with its ability to generalize. In theory, a neural network with one hidden layer with a sufficient number of hidden neurons is capable of approximating any continuous function. In practice, neural network with one and occasionally two hidden layers are widely used and have to perform very well. (b) Number of hidden nodes: There is no magic formula for selecting the optimum number of hidden neurons. However, some thumb rules are available for calculating number of hidden neurons. A rough approximation can be obtained by the geometric pyramid rule 163

8 (c) (d) proposed by Masters (1993). For a three layer network with n input and m output neurons, the hidden layer would have sqrt(n*m) neurons. Number of output nodes: Neural networks with multiple outputs, especially if these outputs are widely spaced, will produce inferior results as compared to a network with a single output. Activation function: Activation functions are mathematical formulae that determine the output of a processing node. Most units in neural network transform their net inputs by using a scalar-to-scalar function called an activation function, yielding a value called the unit's activation. Except possibly for output units, the activation value is fed to one or more other units. Activation functions with a bounded range are often called squashing functions. Appropriate differentiable function will be used as activation function. Some of the most commonly used activation functions are : - The sigmoid (logistic) function f ( x) ( 1 exp( x)) 1 - The hyperbolic tangent (tanh) function f ( x) (exp( x) exp( x))/ (exp( x) exp( x)) - The sine or cosine function f ( x) sin( x) or f ( x) cos( x) Activation functions for the hidden units are needed to introduce non-linearity into the networks. The reason is that a composition of linear functions is again a linear function. However, it is the non-linearity (i.e. the capability to represent nonlinear functions) that makes multilayer networks so powerful. Almost any nonlinear function does the ob, although for back-propagation learning it must be differentiable and it helps if the function is bounded. Therefore, the sigmoid functions are the most common choices. There are some heuristic rules for selection of the activation function. For example, Klimasauskas (1991) suggests logistic activation functions for classification problems which involve learning about average behaviour, and to use the hyperbolic tangent functions if the problem involves learning about deviations from the average such as the forecasting problem. 8.4 Model building Multilayer feed forward neural network or multi layer perceptron (MLP), is very popular and is used more than other neural network type for a wide variety of tasks. Multilayer feed forward neural network learned by back propagation algorithm is based on supervised procedure, i.e., the network constructs a model based on examples of data with known output. It has to build the model up solely from the examples presented, which are together assumed to implicitly contain the information necessary to establish the relation. An MLP is a powerful system, often capable of modeling complex, relationships between variables. It allows prediction of an output obect for a given input obect. The architecture of MLP is a layered feedforward neural network in which the non-linear elements (neurons) are arranged in successive layers, and the information flow uni-directionally from input layer to output layer through hidden layer(s). An MLP with ust one hidden layer can learn to approximate virtually any function to any degree of accuracy. For this reason MLPs are known as universal approximates and can be used when we have litter prior knowledge of the relationship between input and targets. One hidden layer is always 164

9 sufficient provided we have enough data. Schematic representation of neural network is given in Fig. 5 Outputs Inputs Fig. 5: Schematic representation of neural network Each interconnection in an ANN has a strength that is expressed by a number referred to as weight. This is accomplished by adusting the weights of given interconnection according to some learning algorithm. Learning methods in neural networks can be broadly classified into three basic types (i) supervised learning (ii) unsupervised learning and (iii) reinforced learning. In MLP, the supervised learning will be used for adusting the weights. The graphic representation of this learning is given in Fig. 6 Input vector ANN model Output vector Target vector = Differences Adust weights Fig. 6 A learning cycle in the ANN model 8.5 Neural network training Training a neural network to learn patterns in the data involves iteratively presenting it with examples of the correct known answers. The obective of training is to find the set of weights between the neurons that determine the global minimum of error function. This involves decision regarding the number of iteration i.e., when to stop training a neural network and the selection of learning rate (a constant of proportionality which determines the size of the weight adustments made at each iteration) and momentum values (how past weight changes affect current weight changes). Backpropagation is the most commonly used method for training multilayered feedforward networks. It can be applied to any feed-forward network with differentiable activation functions. For most networks, the learning process is based on a suitable error function, which is then minimized with respect to the weights and bias. If a network has differential activation functions, then the activations of the output units become differentiable functions of input 165

10 variables, the weights and bias. If we also define a differentiable error function of the network outputs such as the sum of square error function, then the error function itself is a differentiable function of the weights. Therefore, we can evaluate the derivative of the error with respect to weights, and these derivatives can then be used to find the weights that minimize the error function by either using optimization method. The algorithm for evaluating the derivative of the error function is known as backpropagation, because it propagates the errors backward through the network. Multilayer feed forward neural network or multilayered perceptron (MLP), is very popular and is used more than other neural network type for a wide variety of tasks. MLP learned by backpropagation algorithm is based on supervised procedure, i.e. the network constructs a model based on examples of data with known output. The Backpropagation Learning Algorithm is based on an error correction learning rule and specifically on the minimization of the mean squared error that is a measure of the difference between the actual and the desired output. As all multilayer feedforward networks, the multilayer perceptrons are constructed of at least three layers (one input layer, one or more hidden layers and one output layer), each layer consisting of elementary processing units (artificial neurons), which incorporate a nonlinear activation function, commonly the logistic sigmoid function. The algorithm calculates the difference between the actual response and the desired output of each neuron of the output layer of the network. Assuming that y(n) is the actual output of the th neuron of the output layer at the iteration n and d(n) is the corresponding desired output, the error signal e(n) is defined as: e (n) d (n) y (n) The instantaneous value of the error for the neuron is defined as (n) / 2 and correspondingly, the instantaneous total error E(n) is obtained by summing the neural error (n) / 2 over all neurons in the output layer. Thus, 1 2 E (n) e (n) 2 In the above formula, runs over all the neurons of the output layer. If we define N to be the total number of training patterns that consist the training set applied to the neural network during the training process, then the average squared error Eav is obtained by summing E(n) over all the training patterns and then normalizing with respect to the size N of the training set. Thus, N 1 E E(n ) av 2 n 1 It is obvious, that the instantaneous error E(n), as well as the average squared error Eav, is a function of all the free parameters of the network. The obective of the learning process is to modify these free parameters of the network in such a way that Eav is minimized. To perform this minimization, a simple training algorithm is utilized. The training algorithm updates the synaptic weights on a pattern-by-pattern basis until one epoch, that is, one complete presentation of the entire training set is completed. The correction (modification) w i (n) that is applied on the synaptic weight w i (indicating the synaptic strength of the synapse originating from neuron i and directing to neuron ), after the application of the n th training pattern is proportional to the E(n) partial derivative. Specifically, the correction applied is given by: w (n) i 166 e 2 e 2

11 E(n) w i w i (n) In the above formula (this is also known as delta rule), η is the learning-rate parameter of the back-propagation algorithm. The use of the minus sign in above equation accounts for the gradient-descent in weight-space, reflecting the seek of a direction for weight change that reduces the value of E(n). The exact value of the learning rate η is of great importance for the convergence of the algorithm since it modulates the changes in the synaptic weights, from iteration to iteration. The smaller the value of η, the smoother the traectory in the weight space and the slower the convergence of the algorithm. On the other hand, if the value of η is too large, the resulting large changes in the synaptic weights may result the network to exhibit unstable (oscillatory) behaviour. Therefore, the momentum term was introduce for generational of the above equation, Thus E(n) w i w i (n 1) w i (n) In this equation α is the is a positive number called the momentum constant is called the Generalized Delta Rule and it includes the Delta Rule as a special case (α =0). The weight update can be obtained as w (n) w i i (n 1) The weight adustment w i is made only after the entire training set has been presented to the network (Konstantinos, A.; 2000). With respect to the convergence rate the back-propagation algorithm is relatively slow. This is related to the stochastic nature of the algorithm that provides an instantaneous estimation of the gradient of the error surface in weight space. In the case that the error surface is fairly flat along a weight dimension, the derivative of the error surface with respect to that weight is small in magnitude, therefore the synaptic adustment applied to the weight is small and consequently many iterations of the algorithms may be required to produce a significant reduction in the error performance of the network. 9. Evaluation criteria The most common error function minimized in neural networks is the sum of squared errors. Other error functions offered by different software include least absolute deviations, least fourth powers, asymmetric least squares and percentage differences. 10. Conclusions ANNs has an ability to learn by example makes them very flexible and powerful which make them quite suitable for a variety of problem areas. Hence, to best utilize ANNs for different problems, it is essential to understand the potential as well as limitations of neural networks. For some tasks, neural networks will never replace conventional methods, but for a growing list of applications, the neural architecture will provide either an alternative or a complement to these existing techniques. ANNs have a huge potential for prediction and classification when they are integrated with Artificial Intelligence, Fuzzy Logic and related subects. (n)y i (n) 167

12 PRACTICAL ON ARTIFICAL NEURAL METWORKS MODELS FOR PREDICTION USING SAS MINER The snapshots for Opening of proect, importing the file from the desire directory and linking of Models (ANNs ) to data file in SAS miner are given below. 168

13 The output of the ANNs models 169

14 References Anderson, J. A. (2003). An Introduction to neural networks. Prentice Hall. Chakraborty, S., Ghosh. R, Ghosh, M., Fernandes, C.D. and Charchar, M.J. (2004). Weatherbased prediction of anthracnose severity using artificial neural network models. Plant Pathology, 53, Cheng, B. and Titterington, D. M. (1994). Neural networks: A review from a statistical perspective. Statistical Science, 9, Dewolf, E.D., and Francl, L.J., (1997). Neural network that distinguish in period of wheat tan spot in an outdoor environment. Phytopathalogy, 87(1) pp Dewolf, E.D. and Francl, L.J. (2000) Neural network classification of tan spot and stagonespore blotch infection period in wheat field environment. Phytopathalogy, 20(2), Gaudart, J. Giusiano, B. and Huiart, L. (2004). Comparison of the performance of multi-layer perceptron and linear regression for epidemiological data. Comput. Statist. & Data Anal., 44, Hassoun, M. H. (1995). Fundamentals of Artificial Neural Networks. Cambridge: MIT Press. Hebb,D.O. (1949) The organization of behaviour: A Neuropsychological Theory, Wiley, New York Hopfield, J.J. (1982). Neural network and physical system with emergent collective computational capabilities. In proceeding of the National Academy of Science (USA) 79, Kaastra, I. and Boyd, M.(1996): Designing a neural network for forecasting financial and economic time series. Neurocomputing, 10(3), pp (1996) Klimasauskas, C.C. (1991). Applying neural networks. Part 3: Training a neural network, PC-AI, May/ June, Konstantinos, A. (2000). Application of Back Propagation Learning Algorithms on Multilayer Perceptrons, Proect Report, Department of Computing, University of Bradford, England. Mcculloch, W.S. and Pitts, W. (1943) A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophy., 5, Patterson, D. (1996). Artificial Neural Networks. Singapore: Prentice Hall. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage ang organization in the brain. Psychological review, 65, Rumelhart, D.E., Hinton, G.E and Williams, R.J. (1986). Learning internal representation by error propagation, in Parallel distributed processing: Exploration in microstructure of cognition, Vol. (1) ( D.E. Rumelhart, J.L. McClelland and the PDP research gropus, edn.) Cambridge, MA: MIT Press, Saanzogni, Louis and Kerr, Don (2001) Milk production estimate using feed forward artificial neural networks. Computer and Electronics in Agriculture, 32, Schalkoff, R. J. (1997). Artificial neural networks. The Mc Graw-Hall Warner, B. and Misra, M. (1996). Understanding neural networks as statistical tools. American Statistician, 50, Widrow, B. and Hoff, M.E. (1960). Adapative switching circuit. IREWESCON convention record, 4, Yegnanarayana, B. (1999). Artificial Neural Networks. Prentice Hall Zhang, G., Patuwo, B. E. and Hu, M. Y. (1998). Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting,14,

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

More information

A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY

A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY ARTIFICIAL NEURAL NETWORKS: A TUTORIAL BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY Contents Introduction Origin Of Neural Network Biological Neural Networks ANN Overview

More information

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)

More information

Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons

Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion

More information

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons PMR5406 Redes Neurais e Aula 3 Multilayer Percetrons Baseado em: Neural Networks, Simon Haykin, Prentice-Hall, 2 nd edition Slides do curso por Elena Marchiori, Vrie Unviersity Multilayer Perceptrons Architecture

More information

Neural Network Design in Cloud Computing

Neural Network Design in Cloud Computing International Journal of Computer Trends and Technology- volume4issue2-2013 ABSTRACT: Neural Network Design in Cloud Computing B.Rajkumar #1,T.Gopikiran #2,S.Satyanarayana *3 #1,#2Department of Computer

More information

Introduction to Artificial Neural Networks. Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks. Introduction to Artificial Neural Networks Introduction to Artificial Neural Networks v.3 August Michel Verleysen Introduction - Introduction to Artificial Neural Networks p Why ANNs? p Biological inspiration p Some examples of problems p Historical

More information

Stock Prediction using Artificial Neural Networks

Stock Prediction using Artificial Neural Networks Stock Prediction using Artificial Neural Networks Abhishek Kar (Y8021), Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach to predict

More information

Neural networks. Chapter 20, Section 5 1

Neural networks. Chapter 20, Section 5 1 Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of

More information

An Introduction to Neural Networks

An Introduction to Neural Networks An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,

More information

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

More information

Neural network software tool development: exploring programming language options

Neural network software tool development: exploring programming language options INEB- PSI Technical Report 2006-1 Neural network software tool development: exploring programming language options Alexandra Oliveira aao@fe.up.pt Supervisor: Professor Joaquim Marques de Sá June 2006

More information

Neural Networks and Back Propagation Algorithm

Neural Networks and Back Propagation Algorithm Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important

More information

INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

More information

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING EFFICIENT DATA PRE-PROCESSING FOR DATA MINING USING NEURAL NETWORKS JothiKumar.R 1, Sivabalan.R.V 2 1 Research scholar, Noorul Islam University, Nagercoil, India Assistant Professor, Adhiparasakthi College

More information

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

More information

Lecture 6. Artificial Neural Networks

Lecture 6. Artificial Neural Networks Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

More information

International Journal of Electronics and Computer Science Engineering 1449

International Journal of Electronics and Computer Science Engineering 1449 International Journal of Electronics and Computer Science Engineering 1449 Available Online at www.ijecse.org ISSN- 2277-1956 Neural Networks in Data Mining Priyanka Gaur Department of Information and

More information

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS N. K. Bose HRB-Systems Professor of Electrical Engineering The Pennsylvania State University, University Park P. Liang Associate Professor

More information

SEMINAR OUTLINE. Introduction to Data Mining Using Artificial Neural Networks. Definitions of Neural Networks. Definitions of Neural Networks

SEMINAR OUTLINE. Introduction to Data Mining Using Artificial Neural Networks. Definitions of Neural Networks. Definitions of Neural Networks SEMINAR OUTLINE Introduction to Data Mining Using Artificial Neural Networks ISM 611 Dr. Hamid Nemati Introduction to and Characteristics of Neural Networks Comparison of Neural Networks to traditional

More information

Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks POLYTECHNIC UNIVERSITY Department of Computer and Information Science Introduction to Artificial Neural Networks K. Ming Leung Abstract: A computing paradigm known as artificial neural network is introduced.

More information

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING Abhishek Agrawal*, Vikas Kumar** 1,Ashish Pandey** 2,Imran Khan** 3 *(M. Tech Scholar, Department of Computer Science, Bhagwant University,

More information

Machine Learning and Data Mining -

Machine Learning and Data Mining - Machine Learning and Data Mining - Perceptron Neural Networks Nuno Cavalheiro Marques (nmm@di.fct.unl.pt) Spring Semester 2010/2011 MSc in Computer Science Multi Layer Perceptron Neurons and the Perceptron

More information

Introduction to Neural Networks

Introduction to Neural Networks Introduction to Neural Networks 2nd Year UG, MSc in Computer Science http://www.cs.bham.ac.uk/~jxb/inn.html Lecturer: Dr. John A. Bullinaria http://www.cs.bham.ac.uk/~jxb John A. Bullinaria, 2004 Module

More information

One Solution to XOR problem using Multilayer Perceptron having Minimum Configuration

One Solution to XOR problem using Multilayer Perceptron having Minimum Configuration International Journal of Science and Engineering Volume 3, Number 2-2015 PP: 32-41 IJSE Available at www.ijse.org ISSN: 2347-2200 One Solution to XOR problem using Multilayer Perceptron having Minimum

More information

APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED VARIABLES TO PREDICT STOCK TREND DIRECTION

APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED VARIABLES TO PREDICT STOCK TREND DIRECTION LJMS 2008, 2 Labuan e-journal of Muamalat and Society, Vol. 2, 2008, pp. 9-16 Labuan e-journal of Muamalat and Society APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED

More information

6.2.8 Neural networks for data mining

6.2.8 Neural networks for data mining 6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural

More information

Artificial neural networks

Artificial neural networks Artificial neural networks Now Neurons Neuron models Perceptron learning Multi-layer perceptrons Backpropagation 2 It all starts with a neuron 3 Some facts about human brain ~ 86 billion neurons ~ 10 15

More information

Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations

Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations Volume 3, No. 8, August 2012 Journal of Global Research in Computer Science REVIEW ARTICLE Available Online at www.jgrcs.info Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations

More information

6. Feed-forward mapping networks

6. Feed-forward mapping networks 6. Feed-forward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer

More information

Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks

Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Ph. D. Student, Eng. Eusebiu Marcu Abstract This paper introduces a new method of combining the

More information

SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS

SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS UDC: 004.8 Original scientific paper SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS Tonimir Kišasondi, Alen Lovren i University of Zagreb, Faculty of Organization and Informatics,

More information

Introduction to Neural Networks for Senior Design

Introduction to Neural Networks for Senior Design Introduction to Neural Networks for Senior Design Intro-1 Neural Networks: The Big Picture Artificial Intelligence Neural Networks Expert Systems Machine Learning not ruleoriented ruleoriented Intro-2

More information

A Time Series ANN Approach for Weather Forecasting

A Time Series ANN Approach for Weather Forecasting A Time Series ANN Approach for Weather Forecasting Neeraj Kumar 1, Govind Kumar Jha 2 1 Associate Professor and Head Deptt. Of Computer Science,Nalanda College Of Engineering Chandi(Bihar) 2 Assistant

More information

Neural Networks in Data Mining

Neural Networks in Data Mining IOSR Journal of Engineering (IOSRJEN) ISSN (e): 2250-3021, ISSN (p): 2278-8719 Vol. 04, Issue 03 (March. 2014), V6 PP 01-06 www.iosrjen.org Neural Networks in Data Mining Ripundeep Singh Gill, Ashima Department

More information

Introduction to Artificial Neural Networks MAE-491/591

Introduction to Artificial Neural Networks MAE-491/591 Introduction to Artificial Neural Networks MAE-491/591 Artificial Neural Networks: Biological Inspiration The brain has been extensively studied by scientists. Vast complexity prevents all but rudimentary

More information

An Introduction to Artificial Neural Networks (ANN) - Methods, Abstraction, and Usage

An Introduction to Artificial Neural Networks (ANN) - Methods, Abstraction, and Usage An Introduction to Artificial Neural Networks (ANN) - Methods, Abstraction, and Usage Introduction An artificial neural network (ANN) reflects a system that is based on operations of biological neural

More information

Neural Networks and Support Vector Machines

Neural Networks and Support Vector Machines INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

More information

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis Contemporary Engineering Sciences, Vol. 4, 2011, no. 4, 149-163 Performance Evaluation of Artificial Neural Networks for Spatial Data Analysis Akram A. Moustafa Department of Computer Science Al al-bayt

More information

Cash Forecasting: An Application of Artificial Neural Networks in Finance

Cash Forecasting: An Application of Artificial Neural Networks in Finance International Journal of Computer Science & Applications Vol. III, No. I, pp. 61-77 2006 Technomathematics Research Foundation Cash Forecasting: An Application of Artificial Neural Networks in Finance

More information

Introduction to Neural Computation. Neural Computation

Introduction to Neural Computation. Neural Computation Introduction to Neural Computation Level 4/M Neural Computation Level 3 Website: http://www.cs.bham.ac.uk/~jxb/inc.html Lecturer: Dr. John A. Bullinaria John A. Bullinaria, 2015 Module Administration and

More information

Back Propagation Neural Network for Wireless Networking

Back Propagation Neural Network for Wireless Networking International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-4 E-ISSN: 2347-2693 Back Propagation Neural Network for Wireless Networking Menal Dahiya Maharaja Surajmal

More information

Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57. Application of Intelligent System for Water Treatment Plant Operation.

Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57. Application of Intelligent System for Water Treatment Plant Operation. Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57 Application of Intelligent System for Water Treatment Plant Operation *A Mirsepassi Dept. of Environmental Health Engineering, School of Public

More information

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK N M Allinson and D Merritt 1 Introduction This contribution has two main sections. The first discusses some aspects of multilayer perceptrons,

More information

Role of Neural network in data mining

Role of Neural network in data mining Role of Neural network in data mining Chitranjanjit kaur Associate Prof Guru Nanak College, Sukhchainana Phagwara,(GNDU) Punjab, India Pooja kapoor Associate Prof Swami Sarvanand Group Of Institutes Dinanagar(PTU)

More information

Analecta Vol. 8, No. 2 ISSN 2064-7964

Analecta Vol. 8, No. 2 ISSN 2064-7964 EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

More information

Neural Networks algorithms and applications

Neural Networks algorithms and applications Neural Networks algorithms and applications By Fiona Nielsen 4i 12/12-2001 Supervisor: Geert Rasmussen Niels Brock Business College 1 Introduction Neural Networks is a field of Artificial Intelligence

More information

129: Artificial Neural Networks. Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS

129: Artificial Neural Networks. Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS 129: Artificial Neural Networks Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 Introduction to Artificial Neural Networks 901 2 Neural Network Architectures 902 3 Neural Network Learning

More information

IAI : Biological Intelligence and Neural Networks

IAI : Biological Intelligence and Neural Networks IAI : Biological Intelligence and Neural Networks John A. Bullinaria, 2005 1. How do Humans do Intelligent Things? 2. What are Neural Networks? 3. What are Artificial Neural Networks used for? 4. Introduction

More information

Recurrent Neural Networks

Recurrent Neural Networks Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

More information

Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification

Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification R. Sathya Professor, Dept. of MCA, Jyoti Nivas College (Autonomous), Professor and Head, Dept. of Mathematics, Bangalore,

More information

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College

More information

NEURAL NETWORKS IN DATA MINING

NEURAL NETWORKS IN DATA MINING NEURAL NETWORKS IN DATA MINING 1 DR. YASHPAL SINGH, 2 ALOK SINGH CHAUHAN 1 Reader, Bundelkhand Institute of Engineering & Technology, Jhansi, India 2 Lecturer, United Institute of Management, Allahabad,

More information

FORECASTING THE JORDANIAN STOCK PRICES USING ARTIFICIAL NEURAL NETWORK

FORECASTING THE JORDANIAN STOCK PRICES USING ARTIFICIAL NEURAL NETWORK 1 FORECASTING THE JORDANIAN STOCK PRICES USING ARTIFICIAL NEURAL NETWORK AYMAN A. ABU HAMMAD Civil Engineering Department Applied Science University P. O. Box: 926296, Amman 11931 Jordan SOUMA M. ALHAJ

More information

Data Mining and Neural Networks in Stata

Data Mining and Neural Networks in Stata Data Mining and Neural Networks in Stata 2 nd Italian Stata Users Group Meeting Milano, 10 October 2005 Mario Lucchini e Maurizo Pisati Università di Milano-Bicocca mario.lucchini@unimib.it maurizio.pisati@unimib.it

More information

Application of Neural Network in User Authentication for Smart Home System

Application of Neural Network in User Authentication for Smart Home System Application of Neural Network in User Authentication for Smart Home System A. Joseph, D.B.L. Bong, D.A.A. Mat Abstract Security has been an important issue and concern in the smart home systems. Smart

More information

Neural Nets. General Model Building

Neural Nets. General Model Building Neural Nets To give you an idea of how new this material is, let s do a little history lesson. The origins are typically dated back to the early 1940 s and work by two physiologists, McCulloch and Pitts.

More information

Introduction to Neural Networks : Revision Lectures

Introduction to Neural Networks : Revision Lectures Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons

More information

Chapter 12 Discovering New Knowledge Data Mining

Chapter 12 Discovering New Knowledge Data Mining Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to

More information

Introduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011

Introduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 Introduction to Machine Learning Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 1 Outline 1. What is machine learning? 2. The basic of machine learning 3. Principles and effects of machine learning

More information

NEURAL NETWORKS A Comprehensive Foundation

NEURAL NETWORKS A Comprehensive Foundation NEURAL NETWORKS A Comprehensive Foundation Second Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Prentice Hall Prentice Hall Upper Saddle River; New Jersey 07458 Preface xii Acknowledgments

More information

Power Prediction Analysis using Artificial Neural Network in MS Excel

Power Prediction Analysis using Artificial Neural Network in MS Excel Power Prediction Analysis using Artificial Neural Network in MS Excel NURHASHINMAH MAHAMAD, MUHAMAD KAMAL B. MOHAMMED AMIN Electronic System Engineering Department Malaysia Japan International Institute

More information

Prediction Model for Crude Oil Price Using Artificial Neural Networks

Prediction Model for Crude Oil Price Using Artificial Neural Networks Applied Mathematical Sciences, Vol. 8, 2014, no. 80, 3953-3965 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2014.43193 Prediction Model for Crude Oil Price Using Artificial Neural Networks

More information

Self Organizing Maps: Fundamentals

Self Organizing Maps: Fundamentals Self Organizing Maps: Fundamentals Introduction to Neural Networks : Lecture 16 John A. Bullinaria, 2004 1. What is a Self Organizing Map? 2. Topographic Maps 3. Setting up a Self Organizing Map 4. Kohonen

More information

Rain prediction from meteoradar images

Rain prediction from meteoradar images 2015 http://excel.fit.vutbr.cz Rain prediction from meteoradar images Michael Vlček t + 1 t + 2... t - 2 t - 1 t t - 3... input layer hidden layers output layer Abstract This paper presents a software

More information

Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016

Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016 Rate-based artificial neural networks and error backpropagation learning Scott Murdison Machine learning journal club May 16, 2016 Murdison, Leclercq, Lefèvre and Blohm J Neurophys 2015 Neural networks???

More information

Data Mining Techniques Chapter 7: Artificial Neural Networks

Data Mining Techniques Chapter 7: Artificial Neural Networks Data Mining Techniques Chapter 7: Artificial Neural Networks Artificial Neural Networks.................................................. 2 Neural network example...................................................

More information

degrees of freedom and are able to adapt to the task they are supposed to do [Gupta].

degrees of freedom and are able to adapt to the task they are supposed to do [Gupta]. 1.3 Neural Networks 19 Neural Networks are large structured systems of equations. These systems have many degrees of freedom and are able to adapt to the task they are supposed to do [Gupta]. Two very

More information

INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr.

INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr. INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr. Meisenbach M. Hable G. Winkler P. Meier Technology, Laboratory

More information

Prediction of Cancer Count through Artificial Neural Networks Using Incidence and Mortality Cancer Statistics Dataset for Cancer Control Organizations

Prediction of Cancer Count through Artificial Neural Networks Using Incidence and Mortality Cancer Statistics Dataset for Cancer Control Organizations Using Incidence and Mortality Cancer Statistics Dataset for Cancer Control Organizations Shivam Sidhu 1,, Upendra Kumar Meena 2, Narina Thakur 3 1,2 Department of CSE, Student, Bharati Vidyapeeth s College

More information

An Artificial Neural Networks-Based on-line Monitoring Odor Sensing System

An Artificial Neural Networks-Based on-line Monitoring Odor Sensing System Journal of Computer Science 5 (11): 878-882, 2009 ISSN 1549-3636 2009 Science Publications An Artificial Neural Networks-Based on-line Monitoring Odor Sensing System Yousif Al-Bastaki The College of Information

More information

Price Prediction of Share Market using Artificial Neural Network (ANN)

Price Prediction of Share Market using Artificial Neural Network (ANN) Prediction of Share Market using Artificial Neural Network (ANN) Zabir Haider Khan Department of CSE, SUST, Sylhet, Bangladesh Tasnim Sharmin Alin Department of CSE, SUST, Sylhet, Bangladesh Md. Akter

More information

SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96. Özer CIFTCIOGLU Istanbul Technical University, ITU. and

SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96. Özer CIFTCIOGLU Istanbul Technical University, ITU. and NEA/NSC-DOC (96)29 AUGUST 1996 SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96 Özer CIFTCIOGLU Istanbul Technical University, ITU and Erdinç TÜRKCAN Netherlands Energy Research

More information

Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007

Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007 Neural Networks Introduction to Artificial Intelligence CSE 150 May 29, 2007 Administration Last programming assignment has been posted! Final Exam: Tuesday, June 12, 11:30-2:30 Last Lecture Naïve Bayes

More information

COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS

COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS Iris Ginzburg and David Horn School of Physics and Astronomy Raymond and Beverly Sackler Faculty of Exact Science Tel-Aviv University Tel-A viv 96678,

More information

Data quality in Accounting Information Systems

Data quality in Accounting Information Systems Data quality in Accounting Information Systems Comparing Several Data Mining Techniques Erjon Zoto Department of Statistics and Applied Informatics Faculty of Economy, University of Tirana Tirana, Albania

More information

Biological Neurons and Neural Networks, Artificial Neurons

Biological Neurons and Neural Networks, Artificial Neurons Biological Neurons and Neural Networks, Artificial Neurons Neural Computation : Lecture 2 John A. Bullinaria, 2015 1. Organization of the Nervous System and Brain 2. Brains versus Computers: Some Numbers

More information

Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network

Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network Prince Gupta 1, Satanand Mishra 2, S.K.Pandey 3 1,3 VNS Group, RGPV, Bhopal, 2 CSIR-AMPRI, BHOPAL prince2010.gupta@gmail.com

More information

Chapter 4: Artificial Neural Networks

Chapter 4: Artificial Neural Networks Chapter 4: Artificial Neural Networks CS 536: Machine Learning Littman (Wu, TA) Administration icml-03: instructional Conference on Machine Learning http://www.cs.rutgers.edu/~mlittman/courses/ml03/icml03/

More information

Lecture 8 February 4

Lecture 8 February 4 ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

More information

SURVIVABILITY ANALYSIS OF PEDIATRIC LEUKAEMIC PATIENTS USING NEURAL NETWORK APPROACH

SURVIVABILITY ANALYSIS OF PEDIATRIC LEUKAEMIC PATIENTS USING NEURAL NETWORK APPROACH 330 SURVIVABILITY ANALYSIS OF PEDIATRIC LEUKAEMIC PATIENTS USING NEURAL NETWORK APPROACH T. M. D.Saumya 1, T. Rupasinghe 2 and P. Abeysinghe 3 1 Department of Industrial Management, University of Kelaniya,

More information

Utilization of Neural Network for Disease Forecasting

Utilization of Neural Network for Disease Forecasting Utilization of Neural Network for Disease Forecasting Oyas Wahyunggoro 1, Adhistya Erna Permanasari 1, and Ahmad Chamsudin 1,2 1 Department of Electrical Engineering and Information Technology, Gadjah

More information

TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC

TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC 777 TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC M.R. Walker. S. Haghighi. A. Afghan. and L.A. Akers Center for Solid State Electronics Research Arizona State University Tempe. AZ 85287-6206 mwalker@enuxha.eas.asu.edu

More information

Data Mining. Supervised Methods. Ciro Donalek donalek@astro.caltech.edu. Ay/Bi 199ab: Methods of Computa@onal Sciences hcp://esci101.blogspot.

Data Mining. Supervised Methods. Ciro Donalek donalek@astro.caltech.edu. Ay/Bi 199ab: Methods of Computa@onal Sciences hcp://esci101.blogspot. Data Mining Supervised Methods Ciro Donalek donalek@astro.caltech.edu Supervised Methods Summary Ar@ficial Neural Networks Mul@layer Perceptron Support Vector Machines SoLwares Supervised Models: Supervised

More information

TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA

TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA www.arpapress.com/volumes/vol9issue3/ijrras_9_3_16.pdf TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA 1 Akintola K.G., 2 Alese B.K. & 2 Thompson

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

American International Journal of Research in Science, Technology, Engineering & Mathematics

American International Journal of Research in Science, Technology, Engineering & Mathematics American International Journal of Research in Science, Technology, Engineering & Mathematics Available online at http://www.iasir.net ISSN (Print): 2328-349, ISSN (Online): 2328-3580, ISSN (CD-ROM): 2328-3629

More information

2. IMPLEMENTATION. International Journal of Computer Applications (0975 8887) Volume 70 No.18, May 2013

2. IMPLEMENTATION. International Journal of Computer Applications (0975 8887) Volume 70 No.18, May 2013 Prediction of Market Capital for Trading Firms through Data Mining Techniques Aditya Nawani Department of Computer Science, Bharati Vidyapeeth s College of Engineering, New Delhi, India Himanshu Gupta

More information

Tolerance of Radial Basis Functions against Stuck-At-Faults

Tolerance of Radial Basis Functions against Stuck-At-Faults Tolerance of Radial Basis Functions against Stuck-At-Faults Ralf Eickhoff 1 and Ulrich Rückert 1 Heinz Nixdorf Institute System and Circuit Technology University of Paderborn, Germany eickhoff,rueckert@hni.upb.de

More information

CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER

CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER 93 CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER 5.1 INTRODUCTION The development of an active trap based feeder for handling brakeliners was discussed

More information

A Multi-level Artificial Neural Network for Residential and Commercial Energy Demand Forecast: Iran Case Study

A Multi-level Artificial Neural Network for Residential and Commercial Energy Demand Forecast: Iran Case Study 211 3rd International Conference on Information and Financial Engineering IPEDR vol.12 (211) (211) IACSIT Press, Singapore A Multi-level Artificial Neural Network for Residential and Commercial Energy

More information

Impelling Heart Attack Prediction System using Data Mining and Artificial Neural Network

Impelling Heart Attack Prediction System using Data Mining and Artificial Neural Network General Article International Journal of Current Engineering and Technology E-ISSN 2277 4106, P-ISSN 2347-5161 2014 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Impelling

More information

NTC Project: S01-PH10 (formerly I01-P10) 1 Forecasting Women s Apparel Sales Using Mathematical Modeling

NTC Project: S01-PH10 (formerly I01-P10) 1 Forecasting Women s Apparel Sales Using Mathematical Modeling 1 Forecasting Women s Apparel Sales Using Mathematical Modeling Celia Frank* 1, Balaji Vemulapalli 1, Les M. Sztandera 2, Amar Raheja 3 1 School of Textiles and Materials Technology 2 Computer Information

More information

Artificial Neural Network and Non-Linear Regression: A Comparative Study

Artificial Neural Network and Non-Linear Regression: A Comparative Study International Journal of Scientific and Research Publications, Volume 2, Issue 12, December 2012 1 Artificial Neural Network and Non-Linear Regression: A Comparative Study Shraddha Srivastava 1, *, K.C.

More information

Use of Artificial Neural Network in Data Mining For Weather Forecasting

Use of Artificial Neural Network in Data Mining For Weather Forecasting Use of Artificial Neural Network in Data Mining For Weather Forecasting Gaurav J. Sawale #, Dr. Sunil R. Gupta * # Department Computer Science & Engineering, P.R.M.I.T& R, Badnera. 1 gaurav.sawale@yahoo.co.in

More information

Neural Network Applications in Stock Market Predictions - A Methodology Analysis

Neural Network Applications in Stock Market Predictions - A Methodology Analysis Neural Network Applications in Stock Market Predictions - A Methodology Analysis Marijana Zekic, MS University of Josip Juraj Strossmayer in Osijek Faculty of Economics Osijek Gajev trg 7, 31000 Osijek

More information

ARTIFICIAL INTELLIGENCE METHODS IN EARLY MANUFACTURING TIME ESTIMATION

ARTIFICIAL INTELLIGENCE METHODS IN EARLY MANUFACTURING TIME ESTIMATION 1 ARTIFICIAL INTELLIGENCE METHODS IN EARLY MANUFACTURING TIME ESTIMATION B. Mikó PhD, Z-Form Tool Manufacturing and Application Ltd H-1082. Budapest, Asztalos S. u 4. Tel: (1) 477 1016, e-mail: miko@manuf.bme.hu

More information

THE HUMAN BRAIN. observations and foundations

THE HUMAN BRAIN. observations and foundations THE HUMAN BRAIN observations and foundations brains versus computers a typical brain contains something like 100 billion miniscule cells called neurons estimates go from about 50 billion to as many as

More information

Forecasting of Indian Rupee (INR) / US Dollar (USD) Currency Exchange Rate Using Artificial Neural Network

Forecasting of Indian Rupee (INR) / US Dollar (USD) Currency Exchange Rate Using Artificial Neural Network Forecasting of Indian Rupee (INR) / US Dollar (USD) Currency Exchange Rate Using Artificial Neural Network Yusuf Perwej 1 and Asif Perwej 2 1 M.Tech, MCA, Department of Computer Science & Information System,

More information

Comparison of K-means and Backpropagation Data Mining Algorithms

Comparison of K-means and Backpropagation Data Mining Algorithms Comparison of K-means and Backpropagation Data Mining Algorithms Nitu Mathuriya, Dr. Ashish Bansal Abstract Data mining has got more and more mature as a field of basic research in computer science and

More information