# Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis

Save this PDF as:

Size: px
Start display at page:

Download "Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis"

## Transcription

1 Contemporary Engineering Sciences, Vol. 4, 2011, no. 4, Performance Evaluation of Artificial Neural Networks for Spatial Data Analysis Akram A. Moustafa Department of Computer Science Al al-bayt University, P.O. Box Mafraq, 11192, Jordan Abstract In this paper the artificial neural network training algorithm is implemented in MATLAB language. This implementation is focused on the network parameters in order to get the optimal architecture of the network that means (the optimal neural network is the network that can reach the goals in minimum number of training iterations and minimum time of training). Many examples were tested and it was shown that using one hidden layer with number of neuron equal to the square of the number of inputs will lead to optimal neural network by mean of reducing the number of training stages (number of training iterations) and thus the processing time needed to train the network. Our goal in this paper is reaching the optimal solution by reaching minimum training time and minimum number of training iterations. Keywords: Artificial neural network (ANN), Back-propagation, training rate and training iteration (epochs), hidden layer, net simulation, multilayer perceptron (MLP). 1. Introduction Artificial Neural Network (ANN) is a Mathematical model designed to train, visualize, and validate neural network models [10], [12], [13] and the Artificial Neural Network (ANN) is a model-free estimator as it does not rely on an assumed form of the underlying data [2]. And we can define the neural network model is a data structure that can be adjusted to produce a mapping from a given set of input data to features of or relationships among the data. The model is adjusted, or trained, using a collection of data from a given source as input,

2 150 A. A. Moustafa typically referred to as the training set. Note that the number of inputs to a network is set by the external specifications of the problem.to experiment with a two-input neuron; use the Neural Network Design Demonstration Two-Input Neuron (nnd2n2) [3]. After successful training, the neural network will be able to perform classification, estimation, prediction, or simulation on new data from the same or similar sources. The Neural Networks package supports different types of training or learning algorithms. More specifically, the Neural Networks model uses numerical data to specify and evaluate artificial neural network models. Given a set of data,{ x, } N i yi i = 1, from an unknown function, y=f(x), this model uses numerical algorithms to derive reasonable estimates of the function, f(x). This involves three basic steps. First, a neural network structure is chosen that is considered suitable for the type of data and underlying process to be modeled. Second, the neural network is trained by using a sufficiently representative set of data. Third, the trained network is tested with different data, from the same or related sources, to validate that the mapping is of acceptable quality. Choosing the proper type of neural network for a certain problem can be a critical issue. So it is very important to define the following in order to achieve a good model: The input sets, the target sets, the network architecture, the activation functions, the training function, the training rate, the goal, the number of iterations used to train the network. Neural networks have arisen from analogies with models of the way that humans might approach pattern recognition tasks, although they have developed a long way from the biological roots. Great claims have been made for these procedures, and although few of these claims have withstood careful scrutiny, neural network methods have had great impact on pattern recognition practice [6]. 2 Back-Propagation There are many different types of ANN and techniques for training them but we are just going to focus on the most basic one all of them is classic back propagation neural network (BPN) [7]. Back propagation is by far the most widely used and understood neural network paradigm. Its popularity arises from its simple architecture and easy to understand learning process, the back propagation scheme consists of two major steps. These are the forward activation and the backward error flows. The training process begins with the assignment of random weights to the connections between the nodes of the various layers. The various input patterns are then presented to the network, and the forward activation flow produces the output patterns. These output patterns will not be the same as the desired output patterns. The errors in the outputs are calculated for the output layer nodes as the

3 Performance evaluation of artificial neural networks 151 difference between the desired and actual outputs. For the hidden layers, the errors are calculated by back propagating the errors in the output layer to the hidden layers. The errors of each of the nodes are summed over the whole set of training patterns. These errors are used to change the weights in the interconnections between the layers. The weights connecting to the output layer are changed according to the delta rule, whereas for the weights in the hidden layers the generalized delta rule is used. There are many good references which describe the mathematics of the back propagation approach in detail including [17], [18]. The neural network is a computerized software program, which can be used to develop a non-linear statistical model in which there are many parameters, called weights. These weights control the output of the model given the input. The type of network used was a fully connected feed forward back propagation multi-layer perceptron (MLP) [15]. Multilayer perceptron & BP (Back-propagation) model Standard multilayer perceptron (MLP) architecture consists more than 2 layers; A MLP can have any number of layers, units per layer, network inputs, and network outputs [16]. In this paper, a Multi Layer Perceptron (MLP) network with a Back Propagation (BP) algorithm, a network with an optimum learning rate is proposed. The MLP consists of neurons that are arranged in multiple layers with connections only between nodes in the adjacent layers by weights. The layer where the optimum learning rate in BP Network input information is presented is known as the input layer and the layer where the processed information is retrieved is called the output layer. All layers between the input and output layers are known as hidden layers. For all neurons in the network, except the input layer neurons, the total input of each neuron is the sum of the weighted outputs of the neurons in the previous layer. Each neuron is activated with input to the neuron and by the activation function of the neuron [18]. The input and output of the neuron, i, (except for the input layer) in a MLP mode, according to the BP algorithm [19], are: Input x = w o + b (1) i ij j i Output o = f x ) (2) i ( i where Wij is the weight of the connection from neuron i to node j, bi is the numerical value and f is the activation function. The sum in Equation 1 is over all neurons, j, in the previous layer. The output function is a nonlinear function, which allows a network to solve problems that a linear network can't [20]. In this study, the tan-sigmoid and linear transfer functions are used to determine the output.

4 152 A. A. Moustafa A Back-Propagation (BP) algorithm is designed to reduce error between the actual output and the desired output of the network in a gradient descent manner. Finally the back propagation algorithm for training neural networks has been discussed in many papers [1]. The back propagation refers to the fact that any mistakes made by the network during training get sent backwards through it in an attempt to correct it and so teach the network what right and wrong [8]. The BPN learns during a training epoch, you will probably go through several epochs before the network has sufficiently learnt to handle all the data you have provided it and the end result is satisfactory. A training epoch is described below [7],[14]: For each input entry in the training data set: Feed input data in (feed forward). Check output against desired value and feed back error (back-propagate). Where back-propagation consists of: Calculate error gradients. Update weights. Thus we can summarize the training Back-propagation algorithm into the following steps: Step 1: Input training vector. Step 2: Hidden nodes calculate their outputs. Step 3: Output nodes calculate their outputs on the basis of Step 2. Step 4: Calculate the differences between the results of Step 3 and targets. Step 5: Apply the first part of the training rule using the results of Step 4. Step 6: For each hidden node, n, calculate d(n). Step 7: Apply the second part of the training rule using the results of Step 6. Steps 1 through 3 are often called the forward pass, and steps 4 through 7 are often called the backward pass. Hence, the name: back-propagation. An actual algorithm for a 3-layer network (only one hidden layer) [9] [13]: Initialize the weights in the network (often randomly) Do For each example e in the training set O = neural-net-output (network, e); forward pass T = teacher output for e Calculate error (T - O) at the output units Compute delta_wh for all weights from hidden layer to output layer; backward pass Compute delta_wi for all weights from input layer to hidden layer; backward pass continued Update the weights in the network

5 Performance evaluation of artificial neural networks 153 Until all examples classified correctly or satisfied Return the network stopping criterion 3 Experimental Results We used MATLAB because it is already in use in many institutions, And it is useful for statistical task to see the demeanor of the system, It is used in research in academia and industry, MATLAB is useful for data analysis, data extraction and data processing, we can perform operations from MATLAB help command,. It is used to generate stimulus for verification of the system. Prototype solutions are usually obtained faster in MATLAB than solving a problem by using other programming languages [11]. The artificial neural network back propagation algorithm is implemented in Matlab language. This implementation is compared with several other software packages. The effect of reducing the number of iterations in the performance of the algorithm is studied. The speed of the back propagation program, written in Matlab language is compared with the speed of several other back propagation programs which are written in the C language. The speed of the Matlab program is, also compared with the C program which is a variant of the back propagation algorithm, As the next test shows, MATLAB was about 3 times faster than a C++ programming both doing a matrix multiply [10]. A program in C++ was written to multiply two matrices containing double precision numbers. The result of the multiplication is assigned into a third matrix. Each matrix contained 1000 rows and 1000 columns. A MATLAB M file was written to do the same multiply as C++ program did. Only the segment of the code which does the multiplication is timed. The test was run on an IBM PC P4 computer, the result is shown in Table 1 Table 1 Speed Comparison of Matrix Multiply In Matlab and a C++ Program Tools Execution time in seconds 1000 X 1000 multiply C MATLAB As appears from the above table that the MATLAB runs 2.63 times faster than the C++ program.

6 154 A. A. Moustafa We can summarize the methodology of the experiment: Training, Testing and Validation Datasets into: In the ANN methodology, the sample data is often subdivided into training, validation, and test sets [14]. The distinctions among these subsets are crucial. defines the following : o Training set: A set of examples used for learning that is to fit the parameters [weights] of the classifier [14]. o Validation set: A set of examples used to tune the parameters of a classifier, for example to choose the number of hidden units in a neural network. o Test set: A set of examples used only to assess the performance [generalization] of a fully-specified classifier [14]. A popular Artificial neural network architectures chosen are multilayer perceptrons using the backpropagation algorithm [5]. In overview, a MLP is composed of layers of processing units that are interconnected through weighted connections: o The first layer consists of the input vector. o The last layer consists of the output vector representing the output class. o Intermediate layers called `hidden` layers receive the entire input pattern that is modified by the passage through the weighted connections. The hidden layer provides the internal representation of neural pathways. The network is trained using back propagation with three major phases. o First phase: an input vector is presented to the network which leads via the forward pass to the activation of the network as a whole. This generates a difference (error) between the output of the network and the desired output. o Second phase: compute the error factor (signal) for the output unit and propagates this factor successively back through the network (error backward pass). o Third phase: compute the changes for the connection weights by feeding the summed squared errors from the output layer back through the hidden layers to the input layer. Continue this process until the connection weights in the network have been adjusted so that the network output has converged, to an acceptable level, with the desired output. Assign "unseen" or new data: o The trained network is then given the new data and processing and flow of information through the activated network should lead to the assignment of the input data to the output class. For the basic equations relevant to the back propagation model based on generalized delta rule, the training algorithm was known [11].

7 Performance evaluation of artificial neural networks 155 We have to note the following things while using the back propagation algorithm: Learning rate: o Standard back propagation can be used for incremental (on-line) training (in which the weights are updated after processing each case) but it does not converge to a stationary point of the error surface. To obtain convergence, the learning rate must be slowly reduced. This methodology is called "stochastic approximation." o In standard back propagation, too low a learning rate makes the network learn very slowly. Too high a learning rate makes the weights and error function diverge, so there is no learning at all. o Trying to train a NN using a constant learning rate is usually a tedious process requiring much trial and error. There are many variations proposed to improve the standard back propagation as well as other learning algorithms that don't suffer from these limitations [11]. Output Representation: Input Data: o Normalize or transform the data into [0,1] range. This can help for various reasons. Number of Hidden Units: o Simply try many networks with different numbers of hidden units, estimate the generalization error for each one, and choose the network with the minimum estimated generalization error. Activation functions: o For the hidden units, are needed to introduce nonlinearity into the network. o Without nonlinearity, hidden units would not make nets more powerful than just plain perceptrons (which do not have any hidden units, just input and output units). o The sigmoidal functions such as logistic and tanh and the Gaussian function are the most common choices [4]. To get some results we choose the following examples: Example 1: Let us take an XOR operation with 2 inputs and 1 output and study the implementation of the neural model for this operation [9,12]: Step 1: Generate the training data. p= [ ; ]; % input for training Step 2: Generate the target output. t= [ ]; % output for training

8 156 A. A. Moustafa Step 3: Create the neural network and define the activation functions and training function. net = newff([0 1;0 1],[4 1],{'logsig' 'purelin'}); Step 4: Initialize the network. net = init(net); Step 5: Define the parameters of the network. net.trainparam.epochs = 10000; % Maximum epochs are (iteration number of optimization) net.trainparam.goal = ; % Error limit is % training the MLP by using a training pattern net.trainparam.lr = 0.05; Step 6: Train the network. [net tr] = train(net,p,t); % training the MLP neural network by using a train function % to draw the result making the test pattern Step 7: Find the output of the network. Output=sim(net,p); The previous steps were implemented using different network architecture and the optimal performance was achieved using 1 hidden layer with 4 neurons and 1 output layer with 1 neuron as shown in figure 1. Performance is e-009, Goal is 1e Training-Blue Goal-Black Epochs Fig. 1 Optimal solution to reach the goal for example1

9 Performance evaluation of artificial neural networks 157 The results of implementation the network using different network architecture are shown in Table 2. Number of neurons in the hidden layer Table 2 Simulation Resells for Example 1 Average training time in seconds Number of training iterations No Yes Yes Yes Yes Yes Yes 2X2(2 hidden layers each with 2 neurons) Yes The goal was met Example 2: Let us now take an XOR operation with 3 inputs and 1 output and study the implementation of the neural model for this operation: Step 1: Generate the training data. p=[ ; ; ]; % input for training Step 2:: Generate the target output. t=[ ]; % output for training Step 3: Create the neural network and define the activation functions and training function. net = newff([0 1;0 1;0 1],[9 1],{'logsig' 'purelin'}); Step 4: Initialize the network. net = init(net); Step 5: Define the parameters of the network. net.trainparam.epochs = 10000; % Maximum epochs is ( iteration number of optimization ) net.trainparam.goal = ; % Error limit is % training the MLP by using a training pattern net.trainparam.lr = 0.05;

10 158 A. A. Moustafa Step 6: Train the network. [net tr] = train(net,p,t); % training the MLP neural network by using a train function % To draw the result making the test pattern Step 7: Find the output of the network. Output=sim(net,p); The previous steps were implemented using different network architecture and the optimal performance was achieved using 1 hidden layer with 9 neurons and 1 output layer with 1 neuron as shown in figure 2. Performance is e-012, Goal is 1e Training-Blue Goal-Black Epochs Fig. 2 Optimal solution to reach the goal for example 2 The results of implementation the network using different network architecture are shown in Table 3.

11 Performance evaluation of artificial neural networks 159 Table 3 Simulation resells for example 2. Number of neurons Average training Number of The goal was met in the hidden layer time in seconds training iterations No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 2 hidden Yes layers(6,3) 2 hidden Yes layers(3,6) Example 3: Let us now take an XOR operation with 4 inputs and 1 output and study the implementation of the neural model for this operation: Step 1: Generate the training data. p=[ ; ; ; ]; % input for training Step 2:: Generate the target output t=[ ]; % output for training Step 3: Create the neural network and define the activation functions and training function net = newff([0 1;0 1;0 1;0 1],[16 1],{'logsig' 'purelin'}); Step 4: Initialize the network net = init(net); Step 5: Define the parameters of the network net.trainparam.epochs = 10000;

12 160 A. A. Moustafa % Maximum epochs is ( iteration number of optimization ) net.trainparam.goal = ; % Error limit is % training the MLP by using a training pattern net.trainparam.lr = 0.05; Step 6: Train the network [net tr] = train(net,p,t); % training the MLP neural network by using a train function % To draw the result making the test pattern Step 7: Find the output of the network Output=sim(net,p); The previous steps were implemented using different network architecture and the optimal performance was achieved using 1 hidden layer with 9 neurons and 1 output layer with 1 neuron as shown in figure Performance is e-007, Goal is 1e Training-Blue Goal-Black Epochs Fig. 3 Optimal solution to reach the goal for example 3 The results of implementation the network using different network architecture are shown in Table 4.

13 Performance evaluation of artificial neural networks 161 Table 4 Simulation Resells for Example 3. Number of neurons Average Number of The goal was met in the hidden layer training time in training seconds iterations No No No No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes 2 hidden Yes layers(8x8)

14 162 A. A. Moustafa 4 Results Discussions The optimal neural network is the network that can reach the goals in minimum number of training iterations and minimum time of training. From the experimental results shown in the Tables 2, 3, and 4 we can see that we can reach the goals by adding a hidden layer to the network, and an optimized solution can be achieved if the number of neurons is equal to the power of 2 of the number of inputs. Splitting the number of neurons in the hidden layer to 2 or more hidden layers make the network not optimal by mean of increasing the number of training iterations and increasing the training time. During the experiments implementations it was seen that changing the training rate around 0.05 does not much affect the network performance. 5 Conclusions This paper proves that the efficiency of Artificial Neural Networks to attain the optimal Neural network and optimal solution. That means we can attaining the goals in minimum training time and minimum number of training iterations and that are depends on the network architecture. Using one hidden layer with number of neurons equal to the power of 2 of the number of inputs leads to optimal and efficient Artificial Neural Networks. References [1] P.J. Werbos, Backpropagation through Time: What It Does and How to Do it" Proceedings of the IEEE, 78_10_, pp , [2] D. H. Chang and S. Islam, "Estimation of Soil Physical Properties Using Remote Sensing and Artificial Neural Network," Remote Sensing of Environment, vol. 74, pp , [3] M.Hagan, H. Demuth, M. Beele, Neural Network Design, University of Colorado Bookstore, 2002, ISBN: [4] Sucharita Gopal. Artificial Neural Networks for Spatial Data Analysis, NCGIA Core Curriculum in GIScience, [5] Benediktsson, J. A.; Swain, P. H. and Ersoy, O. K., 1990, Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Trans. on Geoscience and Remote Sensing, GE-28, Pp

15 Performance evaluation of artificial neural networks 163 [6] Ripley, B.D., Pattern Recognition and Neural Networks, Cambridge: Cambridge University Press, ISBN (hardback), (1996), pp 354 and pp 403. [7] Bishop, C.M. Neural Networks for Pattern Recognition, Oxford: Oxford University Press. ISBN (hardback) or , (1995), pp.482. [8] W. Chang, B. Bosworth, G.C. Carter, On using back propagation neural networks to separate single echoes from multiple echoes, Acoustics, Speech, and Signal Processing, ICASSP, IEEE International Conference on, (27-30 Apr, 1993), vol. 1, pp [9] Kwan, H.K., simple sigmoid. Like activation function suitable for digital hardware implementation, Electronic Letters, vol.28, no.15, (1992), pp [10] Jamshid nazari and okan k.ersoy, implementation of back propagation neural networks with Matlab, school of electrical engineering purdue university1992. [11] The MathWorks Inc. PRO-MATLAB for Sun Workstations, User's Guide, January [12] Paul T. Baffes. NETS Users's Guide, Version 2.0 of NETS. Technical Report JSC-23366, NASA, Software Technology Branch, Lyndon B. Johnson Space Center, September [13] E. Fahlman, Scott. An Empirical Study of Learning Speed in Back Propagation Networks. Technical Report CMU-CS , CMU, CMU, September [14] D. C. Silverman, Discussion of The Corrosion of Nickel 270 In Phosphate Containing Solutions, Corrosion, vol. 41, no. 4, (1985), pp [15] Patterson DW. Artificial Neural Networks, Theory and Applications. Singapore: Prentice Hall, 1996 pp [16] Insung Jung, and Gi-Nam Wang, "Pattern Classification of Back-Propagation Algorithm Using Exclusive Connecting Network, World Academy of Science, Engineering and Technology, vol 36, 2007, pp [17] R. Hecht-Nielsen, "Theory of the Backpropagation Neural Network", IJCNN International Conference on Neural Networks, 1, 1989, pp [18] D. E. Rumelhart, J. L.McClell and the PDP Research Group institute for Cognitive Science, University of California, SanDiego, and Parallel Distributed Processing: Explorations in the Microstructure of Cognition, (MIT Press, Cambridge, Mass. 1986). [19] Pao, Y.H. Adaptive Pattern Recognition and Neural Network, Addison- Wesley Publishing Company, Inc [20] Heermann, P. and Khazenie, N. Classification of Multi spectral remote sensing data using a backpropagation neural network", IEEE Trans. on Geo science and Remote Sensing, 30, pp , 1992.

### Analecta Vol. 8, No. 2 ISSN 2064-7964

EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

### Lecture 6. Artificial Neural Networks

Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

### INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

### CHAPTER 6 NEURAL NETWORK BASED SURFACE ROUGHNESS ESTIMATION

CHAPTER 6 NEURAL NETWORK BASED SURFACE ROUGHNESS ESTIMATION 6.1. KNOWLEDGE REPRESENTATION The function of any representation scheme is to capture the essential features of a problem domain and make that

### Chapter 4: Artificial Neural Networks

Chapter 4: Artificial Neural Networks CS 536: Machine Learning Littman (Wu, TA) Administration icml-03: instructional Conference on Machine Learning http://www.cs.rutgers.edu/~mlittman/courses/ml03/icml03/

### Learning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning.

Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Artificial Intelligence Learning Chapter 8 Learning is useful as a system construction method, i.e., expose

### Keywords: Image complexity, PSNR, Levenberg-Marquardt, Multi-layer neural network.

Global Journal of Computer Science and Technology Volume 11 Issue 3 Version 1.0 Type: Double Blind Peer Reviewed International Research Journal Publisher: Global Journals Inc. (USA) Online ISSN: 0975-4172

### OPTIMUM LEARNING RATE FOR CLASSIFICATION PROBLEM

OPTIMUM LEARNING RATE FOR CLASSIFICATION PROBLEM WITH MLP IN DATA MINING Lalitha Saroja Thota 1 and Suresh Babu Changalasetty 2 1 Department of Computer Science, King Khalid University, Abha, KSA 2 Department

### Open Access Research on Application of Neural Network in Computer Network Security Evaluation. Shujuan Jin *

Send Orders for Reprints to reprints@benthamscience.ae 766 The Open Electrical & Electronic Engineering Journal, 2014, 8, 766-771 Open Access Research on Application of Neural Network in Computer Network

### Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57. Application of Intelligent System for Water Treatment Plant Operation.

Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57 Application of Intelligent System for Water Treatment Plant Operation *A Mirsepassi Dept. of Environmental Health Engineering, School of Public

### Solving Nonlinear Equations Using Recurrent Neural Networks

Solving Nonlinear Equations Using Recurrent Neural Networks Karl Mathia and Richard Saeks, Ph.D. Accurate Automation Corporation 71 Shallowford Road Chattanooga, Tennessee 37421 Abstract A class of recurrent

### Artificial Neural Computation Systems

Artificial Neural Computation Systems Spring 2003 Technical University of Szczecin Department of Electrical Engineering Lecturer: Prof. Adam Krzyzak,PS 5. Lecture 15.03.2003 147 1. Multilayer Perceptrons............

### Neural network software tool development: exploring programming language options

INEB- PSI Technical Report 2006-1 Neural network software tool development: exploring programming language options Alexandra Oliveira aao@fe.up.pt Supervisor: Professor Joaquim Marques de Sá June 2006

### PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons

PMR5406 Redes Neurais e Aula 3 Multilayer Percetrons Baseado em: Neural Networks, Simon Haykin, Prentice-Hall, 2 nd edition Slides do curso por Elena Marchiori, Vrie Unviersity Multilayer Perceptrons Architecture

### Power Prediction Analysis using Artificial Neural Network in MS Excel

Power Prediction Analysis using Artificial Neural Network in MS Excel NURHASHINMAH MAHAMAD, MUHAMAD KAMAL B. MOHAMMED AMIN Electronic System Engineering Department Malaysia Japan International Institute

### Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

### An Introduction to Neural Networks

An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,

### Building MLP networks by construction

University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2000 Building MLP networks by construction Ah Chung Tsoi University of

### Neural Network Design in Cloud Computing

International Journal of Computer Trends and Technology- volume4issue2-2013 ABSTRACT: Neural Network Design in Cloud Computing B.Rajkumar #1,T.Gopikiran #2,S.Satyanarayana *3 #1,#2Department of Computer

### A Multi-level Artificial Neural Network for Residential and Commercial Energy Demand Forecast: Iran Case Study

211 3rd International Conference on Information and Financial Engineering IPEDR vol.12 (211) (211) IACSIT Press, Singapore A Multi-level Artificial Neural Network for Residential and Commercial Energy

### Forecasting Demand in the Clothing Industry. Eduardo Miguel Rodrigues 1, Manuel Carlos Figueiredo 2 2

XI Congreso Galego de Estatística e Investigación de Operacións A Coruña, 24-25-26 de outubro de 2013 Forecasting Demand in the Clothing Industry Eduardo Miguel Rodrigues 1, Manuel Carlos Figueiredo 2

### Application of Neural Network in User Authentication for Smart Home System

Application of Neural Network in User Authentication for Smart Home System A. Joseph, D.B.L. Bong, D.A.A. Mat Abstract Security has been an important issue and concern in the smart home systems. Smart

### ANN Based Fault Classifier and Fault Locator for Double Circuit Transmission Line

International Journal of Computer Sciences and Engineering Open Access Research Paper Volume-4, Special Issue-2, April 2016 E-ISSN: 2347-2693 ANN Based Fault Classifier and Fault Locator for Double Circuit

### CLOUD FREE MOSAIC IMAGES

CLOUD FREE MOSAIC IMAGES T. Hosomura, P.K.M.M. Pallewatta Division of Computer Science Asian Institute of Technology GPO Box 2754, Bangkok 10501, Thailand ABSTRACT Certain areas of the earth's surface

### Price Prediction of Share Market using Artificial Neural Network (ANN)

Prediction of Share Market using Artificial Neural Network (ANN) Zabir Haider Khan Department of CSE, SUST, Sylhet, Bangladesh Tasnim Sharmin Alin Department of CSE, SUST, Sylhet, Bangladesh Md. Akter

### SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS

UDC: 004.8 Original scientific paper SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS Tonimir Kišasondi, Alen Lovren i University of Zagreb, Faculty of Organization and Informatics,

### TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA

www.arpapress.com/volumes/vol9issue3/ijrras_9_3_16.pdf TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA 1 Akintola K.G., 2 Alese B.K. & 2 Thompson

### NEURAL NETWORKS IN DATA MINING

NEURAL NETWORKS IN DATA MINING 1 DR. YASHPAL SINGH, 2 ALOK SINGH CHAUHAN 1 Reader, Bundelkhand Institute of Engineering & Technology, Jhansi, India 2 Lecturer, United Institute of Management, Allahabad,

### 6.2.8 Neural networks for data mining

6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural

### International Journal of Electronics and Computer Science Engineering 1449

International Journal of Electronics and Computer Science Engineering 1449 Available Online at www.ijecse.org ISSN- 2277-1956 Neural Networks in Data Mining Priyanka Gaur Department of Information and

### Multiple Layer Perceptron Training Using Genetic Algorithms

Multiple Layer Perceptron Training Using Genetic Algorithms Udo Seiffert University of South Australia, Adelaide Knowledge-Based Intelligent Engineering Systems Centre (KES) Mawson Lakes, 5095, Adelaide,

### Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016

Rate-based artificial neural networks and error backpropagation learning Scott Murdison Machine learning journal club May 16, 2016 Murdison, Leclercq, Lefèvre and Blohm J Neurophys 2015 Neural networks???

### Comparison of K-means and Backpropagation Data Mining Algorithms

Comparison of K-means and Backpropagation Data Mining Algorithms Nitu Mathuriya, Dr. Ashish Bansal Abstract Data mining has got more and more mature as a field of basic research in computer science and

### A New Approach For Estimating Software Effort Using RBFN Network

IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.7, July 008 37 A New Approach For Estimating Software Using RBFN Network Ch. Satyananda Reddy, P. Sankara Rao, KVSVN Raju,

### SOFTWARE EFFORT ESTIMATION USING RADIAL BASIS FUNCTION NEURAL NETWORKS Ana Maria Bautista, Angel Castellanos, Tomas San Feliu

International Journal Information Theories and Applications, Vol. 21, Number 4, 2014 319 SOFTWARE EFFORT ESTIMATION USING RADIAL BASIS FUNCTION NEURAL NETWORKS Ana Maria Bautista, Angel Castellanos, Tomas

### Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

### AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING Abhishek Agrawal*, Vikas Kumar** 1,Ashish Pandey** 2,Imran Khan** 3 *(M. Tech Scholar, Department of Computer Science, Bhagwant University,

### Neural Networks and Support Vector Machines

INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

### Neural Networks and Back Propagation Algorithm

Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important

### Neural Network Toolbox

Neural Network Toolbox A Tutorial for the Course Computational Intelligence http://www.igi.tugraz.at/lehre/ci Stefan Häusler Institute for Theoretical Computer Science Inffeldgasse 16b/I Abstract This

### Bank Customers (Credit) Rating System Based On Expert System and ANN

Bank Customers (Credit) Rating System Based On Expert System and ANN Project Review Yingzhen Li Abstract The precise rating of customers has a decisive impact on loan business. We constructed the BP network,

### American International Journal of Research in Science, Technology, Engineering & Mathematics

American International Journal of Research in Science, Technology, Engineering & Mathematics Available online at http://www.iasir.net ISSN (Print): 2328-349, ISSN (Online): 2328-3580, ISSN (CD-ROM): 2328-3629

### Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons

Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion

### Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations

Volume 3, No. 8, August 2012 Journal of Global Research in Computer Science REVIEW ARTICLE Available Online at www.jgrcs.info Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations

### CONNECTIONIST THEORIES OF LEARNING

CONNECTIONIST THEORIES OF LEARNING Themis N. Karaminis, Michael S.C. Thomas Department of Psychological Sciences, Birkbeck College, University of London London, WC1E 7HX UK tkaram01@students.bbk.ac.uk,

### Recurrent Neural Networks

Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

### Neural Computation - Assignment

Neural Computation - Assignment Analysing a Neural Network trained by Backpropagation AA SSt t aa t i iss i t i icc aa l l AA nn aa l lyy l ss i iss i oo f vv aa r i ioo i uu ss l lee l aa r nn i inn gg

### ANN-DEA Integrated Approach for Sensitivity Analysis in Efficiency Models

Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 14-24 ANN-DEA Integrated Approach for Sensitivity Analysis in Efficiency Models L. Karamali *,1, A. Memariani 2, G.R. Jahanshahloo 3 Here,

### 6. Feed-forward mapping networks

6. Feed-forward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer

### Neural networks. Chapter 20, Section 5 1

Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of

### Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007

Neural Networks Introduction to Artificial Intelligence CSE 150 May 29, 2007 Administration Last programming assignment has been posted! Final Exam: Tuesday, June 12, 11:30-2:30 Last Lecture Naïve Bayes

### The Need for Small Learning Rates on Large Problems

In Proceedings of the 200 International Joint Conference on Neural Networks (IJCNN 0), -9. The Need for Small Learning Rates on Large Problems D. Randall Wilson fonix Corporation 80 West Election Road

### Analysis of Multilayer Neural Networks with Direct and Cross-Forward Connection

Analysis of Multilayer Neural Networks with Direct and Cross-Forward Connection Stanis law P laczek and Bijaya Adhikari Vistula University, Warsaw, Poland stanislaw.placzek@wp.pl,bijaya.adhikari1991@gmail.com

### Data quality in Accounting Information Systems

Data quality in Accounting Information Systems Comparing Several Data Mining Techniques Erjon Zoto Department of Statistics and Applied Informatics Faculty of Economy, University of Tirana Tirana, Albania

### Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

### PLAANN as a Classification Tool for Customer Intelligence in Banking

PLAANN as a Classification Tool for Customer Intelligence in Banking EUNITE World Competition in domain of Intelligent Technologies The Research Report Ireneusz Czarnowski and Piotr Jedrzejowicz Department

### A Time Series ANN Approach for Weather Forecasting

A Time Series ANN Approach for Weather Forecasting Neeraj Kumar 1, Govind Kumar Jha 2 1 Associate Professor and Head Deptt. Of Computer Science,Nalanda College Of Engineering Chandi(Bihar) 2 Assistant

### Chapter 6. The stacking ensemble approach

82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described

### Back Propagation Neural Network for Wireless Networking

International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-4 E-ISSN: 2347-2693 Back Propagation Neural Network for Wireless Networking Menal Dahiya Maharaja Surajmal

### SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK N M Allinson and D Merritt 1 Introduction This contribution has two main sections. The first discusses some aspects of multilayer perceptrons,

### Horse Racing Prediction Using Artificial Neural Networks

Horse Racing Prediction Using Artificial Neural Networks ELNAZ DAVOODI, ALI REZA KHANTEYMOORI Mathematics and Computer science Department Institute for Advanced Studies in Basic Sciences (IASBS) Gavazang,

### CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER

93 CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER 5.1 INTRODUCTION The development of an active trap based feeder for handling brakeliners was discussed

### FORECASTING THE JORDANIAN STOCK PRICES USING ARTIFICIAL NEURAL NETWORK

1 FORECASTING THE JORDANIAN STOCK PRICES USING ARTIFICIAL NEURAL NETWORK AYMAN A. ABU HAMMAD Civil Engineering Department Applied Science University P. O. Box: 926296, Amman 11931 Jordan SOUMA M. ALHAJ

### Impact of Feature Selection on the Performance of Wireless Intrusion Detection Systems

2009 International Conference on Computer Engineering and Applications IPCSIT vol.2 (2011) (2011) IACSIT Press, Singapore Impact of Feature Selection on the Performance of ireless Intrusion Detection Systems

### A Simple Feature Extraction Technique of a Pattern By Hopfield Network

A Simple Feature Extraction Technique of a Pattern By Hopfield Network A.Nag!, S. Biswas *, D. Sarkar *, P.P. Sarkar *, B. Gupta **! Academy of Technology, Hoogly - 722 *USIC, University of Kalyani, Kalyani

### Data Mining using Artificial Neural Network Rules

Data Mining using Artificial Neural Network Rules Pushkar Shinde MCOERC, Nasik Abstract - Diabetes patients are increasing in number so it is necessary to predict, treat and diagnose the disease. Data

### Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification

Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification R. Sathya Professor, Dept. of MCA, Jyoti Nivas College (Autonomous), Professor and Head, Dept. of Mathematics, Bangalore,

### Introduction to Neural Networks : Revision Lectures

Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons

### Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks

Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Ph. D. Student, Eng. Eusebiu Marcu Abstract This paper introduces a new method of combining the

### A Content based Spam Filtering Using Optical Back Propagation Technique

A Content based Spam Filtering Using Optical Back Propagation Technique Sarab M. Hameed 1, Noor Alhuda J. Mohammed 2 Department of Computer Science, College of Science, University of Baghdad - Iraq ABSTRACT

### A Neural Network Based System for Intrusion Detection and Classification of Attacks

A Neural Network Based System for Intrusion Detection and Classification of Attacks Mehdi MORADI and Mohammad ZULKERNINE Abstract-- With the rapid expansion of computer networks during the past decade,

### Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network

Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network Prince Gupta 1, Satanand Mishra 2, S.K.Pandey 3 1,3 VNS Group, RGPV, Bhopal, 2 CSIR-AMPRI, BHOPAL prince2010.gupta@gmail.com

### COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS

COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS B.K. Mohan and S. N. Ladha Centre for Studies in Resources Engineering IIT

### An Artificial Neural Networks-Based on-line Monitoring Odor Sensing System

Journal of Computer Science 5 (11): 878-882, 2009 ISSN 1549-3636 2009 Science Publications An Artificial Neural Networks-Based on-line Monitoring Odor Sensing System Yousif Al-Bastaki The College of Information

### TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC

777 TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC M.R. Walker. S. Haghighi. A. Afghan. and L.A. Akers Center for Solid State Electronics Research Arizona State University Tempe. AZ 85287-6206 mwalker@enuxha.eas.asu.edu

### Face Recognition For Remote Database Backup System

Face Recognition For Remote Database Backup System Aniza Mohamed Din, Faudziah Ahmad, Mohamad Farhan Mohamad Mohsin, Ku Ruhana Ku-Mahamud, Mustafa Mufawak Theab 2 Graduate Department of Computer Science,UUM

### Supply Chain Forecasting Model Using Computational Intelligence Techniques

CMU.J.Nat.Sci Special Issue on Manufacturing Technology (2011) Vol.10(1) 19 Supply Chain Forecasting Model Using Computational Intelligence Techniques Wimalin S. Laosiritaworn Department of Industrial

### Utilization of Neural Network for Disease Forecasting

Utilization of Neural Network for Disease Forecasting Oyas Wahyunggoro 1, Adhistya Erna Permanasari 1, and Ahmad Chamsudin 1,2 1 Department of Electrical Engineering and Information Technology, Gadjah

### Novelty Detection in image recognition using IRF Neural Networks properties

Novelty Detection in image recognition using IRF Neural Networks properties Philippe Smagghe, Jean-Luc Buessler, Jean-Philippe Urban Université de Haute-Alsace MIPS 4, rue des Frères Lumière, 68093 Mulhouse,

### Anupam Tarsauliya Shoureya Kant Rahul Kala Researcher Researcher Researcher IIITM IIITM IIITM Gwalior Gwalior Gwalior

Analysis of Artificial Neural Network for Financial Time Series Forecasting Anupam Tarsauliya Shoureya Kant Rahul Kala Researcher Researcher Researcher IIITM IIITM IIITM Gwalior Gwalior Gwalior Ritu Tiwari

### FRAUD DETECTION IN ELECTRIC POWER DISTRIBUTION NETWORKS USING AN ANN-BASED KNOWLEDGE-DISCOVERY PROCESS

FRAUD DETECTION IN ELECTRIC POWER DISTRIBUTION NETWORKS USING AN ANN-BASED KNOWLEDGE-DISCOVERY PROCESS Breno C. Costa, Bruno. L. A. Alberto, André M. Portela, W. Maduro, Esdras O. Eler PDITec, Belo Horizonte,

### Stock Prediction using Artificial Neural Networks

Stock Prediction using Artificial Neural Networks Abhishek Kar (Y8021), Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach to predict

### Neural Network Predictor for Fraud Detection: A Study Case for the Federal Patrimony Department

DOI: 10.5769/C2012010 or http://dx.doi.org/10.5769/c2012010 Neural Network Predictor for Fraud Detection: A Study Case for the Federal Patrimony Department Antonio Manuel Rubio Serrano (1,2), João Paulo

### Lecture 8 February 4

ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

### MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College

### Speech Synthesis by Artificial Neural Networks (AI / Speech processing / Signal processing)

Speech Synthesis by Artificial Neural Networks (AI / Speech processing / Signal processing) Christos P. Yiakoumettis Department of Informatics University of Sussex, UK (Email: c.yiakoumettis@sussex.ac.uk)

### Feature Extraction by Neural Network Nonlinear Mapping for Pattern Classification

Lerner et al.:feature Extraction by NN Nonlinear Mapping 1 Feature Extraction by Neural Network Nonlinear Mapping for Pattern Classification B. Lerner, H. Guterman, M. Aladjem, and I. Dinstein Department

### Planning Workforce Management for Bank Operation Centers with Neural Networks

Plaing Workforce Management for Bank Operation Centers with Neural Networks SEFIK ILKIN SERENGIL Research and Development Center SoftTech A.S. Tuzla Teknoloji ve Operasyon Merkezi, Tuzla 34947, Istanbul

### New Ensemble Combination Scheme

New Ensemble Combination Scheme Namhyoung Kim, Youngdoo Son, and Jaewook Lee, Member, IEEE Abstract Recently many statistical learning techniques are successfully developed and used in several areas However,

### Estimation of Sympathetic and Parasympathetic Level during Orthostatic Stress using Artificial Neural Networks

Estimation of Sympathetic and Parasympathetic Level during Orthostatic Stress using Artificial Neural Networks M. Kana 1, M. Jirina 1, J. Holcik 2 1 Czech Technical University Prague, Faculty of Biomedical

### EFFICIENT DATA PRE-PROCESSING FOR DATA MINING

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING USING NEURAL NETWORKS JothiKumar.R 1, Sivabalan.R.V 2 1 Research scholar, Noorul Islam University, Nagercoil, India Assistant Professor, Adhiparasakthi College

### Machine Learning: Multi Layer Perceptrons

Machine Learning: Multi Layer Perceptrons Prof. Dr. Martin Riedmiller Albert-Ludwigs-University Freiburg AG Maschinelles Lernen Machine Learning: Multi Layer Perceptrons p.1/61 Outline multi layer perceptrons

### Artificial Neural Network and Non-Linear Regression: A Comparative Study

International Journal of Scientific and Research Publications, Volume 2, Issue 12, December 2012 1 Artificial Neural Network and Non-Linear Regression: A Comparative Study Shraddha Srivastava 1, *, K.C.

### International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May ISSN

International Journal of Scientific & Engineering Research, Volume 4, Issue 5, May-213 737 Letter Recognition Data Using Neural Network Hussein Salim Qasim Abstract The letters dataset from the UCI repository

### Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

### Impelling Heart Attack Prediction System using Data Mining and Artificial Neural Network

General Article International Journal of Current Engineering and Technology E-ISSN 2277 4106, P-ISSN 2347-5161 2014 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Impelling

### Machine Learning and Data Mining -

Machine Learning and Data Mining - Perceptron Neural Networks Nuno Cavalheiro Marques (nmm@di.fct.unl.pt) Spring Semester 2010/2011 MSc in Computer Science Multi Layer Perceptron Neurons and the Perceptron

### Introduction to Neural Networks for Senior Design

Introduction to Neural Networks for Senior Design Intro-1 Neural Networks: The Big Picture Artificial Intelligence Neural Networks Expert Systems Machine Learning not ruleoriented ruleoriented Intro-2