Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016

Size: px
Start display at page:

Download "Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016"

Transcription

1 Rate-based artificial neural networks and error backpropagation learning Scott Murdison Machine learning journal club May 16, 2016 Murdison, Leclercq, Lefèvre and Blohm J Neurophys 2015

2 Neural networks??? Why would we use those?! Some problems are just too complicated (i.e. nonlinear) to solve with simple programming Computer vision/audition applications: - digit recognition - facial recognition - reading - speech recognition - object recognition Others: - detection of medical disorders - industrial process control - stock market predictions - the list goes on Could theoretically write a program to solve each of these problems, but the rules governing such a program would be incredibly complicated! Geoff Hinton sweet intro vid 2

3 Neural networks??? Why would we use those?! Some problems are just too complicated to solve with simple programming Computer vision/audition applications: - facial recognition Paul Debevec. A Neural Network for Facial Feature Location. UC Berkeley CS283 Project Report, December

4 Neural networks??? Why would we use those?! Some problems are just too complicated to solve with simple programming Computer vision/audition applications: - facial recognition manually located left eye, nose and mh polar-transformed images around each feature Paul Debevec. A Neural Network for Facial Feature Location. UC Berkeley CS283 Project Report, December subsampled images to feed to neural network A. Avoids local minima Why? B. 4MB RAM in 1992 = $150 USD 4

5 Neural networks??? Why would we use those?! Some problems are just too complicated to solve with simple programming Computer vision/audition applications: - facial recognition After training a simple, feedforward network with backprop using yearbook photos it could successfully detect each eye, nose and mh in previously unseen photo! Paul Debevec. A Neural Network for Facial Feature Location. UC Berkeley CS283 Project Report, December

6 The choice for rate-based over spiking for machine learning spiking rate-based Pros neuron-level resolution time-resolved (spiking dynamics, variability) several levels of abstraction available (H-H, leaky integrate-and-fire, Izhikevich) Cons computationally expensive for large populations of neurons intractable for simulations of whole brain areas not obviously useful for machine learning Pros describes average firing of functional populations of neurons computationally cheap can address network structure/function mathematical simplicity Universal function approximator Cons lose benefits of spiking spatiotemporal resolution averages over timing/dynamics/variability/ etc. biological analogs not always obvious 6

7 General architecture Simplest network is the perceptron r1 in r2 in w1 w2 g r Transfer function g( ) general engineering term used to describe put of some processing unit as a function of input in if r i = x i and r = y then y = g(w 1 x 1 + w 2 x 2 ) and if g( ) = (i.e. g is purely linear) then y = w1 x1 + w2 x2 General single-layer mapping r i = g( w ij r in j ) + r = g(wr in ) i 7

8 r1 in The XOR problem w1 XOR x1 x2 y x2 r2 in w2 Boolean OR x1 x2 y x2 g r x not linearly separable: more complicated! x1 linearly separable: can be easily separated with single threshold! g(x)= G 1 if x > H 0 else 8

9 r1 in The XOR problem w1 XOR x1 x2 y x1 r2 in w2 g r becomes not linearly separable: more complicated! x2 With enough hidden units, multilayer perceptron is the universal function approximator! 9

10 The XOR problem r1 in w1 g r r2 in w2 becomes With enough hidden units, multilayer perceptron is the universal function approximator! 10

11 Multi-layer perceptron HLU1 HLU2 put HLU3 E.g. sine wave approximation using logistic transfer function With enough hidden units, multilayer perceptron is the universal function approximator! 11

12 Multi-layer perceptron Limitations - Brain-like performance doesn t equate with actual performance - Training rules are non-physiologic Strengths - Hidden layer activity might resemble brain function (with appropriate inputs/puts) - Brain = mapping network - Self-Organization, like the brain - High flexibility in possible computations Point: MLP is usually good for machine learning purposes, but is not necessarily good for neuroscience theory all the time! 12

13 Two types of NNets feedforward rate-based recurrent Feed-forward: information only flows forward, simplest connectivity - Kernel machines - Radial basis function networks - Probabilistic mapping networks - Bayesian nets - Stochastic nets - Modular nets - Committee machines - Associative neural nets - Holographic associative nets (???) - Fuzzy nets - Compositional pattern-producing nets - etc. Recurrent: information can flow forward and backward, useful for time-resolved problems 13

14 How do we train a neural network? Minimize some cost function gradient descent! r1 in w1 MSE g r 1 E = 2 r2 in w2 i (r i - y i ) 2 desired put (data, training set) w ij! w ij + Dw ij 2E Dw ij = -f( ) 2wij learning gradient of MSE rate wrt weights 14

15 How do we train a neural network? r1 in r2 in w1 w2 g r E = 2 1 i (r i - y i ) 2 w ij! w ij + Dw ij 2E Dw ij = -f( ) 2wij 2E 1 2 2w = ij 2 2wij i j in (g( w ij r j ) - y i ) hi f change in weight i,j depends on learning rate and dependence of error on weight change at i,j 2f 2w ij 2f = 2g 2g 2h (using chain rule) 2h 2w ij error s dependence on weight i,j, rewritten using MSE equation Dw ij = f(g'(h i )(y i - r i )r j in ) learning rule (derivation?) 15

16 Adding layers Start by finding the put rates 2-layer (1 hidden) perceptron r i = g( w r ) r = g( w r ) j h ij h j E = 2 1 i (r i - y i ) 2 Dw ij = f(g'(h i )(y i - r i )r j in ) h h in r = g ( w g ( w r )) 3-layer (2 hidden) perceptron h h h h in r = g ( w g -1( w -1g -2( w -2r ))) n-layer (n-1 hidden) perceptron h h h h h h h h in r = g ( w g -1( w -1g -2( w g - n+ 1( w - n+ 1g -n( w -nr ))))) Idea is to nest each layer s put rates within the next 16

17 Training through the layers Generalized delta rule (put wts) 2E 2wij with d i 2E 2w 2E 2w with d h i 1 2 = 2 ( ri y) w - i 2 ij i h = di r j = g '( h h i)( r i -yi) h ij h ij = E = ( r ) w 2 = h i -yi 2 ij i 1 2 h 2 ( g ( wij g ( w r )) yi) w h = h in jk k - 2 ij i j k h in = dir j h g '( h in i ) w k ik d k 2 i delta rule for put weights (r i - y i ) 2 Dw ij = f(g'(h i )(y i - r i )r j in ) Hidden layer weights Error propagates BACKWARDS through the layers! delta rule for hidden wts (depends on delta rule for put wts) 2 17

18 Training protocol (gradient descent) Training set Test set 1. Train until gradient of error function reaches minimum. A. Batch: use full training set with each iteration (smooth convergence, but more prone to local minima) B. Online: use different sample of training set with each iteration (more memory efficient, but messy convergence) 2. Test generalization of network using previously unseen test data set. 18

19 Caveats Local minima Can use momentum term in weight update to incorporate history of weight changes. Overfitting Use heuristics to determine appropriate number of nodes for solving particular problem. Can usually use 2*number of nodes for training set. 19

20 Recurrent networks are a whole new game! but I ll spare you Backprop. through time (BPTT) 20

Neural networks. Chapter 20, Section 5 1

Neural networks. Chapter 20, Section 5 1 Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of

More information

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

More information

An Introduction to Neural Networks

An Introduction to Neural Networks An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,

More information

INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

More information

6. Feed-forward mapping networks

6. Feed-forward mapping networks 6. Feed-forward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer

More information

Lecture 6. Artificial Neural Networks

Lecture 6. Artificial Neural Networks Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

More information

Introduction to Neural Networks : Revision Lectures

Introduction to Neural Networks : Revision Lectures Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons

More information

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

More information

Neural Networks and Support Vector Machines

Neural Networks and Support Vector Machines INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

More information

Feedforward Neural Networks and Backpropagation

Feedforward Neural Networks and Backpropagation Feedforward Neural Networks and Backpropagation Feedforward neural networks Architectural issues, computational capabilities Sigmoidal and radial basis functions Gradient-based learning and Backprogation

More information

Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons

Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion

More information

Chapter 4: Artificial Neural Networks

Chapter 4: Artificial Neural Networks Chapter 4: Artificial Neural Networks CS 536: Machine Learning Littman (Wu, TA) Administration icml-03: instructional Conference on Machine Learning http://www.cs.rutgers.edu/~mlittman/courses/ml03/icml03/

More information

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

More information

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons PMR5406 Redes Neurais e Aula 3 Multilayer Percetrons Baseado em: Neural Networks, Simon Haykin, Prentice-Hall, 2 nd edition Slides do curso por Elena Marchiori, Vrie Unviersity Multilayer Perceptrons Architecture

More information

Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007

Neural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007 Neural Networks Introduction to Artificial Intelligence CSE 150 May 29, 2007 Administration Last programming assignment has been posted! Final Exam: Tuesday, June 12, 11:30-2:30 Last Lecture Naïve Bayes

More information

Learning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning.

Learning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning. Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Artificial Intelligence Learning Chapter 8 Learning is useful as a system construction method, i.e., expose

More information

Recurrent Neural Networks

Recurrent Neural Networks Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

More information

Introduction to Artificial Neural Networks MAE-491/591

Introduction to Artificial Neural Networks MAE-491/591 Introduction to Artificial Neural Networks MAE-491/591 Artificial Neural Networks: Biological Inspiration The brain has been extensively studied by scientists. Vast complexity prevents all but rudimentary

More information

Learning. CS461 Artificial Intelligence Pinar Duygulu. Bilkent University, Spring 2007. Slides are mostly adapted from AIMA and MIT Open Courseware

Learning. CS461 Artificial Intelligence Pinar Duygulu. Bilkent University, Spring 2007. Slides are mostly adapted from AIMA and MIT Open Courseware 1 Learning CS 461 Artificial Intelligence Pinar Duygulu Bilkent University, Slides are mostly adapted from AIMA and MIT Open Courseware 2 Learning What is learning? 3 Induction David Hume Bertrand Russell

More information

Neural Network Design in Cloud Computing

Neural Network Design in Cloud Computing International Journal of Computer Trends and Technology- volume4issue2-2013 ABSTRACT: Neural Network Design in Cloud Computing B.Rajkumar #1,T.Gopikiran #2,S.Satyanarayana *3 #1,#2Department of Computer

More information

Machine Learning and Data Mining -

Machine Learning and Data Mining - Machine Learning and Data Mining - Perceptron Neural Networks Nuno Cavalheiro Marques (nmm@di.fct.unl.pt) Spring Semester 2010/2011 MSc in Computer Science Multi Layer Perceptron Neurons and the Perceptron

More information

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)

More information

Neural Computation - Assignment

Neural Computation - Assignment Neural Computation - Assignment Analysing a Neural Network trained by Backpropagation AA SSt t aa t i iss i t i icc aa l l AA nn aa l lyy l ss i iss i oo f vv aa r i ioo i uu ss l lee l aa r nn i inn gg

More information

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

More information

Introduction to Neural Computation. Neural Computation

Introduction to Neural Computation. Neural Computation Introduction to Neural Computation Level 4/M Neural Computation Level 3 Website: http://www.cs.bham.ac.uk/~jxb/inc.html Lecturer: Dr. John A. Bullinaria John A. Bullinaria, 2015 Module Administration and

More information

NEURAL NETWORKS A Comprehensive Foundation

NEURAL NETWORKS A Comprehensive Foundation NEURAL NETWORKS A Comprehensive Foundation Second Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Prentice Hall Prentice Hall Upper Saddle River; New Jersey 07458 Preface xii Acknowledgments

More information

Machine Learning: Multi Layer Perceptrons

Machine Learning: Multi Layer Perceptrons Machine Learning: Multi Layer Perceptrons Prof. Dr. Martin Riedmiller Albert-Ludwigs-University Freiburg AG Maschinelles Lernen Machine Learning: Multi Layer Perceptrons p.1/61 Outline multi layer perceptrons

More information

Lecture 8 February 4

Lecture 8 February 4 ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

More information

One Solution to XOR problem using Multilayer Perceptron having Minimum Configuration

One Solution to XOR problem using Multilayer Perceptron having Minimum Configuration International Journal of Science and Engineering Volume 3, Number 2-2015 PP: 32-41 IJSE Available at www.ijse.org ISSN: 2347-2200 One Solution to XOR problem using Multilayer Perceptron having Minimum

More information

Data Mining Techniques Chapter 7: Artificial Neural Networks

Data Mining Techniques Chapter 7: Artificial Neural Networks Data Mining Techniques Chapter 7: Artificial Neural Networks Artificial Neural Networks.................................................. 2 Neural network example...................................................

More information

Designing a neural network for forecasting financial time series

Designing a neural network for forecasting financial time series Designing a neural network for forecasting financial time series 29 février 2008 What a Neural Network is? Each neurone k is characterized by a transfer function f k : output k = f k ( i w ik x k ) From

More information

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS N. K. Bose HRB-Systems Professor of Electrical Engineering The Pennsylvania State University, University Park P. Liang Associate Professor

More information

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis Contemporary Engineering Sciences, Vol. 4, 2011, no. 4, 149-163 Performance Evaluation of Artificial Neural Networks for Spatial Data Analysis Akram A. Moustafa Department of Computer Science Al al-bayt

More information

6.2.8 Neural networks for data mining

6.2.8 Neural networks for data mining 6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural

More information

Forecasting Demand in the Clothing Industry. Eduardo Miguel Rodrigues 1, Manuel Carlos Figueiredo 2 2

Forecasting Demand in the Clothing Industry. Eduardo Miguel Rodrigues 1, Manuel Carlos Figueiredo 2 2 XI Congreso Galego de Estatística e Investigación de Operacións A Coruña, 24-25-26 de outubro de 2013 Forecasting Demand in the Clothing Industry Eduardo Miguel Rodrigues 1, Manuel Carlos Figueiredo 2

More information

Introduction to Neural Networks for Senior Design

Introduction to Neural Networks for Senior Design Introduction to Neural Networks for Senior Design Intro-1 Neural Networks: The Big Picture Artificial Intelligence Neural Networks Expert Systems Machine Learning not ruleoriented ruleoriented Intro-2

More information

Artificial neural networks

Artificial neural networks Artificial neural networks Now Neurons Neuron models Perceptron learning Multi-layer perceptrons Backpropagation 2 It all starts with a neuron 3 Some facts about human brain ~ 86 billion neurons ~ 10 15

More information

A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY

A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY ARTIFICIAL NEURAL NETWORKS: A TUTORIAL BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY Contents Introduction Origin Of Neural Network Biological Neural Networks ANN Overview

More information

Stock Prediction using Artificial Neural Networks

Stock Prediction using Artificial Neural Networks Stock Prediction using Artificial Neural Networks Abhishek Kar (Y8021), Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach to predict

More information

1 SELF-ORGANIZATION MECHANISM IN THE NETWORKS

1 SELF-ORGANIZATION MECHANISM IN THE NETWORKS Mathematical literature reveals that the number of neural network structures, concepts, methods, and their applications have been well known in neural modeling literature for sometime. It started with

More information

Chapter 12 Discovering New Knowledge Data Mining

Chapter 12 Discovering New Knowledge Data Mining Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to

More information

Introduction to Neural Networks

Introduction to Neural Networks Introduction to Neural Networks 2nd Year UG, MSc in Computer Science http://www.cs.bham.ac.uk/~jxb/inn.html Lecturer: Dr. John A. Bullinaria http://www.cs.bham.ac.uk/~jxb John A. Bullinaria, 2004 Module

More information

Neural Networks algorithms and applications

Neural Networks algorithms and applications Neural Networks algorithms and applications By Fiona Nielsen 4i 12/12-2001 Supervisor: Geert Rasmussen Niels Brock Business College 1 Introduction Neural Networks is a field of Artificial Intelligence

More information

Machine learning in financial forecasting. Haindrich Henrietta Vezér Evelin

Machine learning in financial forecasting. Haindrich Henrietta Vezér Evelin Machine learning in financial forecasting Haindrich Henrietta Vezér Evelin Contents Financial forecasting Window Method Machine learning-past and future MLP (Multi-layer perceptron) Gaussian Process Bibliography

More information

Data Mining Prediction

Data Mining Prediction Data Mining Prediction Jingpeng Li 1 of 23 What is Prediction? Predicting the identity of one thing based purely on the description of another related thing Not necessarily future events, just unknowns

More information

APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED VARIABLES TO PREDICT STOCK TREND DIRECTION

APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED VARIABLES TO PREDICT STOCK TREND DIRECTION LJMS 2008, 2 Labuan e-journal of Muamalat and Society, Vol. 2, 2008, pp. 9-16 Labuan e-journal of Muamalat and Society APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED

More information

Building MLP networks by construction

Building MLP networks by construction University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2000 Building MLP networks by construction Ah Chung Tsoi University of

More information

129: Artificial Neural Networks. Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS

129: Artificial Neural Networks. Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS 129: Artificial Neural Networks Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 Introduction to Artificial Neural Networks 901 2 Neural Network Architectures 902 3 Neural Network Learning

More information

Neural network software tool development: exploring programming language options

Neural network software tool development: exploring programming language options INEB- PSI Technical Report 2006-1 Neural network software tool development: exploring programming language options Alexandra Oliveira aao@fe.up.pt Supervisor: Professor Joaquim Marques de Sá June 2006

More information

University of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons

University of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons University of Cambridge Engineering Part IIB Module 4F0: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons x y (x) Inputs x 2 y (x) 2 Outputs x d First layer Second Output layer layer y

More information

TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA

TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA www.arpapress.com/volumes/vol9issue3/ijrras_9_3_16.pdf TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA 1 Akintola K.G., 2 Alese B.K. & 2 Thompson

More information

SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS

SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS UDC: 004.8 Original scientific paper SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS Tonimir Kišasondi, Alen Lovren i University of Zagreb, Faculty of Organization and Informatics,

More information

Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks

Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Ph. D. Student, Eng. Eusebiu Marcu Abstract This paper introduces a new method of combining the

More information

Improving Generalization

Improving Generalization Improving Generalization Introduction to Neural Networks : Lecture 10 John A. Bullinaria, 2004 1. Improving Generalization 2. Training, Validation and Testing Data Sets 3. Cross-Validation 4. Weight Restriction

More information

TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC

TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC 777 TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC M.R. Walker. S. Haghighi. A. Afghan. and L.A. Akers Center for Solid State Electronics Research Arizona State University Tempe. AZ 85287-6206 mwalker@enuxha.eas.asu.edu

More information

IFT3395/6390. Machine Learning from linear regression to Neural Networks. Machine Learning. Training Set. t (3.5, -2,..., 127, 0,...

IFT3395/6390. Machine Learning from linear regression to Neural Networks. Machine Learning. Training Set. t (3.5, -2,..., 127, 0,... IFT3395/6390 Historical perspective: back to 1957 (Prof. Pascal Vincent) (Rosenblatt, Perceptron ) Machine Learning from linear regression to Neural Networks Computer Science Artificial Intelligence Symbolic

More information

Introduction to Artificial Neural Networks

Introduction to Artificial Neural Networks POLYTECHNIC UNIVERSITY Department of Computer and Information Science Introduction to Artificial Neural Networks K. Ming Leung Abstract: A computing paradigm known as artificial neural network is introduced.

More information

Lecture 14: Convolutional neural networks for computer vision

Lecture 14: Convolutional neural networks for computer vision Lecture 14: Convolutional neural networks for computer vision Dr. Richard E. Turner (ret26@cam.ac.uk) November 20, 2014 Big picture Goal: how to produce good internal representations of the visual world

More information

Back Propagation Neural Network for Wireless Networking

Back Propagation Neural Network for Wireless Networking International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-4 E-ISSN: 2347-2693 Back Propagation Neural Network for Wireless Networking Menal Dahiya Maharaja Surajmal

More information

SEMINAR OUTLINE. Introduction to Data Mining Using Artificial Neural Networks. Definitions of Neural Networks. Definitions of Neural Networks

SEMINAR OUTLINE. Introduction to Data Mining Using Artificial Neural Networks. Definitions of Neural Networks. Definitions of Neural Networks SEMINAR OUTLINE Introduction to Data Mining Using Artificial Neural Networks ISM 611 Dr. Hamid Nemati Introduction to and Characteristics of Neural Networks Comparison of Neural Networks to traditional

More information

Horse Racing Prediction Using Artificial Neural Networks

Horse Racing Prediction Using Artificial Neural Networks Horse Racing Prediction Using Artificial Neural Networks ELNAZ DAVOODI, ALI REZA KHANTEYMOORI Mathematics and Computer science Department Institute for Advanced Studies in Basic Sciences (IASBS) Gavazang,

More information

SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96. Özer CIFTCIOGLU Istanbul Technical University, ITU. and

SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96. Özer CIFTCIOGLU Istanbul Technical University, ITU. and NEA/NSC-DOC (96)29 AUGUST 1996 SMORN-VII REPORT NEURAL NETWORK BENCHMARK ANALYSIS RESULTS & FOLLOW-UP 96 Özer CIFTCIOGLU Istanbul Technical University, ITU and Erdinç TÜRKCAN Netherlands Energy Research

More information

Modeling and Forecasting Stock Prices Using an Artificial Neural Network and Imperialist Competitive Algorithm

Modeling and Forecasting Stock Prices Using an Artificial Neural Network and Imperialist Competitive Algorithm Vol. 3, No.1, January 2013, pp. 296 302 ISSN: 2225-8329 2013 HRMARS www.hrmars.com Modeling and Forecasting Stock Prices Using an Artificial Neural Network and Imperialist Competitive Algorithm Hossein

More information

Follow links Class Use and other Permissions. For more information, send email to: permissions@pupress.princeton.edu

Follow links Class Use and other Permissions. For more information, send email to: permissions@pupress.princeton.edu COPYRIGHT NOTICE: David A. Kendrick, P. Ruben Mercado, and Hans M. Amman: Computational Economics is published by Princeton University Press and copyrighted, 2006, by Princeton University Press. All rights

More information

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK N M Allinson and D Merritt 1 Introduction This contribution has two main sections. The first discusses some aspects of multilayer perceptrons,

More information

Introduction to Machine Learning CMU-10701

Introduction to Machine Learning CMU-10701 Introduction to Machine Learning CMU-10701 Deep Learning Barnabás Póczos & Aarti Singh Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio Geoffrey

More information

The Need for Small Learning Rates on Large Problems

The Need for Small Learning Rates on Large Problems In Proceedings of the 200 International Joint Conference on Neural Networks (IJCNN 0), -9. The Need for Small Learning Rates on Large Problems D. Randall Wilson fonix Corporation 80 West Election Road

More information

Dynamic neural network with adaptive Gauss neuron activation function

Dynamic neural network with adaptive Gauss neuron activation function Dynamic neural network with adaptive Gauss neuron activation function Dubravko Majetic, Danko Brezak, Branko Novakovic & Josip Kasac Abstract: An attempt has been made to establish a nonlinear dynamic

More information

Supply Chain Forecasting Model Using Computational Intelligence Techniques

Supply Chain Forecasting Model Using Computational Intelligence Techniques CMU.J.Nat.Sci Special Issue on Manufacturing Technology (2011) Vol.10(1) 19 Supply Chain Forecasting Model Using Computational Intelligence Techniques Wimalin S. Laosiritaworn Department of Industrial

More information

Analecta Vol. 8, No. 2 ISSN 2064-7964

Analecta Vol. 8, No. 2 ISSN 2064-7964 EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

More information

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING Abhishek Agrawal*, Vikas Kumar** 1,Ashish Pandey** 2,Imran Khan** 3 *(M. Tech Scholar, Department of Computer Science, Bhagwant University,

More information

Neural Networks and Back Propagation Algorithm

Neural Networks and Back Propagation Algorithm Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important

More information

Prediction Model for Crude Oil Price Using Artificial Neural Networks

Prediction Model for Crude Oil Price Using Artificial Neural Networks Applied Mathematical Sciences, Vol. 8, 2014, no. 80, 3953-3965 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2014.43193 Prediction Model for Crude Oil Price Using Artificial Neural Networks

More information

Neural Networks in Quantitative Finance

Neural Networks in Quantitative Finance Neural Networks in Quantitative Finance Master Thesis submitted to Prof. Dr. Wolfgang Härdle Institute for Statistics and Econometrics CASE - Center for Applied Statistics and Economics Humboldt-Universität

More information

Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network

Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network Time Series Data Mining in Rainfall Forecasting Using Artificial Neural Network Prince Gupta 1, Satanand Mishra 2, S.K.Pandey 3 1,3 VNS Group, RGPV, Bhopal, 2 CSIR-AMPRI, BHOPAL prince2010.gupta@gmail.com

More information

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING

EFFICIENT DATA PRE-PROCESSING FOR DATA MINING EFFICIENT DATA PRE-PROCESSING FOR DATA MINING USING NEURAL NETWORKS JothiKumar.R 1, Sivabalan.R.V 2 1 Research scholar, Noorul Islam University, Nagercoil, India Assistant Professor, Adhiparasakthi College

More information

A Time Series ANN Approach for Weather Forecasting

A Time Series ANN Approach for Weather Forecasting A Time Series ANN Approach for Weather Forecasting Neeraj Kumar 1, Govind Kumar Jha 2 1 Associate Professor and Head Deptt. Of Computer Science,Nalanda College Of Engineering Chandi(Bihar) 2 Assistant

More information

Multiple Layer Perceptron Training Using Genetic Algorithms

Multiple Layer Perceptron Training Using Genetic Algorithms Multiple Layer Perceptron Training Using Genetic Algorithms Udo Seiffert University of South Australia, Adelaide Knowledge-Based Intelligent Engineering Systems Centre (KES) Mawson Lakes, 5095, Adelaide,

More information

Classification and Prediction

Classification and Prediction Classification and Prediction 1. Objectives...2 2. Classification vs. Prediction...3 2.1. Definitions...3 2.2. Supervised vs. Unsupervised Learning...3 2.3. Classification and Prediction Related Issues...4

More information

Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification

Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification Comparison of Supervised and Unsupervised Learning Algorithms for Pattern Classification R. Sathya Professor, Dept. of MCA, Jyoti Nivas College (Autonomous), Professor and Head, Dept. of Mathematics, Bangalore,

More information

COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS

COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS Iris Ginzburg and David Horn School of Physics and Astronomy Raymond and Beverly Sackler Faculty of Exact Science Tel-Aviv University Tel-A viv 96678,

More information

Università degli Studi di Bologna

Università degli Studi di Bologna Università degli Studi di Bologna DEIS Biometric System Laboratory Incremental Learning by Message Passing in Hierarchical Temporal Memory Davide Maltoni Biometric System Laboratory DEIS - University of

More information

American International Journal of Research in Science, Technology, Engineering & Mathematics

American International Journal of Research in Science, Technology, Engineering & Mathematics American International Journal of Research in Science, Technology, Engineering & Mathematics Available online at http://www.iasir.net ISSN (Print): 2328-349, ISSN (Online): 2328-3580, ISSN (CD-ROM): 2328-3629

More information

Role of Neural network in data mining

Role of Neural network in data mining Role of Neural network in data mining Chitranjanjit kaur Associate Prof Guru Nanak College, Sukhchainana Phagwara,(GNDU) Punjab, India Pooja kapoor Associate Prof Swami Sarvanand Group Of Institutes Dinanagar(PTU)

More information

ARTIFICIAL NEURAL NETWORKS FOR DATA MINING

ARTIFICIAL NEURAL NETWORKS FOR DATA MINING ARTIFICIAL NEURAL NETWORKS FOR DATA MINING Amrender Kumar I.A.S.R.I., Library Avenue, Pusa, New Delhi-110 012 akha@iasri.res.in 1. Introduction Neural networks, more accurately called Artificial Neural

More information

Bank Customers (Credit) Rating System Based On Expert System and ANN

Bank Customers (Credit) Rating System Based On Expert System and ANN Bank Customers (Credit) Rating System Based On Expert System and ANN Project Review Yingzhen Li Abstract The precise rating of customers has a decisive impact on loan business. We constructed the BP network,

More information

Neural Network Architectures

Neural Network Architectures 6 Neural Network Architectures Bogdan M. Wilamowski Auburn University 6. Introduction... 6-6. Special Easy-to-Train Neural Network Architectures... 6- Polynomial Networks Functional Link Networks Sarajedini

More information

Power Prediction Analysis using Artificial Neural Network in MS Excel

Power Prediction Analysis using Artificial Neural Network in MS Excel Power Prediction Analysis using Artificial Neural Network in MS Excel NURHASHINMAH MAHAMAD, MUHAMAD KAMAL B. MOHAMMED AMIN Electronic System Engineering Department Malaysia Japan International Institute

More information

Neural Nets. General Model Building

Neural Nets. General Model Building Neural Nets To give you an idea of how new this material is, let s do a little history lesson. The origins are typically dated back to the early 1940 s and work by two physiologists, McCulloch and Pitts.

More information

3 An Illustrative Example

3 An Illustrative Example Objectives An Illustrative Example Objectives - Theory and Examples -2 Problem Statement -2 Perceptron - Two-Input Case -4 Pattern Recognition Example -5 Hamming Network -8 Feedforward Layer -8 Recurrent

More information

NEURAL NETWORKS AND THEIR APPLICATION TO FINANCE

NEURAL NETWORKS AND THEIR APPLICATION TO FINANCE 2008 Martin P. Wallace 67 NEURAL NETWORKS AND THEIR APPLICATION TO FINANCE Martin P. Wallace (PHD) Applied Technologies Centre London, UK Tel. +44 (0) 20 7732 2312 mw.sat@apcentre.edu Abstract Neural networks

More information

Neural Network Toolbox

Neural Network Toolbox Neural Network Toolbox A Tutorial for the Course Computational Intelligence http://www.igi.tugraz.at/lehre/ci Stefan Häusler Institute for Theoretical Computer Science Inffeldgasse 16b/I Abstract This

More information

CSC384 Intro to Artificial Intelligence

CSC384 Intro to Artificial Intelligence CSC384 Intro to Artificial Intelligence Artificial Intelligence A branch of Computer Science. Examines how we can achieve intelligent behaviour through computation. What is intelligence? Are these Intelligent?

More information

Hot Resistance Estimation for Dry Type Transformer Using Multiple Variable Regression, Multiple Polynomial Regression and Soft Computing Techniques

Hot Resistance Estimation for Dry Type Transformer Using Multiple Variable Regression, Multiple Polynomial Regression and Soft Computing Techniques American Journal of Applied Sciences 9 (2): 231-237, 2012 ISSN 1546-9239 2012 Science Publications Hot Resistance Estimation for Dry Type Transformer Using Multiple Variable Regression, Multiple Polynomial

More information

OPTIMUM LEARNING RATE FOR CLASSIFICATION PROBLEM

OPTIMUM LEARNING RATE FOR CLASSIFICATION PROBLEM OPTIMUM LEARNING RATE FOR CLASSIFICATION PROBLEM WITH MLP IN DATA MINING Lalitha Saroja Thota 1 and Suresh Babu Changalasetty 2 1 Department of Computer Science, King Khalid University, Abha, KSA 2 Department

More information

Neural Network Based Forecasting of Foreign Currency Exchange Rates

Neural Network Based Forecasting of Foreign Currency Exchange Rates Neural Network Based Forecasting of Foreign Currency Exchange Rates S. Kumar Chandar, PhD Scholar, Madurai Kamaraj University, Madurai, India kcresearch2014@gmail.com Dr. M. Sumathi, Associate Professor,

More information

Learning to Process Natural Language in Big Data Environment

Learning to Process Natural Language in Big Data Environment CCF ADL 2015 Nanchang Oct 11, 2015 Learning to Process Natural Language in Big Data Environment Hang Li Noah s Ark Lab Huawei Technologies Part 1: Deep Learning - Present and Future Talk Outline Overview

More information

Machine Learning. 01 - Introduction

Machine Learning. 01 - Introduction Machine Learning 01 - Introduction Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge

More information

Q1. What is Artificial Intelligence? Explain how an AI system is different from a conventional Computing system.

Q1. What is Artificial Intelligence? Explain how an AI system is different from a conventional Computing system. Unit I: ARTIFICIAL INTELLIGENCE Q1. What is Artificial Intelligence? Explain how an AI system is different from a conventional Computing system. Q2. What are the AI techniques? Discuss with example. Q3.

More information

Predictive Dynamix Inc

Predictive Dynamix Inc Predictive Modeling Technology Predictive modeling is concerned with analyzing patterns and trends in historical and operational data in order to transform data into actionable decisions. This is accomplished

More information