Introduction to Artificial Neural Networks. Introduction to Artificial Neural Networks


 Cornelia Wilson
 2 years ago
 Views:
Transcription
1 Introduction to Artificial Neural Networks v.3 August Michel Verleysen Introduction  Introduction to Artificial Neural Networks p Why ANNs? p Biological inspiration p Some examples of problems p Historical background p Neural computation p feature extraction p classification regression adaptive filtering projection p principle of learning (nonlinearity, error criterion) p difficulties: learning and generalization, the curse of dimensionality, overfitting and model complexity p NN structure and learning rule p How to use ANN s? Michel Verleysen Introduction 
2 Why Anns? Von Von Neumann s computer determinism sequence of of instructions high high speed repetitive tasks programming uniqueness of of solutions ex: ex: matrix product (Human) brain fuzzy behaviour parallelism slow slow speed adaptation to to situations learning different solutions ex: ex: face face recognition Michel Verleysen Introduction  3 Brain Properties p Robust (cells die!) and faulttolerant p Flexible (does not need C or java ) p Information is fuzzy, probabilistic, noisy, or inconsistent p Highly parallel p Learning p Slow p Small, compact, dissipates little power p Autoorganization p Nonunique behavior Michel Verleysen Introduction  4
3 Neurosciences  Neuroengineering From Handbook of neural computation, E. Fiesler, R. Beale eds., IOP and Oxford University Press, 996, p. A.:3 Michel Verleysen Introduction  5 Some examples of problems From DARPA Neural Network Study, 988 Michel Verleysen Introduction  6 3
4 Examples of tasks p face recognition p timeseries prediction p process identification p process control p optical character recognition p adaptive filtering p etc. Michel Verleysen Introduction  7 Historical Background Hebb biological learning rule rule McCulloch & Pitts Pitts binary decision units Rosenblatt Perceptron  learning 969 Minsky & Papert limits to to Perceptrons 974 Werbos backpropagation 98 Hopfield feedback networks Parker, LeCun, Rumelhart, backpropagation McClelland Kohonen SelfOrganizing Maps Michel Verleysen Introduction  8 4
5 Biological Neuron  Hebb s Rule p Basic scheme p Hebbs rule: how coupling between neurons (synaptic values) influences and is influenced by the operation performed by a group of neurons Michel Verleysen Introduction  9 Hebb s rule p When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A s efficiency, as one of the cells firing B, is increased. weight p Symmetrical Hebb s rule for artificial networks: p to avoid saturation p because and are conventional weight weight no change Michel Verleysen Introduction  5
6 Similarity levels p elements level p neurons sums & nonlinear functions p synapses multiplication p other elements not useful for computation p functional blocks level p visual cortex pattern recognition p sensorymotor cortex control p p structure level p distributed information p... Michel Verleysen Introduction  Artificial Neural Networks p Artificial neural networks ARE NOT : p an attempt to understand biological systems p an attempt to reproduce the behavior of a biological system p Artificial neural networks ARE : p a set of tools aimed to solve perceptive problems ( perceptive <> rulebased ) At least in the framework of this lecture, i.e. Neural Computation Michel Verleysen Introduction  6
7 Why Artificial neural networks? p Structure: many computation units in parallel (McCulloch & Pitts 943) p Learning rule (adaptation of parameters): similar to Hebb s rule (949) inputs And that s all... Michel Verleysen Introduction  3 Learning inputs p Give  many  examples (inputoutput pairs, or training samples) p Compute the parameters to fit the examples p Test the result! Michel Verleysen Introduction  4 7
8 Why Neural Computation? p Hypothetical problem of optical character recognition (OCR) p If: then:  image 56 x 56 pixels  8bits pixel values (56 gray levels)  58 different images p Necessity to work with features! Michel Verleysen Introduction  5 Feature Extraction p Example of feature in OCR: ratio height/width (x ) of the character p Histogram of feature value a class b class x Michel Verleysen Introduction  6 8
9 Multidimensional Features p Necessity to classify according to several, but not too much features x C C x p Several: because of the overlap in histogram with feature p Not too much: because classification methods are easier in small dimensional spaces Michel Verleysen Introduction  7 What can be done with ANNs? p regression p classification p adaptive filtering p projection Michel Verleysen Introduction  8 9
10 What can be done with ANNs? p regression p classification p adaptive filtering p projection Michel Verleysen Introduction  9 Classification  Regression y features regression Output value (continuous variable) x x x C features classification Class label (discrete variable) C x Michel Verleysen Introduction 
11 What can be done with ANNs? p regression.5.5 p classification p adaptive filtering .5 p projection Michel Verleysen Introduction  What can be done with ANNs? p regression p classification p adaptive filtering p projection Michel Verleysen Introduction 
12 Principle inputs Nonlinear parametric regression model Parameter adaptation _ desired (error criteria) Michel Verleysen Introduction  3 Principle inputs Nonlinear parametric regression model Parameter adaptation _ desired (error criteria) p Model: restrictions about the possible relation Michel Verleysen Introduction  4
13 Principle inputs Nonlinear parametric regression model Parameter adaptation _ desired (error criteria) p Parametric: learning parameters to adjust (contain the information) Michel Verleysen Introduction  5 Principle inputs Nonlinear parametric regression model Parameter adaptation _ desired (error criteria) p Nonlinear: more powerful than linear Michel Verleysen Introduction  6 3
14 Principle inputs Nonlinear parametric regression model Parameter adaptation _ desired (error criteria) Universal approximator! Michel Verleysen Introduction  7 p If world was linear: Why nonlinear? Push on a flexible bar, and it will shrink (one equilibrium point, stability, linearity, etc.) Michel Verleysen Introduction  8 4
15 Why nonlinear? p But world is nonlinear: Push on a flexible bar, and it will bend, to the left or to the right! (unstable equilibrium point, bifurcation, nonlinearity, etc.) Michel Verleysen Introduction  9 Error criterion inputs _ desired p regression p classification p adaptive filtering p projection i i Michel Verleysen Introduction  3 5
16 Error criterion inputs _ desired p regression p classification p adaptive filtering p projection Wrong classifications Michel Verleysen Introduction  3 Error criterion inputs _ desired p regression p classification p adaptive filtering p projection Independence between Michel Verleysen Introduction  3 6
17 Error criterion inputs _ desired p regression p classification p adaptive filtering p projection Independence between Michel Verleysen Introduction  33 Polynomial Curve Fitting p Strong similarity between polynomial curve fitting and regression with neural networks p polynoms y = w + wx + wx + L+ wmx p neural networks y = f ( w,x ) p Error criterion E = P p= p p ( y t ) p Polynoms: E is quadratic in w min(e) is the solution of a set of linear equations Neural networks: E is nonlinear in w local minima M = M j= w j x desired output (target) j Michel Verleysen Introduction
18 t t t t Learning  generalization p Learning: process of p finding parameters w which p minimize the error function E p given a set of inputoutput pairs (x p,t p ) w x x Michel Verleysen Introduction  35 Learning  generalization p generalization: process of p assigning an output value y to p a new input x p given the parameters vector w w x x Michel Verleysen Introduction
19 p Compromise between pmodel complexity and pnumber of learning pairs Overfitting p Neural networks are puniversal approximation models with pa (relatively) low complexity p But overfitting must be a serious concern! y y x x Michel Verleysen Introduction  37 Overfitting in classification x C C x C x C x C x C x Michel Verleysen Introduction
20 Model complexity p Compromise between performance and complexity: difficult to evaluate a priori p Alternative: to control the effective complexity: p new error function p penalty term (example) E ~ = E + λω d y dx dx Ω = p But: p many possible forms of penalty terms p λ difficult to choose/adjust Michel Verleysen Introduction  39 The Curse of Dimensionality p Histograms: the number of boxes is exponential with the dimension p More features reduced performances p If # data is limited dimension must be low p Concept of intrinsic dimensionality of data Michel Verleysen Introduction  4
21 Supervised  unsupervised learning p Supervised learning: building an inputoutput relation known through examples (inputoutput pairs) Observed phenomenon Neural network output  Desired p Unsupervised learning: modeling a property of the input data (for example density) Observed phenomenon Neural network output p Reinforcement learning: are good/bad (but how much good or bad is not know!) Michel Verleysen Introduction  4 What is a neural network? p Model = structure + learning rule p structure p learning rule How to compute (estimate) the parameters of the network? p Several learning rules for one structure p Similar learning rules for several structures Michel Verleysen Introduction  4
22 How to use ANNs ) examine problem Michel Verleysen Introduction  43 How to use ANNs ) examine problem ) formulate problem in terms of data analysis Michel Verleysen Introduction  44
23 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning Michel Verleysen Introduction  45 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data Michel Verleysen Introduction
24 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model Michel Verleysen Introduction  47 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model 6) learn Michel Verleysen Introduction
25 How to use ANNs + ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model 6) learn 7) test the results Michel Verleysen Introduction  49 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model 6) learn 7) test the results unsuccessful : Michel Verleysen Introduction  5 5
26 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model 6) learn 7) test the results successful! Michel Verleysen Introduction  5 How to use ANNs ) examine problem ) formulate problem in terms of data analysis 3) collect many data for learning 4) understand data 5) choose an ANN model 6) learn 7) test the results successful! Michel Verleysen Introduction  5 6
27 Sources and References p Some ideas developed in these slides come from: p Handbook of neural computation, E. Fiesler, R. Beale eds., IOP and oxford university press, 996 p Neural networks for pattern recognition, C. Bishop, Clarendon press, Oxford, 995 p Other references: p Les Réseaux de Neurones Artificiels, F. Blayo, M. Verleysen, Presses Universitaires de France, collection "Que saisje?", Paris (996), 6 pages. Short introduction in French, not sufficient, but easy to read. p Pattern classification, R.O. Duda, P.E. Hart, D.G. Strok, second edition, Wiley, (a good book on pattern classification, including, but not limited to, neural networks). Michel Verleysen Introduction
Artificial neural networks
Artificial neural networks Now Neurons Neuron models Perceptron learning Multilayer perceptrons Backpropagation 2 It all starts with a neuron 3 Some facts about human brain ~ 86 billion neurons ~ 10 15
More informationLecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham
Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)
More informationArtificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence
Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network?  Perceptron learners  Multilayer networks What is a Support
More informationIntroduction to Neural Networks
Introduction to Neural Networks 2nd Year UG, MSc in Computer Science http://www.cs.bham.ac.uk/~jxb/inn.html Lecturer: Dr. John A. Bullinaria http://www.cs.bham.ac.uk/~jxb John A. Bullinaria, 2004 Module
More informationNeural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons
Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion
More informationA TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY
ARTIFICIAL NEURAL NETWORKS: A TUTORIAL BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY Contents Introduction Origin Of Neural Network Biological Neural Networks ANN Overview
More informationIntroduction to Neural Computation. Neural Computation
Introduction to Neural Computation Level 4/M Neural Computation Level 3 Website: http://www.cs.bham.ac.uk/~jxb/inc.html Lecturer: Dr. John A. Bullinaria John A. Bullinaria, 2015 Module Administration and
More informationIntroduction to Artificial Neural Networks
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Introduction to Artificial Neural Networks K. Ming Leung Abstract: A computing paradigm known as artificial neural network is introduced.
More information6.2.8 Neural networks for data mining
6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural
More informationLearning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning.
Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Artificial Intelligence Learning Chapter 8 Learning is useful as a system construction method, i.e., expose
More informationIntroduction to Machine Learning Using Python. Vikram Kamath
Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression
More informationNeural networks. Chapter 20, Section 5 1
Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of
More information6. Feedforward mapping networks
6. Feedforward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation ByoungTak Zhang Biointelligence Laboratory School of Computer
More informationInternational Journal of Electronics and Computer Science Engineering 1449
International Journal of Electronics and Computer Science Engineering 1449 Available Online at www.ijecse.org ISSN 22771956 Neural Networks in Data Mining Priyanka Gaur Department of Information and
More informationLecture 8 Artificial neural networks: Unsupervised learning
Lecture 8 Artificial neural networks: Unsupervised learning Introduction Hebbian learning Generalised Hebbian learning algorithm Competitive learning Selforganising computational map: Kohonen network
More informationFeedForward mapping networks KAIST 바이오및뇌공학과 정재승
FeedForward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Tradeoff between energy consumption and wiring cost Tradeoff between energy consumption
More informationINTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbookchapterslides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An
More informationCS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.
Lecture Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott
More informationNeural Networks. Introduction to Artificial Intelligence CSE 150 May 29, 2007
Neural Networks Introduction to Artificial Intelligence CSE 150 May 29, 2007 Administration Last programming assignment has been posted! Final Exam: Tuesday, June 12, 11:302:30 Last Lecture Naïve Bayes
More informationData Mining and Neural Networks in Stata
Data Mining and Neural Networks in Stata 2 nd Italian Stata Users Group Meeting Milano, 10 October 2005 Mario Lucchini e Maurizo Pisati Università di MilanoBicocca mario.lucchini@unimib.it maurizio.pisati@unimib.it
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationIntroduction to Pattern Recognition
Introduction to Pattern Recognition Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationCS 2750 Machine Learning. Lecture 1. Machine Learning. CS 2750 Machine Learning.
Lecture 1 Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott
More informationEFFICIENT DATA PREPROCESSING FOR DATA MINING
EFFICIENT DATA PREPROCESSING FOR DATA MINING USING NEURAL NETWORKS JothiKumar.R 1, Sivabalan.R.V 2 1 Research scholar, Noorul Islam University, Nagercoil, India Assistant Professor, Adhiparasakthi College
More informationMachine Learning. CUNY Graduate Center, Spring 2013. Professor Liang Huang. huang@cs.qc.cuny.edu
Machine Learning CUNY Graduate Center, Spring 2013 Professor Liang Huang huang@cs.qc.cuny.edu http://acl.cs.qc.edu/~lhuang/teaching/machinelearning Logistics Lectures M 9:3011:30 am Room 4419 Personnel
More informationNEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS
NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS N. K. Bose HRBSystems Professor of Electrical Engineering The Pennsylvania State University, University Park P. Liang Associate Professor
More informationIntroduction to Artificial Neural Networks MAE491/591
Introduction to Artificial Neural Networks MAE491/591 Artificial Neural Networks: Biological Inspiration The brain has been extensively studied by scientists. Vast complexity prevents all but rudimentary
More informationA Partially Supervised Metric Multidimensional Scaling Algorithm for Textual Data Visualization
A Partially Supervised Metric Multidimensional Scaling Algorithm for Textual Data Visualization Ángela Blanco Universidad Pontificia de Salamanca ablancogo@upsa.es Spain Manuel MartínMerino Universidad
More informationLecture 6. Artificial Neural Networks
Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm
More informationSEMINAR OUTLINE. Introduction to Data Mining Using Artificial Neural Networks. Definitions of Neural Networks. Definitions of Neural Networks
SEMINAR OUTLINE Introduction to Data Mining Using Artificial Neural Networks ISM 611 Dr. Hamid Nemati Introduction to and Characteristics of Neural Networks Comparison of Neural Networks to traditional
More information129: Artificial Neural Networks. Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS
129: Artificial Neural Networks Ajith Abraham Oklahoma State University, Stillwater, OK, USA 1 Introduction to Artificial Neural Networks 901 2 Neural Network Architectures 902 3 Neural Network Learning
More informationIntroduction to Machine Learning Lecture 1. Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu
Introduction to Machine Learning Lecture 1 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Introduction Logistics Prerequisites: basics concepts needed in probability and statistics
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher
More informationIntroduction to Machine Learning. Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011
Introduction to Machine Learning Speaker: Harry Chao Advisor: J.J. Ding Date: 1/27/2011 1 Outline 1. What is machine learning? 2. The basic of machine learning 3. Principles and effects of machine learning
More informationMonotonicity Hints. Abstract
Monotonicity Hints Joseph Sill Computation and Neural Systems program California Institute of Technology email: joe@cs.caltech.edu Yaser S. AbuMostafa EE and CS Deptartments California Institute of Technology
More informationSelf Organizing Maps: Fundamentals
Self Organizing Maps: Fundamentals Introduction to Neural Networks : Lecture 16 John A. Bullinaria, 2004 1. What is a Self Organizing Map? 2. Topographic Maps 3. Setting up a Self Organizing Map 4. Kohonen
More informationProgramming Neural Networks in Java
Programming Neural Networks in Java Programming Neural Networks in Java will show the intermediate to advanced Java programmer how to create neural networks. This book attempts to teach neural network
More informationIntroduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk
Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems
More informationIntroduction to Neural Networks : Revision Lectures
Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons
More informationNeural Networks and Support Vector Machines
INF5390  Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF539013 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines
More informationModels of Cortical Maps II
CN510: Principles and Methods of Cognitive and Neural Modeling Models of Cortical Maps II Lecture 19 Instructor: Anatoli Gorchetchnikov dy dt The Network of Grossberg (1976) Ay B y f (
More informationIntroduction to Neural Networks for Senior Design
Introduction to Neural Networks for Senior Design Intro1 Neural Networks: The Big Picture Artificial Intelligence Neural Networks Expert Systems Machine Learning not ruleoriented ruleoriented Intro2
More informationLecture 14: Convolutional neural networks for computer vision
Lecture 14: Convolutional neural networks for computer vision Dr. Richard E. Turner (ret26@cam.ac.uk) November 20, 2014 Big picture Goal: how to produce good internal representations of the visual world
More informationNEURAL NETWORKS IN DATA MINING
NEURAL NETWORKS IN DATA MINING 1 DR. YASHPAL SINGH, 2 ALOK SINGH CHAUHAN 1 Reader, Bundelkhand Institute of Engineering & Technology, Jhansi, India 2 Lecturer, United Institute of Management, Allahabad,
More information203.4770: Introduction to Machine Learning Dr. Rita Osadchy
203.4770: Introduction to Machine Learning Dr. Rita Osadchy 1 Outline 1. About the Course 2. What is Machine Learning? 3. Types of problems and Situations 4. ML Example 2 About the course Course Homepage:
More informationRecurrent Neural Networks
Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time
More informationIntroduction to Machine Learning
Introduction to Machine Learning Isabelle Guyon isabelle@clopinet.com What is Machine Learning? Learning algorithm Trained machine TRAINING DATA Answer Query What for? Classification Time series prediction
More informationARTIFICIAL NEURAL NETWORKS FOR DATA MINING
ARTIFICIAL NEURAL NETWORKS FOR DATA MINING Amrender Kumar I.A.S.R.I., Library Avenue, Pusa, New Delhi110 012 akha@iasri.res.in 1. Introduction Neural networks, more accurately called Artificial Neural
More informationThe Hebbian rule. The neural network stores and retrieves associations, which are learned as synaptic connection.
Hopfield Networks The Hebbian rule Donald Hebb hypothesised in 1949 how neurons are connected with each other in the brain: When an axon of cell A is near enough to excite a cell B and repeatedly or persistently
More informationIFT3395/6390. Machine Learning from linear regression to Neural Networks. Machine Learning. Training Set. t (3.5, 2,..., 127, 0,...
IFT3395/6390 Historical perspective: back to 1957 (Prof. Pascal Vincent) (Rosenblatt, Perceptron ) Machine Learning from linear regression to Neural Networks Computer Science Artificial Intelligence Symbolic
More informationPredictive Dynamix Inc
Predictive Modeling Technology Predictive modeling is concerned with analyzing patterns and trends in historical and operational data in order to transform data into actionable decisions. This is accomplished
More information/SOLUTIONS/ where a, b, c and d are positive constants. Study the stability of the equilibria of this system based on linearization.
echnische Universiteit Eindhoven Faculteit Elektrotechniek NIELINEAIRE SYSEMEN / NEURALE NEWERKEN (P6) gehouden op donderdag maart 7, van 9: tot : uur. Dit examenonderdeel bestaat uit 8 opgaven. /SOLUIONS/
More informationMANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL
MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College
More informationNeural Networks for Machine Learning. Lecture 13a The ups and downs of backpropagation
Neural Networks for Machine Learning Lecture 13a The ups and downs of backpropagation Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdelrahman Mohamed A brief history of backpropagation
More informationNeural Networks algorithms and applications
Neural Networks algorithms and applications By Fiona Nielsen 4i 12/122001 Supervisor: Geert Rasmussen Niels Brock Business College 1 Introduction Neural Networks is a field of Artificial Intelligence
More informationAn Introduction to Artificial Neural Networks (ANN)  Methods, Abstraction, and Usage
An Introduction to Artificial Neural Networks (ANN)  Methods, Abstraction, and Usage Introduction An artificial neural network (ANN) reflects a system that is based on operations of biological neural
More informationUsing Neural Networks for Pattern Classification Problems
Using Neural Networks for Pattern Classification Problems Converting an Image Camera captures an image Image needs to be converted to a form that can be processed by the Neural Network Converting an Image
More informationNeural Networks in Data Mining
IOSR Journal of Engineering (IOSRJEN) ISSN (e): 22503021, ISSN (p): 22788719 Vol. 04, Issue 03 (March. 2014), V6 PP 0106 www.iosrjen.org Neural Networks in Data Mining Ripundeep Singh Gill, Ashima Department
More informationSynaptic Learning Rules
Synaptic Learning Rules Computational Models of Neural Systems Lecture 4.1 David S. Touretzky October, 2013 Why Study Synaptic Plasticity? Synaptic learning rules determine the information processing capabilities
More informationAn Introduction to Neural Networks
An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,
More informationIntroduction To Artificial Neural Network
Introduction To Artificial Neural Network Marcello Pelillo Department of Computer Science University of Venice http://www.unive.it http://www.dsi.unive.it DARPA Neural Network Study Over the history of
More informationIntroduction to Learning & Decision Trees
Artificial Intelligence: Representation and Problem Solving 538 April 0, 2007 Introduction to Learning & Decision Trees Learning and Decision Trees to learning What is learning?  more than just memorizing
More informationNeural Network Addin
Neural Network Addin Version 1.5 Software User s Guide Contents Overview... 2 Getting Started... 2 Working with Datasets... 2 Open a Dataset... 3 Save a Dataset... 3 Data Preprocessing... 3 Lagging...
More informationAttribution. Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley)
Machine Learning 1 Attribution Modified from Stuart Russell s slides (Berkeley) Parts of the slides are inspired by Dan Klein s lecture material for CS 188 (Berkeley) 2 Outline Inductive learning Decision
More informationWHAT IS AN ARTIFICIAL INTELLIGENCE?
ARTIFICIAL INTELLIGENCE AND NEURAL NETWORKS WHAT IS AN ARTIFICIAL INTELLIGENCE? It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related
More informationSteven C.H. Hoi School of Information Systems Singapore Management University Email: chhoi@smu.edu.sg
Steven C.H. Hoi School of Information Systems Singapore Management University Email: chhoi@smu.edu.sg Introduction http://stevenhoi.org/ Finance Recommender Systems Cyber Security Machine Learning Visual
More informationIntroduction to Machine Learning
Introduction to Machine Learning Prof. Alexander Ihler Prof. Max Welling icamp Tutorial July 22 What is machine learning? The ability of a machine to improve its performance based on previous results:
More informationData Mining Part 5. Prediction
Data Mining Part 5. Prediction 5.1 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Classification vs. Numeric Prediction Prediction Process Data Preparation Comparing Prediction Methods References Classification
More informationBiological Neurons and Neural Networks, Artificial Neurons
Biological Neurons and Neural Networks, Artificial Neurons Introduction to Neural Networks : Lecture 2 John A. Bullinaria, 2004 1. Organization of the Nervous System and Brain 2. Brains versus Computers:
More informationNeural Nets. General Model Building
Neural Nets To give you an idea of how new this material is, let s do a little history lesson. The origins are typically dated back to the early 1940 s and work by two physiologists, McCulloch and Pitts.
More informationMethod of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks
Method of Combining the Degrees of Similarity in Handwritten Signature Authentication Using Neural Networks Ph. D. Student, Eng. Eusebiu Marcu Abstract This paper introduces a new method of combining the
More informationComparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations
Volume 3, No. 8, August 2012 Journal of Global Research in Computer Science REVIEW ARTICLE Available Online at www.jgrcs.info Comparison of Supervised and Unsupervised Learning Classifiers for Travel Recommendations
More informationData Mining Algorithms Part 1. Dejan Sarka
Data Mining Algorithms Part 1 Dejan Sarka Join the conversation on Twitter: @DevWeek #DW2015 Instructor Bio Dejan Sarka (dsarka@solidq.com) 30 years of experience SQL Server MVP, MCT, 13 books 7+ courses
More informationComputational Intelligence Introduction
Computational Intelligence Introduction Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Fall 2011 Farzaneh Abdollahi Neural Networks 1/21 Fuzzy Systems What are
More informationLearning to Process Natural Language in Big Data Environment
CCF ADL 2015 Nanchang Oct 11, 2015 Learning to Process Natural Language in Big Data Environment Hang Li Noah s Ark Lab Huawei Technologies Part 1: Deep Learning  Present and Future Talk Outline Overview
More informationIAI : Biological Intelligence and Neural Networks
IAI : Biological Intelligence and Neural Networks John A. Bullinaria, 2005 1. How do Humans do Intelligent Things? 2. What are Neural Networks? 3. What are Artificial Neural Networks used for? 4. Introduction
More informationTRAINING A LIMITEDINTERCONNECT, SYNTHETIC NEURAL IC
777 TRAINING A LIMITEDINTERCONNECT, SYNTHETIC NEURAL IC M.R. Walker. S. Haghighi. A. Afghan. and L.A. Akers Center for Solid State Electronics Research Arizona State University Tempe. AZ 852876206 mwalker@enuxha.eas.asu.edu
More informationNeural Networks. CAP5610 Machine Learning Instructor: GuoJun Qi
Neural Networks CAP5610 Machine Learning Instructor: GuoJun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector
More informationComparative study of biological and artificial neural networks
Available online at www.scholarsresearchlibrary.com European Journal of Applied Engineering and Scientific Research, 2013, 2 (1):3646 (http://scholarsresearchlibrary.com/archive.html) ISSN: 2278 0041
More informationFoundations. Computational Intelligence
Foundations Computational Intelligence Intelligence A standard dictionary definition of intelligence is: "1 a (1): The ability to learn or understand or to deal with new or trying situations : REASON;
More informationMACHINE LEARNING. Introduction. Alessandro Moschitti
MACHINE LEARNING Introduction Alessandro Moschitti Department of Computer Science and Information Engineering University of Trento Email: moschitti@disi.unitn.it Course Schedule Lectures Tuesday, 14:0016:00
More informationNEURAL NETWORKS A Comprehensive Foundation
NEURAL NETWORKS A Comprehensive Foundation Second Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Prentice Hall Prentice Hall Upper Saddle River; New Jersey 07458 Preface xii Acknowledgments
More informationCSC321 Introduction to Neural Networks and Machine Learning. Lecture 21 Using Boltzmann machines to initialize backpropagation.
CSC321 Introduction to Neural Networks and Machine Learning Lecture 21 Using Boltzmann machines to initialize backpropagation Geoffrey Hinton Some problems with backpropagation The amount of information
More informationBiological Neurons and Neural Networks, Artificial Neurons
Biological Neurons and Neural Networks, Artificial Neurons Neural Computation : Lecture 2 John A. Bullinaria, 2015 1. Organization of the Nervous System and Brain 2. Brains versus Computers: Some Numbers
More informationNeural Network Design in Cloud Computing
International Journal of Computer Trends and Technology volume4issue22013 ABSTRACT: Neural Network Design in Cloud Computing B.Rajkumar #1,T.Gopikiran #2,S.Satyanarayana *3 #1,#2Department of Computer
More informationData, Measurements, Features
Data, Measurements, Features Middle East Technical University Dep. of Computer Engineering 2009 compiled by V. Atalay What do you think of when someone says Data? We might abstract the idea that data are
More information1 SELFORGANIZATION MECHANISM IN THE NETWORKS
Mathematical literature reveals that the number of neural network structures, concepts, methods, and their applications have been well known in neural modeling literature for sometime. It started with
More informationChapter 12 Discovering New Knowledge Data Mining
Chapter 12 Discovering New Knowledge Data Mining BecerraFernandez, et al.  Knowledge Management 1/e  2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to
More informationMachine Learning. 01  Introduction
Machine Learning 01  Introduction Machine learning course One lecture (Wednesday, 9:30, 346) and one exercise (Monday, 17:15, 203). Oral exam, 20 minutes, 5 credit points. Some basic mathematical knowledge
More informationEVALUATION OF NEURAL NETWORK BASED CLASSIFICATION SYSTEMS FOR CLINICAL CANCER DATA CLASSIFICATION
EVALUATION OF NEURAL NETWORK BASED CLASSIFICATION SYSTEMS FOR CLINICAL CANCER DATA CLASSIFICATION K. Mumtaz Vivekanandha Institute of Information and Management Studies, Tiruchengode, India S.A.Sheriff
More informationApplication of Neural Networks to Character Recognition
Proceedings of Students/Faculty Research Day, CSIS, Pace University, May 4th, 2007 Application of Neural Networks to Character Recognition Abstract Dong Xiao Ni Seidenberg School of CSIS, Pace University,
More informationProgramming Exercise 3: Multiclass Classification and Neural Networks
Programming Exercise 3: Multiclass Classification and Neural Networks Machine Learning November 4, 2011 Introduction In this exercise, you will implement onevsall logistic regression and neural networks
More informationClassification algorithm in Data mining: An Overview
Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department
More informationA Time Series ANN Approach for Weather Forecasting
A Time Series ANN Approach for Weather Forecasting Neeraj Kumar 1, Govind Kumar Jha 2 1 Associate Professor and Head Deptt. Of Computer Science,Nalanda College Of Engineering Chandi(Bihar) 2 Assistant
More informationSEARCH AND CLASSIFICATION OF "INTERESTING" BUSINESS APPLICATIONS IN THE WORLD WIDE WEB USING A NEURAL NETWORK APPROACH
SEARCH AND CLASSIFICATION OF "INTERESTING" BUSINESS APPLICATIONS IN THE WORLD WIDE WEB USING A NEURAL NETWORK APPROACH Abstract Karl Kurbel, Kirti Singh, Frank Teuteberg Europe University Viadrina Frankfurt
More informationSupervised and unsupervised learning  1
Chapter 3 Supervised and unsupervised learning  1 3.1 Introduction The science of learning plays a key role in the field of statistics, data mining, artificial intelligence, intersecting with areas in
More informationData Mining Practical Machine Learning Tools and Techniques
Ensemble learning Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Combining multiple models Bagging The basic idea
More informationRain prediction from meteoradar images
2015 http://excel.fit.vutbr.cz Rain prediction from meteoradar images Michael Vlček t + 1 t + 2... t  2 t  1 t t  3... input layer hidden layers output layer Abstract This paper presents a software
More informationAn Artificial Neural NetworksBased online Monitoring Odor Sensing System
Journal of Computer Science 5 (11): 878882, 2009 ISSN 15493636 2009 Science Publications An Artificial Neural NetworksBased online Monitoring Odor Sensing System Yousif AlBastaki The College of Information
More informationScalable Developments for Big Data Analytics in Remote Sensing
Scalable Developments for Big Data Analytics in Remote Sensing Federated Systems and Data Division Research Group High Productivity Data Processing Dr.Ing. Morris Riedel et al. Research Group Leader,
More information