Learning framework for NNs. Introduction to Neural Networks. Learning goal: Inputs/outputs. x 1 x 2. y 1 y 2

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Learning framework for NNs. Introduction to Neural Networks. Learning goal: Inputs/outputs. x 1 x 2. y 1 y 2"

Transcription

1 Introduction to Neura Networks Learning framework for NNs What are neura networks? Noninear function approimators How do they reate to pattern recognition/cassification? Noninear discriminant functions More compe decision boundaries than inear discriminant functions (e.g. Fisher, Gaussians with equa covariances inputs n Unknown mapping G rainabe mode Γ y y y m m mode outputs desired outputs Inputs/outputs Learning goa: Definitions: y y G( (e.g discriminant function we want to earn n ( n inputs y y y m ( m process outputs Find w such that: E( w E( w, w, where E( w error between G and Γ. rainabe mode: Γ(, ( w adjustabe parameters w m ( m mode outputs What shoud E( w be?

2 Error function (idea Error function (practica Input/output data: p input-output training patterns Ideay, E( w y p( d p y y y p How to compute? i y i i i in y i y i y im, i Γ( i, w. E( w p -- y i i i p m -- y ( ij ij i j Artificia neura networks (NNs Neura networks are one type of parametric mode Γ. Noninear function approimators Bioogica inspiration Structure and function oosey based on bioogica neura networks (e.g. brain. Reativey simpe buiding bocks connected together in massive and parae network. Adjustabe (trainabe parameters w (weights Map inputs to outputs dendrites neuron aon Why Neura Network? What does a neuron do?

3 Neuron transfer function Rough approimation: threshod function aon output net stimuus from dendrites Neura networks: crude emuation of bioogy Simpe basic buiding bocks. Individua units are connected massivey and in parae. Individua units have threshod-type activation functions. Learning through adjustment of the strength of connection (weights between individua units Caveat: Artificia neura networks are much, much, much simper than bioogica systems. Eampe: Human brain: neurons connections Basic buiding bocks of neura networks Basic buiding bock: the unit ψ φ φ φ φ q (scaar inputs w ω ω ω q (weights ω ω ω ω q φ φ φ φ q noninear activation function ψ ( w ω i φ i φ q i (output

4 Perceptrons: the simpest neura network hreshod activation function t ( u u θ u < θ ω ω n ω ω n t ( u What is this? θ u Perceptron output Limited mapping capabiity Perceptron mapping: w t w t < ω.5 ω ω OR function where,.8 n.6.4. w ω ω ω n XOR function

5 More genera networks: activation function ( u ( u ( u (sigmoid + e u e u e u e u e u (hyperboic tangent + sigmoid ( u hyperboic tangent signa fow (feedforward More genera networks: mutiayer perceptrons (MLPs m n output ayer input ayer hidden unit ayer u u signa fow (feedforward More genera networks: mutiayer perceptrons (MLPs m n output ayer hidden unit ayer # hidden unit ayer # input ayer MLP appication eampe: ALVINN Sharp Left Straight Ahead Sharp Right 4 Hidden Units 3 Output Units 33 Sensor Input Retina ALVINN: Neura Network for Autonomous Steering

6 A simpe eampe Derivation of function ƒ( f ( c[ t ( a t ( b ] ω 6 ω 7 f ( c t ( a c t ( b ω 5 f ( c ω ω ω 4 ω 3 t ( u ( ku as k f ( c [ k ( a ] c [ k ( b ] for arge k. a b ω 5 + ω 6 ( ω + ω + ω 7 ( ω 3 + ω 4 Weight vaues for simpe eampe Some theoretica properties of NNs set # set # ω ω ω 3 ω 4 ω 5 ω 6 ω 7 kb k ka k c c ka k kb k c c Singe-input functions: what does the previous eampe say about singe-input functions? f ( NN error

7 Muti-input functions: universa function approimator? Does the singe-input eampe hod in genera? Neura networks in practice: 3 basic steps. Coect input/output training data.. Seect an appropriate neura network architecture: Number of hidden ayers Number of hidden units in each ayer. 3. rain (adjust the weights of the neura network to minimie the error measure, E p -- y i i i Neura network training Gradient descent (one parameter Key probem: How to adjust w to minimie E? Answer: use derivative information on error surface.. Initiaie ω to some random initia vaue.. Change ω iterativey at step t according to: E ( ω a b e c ω( t + ω( t η de dω( t d f ω Impies oca, not goba minimum...

8 Genera gradient descent Simpe eampe of gradient computation. Initiaie w to some random initia vaue.. Change w iterativey at step t according to: w( t + w( t η E[ w( t ] E[ w( t ] ω ( t ω ( t ( ωq t Compute ω 4 ω 5 for the neura network beow: ω 6 ω 7 ω ω ω 4 Singe training pattern y, E -- ( y ω 3 Derivation Generaiation to mutipe training patterns: ω j p -- y ω j ( i i i p i -- ( y i i ω j. net ω + ω net ω 3 + ω 4 h ( net h ( net ω 5 + ω 6 h + ω 7 h ω 4 ( y ω4 Derivation ω 5 ω ω ω 4 ω 3 ω 6 ω 7 ω 4 h ( y h net net ω 4

9 net ω + ω net ω 3 + ω 4 h ( net h ( net ω 5 + ω 6 h + ω 7 h ω 4 ω 4 Derivation ω 5 h ( y h net net ω 4 ( yω 7 ' ( net ω ω ω 4 ω 3 ω 6 ω 7 Generaiation: Backpropagation Key probem: Generaie specific resut to compute derivatives in more genera manner. Answer: Backpropagation agorithm [Rumehart and McCeand,986]. Efficient, agorithmic formuation for computing error derivatives Gradient computation without hardcoding derivatives (aows on-the-fy adjustment of NN architectures. ( h k ω kj ω ij k Backpropagation derivation net j ω ij h i ω ij h i ω ij ω ij h i Backpropagation derivation: output units net k ω jk h i unit k ω ij E net k m -- y ( k net k

10 Backpropagation derivation: output units unit k m E -- y ( Backpropagation derivation:output units unit k net k k net k net k ω jk net k k net k net k ω jk ( net k h i ω ij ' ( net net k k h i ω ij ( y k '( net k ( y k ω jk Backpropagation derivation: hidden units unit k Backpropagation derivation: hidden units unit k net k ω jk net k ω jk h i ω ij net net h i ω ij net net net δ

11 Backpropagation derivation: hidden units net k ω jk net unit k δ net s ω s ( net s Backpropagation derivation: hidden units net k ω jk unit k h i ω ij net ω ' ( j δ ω j ' ( h i ω ij δ ω j ' ( netj ω ij h i Output units: ω jk Hidden units: Backpropagation summary ( y k '( net k Basic steps in using neura networks. Coect training data. Preprocess training data 3. Seect neura network architecture 4. Seect earning agorithm 5. Weight initiaiation ω ij h i 6. Forward pass δ ω j ' ( netj 7. Backward pass 8. Repeat steps 6 and 7 unti satisfactory mode is reached.

12 he Forward Pass he Backward Pass. Evauate at the outputs, where,. Appy an input vector to network.. Compute the net input to each hidden unit ( netj. 3. Compute the hidden-unit outputs (, 4. Compute the neura network outputs (. i net k for each output unit k.. Backpropagate the δ vaues from the outputs backwards through the neura network. 3. Compute. ω i 4. Update weights based on the computed gradient, w( t + w( t η E[ w( t ]. Practica issues. What shoud your training data be? Practica issues (continued 5. Seecting the earning parameter Sufficient training data? Biased training data? Deterministic/stochastic task? Stationary/non-stationary? In gradient descent: w( t + w( t η E[ w( t ] what shoud η be?. What shoud your neura network architecture be? 3. Preprocessing of data. 4. Weight initiaiation why sma, random vaues? Difficut question to answer...

13 Seecting the earning parameter: an eampe Sampe error surface: E ω + ω (reaistic? Seecting the earning parameter: an eampe Where is the minimum of this error surface? E ω + ω E ω -.5 ω How many steps to convergence? ( E < 6 Different initia weights Different earning rates Deriving the gradient descent equations Gradient? E ω + ω Convergence eperiments Initia weights: ( ω, ω (, ω ω Gradient descent? ω ( t + ω ( t η ω ( t ω ( t + ω ( t ( 4η ω ( t + ω ( t ( η ω ω # steps to convergence η

14 A coser ook A coser ook η. η ω.5 ω ω ω A coser ook η.5 What happens at η >.5? Gradient descent equations: ω ( t + ω ( t ( 4η.5 ω ( t + ω ( t ( η ω ω Simiar to fied-point iteration: ω( t + cω( t diverges for c >, ω( converges for c <.

15 Convergence of gradient descent equations Learning rate discussion ω ( t + ω ( t ( 4η require that: 4η < < 4η < < η <.5 Why not η <? ω ( t + ω ( t ( η Probematic error surfaces: ong, steep-sided vaeys If earning rate is too sma, sow convergence. If earning rate is too arge, possibe divergence. heoretica bounds not possibe in genera case (ony for specific, trivia eampe. Motivation for ooking at more advanced training agorithms doing more with the gradient information. Any thoughts? Practica issues (continued Good generaiation: wo data sets 6. Pattern vs. batch training 7. Good generaiation y y NN error cross-vaidation data training data Sufficienty constrained neura network architecture. Cross vaidation. eary stopping point training time

Artificial neural networks and deep learning

Artificial neural networks and deep learning February 20, 2015 1 Introduction Artificia Neura Networks (ANNs) are a set of statistica modeing toos originay inspired by studies of bioogica neura networks in animas, for exampe the brain and the centra

More information

INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

More information

An Idiot s guide to Support vector machines (SVMs)

An Idiot s guide to Support vector machines (SVMs) An Idiot s guide to Support vector machines (SVMs) R. Berwick, Viage Idiot SVMs: A New Generation of Learning Agorithms Pre 1980: Amost a earning methods earned inear decision surfaces. Linear earning

More information

Face Hallucination and Recognition

Face Hallucination and Recognition Face Haucination and Recognition Xiaogang Wang and Xiaoou Tang Department of Information Engineering, The Chinese University of Hong Kong {xgwang1, xtang}@ie.cuhk.edu.hk http://mmab.ie.cuhk.edu.hk Abstract.

More information

Simultaneous Routing and Power Allocation in CDMA Wireless Data Networks

Simultaneous Routing and Power Allocation in CDMA Wireless Data Networks Simutaneous Routing and Power Aocation in CDMA Wireess Data Networks Mikae Johansson *,LinXiao and Stephen Boyd * Department of Signas, Sensors and Systems Roya Institute of Technoogy, SE 00 Stockhom,

More information

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons

PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons PMR5406 Redes Neurais e Aula 3 Multilayer Percetrons Baseado em: Neural Networks, Simon Haykin, Prentice-Hall, 2 nd edition Slides do curso por Elena Marchiori, Vrie Unviersity Multilayer Perceptrons Architecture

More information

Lecture 6. Artificial Neural Networks

Lecture 6. Artificial Neural Networks Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

More information

Development of Model Building Procedures in Wavelet Neural Networks for Forecasting Non-Stationary Time Series

Development of Model Building Procedures in Wavelet Neural Networks for Forecasting Non-Stationary Time Series European Journa of Scientific Research ISSN 1450-216X Vo.34 No.3 (2009), pp.416-427 EuroJournas Pubishing, Inc. 2009 http://www.eurojournas.com/ejsr.htm Deveopment of Mode Buiding Procedures in Waveet

More information

FRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS. Karl Skretting and John Håkon Husøy

FRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS. Karl Skretting and John Håkon Husøy FRAME BASED TEXTURE CLASSIFICATION BY CONSIDERING VARIOUS SPATIAL NEIGHBORHOODS Kar Skretting and John Håkon Husøy University of Stavanger, Department of Eectrica and Computer Engineering N-4036 Stavanger,

More information

Introduction to Neural Networks : Revision Lectures

Introduction to Neural Networks : Revision Lectures Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons

More information

Chapter 4: Artificial Neural Networks

Chapter 4: Artificial Neural Networks Chapter 4: Artificial Neural Networks CS 536: Machine Learning Littman (Wu, TA) Administration icml-03: instructional Conference on Machine Learning http://www.cs.rutgers.edu/~mlittman/courses/ml03/icml03/

More information

Pricing Internet Services With Multiple Providers

Pricing Internet Services With Multiple Providers Pricing Internet Services With Mutipe Providers Linhai He and Jean Warand Dept. of Eectrica Engineering and Computer Science University of Caifornia at Berkeey Berkeey, CA 94709 inhai, wr@eecs.berkeey.edu

More information

HYBRID FUZZY LOGIC PID CONTROLLER. Abstract

HYBRID FUZZY LOGIC PID CONTROLLER. Abstract HYBRID FUZZY LOGIC PID CONTROLLER Thomas Brehm and Kudip S. Rattan Department of Eectrica Engineering Wright State University Dayton, OH 45435 Abstract This paper investigates two fuzzy ogic PID controers

More information

ONE of the most challenging problems addressed by the

ONE of the most challenging problems addressed by the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 44, NO. 9, SEPTEMBER 2006 2587 A Mutieve Context-Based System for Cassification of Very High Spatia Resoution Images Lorenzo Bruzzone, Senior Member,

More information

Introduction to Artificial Neural Networks MAE-491/591

Introduction to Artificial Neural Networks MAE-491/591 Introduction to Artificial Neural Networks MAE-491/591 Artificial Neural Networks: Biological Inspiration The brain has been extensively studied by scientists. Vast complexity prevents all but rudimentary

More information

Vibration Reduction of Audio Visual Device Mounted on Automobile due to Gap Vibration

Vibration Reduction of Audio Visual Device Mounted on Automobile due to Gap Vibration Vibration Reduction of Audio Visua Device Mounted on Automobie due to Gap Vibration Nobuyuki OKUBO, Shinji KANADA, Takeshi TOI CAMAL, Department of Precision Mechanics, Chuo University 1-13-27 Kasuga,

More information

The Computation of the Inverse of a Square Polynomial Matrix

The Computation of the Inverse of a Square Polynomial Matrix The Computation of the Inverse of a Square Poynomia Matrix Ky M. Vu, PhD. AuLac Technoogies Inc. c 2008 Emai: kymvu@auactechnoogies.com Abstract An approach to cacuate the inverse of a square poynomia

More information

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

More information

GREEN: An Active Queue Management Algorithm for a Self Managed Internet

GREEN: An Active Queue Management Algorithm for a Self Managed Internet : An Active Queue Management Agorithm for a Sef Managed Internet Bartek Wydrowski and Moshe Zukerman ARC Specia Research Centre for Utra-Broadband Information Networks, EEE Department, The University of

More information

Decision support system for the batching problems of steelmaking and continuous-casting production

Decision support system for the batching problems of steelmaking and continuous-casting production Omega 36 (2008 976 991 www.esevier.com/ocate/omega Decision support system for the batching probems of steemaking and continuous-casting production Lixin Tang, Gongshu Wang The Logistics Institute, Northeastern

More information

CS 688 Pattern Recognition Lecture 4. Linear Models for Classification

CS 688 Pattern Recognition Lecture 4. Linear Models for Classification CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(

More information

An Introduction to Neural Networks

An Introduction to Neural Networks An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,

More information

Experiment #11 BJT filtering

Experiment #11 BJT filtering Jonathan Roderick Hakan Durmus Experiment #11 BJT fitering Introduction: Now that the BJT has bn expored in static and dynamic operation, the BJT, combined with what has bn aready presented, may be used

More information

Neural networks. Chapter 20, Section 5 1

Neural networks. Chapter 20, Section 5 1 Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of

More information

Fast Robust Hashing. ) [7] will be re-mapped (and therefore discarded), due to the load-balancing property of hashing.

Fast Robust Hashing. ) [7] will be re-mapped (and therefore discarded), due to the load-balancing property of hashing. Fast Robust Hashing Manue Urueña, David Larrabeiti and Pabo Serrano Universidad Caros III de Madrid E-89 Leganés (Madrid), Spain Emai: {muruenya,darra,pabo}@it.uc3m.es Abstract As statefu fow-aware services

More information

REAL TIME IMPLEMANTATION OF LMS BEAMFORMER FOR cdma2000 3G SYSTEM USING TI TMS320C6701 DSP

REAL TIME IMPLEMANTATION OF LMS BEAMFORMER FOR cdma2000 3G SYSTEM USING TI TMS320C6701 DSP REAL TIME IMPLEMANTATION OF LMS BEAMFORMER FOR cdma2000 3G SYSTEM USING TI TMS320C6701 DSP Kerem Küçük, Mustafa Karakoç, and Adnan Kavak + Kocaei University + Kocaei University Eectronics and Computer

More information

Lecture 7 Datalink Ethernet, Home. Datalink Layer Architectures

Lecture 7 Datalink Ethernet, Home. Datalink Layer Architectures Lecture 7 Dataink Ethernet, Home Peter Steenkiste Schoo of Computer Science Department of Eectrica and Computer Engineering Carnegie Meon University 15-441 Networking, Spring 2004 http://www.cs.cmu.edu/~prs/15-441

More information

Figure 1. A Simple Centrifugal Speed Governor.

Figure 1. A Simple Centrifugal Speed Governor. ENGINE SPEED CONTROL Peter Westead and Mark Readman, contro systems principes.co.uk ABSTRACT: This is one of a series of white papers on systems modeing, anaysis and contro, prepared by Contro Systems

More information

Neural Networks and Support Vector Machines

Neural Networks and Support Vector Machines INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

More information

Splicing System Based Genetic Algorithms for Developing RBF Networks

Splicing System Based Genetic Algorithms for Developing RBF Networks Chin. J. Chem. Eng., 5() 40 46 (007) Spicing System Based Genetic Agorithms for Deveoping RBF Networks Modes * TAO Jii( 陶吉利 ) and WANG Ning( 王宁 )** Nationa Laboratory of Industria Contro Technoogy, Institute

More information

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

More information

Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016

Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016 Rate-based artificial neural networks and error backpropagation learning Scott Murdison Machine learning journal club May 16, 2016 Murdison, Leclercq, Lefèvre and Blohm J Neurophys 2015 Neural networks???

More information

With the arrival of Java 2 Micro Edition (J2ME) and its industry

With the arrival of Java 2 Micro Edition (J2ME) and its industry Knowedge-based Autonomous Agents for Pervasive Computing Using AgentLight Fernando L. Koch and John-Jues C. Meyer Utrecht University Project AgentLight is a mutiagent system-buiding framework targeting

More information

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK N M Allinson and D Merritt 1 Introduction This contribution has two main sections. The first discusses some aspects of multilayer perceptrons,

More information

Comparison of Misspecification Tests Designed for Non-linear Time Series Models

Comparison of Misspecification Tests Designed for Non-linear Time Series Models ömmföäfsäafaäsfassfassfas fffffffffffffffffffffffffffffffffff Discussion Papers Comparison of Misspecification Tests Designed for Non-inear Time Series Modes Leena Kaiovirta University of Hesinki and HECER

More information

Neural network software tool development: exploring programming language options

Neural network software tool development: exploring programming language options INEB- PSI Technical Report 2006-1 Neural network software tool development: exploring programming language options Alexandra Oliveira aao@fe.up.pt Supervisor: Professor Joaquim Marques de Sá June 2006

More information

ELEVATING YOUR GAME FROM TRADE SPEND TO TRADE INVESTMENT

ELEVATING YOUR GAME FROM TRADE SPEND TO TRADE INVESTMENT Initiatives Strategic Mapping Success in The Food System: Discover. Anayze. Strategize. Impement. Measure. ELEVATING YOUR GAME FROM TRADE SPEND TO TRADE INVESTMENT Foodservice manufacturers aocate, in

More information

Elsevier Editorial System(tm) for Computers in Biology and Medicine Manuscript Draft

Elsevier Editorial System(tm) for Computers in Biology and Medicine Manuscript Draft Esevier Editoria System(tm) for Computers in Bioogy and Medicine Manuscript Draft Manuscript Number: CBM-D--00R1 Tite: CLASSIFICATION AND RETRIEVAL ON MACROINVERTABRATE IMAGE DATABASES Artice Type: Fu

More information

Discounted Cash Flow Analysis (aka Engineering Economy)

Discounted Cash Flow Analysis (aka Engineering Economy) Discounted Cash Fow Anaysis (aka Engineering Economy) Objective: To provide economic comparison of benefits and costs that occur over time Assumptions: Future benefits and costs can be predicted A Benefits,

More information

A Latent Variable Pairwise Classification Model of a Clustering Ensemble

A Latent Variable Pairwise Classification Model of a Clustering Ensemble A atent Variabe Pairwise Cassification Mode of a Custering Ensembe Vadimir Berikov Soboev Institute of mathematics, Novosibirsk State University, Russia berikov@math.nsc.ru http://www.math.nsc.ru Abstract.

More information

Normalization of Database Tables. Functional Dependency. Examples of Functional Dependencies: So Now what is Normalization? Transitive Dependencies

Normalization of Database Tables. Functional Dependency. Examples of Functional Dependencies: So Now what is Normalization? Transitive Dependencies ISM 602 Dr. Hamid Nemati Objectives The idea Dependencies Attributes and Design Understand concepts normaization (Higher-Leve Norma Forms) Learn how to normaize tabes Understand normaization and database

More information

Inductance. Bởi: OpenStaxCollege

Inductance. Bởi: OpenStaxCollege Inductance Bởi: OpenStaxCoege Inductors Induction is the process in which an emf is induced by changing magnetic fux. Many exampes have been discussed so far, some more effective than others. Transformers,

More information

INTRODUCTION TO THE FINITE ELEMENT METHOD

INTRODUCTION TO THE FINITE ELEMENT METHOD INTRODUCTION TO THE FINITE ELEMENT METHOD Evgen Barkanov Institute of Materias and Structures Facut of Civi Engineering Riga Technica Universit Riga, Preface Toda the finite eement method (FEM) is considered

More information

Linear Models for Classification

Linear Models for Classification Linear Models for Classification Sumeet Agarwal, EEL709 (Most figures from Bishop, PRML) Approaches to classification Discriminant function: Directly assigns each data point x to a particular class Ci

More information

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

More information

CONDENSATION. Prabal Talukdar. Associate Professor Department of Mechanical Engineering IIT Delhi E-mail: prabal@mech.iitd.ac.in

CONDENSATION. Prabal Talukdar. Associate Professor Department of Mechanical Engineering IIT Delhi E-mail: prabal@mech.iitd.ac.in CONDENSATION Praba Taukdar Associate Professor Department of Mechanica Engineering IIT Dehi E-mai: praba@mech.iitd.ac.in Condensation When a vapor is exposed to a surface at a temperature beow T sat, condensation

More information

Lecture 5: Solution Method for Beam Deflections

Lecture 5: Solution Method for Beam Deflections Structura Mechanics.080 Lecture 5 Semester Yr Lecture 5: Soution Method for Beam Defections 5.1 Governing Equations So far we have estabished three groups of equations fuy characterizing the response of

More information

Neural Networks in Quantitative Finance

Neural Networks in Quantitative Finance Neural Networks in Quantitative Finance Master Thesis submitted to Prof. Dr. Wolfgang Härdle Institute for Statistics and Econometrics CASE - Center for Applied Statistics and Economics Humboldt-Universität

More information

Artificial neural networks

Artificial neural networks Artificial neural networks Now Neurons Neuron models Perceptron learning Multi-layer perceptrons Backpropagation 2 It all starts with a neuron 3 Some facts about human brain ~ 86 billion neurons ~ 10 15

More information

Background & Motivation. Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures. Layered Architecture

Background & Motivation. Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures. Layered Architecture Background & Motivation Layering as Optimization Decomposition: A Mathematica Theory o Network Architectures Network Architecture? Convex Optimization max U(x) Ax C x 0 Francesco Lo Presti Universita di

More information

Leakage detection in water pipe networks using a Bayesian probabilistic framework

Leakage detection in water pipe networks using a Bayesian probabilistic framework Probabiistic Engineering Mechanics 18 (2003) 315 327 www.esevier.com/ocate/probengmech Leakage detection in water pipe networks using a Bayesian probabiistic framework Z. Pouakis, D. Vaougeorgis, C. Papadimitriou*

More information

University of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons

University of Cambridge Engineering Part IIB Module 4F10: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons University of Cambridge Engineering Part IIB Module 4F0: Statistical Pattern Processing Handout 8: Multi-Layer Perceptrons x y (x) Inputs x 2 y (x) 2 Outputs x d First layer Second Output layer layer y

More information

Recognition of Prior Learning

Recognition of Prior Learning Recognition of Prior Learning Information Guideines for Students EXTENDED CAMPUS This subject materia is issued by Cork Institute of Technoogy on the understanding that: Cork Institute of Technoogy expressy

More information

Simulation-Based Booking Limits for Airline Revenue Management

Simulation-Based Booking Limits for Airline Revenue Management OPERATIONS RESEARCH Vo. 53, No. 1, January February 2005, pp. 90 106 issn 0030-364X eissn 1526-5463 05 5301 0090 informs doi 10.1287/opre.1040.0164 2005 INFORMS Simuation-Based Booking Limits for Airine

More information

Temporal Difference Learning in the Tetris Game

Temporal Difference Learning in the Tetris Game Temporal Difference Learning in the Tetris Game Hans Pirnay, Slava Arabagi February 6, 2009 1 Introduction Learning to play the game Tetris has been a common challenge on a few past machine learning competitions.

More information

The Radix-4 and the Class of Radix-2 s FFTs

The Radix-4 and the Class of Radix-2 s FFTs Chapter 11 The Radix- and the Cass of Radix- s FFTs The divide-and-conuer paradigm introduced in Chapter 3 is not restricted to dividing a probem into two subprobems. In fact, as expained in Section. and

More information

GWPD 4 Measuring water levels by use of an electric tape

GWPD 4 Measuring water levels by use of an electric tape GWPD 4 Measuring water eves by use of an eectric tape VERSION: 2010.1 PURPOSE: To measure the depth to the water surface beow and-surface datum using the eectric tape method. Materias and Instruments 1.

More information

Perfect competition. By the end of this chapter, you should be able to: 7 Perfect competition. 1 Microeconomics. 1 Microeconomics

Perfect competition. By the end of this chapter, you should be able to: 7 Perfect competition. 1 Microeconomics. 1 Microeconomics erfect 7 12 By the end of this chapter, you shoud be abe to: HL expain the assumptions of perfect HL distinguish between the demand curve for the industry and for the firm in perfect HL expain how the

More information

SQL. Ilchul Yoon Assistant Professor State University of New York, Korea. on tables. describing schema. CSE 532 Theory of Database Systems

SQL. Ilchul Yoon Assistant Professor State University of New York, Korea. on tables. describing schema. CSE 532 Theory of Database Systems CSE 532 Theory of Database Systems Lecture 03 SQL Ichu Yoon Assistant Professor State University of New York, Korea Adapted from book authors sides SQL Language for describing database schema & operations

More information

eg Enterprise vs. a Big 4 Monitoring Soution: Comparing Tota Cost of Ownership Restricted Rights Legend The information contained in this document is confidentia and subject to change without notice. No

More information

Linear Classification. Volker Tresp Summer 2015

Linear Classification. Volker Tresp Summer 2015 Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

More information

Math: Fundamentals 100

Math: Fundamentals 100 Math: Fundamentas 100 Wecome to the Tooing University. This course is designed to be used in conjunction with the onine version of this cass. The onine version can be found at http://www.tooingu.com. We

More information

Applying graph theory to automatic vehicle tracking by remote sensing

Applying graph theory to automatic vehicle tracking by remote sensing 0 0 Appying graph theory to automatic vehice tracking by remote sensing *Caros Lima Azevedo Nationa Laboratory for Civi Engineering Department of Transportation Av. Do Brasi, Lisbon, 00-0 Portuga Phone:

More information

WINMAG Graphics Management System

WINMAG Graphics Management System SECTION 10: page 1 Section 10: by Honeywe WINMAG Graphics Management System Contents What is WINMAG? WINMAG Text and Graphics WINMAG Text Ony Scenarios Fire/Emergency Management of Fauts & Disabement Historic

More information

3.14 Lifting Surfaces

3.14 Lifting Surfaces .0 - Marine Hydrodynamics, Spring 005 Lecture.0 - Marine Hydrodynamics Lecture 3.4 Lifting Surfaces 3.4. D Symmetric Streamined Body No separation, even for arge Reynods numbers. stream ine Viscous effects

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

Recurrent Neural Networks

Recurrent Neural Networks Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

More information

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis

Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis Contemporary Engineering Sciences, Vol. 4, 2011, no. 4, 149-163 Performance Evaluation of Artificial Neural Networks for Spatial Data Analysis Akram A. Moustafa Department of Computer Science Al al-bayt

More information

3.5 Pendulum period. 2009-02-10 19:40:05 UTC / rev 4d4a39156f1e. g = 4π2 l T 2. g = 4π2 x1 m 4 s 2 = π 2 m s 2. 3.5 Pendulum period 68

3.5 Pendulum period. 2009-02-10 19:40:05 UTC / rev 4d4a39156f1e. g = 4π2 l T 2. g = 4π2 x1 m 4 s 2 = π 2 m s 2. 3.5 Pendulum period 68 68 68 3.5 Penduum period 68 3.5 Penduum period Is it coincidence that g, in units of meters per second squared, is 9.8, very cose to 2 9.87? Their proximity suggests a connection. Indeed, they are connected

More information

Art of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1-932394-06-0

Art of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1-932394-06-0 IEEE DISTRIBUTED SYSTEMS ONLINE 1541-4922 2005 Pubished by the IEEE Computer Society Vo. 6, No. 5; May 2005 Editor: Marcin Paprzycki, http://www.cs.okstate.edu/%7emarcin/ Book Reviews: Java Toos and Frameworks

More information

An Interactive Fuzzy Satisficing Method for Multiobjective Stochastic Integer Programming Problems through a Probability Maximization Model

An Interactive Fuzzy Satisficing Method for Multiobjective Stochastic Integer Programming Problems through a Probability Maximization Model Cahit Perkgoz et a./asia Pacific Management Review (005) 10(1), 9-35 An Interactive Fuzzy Satisficing Method for Mutiobective Stochastic Integer Programming Probems through a Probabiity Maximization Mode

More information

Secure Network Coding with a Cost Criterion

Secure Network Coding with a Cost Criterion Secure Network Coding with a Cost Criterion Jianong Tan, Murie Médard Laboratory for Information and Decision Systems Massachusetts Institute of Technoogy Cambridge, MA 0239, USA E-mai: {jianong, medard}@mit.edu

More information

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence

Artificial Neural Networks and Support Vector Machines. CS 486/686: Introduction to Artificial Intelligence Artificial Neural Networks and Support Vector Machines CS 486/686: Introduction to Artificial Intelligence 1 Outline What is a Neural Network? - Perceptron learners - Multi-layer networks What is a Support

More information

Virtual trunk simulation

Virtual trunk simulation Virtua trunk simuation Samui Aato * Laboratory of Teecommunications Technoogy Hesinki University of Technoogy Sivia Giordano Laboratoire de Reseaux de Communication Ecoe Poytechnique Federae de Lausanne

More information

500 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 31, NO. 3, MARCH 2013

500 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 31, NO. 3, MARCH 2013 500 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 3, NO. 3, MARCH 203 Cognitive Radio Network Duaity Agorithms for Utiity Maximization Liang Zheng Chee Wei Tan, Senior Member, IEEE Abstract We

More information

The Use of Cooling-Factor Curves for Coordinating Fuses and Reclosers

The Use of Cooling-Factor Curves for Coordinating Fuses and Reclosers he Use of ooing-factor urves for oordinating Fuses and Recosers arey J. ook Senior Member, IEEE S& Eectric ompany hicago, Iinois bstract his paper describes how to precisey coordinate distribution feeder

More information

Interpreting Individual Classifications of Hierarchical Networks

Interpreting Individual Classifications of Hierarchical Networks Interpreting Individua Cassifications of Hierarchica Networks Wi Landecker Michae D. Thomure Luís M. A. Bettencourt Meanie Mitche Garrett T. Kenyon SFI WORKING PAPER: 2013-02-007 SFI Working Papers contain

More information

25.4. Solution Using Fourier Series. Introduction. Prerequisites. Learning Outcomes

25.4. Solution Using Fourier Series. Introduction. Prerequisites. Learning Outcomes Soution Using Fourier Series 25.4 Introduction In this Section we continue to use the separation of variabes method for soving PDEs but you wi find that, to be abe to fit certain boundary conditions, Fourier

More information

Precise assessment of partial discharge in underground MV/HV power cables and terminations

Precise assessment of partial discharge in underground MV/HV power cables and terminations QCM-C-PD-Survey Service Partia discharge monitoring for underground power cabes Precise assessment of partia discharge in underground MV/HV power cabes and terminations Highy accurate periodic PD survey

More information

A Branch-and-Price Algorithm for Parallel Machine Scheduling with Time Windows and Job Priorities

A Branch-and-Price Algorithm for Parallel Machine Scheduling with Time Windows and Job Priorities A Branch-and-Price Agorithm for Parae Machine Scheduing with Time Windows and Job Priorities Jonathan F. Bard, 1 Siwate Rojanasoonthon 2 1 Graduate Program in Operations Research and Industria Engineering,

More information

Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images

Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images Large-Scae Optimization of Hierarchica Features for Saiency Prediction in Natura Images Eeonora Vig Harvard University vig@fas.harvard.edu Michae Dorr Harvard Medica Schoo michae.dorr@schepens.harvard.edu

More information

An FDD Wideband CDMA MAC Protocol for Wireless Multimedia Networks

An FDD Wideband CDMA MAC Protocol for Wireless Multimedia Networks An FDD ideband CDMA MAC Protoco for ireess Mutimedia Networks Xudong ang Broadband and ireess Networking (BN) Lab Schoo of Eectrica and Computer Engineering Georgia Institute of Technoogy Atanta, GA 3332

More information

nπt a n cos y(x) satisfies d 2 y dx 2 + P EI y = 0 If the column is constrained at both ends then we can apply the boundary conditions

nπt a n cos y(x) satisfies d 2 y dx 2 + P EI y = 0 If the column is constrained at both ends then we can apply the boundary conditions Chapter 6: Page 6- Fourier Series 6. INTRODUCTION Fourier Anaysis breaks down functions into their frequency components. In particuar, a function can be written as a series invoving trigonometric functions.

More information

Hamstring strains. What is a hamstring strain? How do hamstring strains occur? what you ll find in this brochure

Hamstring strains. What is a hamstring strain? How do hamstring strains occur? what you ll find in this brochure what you find in this brochure What is a hamstring strain? How do hamstring strains occur? What you shoud do if a hamstring strain occurs. What rehabiitation you shoud do. Exampe of a return to pay strategy.

More information

Lecture 6 Chi Square Distribution (c 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distribution (c 2 ) and Least Squares Fitting Lectre 6 Chi Sqare Distribtion (c ) and Least Sqares Fitting Chi Sqare Distribtion (c ) Sppose: We have a set of measrements {x 1, x, x n }. We know the tre vae of each x i (x t1, x t, x tn ). + We wod

More information

Australian Bureau of Statistics Management of Business Providers

Australian Bureau of Statistics Management of Business Providers Purpose Austraian Bureau of Statistics Management of Business Providers 1 The principa objective of the Austraian Bureau of Statistics (ABS) in respect of business providers is to impose the owest oad

More information

LADDER SAFETY Table of Contents

LADDER SAFETY Table of Contents Tabe of Contents SECTION 1. TRAINING PROGRAM INTRODUCTION..................3 Training Objectives...........................................3 Rationae for Training.........................................3

More information

Market Design & Analysis for a P2P Backup System

Market Design & Analysis for a P2P Backup System Market Design & Anaysis for a P2P Backup System Sven Seuken Schoo of Engineering & Appied Sciences Harvard University, Cambridge, MA seuken@eecs.harvard.edu Denis Chares, Max Chickering, Sidd Puri Microsoft

More information

Safety Manual VEGAPULS series 60

Safety Manual VEGAPULS series 60 Safety Manua VEGAPULS series 60 4 20 ma/hart Radar Contents Contents 1 Functiona safety 1.1 In genera............................ 3 1.2 Panning............................. 5 1.3 Instrument parameter adjustment...........

More information

Advanced ColdFusion 4.0 Application Development - 3 - Server Clustering Using Bright Tiger

Advanced ColdFusion 4.0 Application Development - 3 - Server Clustering Using Bright Tiger Advanced CodFusion 4.0 Appication Deveopment - CH 3 - Server Custering Using Bri.. Page 1 of 7 [Figures are not incuded in this sampe chapter] Advanced CodFusion 4.0 Appication Deveopment - 3 - Server

More information

Budgeting Loans from the Social Fund

Budgeting Loans from the Social Fund Budgeting Loans from the Socia Fund tes sheet Pease read these notes carefuy. They expain the circumstances when a budgeting oan can be paid. Budgeting Loans You may be abe to get a Budgeting Loan if:

More information

The Simple Pendulum. by Dr. James E. Parks

The Simple Pendulum. by Dr. James E. Parks by Dr. James E. Parks Department of Physics and Astronomy 401 Niesen Physics Buidin The University of Tennessee Knoxvie, Tennessee 37996-100 Copyriht June, 000 by James Edar Parks* *A rihts are reserved.

More information

A Supplier Evaluation System for Automotive Industry According To Iso/Ts 16949 Requirements

A Supplier Evaluation System for Automotive Industry According To Iso/Ts 16949 Requirements A Suppier Evauation System for Automotive Industry According To Iso/Ts 16949 Requirements DILEK PINAR ÖZTOP 1, ASLI AKSOY 2,*, NURSEL ÖZTÜRK 2 1 HONDA TR Purchasing Department, 41480, Çayırova - Gebze,

More information

Neural Networks algorithms and applications

Neural Networks algorithms and applications Neural Networks algorithms and applications By Fiona Nielsen 4i 12/12-2001 Supervisor: Geert Rasmussen Niels Brock Business College 1 Introduction Neural Networks is a field of Artificial Intelligence

More information

Design and Analysis of a Hidden Peer-to-peer Backup Market

Design and Analysis of a Hidden Peer-to-peer Backup Market Design and Anaysis of a Hidden Peer-to-peer Backup Market Sven Seuken, Denis Chares, Max Chickering, Mary Czerwinski Kama Jain, David C. Parkes, Sidd Puri, and Desney Tan December, 2015 Abstract We present

More information

TCP/IP Gateways and Firewalls

TCP/IP Gateways and Firewalls Gateways and Firewas 1 Gateways and Firewas Prof. Jean-Yves Le Boudec Prof. Andrzej Duda ICA, EPFL CH-1015 Ecubens http://cawww.epf.ch Gateways and Firewas Firewas 2 o architecture separates hosts and

More information

SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci

SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH Ufuk Cebeci Department of Industria Engineering, Istanbu Technica University, Macka, Istanbu, Turkey - ufuk_cebeci@yahoo.com Abstract An Enterprise

More information

Design of Follow-Up Experiments for Improving Model Discrimination and Parameter Estimation

Design of Follow-Up Experiments for Improving Model Discrimination and Parameter Estimation Design of Foow-Up Experiments for Improving Mode Discrimination and Parameter Estimation Szu Hui Ng 1 Stephen E. Chick 2 Nationa University of Singapore, 10 Kent Ridge Crescent, Singapore 119260. Technoogy

More information

CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society

CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY Course Offered By: Indian Environmenta Society INTRODUCTION The Indian Environmenta Society (IES) a dynamic and fexibe organization with a goba vision

More information

Multi-Robot Task Scheduling

Multi-Robot Task Scheduling Proc of IEEE Internationa Conference on Robotics and Automation, Karsruhe, Germany, 013 Muti-Robot Tas Scheduing Yu Zhang and Lynne E Parer Abstract The scheduing probem has been studied extensivey in

More information