# COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 COMBINED NEURAL NETWORKS FOR TIME SERIES ANALYSIS Iris Ginzburg and David Horn School of Physics and Astronomy Raymond and Beverly Sackler Faculty of Exact Science Tel-Aviv University Tel-A viv 96678, Israel Abstract We propose a method for improving the performance of any network designed to predict the next value of a time series. Vve advocate analyzing the deviations of the network's predictions from the data in the training set. This can be carried out by a secondary network trained on the time series of these residuals. The combined system of the two networks is viewed as the new predictor. We demonstrate the simplicity and success of this method, by applying it to the sunspots data. The small corrections of the secondary network can be regarded as resulting from a Taylor expansion of a complex network which includes the combined system. \\Te find that the complex network is more difficult to train and performs worse than the two-step procedure of the combined system. 1 INTRODUCTION The use of neural networks for computational tasks is based on the idea that the efficient way in which the nervous system handles memory and cognition is worth immitating. Artificial implementations are often based on a single network of mathematical neurons. We note, however, that in biological systems one can find collections of consecutive networks, performing a complicated task in several stages, with later stages refining the performance of earlier ones. Here we propose to follow this strategy in artificial applications. 224

2 Combined Neural Networks for Time Series Analysis 225 We study the analysis of time series, where the problem is to predict the next element on the basis of previous elements of the series. One looks then for a functional relation Yn = f (Yn -1, Yn - 2,..., Yn - m). (1 ) This type of representation is particularly useful for the study of dynamical systems. These are characterized by a common continuous variable, time, and many correlated degrees of freedom which combine into a set of differential equations. Nonetheless, each variable can in principle be described by a lag-space representation of the type 1. This is valid even if the Y = y(t) solution is unpredictable as in chaotic phenomena. Weigend Huberman and Rumelhart (1990) have studied the experimental series of yearly averages of sunspots activity using this approach. They have realized the lag-space representation on an (m, d, 1) network, where the notation implies a hidden layer of d sigmoidal neurons and one linear output. Using m = 12 and a weight-elimination method which led to d = 3, they obtained results which compare favorably with the leading statistical model (Tong and Lim, 1980). Both models do well in predicting the next element of the sunspots series. Recently, Nowlan and Hinton (1992) have shown that a significantly better network can be obtained if the training procedure includes a complexity penalty term in which the distribution of weights is modelled as a mixture of multiple gaussians whose parameters vary in an adaptive manner as the system is being trained. We propose an alternative method which is capable of improving the performance of neural networks: train another network to predict the errors of the first one, to uncover and remove systematic correlations that may be found in the solution given by the trained network, thus correcting the original predictions. This is in agreement with the general philosophy mentioned at the beginning, where we take from Nature the idea that the task does not have to be performed by one complicated network; it is advantageous to break it into stages of consecutive analysis steps. Starting with a network which is trained on the sunspots data with back-propagation, we show that the processed results improve considerably and we find solutions which match the performance of Weigend et. al. 2 CONSTRUCTION OF THE PRIMARY NETWORK Let us start with a simple application of back-propagation to the construction of a neural network describing the sunspots data which are normalized to lie between o and 1. The network is assumed to have one hidden layer of sigmoidal neurons, hi i = 1". " d, which receives the input of the nth vector: m hi = 0'(2: WijYn-j - Oi) j=l (2) The output of the network, Pn, is constructed linearly, d Pn = 2: Wi hi - O. (3)

3 226 Ginzburg and Hom The error-function which we minimize is defined by 1 N E = 2 L (Pn - Yn)2 n=m+l where we try to equate Pn, the prediction or output of the network, with Yn, the nth value of the series. This is the appropriate formulation for a training set of N data points which are viewed as N - m strings of length m used to predict the point following each string. We will work with two sets of data points. One will be labelled T and be used for training the network, and the other P will be used for testing its predictive power. Let us define the average error by 1 {s = jjsfj 2:(Pn - Yn)2 nes where the set S is either Tor P. An alternative parameter was used by Weigend et. al.,in which the error is normalized by the standard deviation of the data. This leads to an average relative variance (arv) which is related to the average error through Following Weigend et. al. we choose m = 12 neurons in the first layer and IITII = 220 data points for the training set. The following IIPII = 35 years are used for testing the predictions of our network. We use three sigmoidal units in the hidden layer and run with a slow convergence rate for 7000 periods. This is roughly where cross-validation would indicate that a minimum is reached. The starting parameters of our networks are chosen randomly. Five examples of such networks are presented in Table 1. 3 THE SECONDARY NETWORK Given the networks constructed above, we investigate their deviations from the desired values qn = Yn - Pn (7) A standard statistical test for the quality of any predictor is the analysis of the correlations between consecutive errors. If such correlations are found, the predictor must be improved. The correlations reflect a systematic deviation of the primary network from the true solution. We propose not to improve the primary network by modifying its architecture but to add to it a secondary network which uses the residuals qn as its new data. The latter is being trained only after the training session of the primary network has been completed. Clearly one may expect some general relation of the type to exist. Looking for a structure of this kind enlarges considerably the original space in which we searched for a solution to 1. We wish the secondary network (4) (5) (6) (8)

4 Combined Neural Networks for Time Series Analysis 227 to do a modest task, therefore we assume that much can be gained by looking at the interdependence of the residuals qn on themselves. This reduces the problem to finding the best values of Tn = b(qn-l, qn-2,"', qn-i) which would minimize the new error function 1 N E2='2 L (Tn-qn)2. n=i+1 Alternatively, one may try to express the residual in terms of the functional values minimizing again the expression 10. (9) (10) Tn =!2(Yn-1, Yn-2,"', Yn-I) (11) When the secondary network completes its training, we propose to view tn = Pn + Tn (12) as the new prediction of the combined system. We will demonstrate that a major improvement can be obtained already with a linear perceptron. This means that the linear regression or 1 Tn = L aiqn-i + /3 1 1 (13) Tn = L a;yn-i + /3 2 (14) is sufficient to account for a large fraction of the systematic deviations of the primary networks from the true function that they were trained to represent. 4 NUMERICAL RESULTS We present in Table 1 five examples of results of (12,5,1) networks, i.e. m = 12 inputs, a hidden layer of three sigmoidal neurons and a linear output neuron. These five examples were chosen from 100 runs of simple back-propagation networks with random initial conditions by selecting the networks with the smallest R values (Ginzburg and Horn, 1992). This is a weak constraint which is based on letting the network generate a large sequence of data by iterating its own predictions, and selecting the networks whose distribution of function values is the closest to the corresponding distribution of the training set. The errors of the primary networks, in particular those of the prediction set p, are quite higher than those quoted by Weigend et. al. who started out from a (12,8,1) network and brought it down through a weight elimination technique to a (12,5,1) structure. They have obtained the values T = p = We can reduce our errors and reach the same range by activating a secondary network with I = 11 to perform the linear regression (3.6) on the residuals of the predictions of the primary network. The results are the primed errors quoted in the table. Characteristically we observe a reduction of T by 3-4% and a reduction of p by more than 10%.

5 228 Ginzburg and Hom # ft f' T {p {' P Table 1 Error parameters of five networks. The unprimed errors are those of the primary networks. The primed errors correspond to the combined system which includes correction of the residuals by a linear perceptron with I = 11, which is an autoregressions of the residuals. Slightly better results for the short term predictions are achieved by corrections based on regression of the residuals on the original input vectors, when the regression length is 13 (Table 2). # {T ft fp f' p Table 2 Error parameters for the same five networks. The primed errors correspond to the combined system which includes correction of the residuals by a linear perceptron based on original input vectors with I = LONG TERM PREDICTIONS When short term prediction is performed, the output of the original network is corrected by the error predicted by the secondary network. This can be easily generalized to perform long term predictions by feeding the corrected output produced by the combined system of both networks back as input to the primary network. The corrected residuals predicted by the secondary network are viewed as the residuals needed as further inputs if the secondary network is the one performing autoregression of residuals. We run both systems based on regression on residuals and regression on functional values to produce long term predictions. In table 3 we present the results of this procedure for the case of a secondary network performing regression on residuals. The errors of the long term predictions are averaged over the test set P of the next 35 years. We see that the errors of the primary networks are reduced by about 20%. The quality of these long term predictions is within the range of results presented by Weigend et. al. Using the regression on (predicted) functional values, as in Eq. 14, the results are improved by up to 15% as shown in Table 4.

6 Combined Neural Networks for Time Series Analysis 229 # f2 fj f5 f~ fll, f Table 3 Long term predictions into the future. fn denotes the average error of n time steps predictions over the P set. The unprimed errors are those of the primary networks. The primed errors correspond to the combined system which includes correction of the residuals by a linear perceptron. # f2 f' 2 f5 f' 5 f11 f , Table 4 Long term predictions into the future. The primed errors correspond to the combined system which includes correction of the residuals by a linear perceptron based on the original inputs. 6 THE COMPLEX NETWORK Since the corrections of the secondary network are much smaller than the characteristic weights of the primary network, the corrections can be regarded as resulting from a Taylor expansion of a complex network which include's the combined system. This can be simply implemented in the case of Eq. 14 which can be incorporated in the complex network as direct linear connections from the input layer to the output neuron, in addition to the non-linear hidden layer, i.e., d m tn = L:: Wihi + L viyn-i - (). (15) We train such a complex network on the same problem to see how it compares with the two-step approach of the combined networks described in the previous chapters. The results depend strongly on the training rates of the direct connections, as compared with the training rates of the primary connections (i.e. those of the primary network). When the direct connections are trained faster than the primary ones, the result is a network that resembles a linear perceptron, with non-linear

7 230 Ginzburg and Hom corrections. In this case, the assumption of the direct connections being small corrections to the primary ones no longer holds. The training error and prediction capability of such a network are worse than those of the primary network. On the other hand, when the primary connections are trained using a faster training rate, we expect the final network to be similar in nature to the combined system. Still, the quality of training and prediction of these solutions is not as good as the quality of the combined system, unless a big effort is made to find the correct rates. Typical results of the various systems are presented in Table 5. type of network primary network learning rate of linear weights = 0.1 learning rate of linear weights = 0.02 combined system Table 5 Short term predictions of various networks. The learning rate of primary weights is The performance of the complex network can be better than that of the primary network by itself, but it is surpassed by the achievements of the combined system. 7 DISCUSSION It is well known that increasing the complexity of a network is not the guaranteed solution to better performance (Geman et. al. 1992). In this paper we propose an alternative which increases very little the number of free parameters, and focuses on the residual errors one wants to eliminate. Still one may raise the question whether this cannot be achieved in one complex network. It can, provided we are allowed to use different updating rates for different connections. In the extreme limit in which one rate supersedes by far the other one, this is equivalent to a disjoint architecture of a combined two-step system. This emphasizes the point that a solution of a feedforward network to any given task depends on the architecture of the network as well as on its training procedure. The secondary network which we have used was linear, hence it defined a simple regression of the residual on a series of residuals or a series of function values. In both cases the minimum which the network looks for is unique. In the case in which the residual is expressed as a regression on function values, the problem can be recast in a complex architecture. However, the combined procedure guarantees that the linear weights will be small, i.e. we look for a small linear correction to the prediction of the primary network. If one trains all weights of the complex network at the same rate this condition is not met, hence the worse results. We advocate therefore the use of the two-step procedure of the combined set of networks. We note that combined set of networks. We note that the secondary networks perform well on all possible tests: they reduce the training errors, they

8 Combined Neural Networks for Time Series Analysis 231 improve short term predictions and they do better on long term predictions as well. Since this approach is quite general and can be applied to any time-series forecasting problem, we believe it should be always tried as a correction procedure. REFERENCES Geman, S., Bienenstock, E., & Doursat, R., bias/variance dilemma. Neural Compo 4, Neural networks and the Ginzburg, I. & Horn, D Learning the rule of a time series. Int. Journal of Neural Systems 3, Nowlan, S. J. & Hinton, G. E Simplifying neural networks by soft weightsharing. Neural Compo 4, Tong, H., & Lim, K. S., Threshold autoregression, limit cycles and cyclical data. J. R. Stat. Soc. B 42, 245. Weigend, A. S., Huberman, B. A. & Rumelhart, D. E., Predicting the Future: A Connectionist Approach, Int. Journal of Neural Systems 1,

### CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER

93 CHAPTER 5 PREDICTIVE MODELING STUDIES TO DETERMINE THE CONVEYING VELOCITY OF PARTS ON VIBRATORY FEEDER 5.1 INTRODUCTION The development of an active trap based feeder for handling brakeliners was discussed

### Stock Prediction using Artificial Neural Networks

Stock Prediction using Artificial Neural Networks Abhishek Kar (Y8021), Dept. of Computer Science and Engineering, IIT Kanpur Abstract In this work we present an Artificial Neural Network approach to predict

### TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC

777 TRAINING A LIMITED-INTERCONNECT, SYNTHETIC NEURAL IC M.R. Walker. S. Haghighi. A. Afghan. and L.A. Akers Center for Solid State Electronics Research Arizona State University Tempe. AZ 85287-6206 mwalker@enuxha.eas.asu.edu

### INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

### Effect of Using Neural Networks in GA-Based School Timetabling

Effect of Using Neural Networks in GA-Based School Timetabling JANIS ZUTERS Department of Computer Science University of Latvia Raina bulv. 19, Riga, LV-1050 LATVIA janis.zuters@lu.lv Abstract: - The school

### Designing a neural network for forecasting financial time series

Designing a neural network for forecasting financial time series 29 février 2008 What a Neural Network is? Each neurone k is characterized by a transfer function f k : output k = f k ( i w ik x k ) From

### Lecture 6. Artificial Neural Networks

Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

### Forecasting of Economic Quantities using Fuzzy Autoregressive Model and Fuzzy Neural Network

Forecasting of Economic Quantities using Fuzzy Autoregressive Model and Fuzzy Neural Network Dušan Marček 1 Abstract Most models for the time series of stock prices have centered on autoregressive (AR)

### Forecasting of Indian Rupee (INR) / US Dollar (USD) Currency Exchange Rate Using Artificial Neural Network

Forecasting of Indian Rupee (INR) / US Dollar (USD) Currency Exchange Rate Using Artificial Neural Network Yusuf Perwej 1 and Asif Perwej 2 1 M.Tech, MCA, Department of Computer Science & Information System,

### 8. Linear least-squares

8. Linear least-squares EE13 (Fall 211-12) definition examples and applications solution of a least-squares problem, normal equations 8-1 Definition overdetermined linear equations if b range(a), cannot

### A Learning Algorithm For Neural Network Ensembles

A Learning Algorithm For Neural Network Ensembles H. D. Navone, P. M. Granitto, P. F. Verdes and H. A. Ceccatto Instituto de Física Rosario (CONICET-UNR) Blvd. 27 de Febrero 210 Bis, 2000 Rosario. República

### Neural Networks. Neural network is a network or circuit of neurons. Neurons can be. Biological neurons Artificial neurons

Neural Networks Neural network is a network or circuit of neurons Neurons can be Biological neurons Artificial neurons Biological neurons Building block of the brain Human brain contains over 10 billion

### Building MLP networks by construction

University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2000 Building MLP networks by construction Ah Chung Tsoi University of

### TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA

www.arpapress.com/volumes/vol9issue3/ijrras_9_3_16.pdf TIME SERIES FORECASTING WITH NEURAL NETWORK: A CASE STUDY OF STOCK PRICES OF INTERCONTINENTAL BANK NIGERIA 1 Akintola K.G., 2 Alese B.K. & 2 Thompson

### 6. Feed-forward mapping networks

6. Feed-forward mapping networks Fundamentals of Computational Neuroscience, T. P. Trappenberg, 2002. Lecture Notes on Brain and Computation Byoung-Tak Zhang Biointelligence Laboratory School of Computer

### Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승

Feed-Forward mapping networks KAIST 바이오및뇌공학과 정재승 How much energy do we need for brain functions? Information processing: Trade-off between energy consumption and wiring cost Trade-off between energy consumption

### SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS

UDC: 004.8 Original scientific paper SELECTING NEURAL NETWORK ARCHITECTURE FOR INVESTMENT PROFITABILITY PREDICTIONS Tonimir Kišasondi, Alen Lovren i University of Zagreb, Faculty of Organization and Informatics,

### Prediction Model for Crude Oil Price Using Artificial Neural Networks

Applied Mathematical Sciences, Vol. 8, 2014, no. 80, 3953-3965 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2014.43193 Prediction Model for Crude Oil Price Using Artificial Neural Networks

### A TUTORIAL. BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY

ARTIFICIAL NEURAL NETWORKS: A TUTORIAL BY: Negin Yousefpour PhD Student Civil Engineering Department TEXAS A&M UNIVERSITY Contents Introduction Origin Of Neural Network Biological Neural Networks ANN Overview

### Combining GLM and datamining techniques for modelling accident compensation data. Peter Mulquiney

Combining GLM and datamining techniques for modelling accident compensation data Peter Mulquiney Introduction Accident compensation data exhibit features which complicate loss reserving and premium rate

### Artificial Neural Network and Non-Linear Regression: A Comparative Study

International Journal of Scientific and Research Publications, Volume 2, Issue 12, December 2012 1 Artificial Neural Network and Non-Linear Regression: A Comparative Study Shraddha Srivastava 1, *, K.C.

### INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr.

INTELLIGENT ENERGY MANAGEMENT OF ELECTRICAL POWER SYSTEMS WITH DISTRIBUTED FEEDING ON THE BASIS OF FORECASTS OF DEMAND AND GENERATION Chr. Meisenbach M. Hable G. Winkler P. Meier Technology, Laboratory

### An Introduction to Neural Networks

An Introduction to Vincent Cheung Kevin Cannons Signal & Data Compression Laboratory Electrical & Computer Engineering University of Manitoba Winnipeg, Manitoba, Canada Advisor: Dr. W. Kinsner May 27,

COPYRIGHT NOTICE: David A. Kendrick, P. Ruben Mercado, and Hans M. Amman: Computational Economics is published by Princeton University Press and copyrighted, 2006, by Princeton University Press. All rights

### NEURAL NETWORKS A Comprehensive Foundation

NEURAL NETWORKS A Comprehensive Foundation Second Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Prentice Hall Prentice Hall Upper Saddle River; New Jersey 07458 Preface xii Acknowledgments

### Modelling and Big Data. Leslie Smith ITNPBD4, October 10 2015. Updated 9 October 2015

Modelling and Big Data Leslie Smith ITNPBD4, October 10 2015. Updated 9 October 2015 Big data and Models: content What is a model in this context (and why the context matters) Explicit models Mathematical

### Monotonicity Hints. Abstract

Monotonicity Hints Joseph Sill Computation and Neural Systems program California Institute of Technology email: joe@cs.caltech.edu Yaser S. Abu-Mostafa EE and CS Deptartments California Institute of Technology

### The CHIR Algorithm for Feed Forward. Networks with Binary Weights

516 Grossman The CHIR Algorithm for Feed Forward Networks with Binary Weights Tal Grossman Department of Electronics Weizmann Institute of Science Rehovot 76100 Israel ABSTRACT A new learning algorithm,

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### AUTOMATION OF ENERGY DEMAND FORECASTING. Sanzad Siddique, B.S.

AUTOMATION OF ENERGY DEMAND FORECASTING by Sanzad Siddique, B.S. A Thesis submitted to the Faculty of the Graduate School, Marquette University, in Partial Fulfillment of the Requirements for the Degree

### Machine Learning and Data Mining -

Machine Learning and Data Mining - Perceptron Neural Networks Nuno Cavalheiro Marques (nmm@di.fct.unl.pt) Spring Semester 2010/2011 MSc in Computer Science Multi Layer Perceptron Neurons and the Perceptron

### Machine Learning Big Data using Map Reduce

Machine Learning Big Data using Map Reduce By Michael Bowles, PhD Where Does Big Data Come From? -Web data (web logs, click histories) -e-commerce applications (purchase histories) -Retail purchase histories

### Linear Models for Classification

Linear Models for Classification Sumeet Agarwal, EEL709 (Most figures from Bishop, PRML) Approaches to classification Discriminant function: Directly assigns each data point x to a particular class Ci

### Data Mining mit der JMSL Numerical Library for Java Applications

Data Mining mit der JMSL Numerical Library for Java Applications Stefan Sineux 8. Java Forum Stuttgart 07.07.2005 Agenda Visual Numerics JMSL TM Numerical Library Neuronale Netze (Hintergrund) Demos Neuronale

International Journal of Computer Trends and Technology- volume4issue2-2013 ABSTRACT: Neural Network Design in Cloud Computing B.Rajkumar #1,T.Gopikiran #2,S.Satyanarayana *3 #1,#2Department of Computer

### ARTIFICIAL NEURAL NETWORKS FOR ADAPTIVE MANAGEMENT TRAFFIC LIGHT OBJECTS AT THE INTERSECTION

The 10 th International Conference RELIABILITY and STATISTICS in TRANSPORTATION and COMMUNICATION - 2010 Proceedings of the 10th International Conference Reliability and Statistics in Transportation and

### PMR5406 Redes Neurais e Lógica Fuzzy Aula 3 Multilayer Percetrons

PMR5406 Redes Neurais e Aula 3 Multilayer Percetrons Baseado em: Neural Networks, Simon Haykin, Prentice-Hall, 2 nd edition Slides do curso por Elena Marchiori, Vrie Unviersity Multilayer Perceptrons Architecture

### Chapter 4: Artificial Neural Networks

Chapter 4: Artificial Neural Networks CS 536: Machine Learning Littman (Wu, TA) Administration icml-03: instructional Conference on Machine Learning http://www.cs.rutgers.edu/~mlittman/courses/ml03/icml03/

### SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK

SUCCESSFUL PREDICTION OF HORSE RACING RESULTS USING A NEURAL NETWORK N M Allinson and D Merritt 1 Introduction This contribution has two main sections. The first discusses some aspects of multilayer perceptrons,

### Chapter 6. The stacking ensemble approach

82 This chapter proposes the stacking ensemble approach for combining different data mining classifiers to get better performance. Other combination techniques like voting, bagging etc are also described

### American International Journal of Research in Science, Technology, Engineering & Mathematics

American International Journal of Research in Science, Technology, Engineering & Mathematics Available online at http://www.iasir.net ISSN (Print): 2328-349, ISSN (Online): 2328-3580, ISSN (CD-ROM): 2328-3629

### Notes on Support Vector Machines

Notes on Support Vector Machines Fernando Mira da Silva Fernando.Silva@inesc.pt Neural Network Group I N E S C November 1998 Abstract This report describes an empirical study of Support Vector Machines

### Rate-based artificial neural networks and error backpropagation learning. Scott Murdison Machine learning journal club May 16, 2016

Rate-based artificial neural networks and error backpropagation learning Scott Murdison Machine learning journal club May 16, 2016 Murdison, Leclercq, Lefèvre and Blohm J Neurophys 2015 Neural networks???

### Predict Influencers in the Social Network

Predict Influencers in the Social Network Ruishan Liu, Yang Zhao and Liuyu Zhou Email: rliu2, yzhao2, lyzhou@stanford.edu Department of Electrical Engineering, Stanford University Abstract Given two persons

### Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

### Neural Networks and Back Propagation Algorithm

Neural Networks and Back Propagation Algorithm Mirza Cilimkovic Institute of Technology Blanchardstown Blanchardstown Road North Dublin 15 Ireland mirzac@gmail.com Abstract Neural Networks (NN) are important

### NEURAL networks [5] are universal approximators [6]. It

Proceedings of the 2013 Federated Conference on Computer Science and Information Systems pp. 183 190 An Investment Strategy for the Stock Exchange Using Neural Networks Antoni Wysocki and Maciej Ławryńczuk

### EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set

EM Clustering Approach for Multi-Dimensional Analysis of Big Data Set Amhmed A. Bhih School of Electrical and Electronic Engineering Princy Johnson School of Electrical and Electronic Engineering Martin

### NTC Project: S01-PH10 (formerly I01-P10) 1 Forecasting Women s Apparel Sales Using Mathematical Modeling

1 Forecasting Women s Apparel Sales Using Mathematical Modeling Celia Frank* 1, Balaji Vemulapalli 1, Les M. Sztandera 2, Amar Raheja 3 1 School of Textiles and Materials Technology 2 Computer Information

### Learning to Play 33 Games: Neural Networks as Bounded-Rational Players Technical Appendix

Learning to Play 33 Games: Neural Networks as Bounded-Rational Players Technical Appendix Daniel Sgroi 1 Daniel J. Zizzo 2 Department of Economics School of Economics, University of Warwick University

### The Combination Forecasting Model of Auto Sales Based on Seasonal Index and RBF Neural Network

, pp.67-76 http://dx.doi.org/10.14257/ijdta.2016.9.1.06 The Combination Forecasting Model of Auto Sales Based on Seasonal Index and RBF Neural Network Lihua Yang and Baolin Li* School of Economics and

### Neural network software tool development: exploring programming language options

INEB- PSI Technical Report 2006-1 Neural network software tool development: exploring programming language options Alexandra Oliveira aao@fe.up.pt Supervisor: Professor Joaquim Marques de Sá June 2006

### degrees of freedom and are able to adapt to the task they are supposed to do [Gupta].

1.3 Neural Networks 19 Neural Networks are large structured systems of equations. These systems have many degrees of freedom and are able to adapt to the task they are supposed to do [Gupta]. Two very

### Chapter 2 The Research on Fault Diagnosis of Building Electrical System Based on RBF Neural Network

Chapter 2 The Research on Fault Diagnosis of Building Electrical System Based on RBF Neural Network Qian Wu, Yahui Wang, Long Zhang and Li Shen Abstract Building electrical system fault diagnosis is the

### http://www.jstor.org This content downloaded on Tue, 19 Feb 2013 17:28:43 PM All use subject to JSTOR Terms and Conditions

A Significance Test for Time Series Analysis Author(s): W. Allen Wallis and Geoffrey H. Moore Reviewed work(s): Source: Journal of the American Statistical Association, Vol. 36, No. 215 (Sep., 1941), pp.

### Neural Networks and Support Vector Machines

INF5390 - Kunstig intelligens Neural Networks and Support Vector Machines Roar Fjellheim INF5390-13 Neural Networks and SVM 1 Outline Neural networks Perceptrons Neural networks Support vector machines

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### 3 An Illustrative Example

Objectives An Illustrative Example Objectives - Theory and Examples -2 Problem Statement -2 Perceptron - Two-Input Case -4 Pattern Recognition Example -5 Hamming Network -8 Feedforward Layer -8 Recurrent

### Neural Network Applications in Stock Market Predictions - A Methodology Analysis

Neural Network Applications in Stock Market Predictions - A Methodology Analysis Marijana Zekic, MS University of Josip Juraj Strossmayer in Osijek Faculty of Economics Osijek Gajev trg 7, 31000 Osijek

### Optimization of technical trading strategies and the profitability in security markets

Economics Letters 59 (1998) 249 254 Optimization of technical trading strategies and the profitability in security markets Ramazan Gençay 1, * University of Windsor, Department of Economics, 401 Sunset,

### Price Prediction of Share Market using Artificial Neural Network (ANN)

Prediction of Share Market using Artificial Neural Network (ANN) Zabir Haider Khan Department of CSE, SUST, Sylhet, Bangladesh Tasnim Sharmin Alin Department of CSE, SUST, Sylhet, Bangladesh Md. Akter

### MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete

### A Brief Study of the Nurse Scheduling Problem (NSP)

A Brief Study of the Nurse Scheduling Problem (NSP) Lizzy Augustine, Morgan Faer, Andreas Kavountzis, Reema Patel Submitted Tuesday December 15, 2009 0. Introduction and Background Our interest in the

Neural Network Add-in Version 1.5 Software User s Guide Contents Overview... 2 Getting Started... 2 Working with Datasets... 2 Open a Dataset... 3 Save a Dataset... 3 Data Pre-processing... 3 Lagging...

### South Carolina College- and Career-Ready (SCCCR) Probability and Statistics

South Carolina College- and Career-Ready (SCCCR) Probability and Statistics South Carolina College- and Career-Ready Mathematical Process Standards The South Carolina College- and Career-Ready (SCCCR)

### APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED VARIABLES TO PREDICT STOCK TREND DIRECTION

LJMS 2008, 2 Labuan e-journal of Muamalat and Society, Vol. 2, 2008, pp. 9-16 Labuan e-journal of Muamalat and Society APPLICATION OF ARTIFICIAL NEURAL NETWORKS USING HIJRI LUNAR TRANSACTION AS EXTRACTED

### Introduction to time series analysis

Introduction to time series analysis Margherita Gerolimetto November 3, 2010 1 What is a time series? A time series is a collection of observations ordered following a parameter that for us is time. Examples

### Chapter 12 Discovering New Knowledge Data Mining

Chapter 12 Discovering New Knowledge Data Mining Becerra-Fernandez, et al. -- Knowledge Management 1/e -- 2004 Prentice Hall Additional material 2007 Dekai Wu Chapter Objectives Introduce the student to

### Recurrent Neural Networks

Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time

### Threshold Autoregressive Models in Finance: A Comparative Approach

University of Wollongong Research Online Applied Statistics Education and Research Collaboration (ASEARC) - Conference Papers Faculty of Informatics 2011 Threshold Autoregressive Models in Finance: A Comparative

### Graduate Certificate in Systems Engineering

Graduate Certificate in Systems Engineering Systems Engineering is a multi-disciplinary field that aims at integrating the engineering and management functions in the development and creation of a product,

### 1. Classification problems

Neural and Evolutionary Computing. Lab 1: Classification problems Machine Learning test data repository Weka data mining platform Introduction Scilab 1. Classification problems The main aim of a classification

### Solving Nonlinear Equations Using Recurrent Neural Networks

Solving Nonlinear Equations Using Recurrent Neural Networks Karl Mathia and Richard Saeks, Ph.D. Accurate Automation Corporation 71 Shallowford Road Chattanooga, Tennessee 37421 Abstract A class of recurrent

### A Time Series ANN Approach for Weather Forecasting

A Time Series ANN Approach for Weather Forecasting Neeraj Kumar 1, Govind Kumar Jha 2 1 Associate Professor and Head Deptt. Of Computer Science,Nalanda College Of Engineering Chandi(Bihar) 2 Assistant

### Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham

Lecture 1: Introduction to Neural Networks Kevin Swingler / Bruce Graham kms@cs.stir.ac.uk 1 What are Neural Networks? Neural Networks are networks of neurons, for example, as found in real (i.e. biological)

### Power Prediction Analysis using Artificial Neural Network in MS Excel

Power Prediction Analysis using Artificial Neural Network in MS Excel NURHASHINMAH MAHAMAD, MUHAMAD KAMAL B. MOHAMMED AMIN Electronic System Engineering Department Malaysia Japan International Institute

### Comparing Neural Networks and ARMA Models in Artificial Stock Market

Comparing Neural Networks and ARMA Models in Artificial Stock Market Jiří Krtek Academy of Sciences of the Czech Republic, Institute of Information Theory and Automation. e-mail: krtek@karlin.mff.cuni.cz

### Joseph Twagilimana, University of Louisville, Louisville, KY

ST14 Comparing Time series, Generalized Linear Models and Artificial Neural Network Models for Transactional Data analysis Joseph Twagilimana, University of Louisville, Louisville, KY ABSTRACT The aim

### Applications of improved grey prediction model for power demand forecasting

Energy Conversion and Management 44 (2003) 2241 2249 www.elsevier.com/locate/enconman Applications of improved grey prediction model for power demand forecasting Che-Chiang Hsu a, *, Chia-Yon Chen b a

### MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL

MANAGING QUEUE STABILITY USING ART2 IN ACTIVE QUEUE MANAGEMENT FOR CONGESTION CONTROL G. Maria Priscilla 1 and C. P. Sumathi 2 1 S.N.R. Sons College (Autonomous), Coimbatore, India 2 SDNB Vaishnav College

### Tolerance of Radial Basis Functions against Stuck-At-Faults

Tolerance of Radial Basis Functions against Stuck-At-Faults Ralf Eickhoff 1 and Ulrich Rückert 1 Heinz Nixdorf Institute System and Circuit Technology University of Paderborn, Germany eickhoff,rueckert@hni.upb.de

### Role of Neural network in data mining

Role of Neural network in data mining Chitranjanjit kaur Associate Prof Guru Nanak College, Sukhchainana Phagwara,(GNDU) Punjab, India Pooja kapoor Associate Prof Swami Sarvanand Group Of Institutes Dinanagar(PTU)

### Example G Cost of construction of nuclear power plants

1 Example G Cost of construction of nuclear power plants Description of data Table G.1 gives data, reproduced by permission of the Rand Corporation, from a report (Mooz, 1978) on 32 light water reactor

### Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57. Application of Intelligent System for Water Treatment Plant Operation.

Iranian J Env Health Sci Eng, 2004, Vol.1, No.2, pp.51-57 Application of Intelligent System for Water Treatment Plant Operation *A Mirsepassi Dept. of Environmental Health Engineering, School of Public

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### More Data Mining with Weka

More Data Mining with Weka Class 5 Lesson 1 Simple neural networks Ian H. Witten Department of Computer Science University of Waikato New Zealand weka.waikato.ac.nz Lesson 5.1: Simple neural networks Class

### Introduction to Neural Networks : Revision Lectures

Introduction to Neural Networks : Revision Lectures John A. Bullinaria, 2004 1. Module Aims and Learning Outcomes 2. Biological and Artificial Neural Networks 3. Training Methods for Multi Layer Perceptrons

### 4. Simple regression. QBUS6840 Predictive Analytics. https://www.otexts.org/fpp/4

4. Simple regression QBUS6840 Predictive Analytics https://www.otexts.org/fpp/4 Outline The simple linear model Least squares estimation Forecasting with regression Non-linear functional forms Regression

### AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING

AN APPLICATION OF TIME SERIES ANALYSIS FOR WEATHER FORECASTING Abhishek Agrawal*, Vikas Kumar** 1,Ashish Pandey** 2,Imran Khan** 3 *(M. Tech Scholar, Department of Computer Science, Bhagwant University,

### Learning is a very general term denoting the way in which agents:

What is learning? Learning is a very general term denoting the way in which agents: Acquire and organize knowledge (by building, modifying and organizing internal representations of some external reality);

### Performance Evaluation of Artificial Neural. Networks for Spatial Data Analysis

Contemporary Engineering Sciences, Vol. 4, 2011, no. 4, 149-163 Performance Evaluation of Artificial Neural Networks for Spatial Data Analysis Akram A. Moustafa Department of Computer Science Al al-bayt

### APPLICATION OF INTELLIGENT METHODS IN COMMERCIAL WEBSITE MARKETING STRATEGIES DEVELOPMENT

ISSN 1392 124X INFORMATION TECHNOLOGY AND CONTROL, 2005, Vol.34, No.2 APPLICATION OF INTELLIGENT METHODS IN COMMERCIAL WEBSITE MARKETING STRATEGIES DEVELOPMENT Algirdas Noreika Department of Practical

### Penalized regression: Introduction

Penalized regression: Introduction Patrick Breheny August 30 Patrick Breheny BST 764: Applied Statistical Modeling 1/19 Maximum likelihood Much of 20th-century statistics dealt with maximum likelihood

### Section 14 Simple Linear Regression: Introduction to Least Squares Regression

Slide 1 Section 14 Simple Linear Regression: Introduction to Least Squares Regression There are several different measures of statistical association used for understanding the quantitative relationship

### Design call center management system of e-commerce based on BP neural network and multifractal

Available online www.jocpr.com Journal of Chemical and Pharmaceutical Research, 2014, 6(6):951-956 Research Article ISSN : 0975-7384 CODEN(USA) : JCPRC5 Design call center management system of e-commerce

### Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 6: Models and Patterns Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Models vs. Patterns Models A model is a high level, global description of a

### Bootstrapping Big Data

Bootstrapping Big Data Ariel Kleiner Ameet Talwalkar Purnamrita Sarkar Michael I. Jordan Computer Science Division University of California, Berkeley {akleiner, ameet, psarkar, jordan}@eecs.berkeley.edu

### Horse Racing Prediction Using Artificial Neural Networks

Horse Racing Prediction Using Artificial Neural Networks ELNAZ DAVOODI, ALI REZA KHANTEYMOORI Mathematics and Computer science Department Institute for Advanced Studies in Basic Sciences (IASBS) Gavazang,