Stock Market Value Prediction Using Neural Networks

Transcription

1 Stock Market Value Preiction Using Neural Networks Mahi Pakaman Naeini IT & Computer Engineering Department Islamic Aza University Paran Branch Hamireza Taremian Engineering Department Islamic Aza University Tehran East Branch Homa Baraaran Hashemi School of Electrical an Computer Engineering University of Tehran Abstract Neural networks, as an intelligent ata mining metho, have been use in many ifferent challenging pattern recognition problems such as stock market preiction. However, there is no formal metho to etermine the optimal neural network for preiction purpose in the literatur. In this paper, two kins of neural networks, a fee forwar multi layer Perceptron (MLP) an an Elman recurrent network, are use to preict a company s stock value base on its stock share value history. The experimental results show that the application of MLP neural network is more promising in preicting stock value changes rather than Elman recurrent network an linear regression metho. However, base on the stanar measures that will be presente in the paper we fin that the Elman recurrent network an linear regression can preict the irection of the changes of the stock value better than the MLP. Keywors- Stock market preiction; Data mining; neural networks I. INTRODUCTION From the beginning of time it has been man s common goal to make his life easier. The prevailing notion in society is that wealth brings comfort an luxury, so it is not surprising that there has been so much work one on ways to preict the markets. Therefore ing stock or financial markets has been one of the biggest challenges to the AI community. Various technical, funamental, an statistical inicators have been propose an use with varying results. However, none of these techniques or combination of techniques has been successful enough. The objective of ing research has been largely beyon the capability of traitional AI research which has mainly focuse on eveloping intelligent systems that are suppose to emulate human intelligence. By its nature the stock market is mostly complex (non-linear) an volatile. With the evelopment of neural networks, researchers an investors are hoping that the market mysteries can be unravele. Artificial Neural networks inspire by human brain cells activity can learn the ata patterns an generalize their knowlege to recognize the future new patterns. Researches on neural networks show that Neural Networks have great capability in pattern recognition an machine learning problems such as classification an regression. These ays Neural Networks are consiere as a common Data Mining metho in ifferent fiels like economy, business, inustry, an science. [6] The application of neural networks in preiction problems is very promising ue to some of their special characteristics. First, traitional methos such as linear regression an logistic regression are moel base while Neural Networks are self-ajusting methos base on training ata, so they have the ability to solve the problem with a little knowlege about its moel an without constraining the preiction moel by aing any extra assumptions. Bsies, neural networks can fin the relationship between the input an output of the system even if this relationship might be very complicate because they are general function approximators. Consequently, neural networks are well applie to the problems in which extracting the relationships among ata is ly ifficult but on the other han there exists a large enough training ata sets. It shoul be mentione that, although sometimes the rules or patterns that we are looking for might not be easily foun or the ata coul be corrupte ue to the process or measurement noise of the system, it is still believe that the inuctive learning or ata riven methos are the best way to eal with worl preiction problems. Secon, Neural Networks have generalization ability meaning that after training they can recognize the new patterns even if they haven t been in training set. Since in most of the pattern recognition problems preicting future events (unseen ata) is base on previous ata (training set), the application of neural networks woul be very beneficial. Thir, neural networks have been claime to be general function approximators. It is prove that an MLP neural network can approximate any complex continuous function that enables us to learn any complicate relationship between the input an the output of the system. The iea of using neural networks for preicting problems was first expresse by Hu in 964 which was use for weather ing [8]. The absence of any learning metho for multi layer networks mae it impossible to apply these networks to complex preiction problems. But in 980s the back propagation algorithm was introuce for training an MLP neural network. Werbos use this technique to train a neural network in 988 an claime that neural networks /0/$26.00 c 200 IEEE 32

2 are better than regression methos an Box-Jenkins moel in preiction problems [5]. The research on neural network applications continue up to the point that all the winners of the preiction contest in Santafa institute ha use neural networks [4]. In the recent ecae so many researches have been one on neural networks to preict the stock market changes. One of the first efforts was by Kimmoto an his colleagues in which they use neural networks to preict the inex of Tokyo stock market [0]. Mizuno an his colleagues also use neural networks to preict the trae of stocks in Tokyo stock market. Their metho was able to preict with 63% precision [2]. By combining Neural Networks an genetic algorithms, Phau an his colleagues manage to preict the irection of Singapore stock market with 8% precision. In this paper we have suggeste a preictive moel base on MLP neural network for preicting stock market changes in Tehran Stock Exchange Corporation (TSEC). Using this moel, one can preict the next ay stock value of a company only base on its stock trae history an without any information of the current market. Our experiments show that the preiction error of this moel is aroun.5%. In the following we will briefly introuce the iea of MLP neural network in the secon section. The thir section presents the architecture of the propose preiction moel, ata preparation methos use in this research an the evaluation criteria use for the evaluation of ifferent moels. In the fourth section the experimental results of the simulations on a company s ata will be analyze using ifferent moels. Finally the fifth section conclues the papers escribing the future works of the stuy. II. NEURAL NETWORKS The iea of neural networks was first inspire by human beings nervous system which consists of a number of simple processing units calle neuron (figure ). Each neuron receives some signals from outsie or from other neurons an then by processing them in activation function prouces its output an sens it to other neurons. Each input impact is ifferent from other inputs. For example in figure two the impact of the i th neuron on j th neuron is shown with w ij, the weight of the connection between neuron i an j. inputs Input layer Hien layer outputs Output layer Figure : Architecture Figure of a. fee P forwar MLP Consequently the more is the weight w ij the stronger woul the connection be an vice versa. In this paper, we focus on fee forwar multi layer neural networks. These networks are mae of layers of neurons. The first layer is the layer connecte to the input ata. After that there coul be one or more mile layers calle hien layers. The last layer is the output layer which shows the results. In feeback networks in contrast with recurrent networks all the connections are towar the output layer. Figure one shows a three layer fee forwar Perceptron network. One of the learning methos in multi layer Perceptron Neural Networks is the error back propagation in which the network learns the pattern in ata set an justifies the weight of the connections in the inverse irection respect to the graient vector of Error function which is usually regularize Figure 2: Perceptron neuron s connections sum of square error. The back propagation metho picks a training vector from training ata set an moves it from the input layer towar the output layer. In the output layer the error is calculate an propagate backwar so the weight of the connections will be correcte. This will usually go on until the error reaches a pre efine value. It s prove that we can approximate any continuous function with a three layer feeback network with any precision. It shoul be sai that the learning spee will ramatically ecrease accoring to the increase of the number of neurons an layers of the networks. III. THE SUGGESTED NEURAL NETWORK In spite of all the features mentione for neural networks, builing a neural network for preiction is somehow complicate. In orer to have a satisfactory performance one must consier some crucial factors in esigning of such a preiction moel. One of the main factors is the network structure incluing number of layers, neurons, an the connections. Other factors to be consiere are the activation functions in each neuron, the training algorithm, ata normalization, selecting training an test set an also evaluation measurements. In the suggeste moel two neural networks, a multilayer Perceptorn fee-forwar an an Elman recurrent are use an the back propagation algorithm is use to train these networks. The inputs to the neural networks are the lowest, the highest an the average value in the previous ays. Other information available about the stock market is not use because our goal is to preict the value of the stock share only base on the stock value history. In other wors, the 200 International Conference on Computer Information Systems an Inustrial Management Applications (CISIM) 33

3 propose moel can be viewe as a time series preiction moel. This moel uses a three layer neural network in which the input layer has 3 neurons which get the lowest, the highest an the average stock value in the last ays. In the hien layer there are h neurons which are fully connecte to the input an output layers. There is one neuron in output layer which preicts the expecte stock value of the next ay of the stock market. A. Data Preparation In this paper the lowest, the highest an the average value of the stock market in the last ays are use to preict the next ay s market value. The stock market ata have been extracte from Tehran Stock Market website. In this metho in contrast with other methos the isorers in the market cause by social or political reasons are not omitte from the ata set because we want to preict the value base on the value history. The simulation ata was extracte in 2000 to In this perio of time 094 companies shares were trae in Tehran Stock Market. The ata use as input to the system are the lowest, the highest, an the average value in the last ays (= {, 2,, 0}). The preiction system preicts the next ay s value using the above ata. In neural networks applications the input ata is usually normalize into the range of [0, ] or [-,] accoring to the activation function of the neurons. So in this paper the value of the stock market is normalize into the range of [-, ] using the () an then the neural networks are traine an teste using the back propagation algorithm. 2 ( Max + Min ) = () Max Min B. Evaluation criteria In preiction problems general criteria like mean absolute eviation, mean absolute percentage error, mean square error, an root mean square error are calculate base on (2, 3, 4, 5). These criteria are preferre to be smaller since they inicate the preiction error of the system. In aition to the above criteria three other measures are use to compare stock value preiction methos. The correct tren measure shows the percentage of correct ssssss Correct Fo recast Tren = T (, ) T (, ) = 0 if ( MAD = (2) (3) MAPE = ( ) 2 MSE = (4) RMSE = ( ) 2 ( 5) preiction of the changes in n+ th ay relative to n th ay (6). When the preiction is completely ranom this number woul be aroun 0.5. As a result, in orer to have a reliable preiction metho this feature shoul be at least above 0.5. Although knowing the irection of the changes is an important factor for ecision making, we also nee to know the amount of the changes. There will be two other criteria to etermine the ratio of correct tren to the tren of stock changes (7) an the ratio of incorrect tren to the tren of stock changes (8). In the Ieal case, the preicte ratio of correct tren to the stock changes in (7) shoul be equal to one. In aition, if the quantity of this ratio is smaller (or greater) than one, it will inicate that the irection of the changes is preicte correctly while the amount of changes has been preicte less (or more). In the other han, when the irection of stock changes is preicte incorrectly, the quantity of the (8) is esire to be closer to one as much as possible which shows the preiction error is minimum in this case. IV. SIMULATION RESULTS In this section the preiction results of the two suggeste methos using multi layer Perceptron neural networks an Elman recurrent network are compare to linear regression metho results. The training algorithm in multi layer neural network is Levenberg-Marquart back propagation which can train any neural networking using ifferentiable activity functions. In this kin of error back propagation algorithm we use both the graient an the Jacobean of the performance measure (error function)of the training set respect to the connection weights, to justify the network weights [9, ]. otherwise ).( ) 0 (6) Correct Forecast Tren = Real Tren Incorrect Re Forecast Tren al Tren = + (7) (8) International Conference on Computer Information Systems an Inustrial Management Applications (CISIM)

4 Mean abstract eviation Mean square error 95 60,00 Mean abstract eviation Mean square error 50,00 40,00 30,00 20, , MLP ( hien layer) Elman ( hien layer) Linear regression 0 Figure 4. comparing mean square error (MSE) in Elman an MLP. Mean absolute percentage error Figure 2. comparing minimum abstract eviation (MAD) in Elman an MLP Mean absolute percentage Figure 3. comparing mean absolute percentage error (MAPE) in Elman an MLP On the other han, to train Elman network, the error back propagation with momentum an aaptive learning rate is use. This algorithm like Levenberg-Marquart has the capability to train any network using ifferentiable activity functions. The weights of the network in this algorithm are ajuste accoring to Graient ecent (with momentum) base on the (9) in which mc is the momentum, Xprev is the previous change in the network weights an lr is the learning rate. pref X = mc Xprev + lr. mc. (9) X In each epoch if the performance measure (Mean square error) is moving towar its goal value, the learning rate will increase (in this simulation lr_inc=.05). On the other han if the performance measure increases more than a threshol (max_perf_inc=.04), leaning rate will ecrease with the rate of lr_ec (in this simulation lr_ec=0.7) an the relate change which has increase the performance measure, will not be applie to the network weights. When one of the below happens the algorithm will stop. The training epochs reach its maximum (in this simulation 000 epochs). The performance measure reaches its goal. (MSE=0to-6) The graient of the performance measure gets uner a threshol. (0to-6) In the following the results of these two methos are going to be compare to the result of the linear regression metho. In figure 3, 4, an 5 it s clearly shown that the MLP Neural Network has less MSE, MAPE, an MSE comparing to Elman an linear regression though this metho cannot preict the irection of the changes as well as Elman an regression (figure 6). However the linear regression metho preicts the irection of the changes well (figure 6), the error in the preiction of the value is much more than multilayer Perceptron an Elman (figure 7). The Elman network can preict the irection of the changes better than multilayer Perceptron (figure 6) but suffers from greater error in preiction (figure 7). V. CONCLUSIONS In this paper we use neural networks moel to preict the value of stock share in the next ay using the previous ata about stock market value. For this purpose two ifferent well known types of neural networks were applie to the problem. The obtaine results show that for preicting the irection of changes of the values in the next ay none of these methos are better than simple linear regression moel. But the error of the preiction of the amount of value changes using MLP neural network is less than both Elman network an linear regression metho. In aition to this, when the fee forwar MLP neural network preicts the irection of the changes correctly, the amount of change is 200 International Conference on Computer Information Systems an Inustrial Management Applications (CISIM) 35

5 Correct irection change Figure 5. Maximum number of correct change irection in Elman an MLP. Ratio of correct irection change to irection change Correct irection change Ratio of correct irection change to irection change Figure 6. Ratio of correct of irection change to irection change in Elman an MLP. completely close to the one in comparison to the other two mentione methos. In future works of this stuy we are going to apply other recently propose regression methos such as Support Vector Regression moels which is newer in the fiel of machine learning researches an claime to have goo generalization ability ue to application of large margin concept. REFERENCES [] Chen An-Sing, Leung Mark, Daouk Hazem, Application of Neural Networks to an Emerging Financial Market: Forecasting an Traing the Taiwan Stock Inex, Computers & Operations Research, vol. 30, 2003, pp [2] G. Armano, M. Marchesi, an A. Murru, A Hybri Genetic-Neural Architecture for Stock Inexes Forecasting, Information Sciences, vol. 70, 2005, pp Ratio of incorrect froecast irection change to irection change Ratio of incorrect irection change to irection change Figure 7. Ratio of incorrect of irection change to irection change in Elman an MLP. [3] Qing Cao, Leggio Karyl, Marc Schnieerjans, A Comparison Between Fama an French s Moel an Artificial Neural Networks in Preicting the Chinese Stock Market, Computers & Operations Research, vol. 32, 2005, pp [4] Olson Dennis, Mossman Charles, Neural Network Forecasts of Canaian Stock Returns Using Accounting Ratios, International Journal of Forecasting, vol.9, 2003, pp [5] C.W.J. Granger an A.P. Anerson, An Introuction to Bilinear Time Series Moels, Vanenhoeck an Ruprecht, Gottingen, 978. [6] G. Grunitzky an L. Osburn, Forecasting S&P an Gol Futures Prices: An Application of Neural Networks, Journal of Futures Markets, vol. 3, No. 6, pp , September 993. [7] Zhang Guoqiang, Patuwo Ey, Hu Michael, Forecasting with Artificial Neural Networks: The State of the Art, International Journal of Forecasting, vol. 4, 998, pp [8] M.J.C. Hu, Application of the Aaline System to Weather Forecasting, Master Thesis, Technical Report 6775-, Stanfor Electronic Laboratories, Stanfor, CA, June 964. [9] Levenberg K., A Metho for the Solution of Certain Problems in Least Squares, Quart. Appl. Math., no. 2, 944, pp [0] T. Kimoto, K. Asakawa, M. Yoa, an M. Takeoka, "Stock market preiction system with moular neural network," Proceeings of the International Joint Conference on Neural Networks, 990, pp. -6. [] D. Marquart, An Algorithm for Least Squares Estimation of Nonlinear Parameters, SIAM Journal of Applie Mathematics, no., 963, pp [2] H. Mizuno, M. Kosaka, H. Yajima, an N. Komoa, Application of Neural Network to Technical Analysis of Stock Market Preiction, Stuies in Informatic an Control, vol.7, no.3, 998, pp.-20. [3] H. Tong an K.S. Lim, Threshol Autoregressive, Limit Cycles an Cyclical Data, Journal of the Royal Statistical Society Series, vol. B- 42, no. 3, 980, pp [4] A.S. Weigen an N.A. Gershenfel, Time Series Preiction: Forecasting the Future an Unerstaning the Past, Aison- Wesley, Reaing, MA, 993. [5] P.J. Werbos, Generalization of back propagation with application to a recurrent gas market moel, Neural Networks, vol., pp , 988. [6] M. Rafiul Hassan an Baikunth Nath, Stock Market Forecasting Using Hien Markov Moel: A New Approach, 5th International Conference on Intelligent Systems Design an Applications, International Conference on Computer Information Systems an Inustrial Management Applications (CISIM)