Artificial Neural Network Time Series Modeling for Revenue Forecasting

Transcription

1 Chiang Mai J. Sci. 2008; 35(3) 411 Chiang Mai J. Sci. 2008; 35(3) : Contributed Paper Artificial Neural Network Time Series Modeling for Revenue Forecasting Siti M. Shamsuddin*, Roselina Sallehuddin and Norfadzila M. Yusof Faculty of Computer Science and Information System, University Technology of Malaysia, Malaysia. *Author for correspondence; mariyam@fsksm.utm.my Received: 4 October 2006 Accepted: 22 May ABSTRACT The obective of this study is to investigate the effect of applying different number of input nodes, activation functions and pre-processing techniques on the performance of backpropagation (BP) network in time series revenue forecasting. In this study, several preprocessing techniques are presented to remove the non-stationary in the time series and their effect on artificial neural network (ANN) model learning and forecast performance are analyzed. Trial and error approach is used to find the sufficient number of input nodes as well as their corresponding number of hidden nodes which obtain using Kolmogorov theorem. This study compares the used of logarithmic function and new proposed ANN model which combines sigmoid function in hidden layer and logarithmic function in output layer, with the standard sigmoid function as the activation function in the nodes. A cross-validation experiment is employed to improve the generalization ability of ANN model. From the empirical findings, it shows that an ANN model which consists of small number of input nodes and smaller corresponding network structure produces accurate forecast result although it suffers from slow convergence. Sigmoid activation function decreases the complexity of ANN and generates fastest convergence and good forecast ability in most cases in this study. This study also shows that the forecasting performance of ANN model can considerably improve by selecting an appropriate pre-processing technique. Keywords : artificial neural network, forecasting, data-preprocessing, input nodes, activation function. 1. INTRODUCTION Knowing future better has attracted many people for thousands of years. The forecasting methods vary greatly and will depend on the data availability, the quality of models available, and the kinds of assumptions made, amongst other things. Generally speaking, forecasting is not an easy task and therefore it has attracted many researchers to explore it. Artificial neural network (ANN) has found increasing consideration in forecasting theory, leading to successful applications in various forecasting domains including economic, business financial [1-3] and many more. ANN can learn from examples (pass data), recognize a hidden pattern in historical observations and use them to forecast future values. In addition to that, they are able to deal with incomplete information

2 412 Chiang Mai J. Sci. 2008; 35(3) or noisy data and can be very effective especially in situations where it is not possible to define the rules or steps that lead to the solution of a problem. Despite of many satisfactory characteristics [4], of ANNs, building an ANN model for a particular forecasting problem is a nontrivial task. Several authors such as [2, 4-8] have provided an insight on issues in developing ANN model for forecasting. These modeling issues must be considered carefully because it may affect the performance of ANNs. Based on their studies, some of the discussed modeling issues in constructing ANN forecasting model are the selection of network architecture, learning parameters and data pre-processing techniques applies to the time series data. The study examines the effectiveness of BP network model as an alternative tool for forecasting. This study provides a practical introductory guide in the design of an ANN for forecasting time series data. We use the time series corresponding to the revenue collection in Royal Malaysian Customs Department to illustrate this process. This research attempts to study the behavior of ANN models when several of its parameters are altered. This study examines the effect of network parameters through trial and error approach by varying network structures based on the number of input nodes, activation functions and data pre-processing techniques in designing of BP network forecasting model. The relevancy of applying difference non-linear activation functions in hidden and output layers of ANN model is also examined in this study. 2. ARTIFICIAL NEURAL NETWORK (ANN) ANN technology is an advanced computerized system of information processing. In many discussions they are credited with operating in the same way as the human brain. This is clearly an overstatement. Although loosely inspired by the structure of the brain, current ANN technology is far from being able to simulate even a simple brain function. Nevertheless, ANNs do offer a computational approach that is potentially very effective in solving problems which do not lend themselves to analysis by conventional means. The key difference between ANNs and other problem solving methods is that ANNs in a real sense learn by example and modify their structure rather than having to be programmed with specific, preconceived rules. In other words, ANNs can be seen as a nonparametric statistical procedure that uses the observed data to estimate the unknown function. The basic unit of any ANN is the neuron or node (processor). Each node is able to sum many inputs x1, x2,, x3 whether these inputs are from a database or from other nodes, with each input modified by an adustable connection weight (Figure 1). The sum of these weighted inputs is added to an adustable threshold for the node and then passed through a modifying (activation) function that determines the final output. Figure 1. Working of node. 2.1 Feed-Forward Network Figure 2. depicts the feed-forward network where computation proceeds along the forward direction only. The output obtained from the output neurons constitutes

3 Chiang Mai J. Sci. 2008; 35(3) 413 the network output. Each layer will have its own index variable: k for output neurons, for hidden, and i for input neurons. In a feedforward network, the input vector, x, is propagated through a weight layer, W i, (2.1) (2.2) horizon. Thus, a one-step-ahead forecast can be performed through a ANN with one output unit and a k-step-ahead forecast can be performed with k output units. Figure 3. illustrates an MLP forecasting model in which two past terms in the series used to forecast the value of the series in a one-step-ahead forecast. (2.3) (2.4) where f is an output function (possibly the same as f above in hidden layer). Figure 3. MLP network for one-step-ahead forecasting based on two lagged terms. Figure 2. A feed-forward network. 2.2 MULTILAYER PERCEPTION (MLP) MLP network is also the type of ANN most commonly used in time series forecasting [6]. The univarite time series modeling is normally carried out through this ANN by using a determined number of the time series lagged terms as inputs and forecasts as outputs. The number of input nodes determines the number of prior time points to be used in each forecast, while the number of output nodes determines the forecast 2.3 Backpropagation (BP) Algorithm The goal of any training algorithm is to find the logical relationship from the given input/output or to reduce the error between the real results and the results obtained with the ANN as small as possible [9]. Any network structure can be trained with BP when desired output patterns exist and each function that has been used to calculate the actual output patterns is differentiable. The BP algorithm is the most computationally straightforward algorithm for training the MLP. As with conventional gradient descent (or ascent), BP works by, for each modifiable weight, calculating the gradient of error (or cost) function with respect to the weight and then adusting it accordingly. The most frequently used error function to be minimized in BP at the output layer is the mean sum squared error, E defined below Error, (2.5) where t k is the target output and o k is the actual

4 414 Chiang Mai J. Sci. 2008; 35(3) outputs of the neuron. For easier understanding, we have summarized the consecutive steps involved in BP algorithm in the following table. Table 1. The summarized steps in BP algorithm. Step 1 Obtain a set of training patterns. Step 2 Set up ANN model that consist of number of input neurons, hidden neurons, and output neurons. Step 3 Set learning rate (h) and momentum rate (a). Step 4 Initialize all connections (W i and W k ) and bias weights (q k and q ) to random values. Step 5 Set the minimum error, E min. Step 6 Start training by applying input patterns one at a time and propagate through the layers then calculate total error. Step 7 Backpropagate error through output and hidden layer and adapt weights, W k and q k. Step 8 Backpropagate error through hidden and input layer and adapt weights W i and q. Step 9 Check if Error <E min. If not, repeat steps 6-9 If yes, stop training. 2.4 Designing an ANN Forecasting Model The eight-steps design methodology presented in Table 2. is the design steps involves in designing ANN for forecasting obtained from [6]. [6] also outlined the common parameters involved in designing a BP network. The large numbers of parameters that must be selected to develop an ANN forecasting model have meant that the design process still involves much trial and error. For example, [10] in his Table 2. Eight steps in designing an ANN forecasting model. Step 1 Variable selection Step 2 Data collection Step 3 Data pre-processing Step 4 Training, testing and validation set Step 5 ANN paradigms number of hidden layers number of hidden nodes number of output nodes activation functions Step 6 Evaluation criteria Step 7 ANN training number of training iterations learning rate and momentum Step 8 Implementation study have implemented trial and error process to find the optimal ANN model for house price prediction. There are various heuristic rules and common practices for selecting the parameters [11], but the selection process remains as an art rather than a science, and varies from problem to problem. 3. EXPERIMENTAL DESIGN This section describes the data set used and explains how the experiment and analysis are accomplished in order to investigate the efficiency and convergence rate of BP learning and the forecasting performance of the approached ANN models by using different number of input nodes and corresponding number of hidden nodes, activation functions and data pre-processing techniques. This study implements a trial and error approaches in order to find sufficient network paradigm (numbers of input node and corresponding hidden node; and activation function) and data data pre-processing technique. In this study the real data of monthly collection of revenue obtains from the Royal

5 Chiang Mai J. Sci. 2008; 35(3) 415 Malaysian Customs Department over the period from January 1990 until December 2004 is used to form as input to the ANN model. The obective or output of ANN model is to forecast following one-monthahead revenue collection. Different data pre-processing techniques are presented to the time series data. Accordingly, we decided to evaluate the following data pre-processing methods, raw values, logs, logs + first differences, logs + seasonal differences, logs + first differences + seasonal differences, logs + one step return and logs + one step return + seasonal differences. Next, different data matrices are generated for the 7 pre-processed data on the basis of the number of lagged terms (from 1 to 11) which would act as input and the number of step-ahead forecast (1) that would act as the ANN output. MLP network with a single hidden layer was chosen for the ANN model since it has been successfully used in forecasting application by many researchers such as [4, 7, 12, 13]. In the experiments, numbers of input node are varied from 1 to 11 nodes, numbers of hidden node are determined using Kolmogorov theorem and 1 output node which represents the forecast horizon were used in ANN model. Because of the potential over-fitting problem, the model adequacy is often checked with a holdout sample that is not used in the model building process. In fact, to build an appropriate ANN model, the available data is often divided into three portions. One approach than can be used to avoid overfitting is cross-validation technique. [14, 15] have outlined the cross-validation operational procedure as below: 1) Divide the available dataset into training set, validation set and test set. 2) Select architecture and training parameters. 3) Evaluate the model using the validation set. 4) Repeat steps 2 through 4 using different architecture and training parameters. 5) Select the best network and train it using data from the training set and validation set. 6) Assess this final model using the test set. Figure 4. Three-way data splits. In this study the dataset is divided into two periods: the first period runs from January 1990 to December 2002 while the second period runs from January 2003 to December 2004 (forecasting target). The former period was again split into training set (January 1990 to December 2000) and validation set (January 2001 to December 2002) (forecasting target) while the second period is represented as test set. For the feed-forward ANN using BP algorithm, a sigmoid function and logarithmic are selected as the activation functions to generate relationship between input and output nodes. The selected activation functions are given below. The sigmoid activation function takes the form: (3.1) The logarithmic activation function takes the form: (3.2)

6 416 Chiang Mai J. Sci. 2008; 35(3) The most common error function used in ANN is mean squared error function (MSE). This error is used to decide when to stop the ANN training process. To examine how close of the forecast to the actual value, the commonly used measures of forecast error, the mean absolute percentage error (MAPE) and the root mean of the squared error (RMSE) are employed: 1 t k o k MAPE = x 100 n t (3.4) k k 1 2 RMSE = ( t k ok ) n (3.5) k where n is equal to total number of observation. RMSE and MAPE produce have produced contradicting outcomes in this study. Thus, to verify the forecasting performance of ANN model we also calculate the R- squared in order to avoid the contradicting problem. R-squared = 1 k k k ( t k t) k ( t o ) (3.6) The obective of the training is to find the set of weights between the neurons that determine the global minimum of the error function. Standard BP learning algorithm has divided into two phases: feed-forward and backward. In forward propagation ation step, the employment of different activation functions will generate different formulation for output generalization in hidden and output layer. While in backward propagation step, applying different activation function will cause slight changes in formulation of error signal in hidden and output layer during weight adaptation. 1) Forward Propagation Step Output generalization: 1) Between input (i) and hidden () layer: ( net ) net = W o + θ (3.7) o = f i ( net ) i o = f net = ln net + 1 net < 0 (3.11) o k = f net k = ln net 1 net k 0 (3.15) o k = f net k = ln net k + 1 net k < 0 (3.16) (3.8) where o is the output in hidden layer. The output in hidden layer, o determines by using sigmoid function is shown in equation 3.9. o = f 1 = net 1+ e (3.9) and logarithmic function is shown in equation 3.10 and ( net ) = ln ( net 1) ( ) ( ) o = f = + net 0 (3.10) 2) Between hidden () and output (k) layer: net k = k W o k = f ( net k ) k o + θ k (3.12) (3.13) where o k is the output in hidden layer. The output in hidden layer, o k, determines by using sigmoid function is as in equation o = f ( net ) 1 = 1+ e o k = k net k (3.14) and logarithmic function is as in equation 3.15 and ( k ) ln( k ) ( ) ( ) k + 2) Backward Propagation Step Weights adaptation: 1) Between hidden () and output (k) layer: Weight changes: ( t + 1) = ηδ o + αδw ( t) Δ W k 1 = (3.17) k k k

7 Chiang Mai J. Sci. 2008; 35(3) 417 New weight: where (3.18), error signal for output neuron is by using sigmoid function and (3.19) (3.20) by using logarithmic function. represents the learning rate and α is the momentum rate. 2) Between input (i) and hidden () layer: Weight changes: New weight: where (3.23) (3.21) (3.22), error signal for hidden neuron is using sigmoid function, and (3.24) using logarithmic function. represents the learning rate and is the momentum rate. Choosing the right value of the learning rate or momentum was difficult [16]. If the value is to too small, a very slow convergence will occur. On the other hand, if the value is too large, the optimization process will diverge [16]. According to [17] a learning rate between 0.05 and 0.5 provides good results in most practical cases while the momentum factor usually has a value close to 1, for example 0.9. In this study, a learning rate of 0.25 and momentum rate of 0.8 are constantly used in all the ANN models during experiments. Another parameter that also needs to be considered is maximum error. The maximum error that will be used is set to In this study, the training process of BP is terminated after 50,000 iterations (epochs), or until the error reach the maximum error within the specified maximum iteration. Overall, a total of 231 ANN models are obtained on the basis of combining the following parameters: 7 datasets (raw, values, logs, logs + first differences, logs + seasonal differences, logs + first + seasonal differences, logs + one step return and logs + one step return + seasonal differences) and 33 ANN paradigms (specifically, 3 types of ANN model (shown in Table 3.) and the number of input (1 to 11)). A total of 11 different network architectures on the basis of vary input nodes (from 1 to 11) are examined with different types of ANN model and preprocessed datasets in this study. The architectures for ANN model are shown in Table 4. Table 3. Features of three types of ANN model. Activation function Activation function Model Type in hidden layer in output layer Sigmoid Sigmoid Sigmoid Logarithmic Logarithmic Logarithmic SigLog Sigmoid Logarithmic 3.1 Cross-Validation Experimental Setup Eleven sets of cross-validation experiment as shown in Table 5. are conducted within 7 different datasets. Based on Figure 5. in each cross-validation experiment, three types of ANN model with same architecture are trained and tested using training set and validation set. The training set is used in ANN learning to find the optimal weights with the BP rule.

8 418 Chiang Mai J. Sci. 2008; 35(3) Table 4. Architectures of ANN model. Number of Number of Number of Model input nodes hidden output nodes in input nodes in in output layer hidden layer layer Table 5. Cross-validation experiments set up. CV Model Model Type CV1 1 Sigmoid Logarithmic SigLog CV2 2 Sigmoid Logarithmic SigLog CV3 3 Sigmoid Logarithmic SigLog CV4 4 Sigmoid Logarithmic SigLog CV5 5 Sigmoid Logarithmic SigLog CV6 6 Sigmoid Logarithmic SigLog CV7 7 Sigmoid Logarithmic SigLog CV8 8 Sigmoid Logarithmic SigLog CV9 9 Sigmoid Logarithmic SigLog CV10 10 Sigmoid Logarithmic SigLog CV11 11 Sigmoid Logarithmic SigLog The validation set is used to select the optimal network parameters for the BP algorithm. Training and validation occurred simultaneously, and the two sets of data are used for exploration of parameters value of ANN configuration. The purpose of the experiment is to search for proper type of ANN model in different architectures using different datasets. The forecast results from each model are evaluated and compared using two forecasting accuracy measures. We select the ANN model which produces the minimum forecast error in validation set to commit an out-of-sample test. Overall, we run 77 set of cross-validation experiments in this study. At the end of the experiments, a total of 77 ANN models with optimal model type are identified to run an out-of-sample test using test set. The forecast result of 77 ANN models are then evaluate and compare. Figure 6. depicts example of cross-validation experiment using ANN model with 2 input nodes. Figure 5. Cross-validation experiment work flow.

9 Chiang Mai J. Sci. 2008; 35(3) 419 Figure 6. Example of cross-validation experiment. 4. EXPERIMENTAL RESULT A simple comparison of the performance of the methods is given by simply comparing the RMSE and MAPE of the forecasts. The model which produces the smallest RMSEs and MAPEs for both the validation data (for model selection) and test data (for true forecasting evaluation) are desirable. The RMSE and MAPE can only be compared between models whose errors are measure in the same units. Thus, each model is evaluated with the training set, validation set and test set on the post-processed data, i.e. converted the forecasted values back to original form. In the experiment, the validation set is required to ascertain the proper type of ANN model. Each type of ANN model is trained and tested with same input data or dataset and network architecture. To determine the final error rate of winner models from validation set, the training and validation sets are used in network training for predicting the revenue of the last 24 months ( ), which was the test set. Before we search for an appropriate dataset or data pre-processing technique, we evaluate the forecasting performances and generalization abilities of 77 estimators on the post-processed data using test set. We chose 7 best ANN models with most accurate forecast results as displayed in Table 6. Forecast results show that sigmoid model outperformed other network models in most datasets except for data after pre-processed with logs + first + seasonal differences. The logs + first + seasonal differences dataset provide better forecast result with the proposed siglog model within network which comprised of 2 inputs, 5 hidden and 1 output nodes. Overall, the forecasting results show that ANN model which consists of small input nodes and their smaller corresponding structures had better forecast results. Table 6. Best ANN models for revenue. series Dataset Model Input Hidden Type Raw values Sigmoid 2 5 Logs Sigmoid 3 7 Logs + first differences Sigmoid 1 3 Logs + seasonal differences Sigmoid 2 5 Log + first + seasonal differences SigLog 2 5 Logs + one step return Sigmoid 1 3 Logs + one step return + seasonal Sigmoid 2 5 differences The final task is to make a selection of an appropriate dataset or data pre-processing for the revenue time series. In our finding, the dataset which provides lowest forecast error in test set is claimed to be the proper data pre-processing technique to produce quality forecast result in time series revenue forecasting. From our empirical studies, logs transformation dataset is seen to produce the lowest

10 420 Chiang Mai J. Sci. 2008; 35(3) learning error rates in training data. According to [6], logarithmic transformations convert multiplicative or ratio relationships to addictive which are believe to simplify and improve network training. However, this form of transformation does not produce accurate forecast result with validation data as shown in Table 7. Table 7. Simulation result for ANN models. Dataset Sample RMSE MAPE Raw values Training % Validation % Test % Training % Logs Validation % Logs + first differences Logs + seasonal differences Test % Training % Validation % Test % Training % Validation % Test % Logs + first + Training % seasonal Validation % differences Test % Logs + one step return Training % Validation % Test % Logs + one step Training % return + seasonal Validation % differences Test % According to [18], an over-fitted model can produce a small RMSE for the training data set, but tends to produce a large RMSE and MAPE for the validation data set. On the other hand, a large RMSE and MAPE for the training data set and a small RMSE and MAPE for the validation data set may indicate that the forecasts for the validation happen to fit the data well. Therefore, [18] recommended using the number of validations which makes smallest the larger of the RMSE for the training and validation data sets. Concerning the impact of the preprocessing techniques, the two performance measures in Table 7. somehow shows contradicting outcomes in test set: the lowest MAPE is obtained by the ANN model using the raw values, where the lowest RMSE is associated with the logs. In order to verify the answer, we have calculated the R-squared of each out-of sample forecast. Table 8. shows the out-ofsample forecast evaluation results for different pre-processing techniques. Again the R- squared of the ANN model using logs transformation is higher than the R-squared of other pre-processing techniques, and the RMSE is lower than other pre-processing techniques. According to [19] the closer a model R-squared value to one, the greater the ability of that model to predict a trend. Therefore, it can be concluded that the ANN model using logs transformation is relatively superior model for revenue forecasting. The results from our finding indicate that selecting an appropriate preprocessing is highly beneficial for improving the ANN learning ability and forecasting quality in this study. Table 8. R-squared of raw values and logs dataset. Model Dataset Model RMSE Type R- squared Raw values 2 Sigmoid Logs 3 Sigmoid DISCUSSION An attempt has been made in this study to investigate the predictive power of neural network by adopting trial and error method to tune the number of input nodes (n) and activation functions. Different network architectures which consist of 2n + 1 number

11 Chiang Mai J. Sci. 2008; 35(3) 421 of hidden nodes are used on the network and their predictive performances are then examined. We also present different preprocessing techniques for removing nonstationary, and evaluate their properties in producing one-step ahead forecasting using ANN model. The first issue that we want to highlight is about ANN learning performance. A number of researchers such [20-22] has subected that BP algorithm has suffers from a number of shortcomings, including a slow learning rate. One of the identified reason is the used of sigmoid function in hidden and output nodes [20, 21]. This study uses logarithmic activation function proposed by [20] in order to accelerate the BP learning speed. A comparison between sigmoid, logarithmic and siglog model shows the accelerated convergence within sigmoid model. Sigmoid model has generated the lowest RMSEs and MAPEs in training data within 99% of the cross-validation experiment. Logarithmic model fails to improve the learning speed and found to produce slowest learning when compared to sigmoid and new proposed siglog model in most of the crossvalidation experiments. The problem occurred because of the too high value of the derivative at the later stage of learning that caused inappropriately large weight changes causing oscillation [22]. Siglog model which applied sigmoid activation function in hidden nodes and logarithmic activation function in output node performs as the second best learning model in this study. In term of error, sigmoid model shows a stable error convergence over 50,000 iterations with less experienced being trapped in local and global minima. Siglog model has shows considerable acceleration of learning speed in comparison with sigmoid model and logarithmic model in 1% of the experiments. Siglog model improves the convergence problem in logarithmic model by decreasing the value of the derivative at the later stage of learning that lead to oscillation problem. Generalization is the ability of the network model to respond correctly to data that not used to train it. Logarithmic model is shown to obtain better generalization when compared to sigmoid and siglog models. This model has produced low RMSEs and MAPEs in validation set and high RMSEs and MAPEs in training set in maorities of the cross-validation experiments. Experimental results show that sigmoid model and siglog model produce higher RMSEs and MAPEs in validation set than RMSEs and MAPEs in training set in most cases. Our finding also shows that siglog model can improve the generalization ability of sigmoid model since the RMSE and MAPE generate in training set and validation are much closer to each other. From the experimental results, its show that sigmoid, logarithmic and siglog models obtain good generalization in network with small number of input nodes. According to [23], too many inputs make the network hard to use and allow the greater chance of overfitting. [6] have provided some suggestion to deal with over-fitting problem that are decreasing the number of hidden nodes or increasing the size of training set. Insufficient training data can cause over-fitting when the network is unnecessarily complex [24]. Finally, the final errors of 77 optimal ANN models obtained from 77 crossvalidation experiments are calculated. The true forecasting evaluation is made based on forest results with test set. Experimental results show that siglog model produces accurate forecast result from 8% of the cross-validation experiments while logarithmic model yields as good predictor in 17% of the crossvalidation experiments. Sigmoid model has

12 422 Chiang Mai J. Sci. 2008; 35(3) been chosen as network that gives lowest RMSEs and MAPEs in validation set from 75% of the cross-validation experiments. ANN model with the best forecast result is chosen from each dataset. Based on experimental results, it indicates that 86% of the best forecast results obtain by sigmoid model using small size network. 14% of the results show that the accurate forecast results obtain from siglog model with small size network. Logarithmic model is seen unable to produce accurate forecast result in test set. In term of network size, the forecast results illustrate that the small size network produces good generalization and better forecast results. Overall, it can be concluded that the small network is more appropriate to be used in the revenue forecasting. Small network decreases the complexity of the network in order to process the input from each network node. The drawback of small size network is it slows convergence. The small network seems to take more time and numbers of iterations to learn and understand the data patterns due to the less information are provided to the network. Large input nodes mean more information is provided to the network and less time is taken by the network to learn the data patterns. From our finding large network produces worst forecast result and bad generalization. According to [23], handling of large amount of data increases the chance of mistake and allows the greater chance of over-fitting. The last issue that we want to discuss in this section is the problem of contradicting outcomes between the two forecasting accuracy measurements used in this study that are the RMSE and MAPE. This contradicting result may cause confusion in order to select which data pre-processing has more accurate forecast result. Either than calculating the R- squared value, [25] have suggested one approach to solve this problem. The approach is to compare the true trend line, obtained with a 5 degree polynomial interpolation of the actual values, with the estimated trend line, based on the prediction values. Contradicting between two forecast performances also exist in test set results between raw values dataset and logs dataset (refer Table 7). The lowest RMSE is obtained in logs dataset while the lowest MAPE is associated with raw values dataset. The graphs in Figures 7(a) and 7(b) help us to understand the reason. The forecasting based on the raw values on Figure 7(a) miss to forecast the new upward trend especially from August 2004 to October On Figure 7(b), the neural forecasts, based on the log transformation, have a higher variance and they swing very much around the trend line, producing high MAPE, but nevertheless they can capture the new trend. Figure 7(a) ANN out-of-sample one-step forecasts using raw data. Figure 7(b) ANN out-of-sample one-step forecasts using data after pre-processing with log transformation.

13 Chiang Mai J. Sci. 2008; 35(3) 423 Moreover the high variance of the estimator resembles the high variance of target data the resulting RMSE is quite low [25]. This is proved by plotting the target data of raw values dataset and target data of logs dataset as shown in Figure 8. The graph in Figure 8. shows that the variance of target data in logs dataset is much higher than target data in raw values. Hence the RMSE for logs dataset is low. Figure 8. Plotting of target data in raw values and logs dataset. According to [26] none of the error measures is ideal for aiding decision making. However, the RMSE describes the magnitude of the error in terms that would be relatively more useful to decision makers. Consistent with this, [27] found that practitioners preferred the RMSE to all other error measures. It is usually best to report the RMSE rather than MSE, because the RMSE is measured in the same units as the data, rather than in squared units, and is representative of the size of a typical error. The MAPE is less appealing because percentages do not have obvious implications for decision making. The MAPE should not be used if large errors are expected because it is biased in favor of low forecasts. The MAPE can only be computed with respect to data that are guaranteed to be strictly positive. The results from our finding indicate that logarithmic transformation is a proper data pre-processing technique in this study. It is shown that this pre-processed technique is highly beneficial to forecast an existing trend and seasonal variation exist in the time series. De-trended, de-seasonalized or de-trended and de-seasonalized data series that are believed could improve an ANN forecast quality does not provides a promising result in this study. This problem maybe due to the unsuitable network architecture and learning parameters used during the experiment. Further experiments could be done with different architecture and learning parameters to identify an appropriate ANN model. 6. CONTRIBUTION Data series that has been pre-processed with logs transformation is able to accelerate the convergence rate and produce better forecast result. Sigmoid model generates better learning and forecast capability when compared to logarithmic and siglog model in most of the cross-validation experiments. Logarithmic model has the slowest learning in all the experiments. Logarithmic model obtains better generalization ability when compared to sigmoid model and siglog model. The proposed siglog model is able to improve the convergence problem in logarithmic model and the generalization problem occurs in sigmoid model. Logarithmic model learns better in network with small number of input nodes while sigmoid and siglog models have better convergence in network with large number of input nodes. Network with small number of input nodes is seen to have good generalization and better forecast performance in sigmoid, logarithmic and siglog models. Comparing the true trend line with 5 degrees polynomial interpolation of the actual values, with the estimated trend line based on the forecast value can help overcome the

14 424 Chiang Mai J. Sci. 2008; 35(3) problem of contradicting values between RMSE and MAPE to verify the best ANN model forecast performance. 7. CONCLUSION Based on the obtained results, we can conclude that employing an appropriate preprocessing technique is highly beneficial to simplify the ANN training and improve the forecasting quality in this study. ANN model with small number of input nodes and their smaller corresponding network structure produces better forecast result and decreases the chance of over-fitting. Sigmoid model decreases the complexity of ANN and generates fastest convergence and good forecast ability in most cases in this study. Illustrating the trend line of actual and forecast values may helps practitioner and researcher to validate the accuracy of forecast result more precisely. 8. SUGGESTION FOR FUTURE WORKS The following are some suggestions that could lead to the improvement of the results that have been obtained and some possible points that could lead to future research. Applying different sample size of training set, validation set and testing set. Appropriate data sampling may help in order to avoid over-fitting problem that occurs in this study. In order to improve the forecast performance of ANN model, more trial and error experiments need to be done on other network parameters that are not studied in this research for example varying the number of hidden nodes to produce better generalization and varying the momentum term and the learning rate to accelerate the BP learning process. Rather than using trial and error method, we can employ genetic algorithm in order to find appropriate parameters for ANN model. Comparison of the two methods can also be made. Combining an early stopping approach with cross-validation experiment in order to reduce over-fitting similar to that used in [12]. During the training procedure, the validation set error normally decreased during the initial phase of training, as did the training set error. However, when the network began to overfit the training data, the error on the validation set typically began to rise. When the validation error increased for a specified number of iterations, the training was stopped, and the weights and biases at the minimum of the validation error were returned. REFERENCES [1] Crone S.F., Kauseh H., and Pre β mar D., Prediction of the CATS benchmark using a business forecasting approach to multilayer perceptron modelling. In: Wunsch, D. et al. (eds.): Proceedings of the International Joint Conference on Neural Networks, 2004; IJCNN 04, Budapest, Hungary, IEEE, New York. [2] Lam M., Neural network techniques for financial performance prediction: integrating fundamental and technical analysis. Decision Support Systems 37, 2004; [3] Yao J.T., Towards a better forecasting model for economic indices. In: Proceedings of the 6th Joint Conference on Information Science, 2002; [4] Zhang G., Patuwo B.E., and Micheal Y.H., Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14, 1998; [5] Bacha H., and Meyer M., A neural network architecture for load forecasting, IEEE, 1992; [6] Kaastra I., and Boyd M., Designing a neural network for forecasting financial

15 Chiang Mai J. Sci. 2008; 35(3) 425 and economic time series. Neurocomputing 10 (3), 1996; [7] Tan C.N.W., and Wittig G.E., A study of the parameters of a backpropagation stock price prediction model. The First New Zealand International Two Stream Conference on Artificial Neural Networks and Expert Systems (ANNES), University of Otago, Dunedin, New Zealand. November 24-26, IEEE Computer Society Press, ISBN , 1993a. [8] Xu L., and Chen W.J., Short-term load forecasting techniques using ANN. In: Proceeeding of the 2001 IEEE International Conference of Control Applications, 2001; [9] Zhang Y., Prediction of traffic fluctuation in telephone networks with neural networks. IEEE, 1992; [10] Limsombunchai V., Gan C., and Lee M., House price prediction: Hedonic price model vs. Artificial neural network. American Journal of Applied Sciences 1(3), 2004; [11] Walczak S., and Cerpa N., Heuristic principles for the design of artificial neural networks. Information and Software Technology 41 (2), 1999; [12] Palmer A., Mantano J.J., and Sese A., Designing an artificial neural network for forecasting tourism time series. Science Direct, [13] Thawornwong S., and Enke D., The adaptive selection of financial and economic variables for use with artificial neural networks. Neurocomputing 56, 2004; [14] Kumar S., Neural Networks: A Classroom Approach, International Edition. McGraw-Hill Publishing Company Limited. ISBN , [15]Osuna R.G., Lecture 13: Validation, Intelligent Sensor System, Wright State University; edu/prism/lectures/iss/iss_l13.pdf. [16] Kong J.H.L., and Martin G.P.M.D., A backpropagation neural network for sales forecasting. IEEE, 1995; [17] Rumelhart D.E., Hinton G.E., and Willams R.J., Learning internal representation by error propagation. Parallel Distributed Processing l, MIT Press, 1986; [18] Kaitani Y., Hiple K.W., and Mcleod A.I., Forecasting Nonlinear Time Series with Feed-Forward Neural Networks: A Case Study of Canadian Lynx Data. John Wiley & Sons, Ltd. J. Forecast. 2005; 24: [19] Kolobe L., What is R-Squared? Connexions. Latest updated on February 22, latest/. [20] Bilski J., The backpropagation learning with logarithmic transfer function. In: Proceeding of The fifth Conference on Neural Network and Soft Computing, Zakopane, Poland, 6-10 June 2000; [21] Fnaiech F., Abid S., Ellala N., and Cheriet, M., A comparative study of fast neural network learning algorithms. IEEE International Conference on Systems, Man and Cybernetics IEEE SMC, [22] Kamruzzaman J., and Aziz S.M., A note on activation function in multilayer feedforward learning. In: Proceeding of International Joint Conference on Neural Network 2002, IEEE, [23] Yu S.W., Forecasting and arbitrage of the nikkei stock index futures: An application of backpropagation networks, Asia- Pasific Finalcial Market 1999, 6(4),

16 426 Chiang Mai J. Sci. 2008; 35(3) [24] Nguyen H.H., and Chan C.W., A comparison of data preprocessing strategies for neural network modeling of oil production prediction. In: Proceeding of the Third IEEE International Conference and Cognitive Informatics (ICCI 04). IEEE Computer Science [25] Virili F., and Freisleben B., Nonstationary and data preprocessing for neural network predictions of an economic time series. IEEE, 2000; [26] Amstrong J.S., and Collopy F., Error measures for generalizing about forecasting methods: empirical comparisons, International Journal of Forecasting, 1992; 8, [27] Carbone R., and Armstrong J.S., Evaluation of extrapolative forecasting methods: Results of a survey of academicians and practitioners, Journal of Forecasting, 1982; 1, Appendix : Example of Revenue Dataset Month Revenue Collection (RM) 1990-Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month Revenue Collection (RM) 1992-Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec