Fuzzy Back-Propagation Network for PCB Sales Forecasting

Transcription

1 Fuzzy Back-Propagation Network for PCB Sales Forecasting Pei-Chann Chang, Yen-Wen Wang, and Chen-Hao Liu Department of Industrial Engineering and Management, Yuan-Ze University, 135 Yuan-Dong Rd., Taoyuan 32026, Taiwan, R.O.C. Abstract. Reliable prediction of sales can improve the quality of business strategy. In this research, fuzzy logic and artificial neural network are integrated into the fuzzy back-propagation network (FBPN) for printed circuit board industry. The fuzzy back propagation network is constructed to incorporate production-control expert judgments in enhancing the model s performance. Parameters chosen as inputs to the FBPN are no longer considered as of equal importance, but some sales managers and production control experts are requested to express their opinions about the importance of each input parameter in predicting the sales with linguistic terms, which can be converted into pre-specified fuzzy numbers, aggregated and corresponding input parameters when fed into the FBPN. The proposed system is evaluated through the real life data provided by a printed circuit board company. Model evaluation results for research indicate that the Fuzzy back-propagation outperforms the other three different forecasting models in MAPE. 1 Introduction Sales forecasting is a very general topic of research. When dealing with the problems of sales forecasting, many researchers have used hybrid artificial intelligent algorithms to forecast, and the most rewarding method is the application integrating artificial neural networks (ANNs) and fuzzy theory. This method is applied by incorporating the experience-based principal and logic-explanation capacity of fuzzy theory and the capacity of memory and error-allowance of ANNs, as well as self learning by numeral data. This research focuses on the sales forecasting of printed circuit board (PCB) and modifies the fuzzy back-propagation network system (FBPN) proposed by Chen[2003], to select variables with a better and more systematic way from expert experience, with the purpose of improving the forecasting accuracy and using this information to help managers make decisions. 2 Literature Review Although the traditional sales forecasting methods have been proved effective, they still have certain shortcomings. (Kuo, 2001, Tang, 2003, Luxhøj, 1996) As claimed L. Wang, K. Chen, and Y.S. Ong (Eds.): ICNC 2005, LNCS 3610, pp , Springer-Verlag Berlin Heidelberg 2005

2 Fuzzy Back-Propagation Network for PCB Sales Forecasting 365 by Kuo[1998], the new developed Artificial Intelligent (AI) models have more flexibilities and can be used to estimate the non-linear relationship, without the limits of traditional Time Series models. Therefore, more and more researchers tend to use AI forecasting models to deal with problem. Fuzzy theory has been broadly applied in forecasting. (Chen, 1999, Hwang, 1998, Huarng, 2001) Fuzzy theory is first combined with ANNs by Lin and Lee[1991], who incorporated the traditional fuzzy controller and ANNs to a network structure to proceed appropriate non-linear planning of unplanned control systems based on the relationship of input and output through the learning capacity of ANNs. Following them, many researchers started doing different relative research based on the combination of fuzzy theory and ANNs. Fuzzy theory combining with ANNs is applied in different areas and has positive performance. (Xue, 1994, Dash, 1995, Chen, 2003, Kuo, 1998) 3 Methodology There are three main stages in this research and the first stage is the variables selection stage. This stage is to select many possible variables, which may influence PCB product sales amount. In order to eliminate the unrelated variables, Stepwise Regression Analysis (SRA) and Fuzzy Delphi Method (FDM) were used to choose the key variables to be considered in the forecasting model. The second stage is the data preprocessing stage and Rescaled Range Analysis (R/S) was used to judge the effects of trend from serial observation values appearing as the time order. If the effect of trend is observed, Winter s method will be applied to remove the trend effect and reduce the forecast error. The third stage is the FBPN forecasting stage, which was developed to forecast the demand of PCB sales amount in this research and will be described in details in the following section. After being compared with other three forecasting models, the superior model will be recommended to the decision makers. The details of each stage will be described as follows: 3.1 Variable Selection Stage In this stage, fewer factors were considered in order to increase the efficiency of network learning. Many researchers have used several methods to select key factors in their forecast system. (Chang, 2000, Kaufmann, 1988, Lin, 2003 and Hsu, 2003) In this research, the following two methods were used to determine the main factors that would influence the PCB sales amount. 1. SRA (Stepwise Regression Analysis) Stepwise regression procedure determines the set of independent variables that most closely determine the dependent variable. This is accomplished by the repetition of a variable selection. At each of these steps, a single variable is either entered or removed from the model. For each step, simple regression is performed using the previously included independent variables and one of the excluded variables. Each of these regressions is subjected to an F-test. If the variable small F value, is greater

3 366 P.-C. Chang, Y.-W. Wang, and C.-H. Liu than a user defined threshold (0.05), it is added to the model. When the variable large F value, is smaller than a user defined threshold (0.1), it is removed from the model. This general procedure is easily applied to polynomials by using powers of the independent variable as pseudo-independent variables. The statistical software SPSS for Windows 10.0 was used for stepwise regression analysis in this research. Variables Selection Stage Data collection 1.Market demand domain 2.Macroeconomics domain 3.Industrial production domain SRA FDM Choose key variables Data Preprocessing Stage R/S analysis no Trend Effect? yes R/S analysis to find the effects of trend Winter s method Winter s method to remove the trend effect FBPN Forecasting Stage Fuzzy input Translate the input signal, which generated by experts, to the fuzzy term General BPN moodel Weights Adjustment Defuzzy Error measurement Defuzzification the output signals to the forecasting value End Training 2. FDM (Fuzzy Delphi Method) Fig. 1. Architecture of Three Main Stages in the Research The modified procedures of the proposed FDM for the search are listed as follows: Step 1: Collect all the possible factors that may affect the PCB product sales quantity. The domain experts select the important factors and give each a fuzzy number. This is the first questionnaire survey. Step 2: Formulate the questionnaire, which is a set of IF-THEN rules.

4 Fuzzy Back-Propagation Network for PCB Sales Forecasting 367 Step 3: Fuzzify the questionnaires that are returned by the domain experts and determine the following indices: (1). Pessimistic (Minimum) index l l 1 + l A + L + l An (1) n A 2 A = where l Ai means the pessimistic index of the i th expert and n is the number of the experts. (2). Optimistic (Maximum) index u A ua1 + ua2 + L+ u An = (2) n where u Ai means the pessimistic index of the i th expert. (3). Average (Most appropriate) index For each interval l Ai + u Ai, calculate the midpoint, m Ai = ( l Ai + u Ai )/ 2, 1/ n then find µ A = ( ma1 ma2 L man ). Step 4: R L Therefore, the fuzzy number A = ( µ, σ, σ ), which represents the mean, right width, and left width, respectively, for an asymmetric bell shaped function that can be determined through the above indices: R A µ A σ = l (3) 3 L u A µ A σ = (4) 3 Step 5: Formulate the next questionnaire with the above indices and conduct the survey. Step 6: Repeat 3 to 5. Use the following formulas as the stopping criteria to confirm that all experts have the consentaneous importance of each factor. = 1 α = 0 δ ( A, B) δ ( A[ α], B[ α]) dα (5) 1 1 L L U u = ( β2 β1) ( A[ α] B[ α] ) + ( A[ α] B[ α] ) dα 2 α = 0 where A and B are the fuzzy numbers, A [ ] and B [ ] denote the membership function of fuzzy numbers. The α -cut of the fuzzy number is defined as A[ α ] = x A[ x] α, x R for 0 < α 1. The distance between the two fuzzy { }

5 368 P.-C. Chang, Y.-W. Wang, and C.-H. Liu numbers is δ ( A, B). β 1 and β 2 are any given convenient values in order to surround both A [ α] = 0 and B [ α ] = Data Preprocessing Stage When the seasonal and trend variation is present in the time serious data, the accuracy of forecasting will be influenced. This stage will use R/S analysis to detect if there is this kind of effects of serious data. If the effects are observed, Winter s exponential smoothing will be used to take the effects of seasonality and trend into consideration. 1. R/S analysis (Rescaled Range Analysis) For eliminating possible trend influence, the rescaled range analysis, invented by Hurst (1965), is used to study records in time or a series of observations in different time. Hurst spent his lifetime studying the Nile and the problems related to water storage. The problem is to determine the design of an ideal reservoir on the basis of the given record of observed discharges from the lake. The detail process of R/S analysis will be omitted here. 2. Winters Exponential Smoothing In order to take the effects of seasonality and trend into consideration, Winter s exponential smoothing is used to preliminarily forecast the quantity of PCB production. According to this method, three components to the model are assumed: a permanent component, a trend, and a seasonal component. Each component is continuously updated using a smoothing constant applied to the most recent observation and the last estimate. Luxh[1996] and Mills[1990] compared Winter s Exponential Smoothing with other forecasting methods, like ARIMA, and all showed that the Winter s method had a superior performance. In this research we assume α = 0.1, β = 0. 1 and γ = Fuzzy Neural Network Forecasting Stage There are three main layers, input layer, hidden layer and output layer, and two training stages in our FBPN. In the feedforward stage, FBPN use the data on hand to forecast the PCB sales amount, and the forecasting error will be recalled to adjust the weights between layers in the backprooagation of error stage. The details will be described in the following: Step0. Initial weights between layers are randomly generated. Step1. While stopping condition is false, do step Step2. For each training pair, do step 3-8. Feedforward stage: Step3. Each input unit I j, which was generated by many experts, receives input signal ~ s ( i) x( i) and broadcasts this signal to all units in the hidden layer.

6 Fuzzy Back-Propagation Network for PCB Sales Forecasting 369 ~ i Where s ( ) is the fuzzy membership function, which is supported by the experts, and x is the normalized input signal. (i ) Step4. Sum the weighted input signals of each hidden unit. Step5. Apply the translation function to compute its output signals. Step6. Sum the weighted input signals of each output unit. Step7. Apply the translation function to compute its output signals. Step8. Defuzzify the output signals to the forecasting value, and compute its MAPE. Backpropagation of error stage: Step9.Compare the forecasted output with the actual sales amount and compute the error term between hidden layer and output layer. Next, calculate its weight correction term, (used to update connection weights latter). Finally, calculate its bias correction term, and update weights and biases. Step10. Compute its error information term for hidden nodes. Then, update the information term of each hidden node. Step11.Calculate its weight correction term between hidden layer and input layer. Then, calculate its bias correction term. Finally, update weights and biases. Generate the initial weights of the network Input layer node receives fuzzy input signal Feedforward stage Translate and compute the forecasting Defuzzification the output signals Compute the MAPE of forecasting Backpropagation of error stage Computes the error information term Update all weights and biases no Satisfy stopping condition? yes Stop Network Training Fig. 2. The detailed flow diagram of FBPN

7 370 P.-C. Chang, Y.-W. Wang, and C.-H. Liu The configuration of the FBPN is established as follows: number of neurons in the input layer: 5 number of neurons in the output layer: 1 single hidden layer number of neurons in the hidden layer: 5 network-learning rule: delta rule transformation function: sigmoid function learning rate: 0.1 learning times: Experimental Results The data in this research are from an electronic company in Taiwan from 1999/1 to 2003/12. Monthly sales amount is considered as an objective of the forecasting model. This research develops a FBPN for sales forecasting in PCB industries and we will compare this method with other traditional methods such as Grey Forecasting (GF), Multiple Regression Analysis (MRA) and Back-propagation network (BPN), etc. Mean average percentage error (MAPE) was applied as a standard performance measure for all four different models in this research. After the intensive experimental test, the MAPEs of four different models are 15.04%, 8.86%, 6.19% and 3.09% (as shown in table 1). Among that, the grey forecasting has the largest errors, and then MRA, BPN, and the least is FBPN. Table 1. Comparisons among Four Different Forecasting Models MAPE Improvement Rate GF 15.04% 74.95% MRA 8.86% 65.21% BPN 6.19% 50.08% FBPN 3.09% - As can be seen in fig 3, the GF has a significant up and down in the beginning and it also over estimate the data up to the end. Thus the overall MAPE is high. As for MRA, the tendency is formed and the up and down is minor compared with GF. The overall MAPE is around 0~20% and it is also a little higher. Traditional BPN model is in a stable situation and the overall MAPE is smaller than MRA and it is around 0~10%. The same situation exist for FBPN although in the beginning it has a larger error however it converge quickly and the overall MAPE is still around 0~10%. Especially, it performs very well in the end since it is very close to the real data. Therefore, the FBPN performs the best among others.

8 Fuzzy Back-Propagation Network for PCB Sales Forecasting 371 Fig. 3. The MAPE Values for Four Different Forecasting Models According to the various criteria, i.e., encompassing test, MAPE, and forecasting accuracy, the best model among these four different models is FBPN with a MAPE of 3.09% and accuracy of 97.61%. Therefore, we can claim that by combining the fuzzy theory and BPN the hybrid model can be applied in forecasting the sales of PCB industry and the result is very convincing and deserve further investigation in the future for applications in other areas. Although, the GF and MRA is very powerful when the data is very scarce and they claim that with only four data points and they can be applied to forecast the future result. However, after intensive experimental test, these two methods did not perform very well especially for those non-linear and highly dynamic data. As for the fuzzy Delphi back-propagation model since it can include the opinion from various experts in PCB sales and production department. It seems the assignment of different weight to the factor really improve the forecasting errors and perform much better than other models. 5 Conclusions The experimental results in section 4 demonstrated the effectiveness of the FBPN that is superior to other traditional approaches. The FBPN approach also provides another informing tool to the decision maker in PCB industries. In summary, this research has the following important contribution in the sales forecasting area and these contributions might be interested to other academic researchers and industrial practitioners: 1. Feature Selections: To filter out significant factors from a series of input variables, the FDM is superior to the SRA method. FDM will collect the opinion from various experts and assign different weights to these variables according to their experiences in this field.

9 372 P.-C. Chang, Y.-W. Wang, and C.-H. Liu Therefore, it is very easy to extract important factors from these various variables. In contrary, gradual regression analysis may come out with a combination of various variables which is mutually correlated. However, the effect of these selected variables may not significant enough to be included in the final inputs. The errors for input from fuzzy Delphi is 12.88%and errors from SRA is 13.87%. It is obvious to see that FDM is more effective for applications. 2. The effect of tendency: When take tendency effect into consideration, the overall errors are decreased. Tendency and seasonality are included in the time series data and these two factors will affect the accuracy of the forecasting method dramatically. This research applies the ÃWinters trend and seasonality exponential smoothing modelãto forecast the sales and then convert this data as an input to the BPN model. After the training procedure, the final errors, no matter it is from FDM or SRA, are decreased significantly. Errors from gradual regression analysis decreased from 13.84% to 7.15% and FDM from 12.88% down to 6.19%%. This shows the significance of including ÃWinters trend and seasonality exponential smoothing modelãin the model. 3. Comparisons of different forecasting models: This research applies three different performance measures, i.e., encompassing test, forecasting errors and accuracy of forecasting to compare the FBPN with other three methods, i.e., GF, MRA and BPN. The intensive experimental results show the following: 1. In encompassing test, FBPN and BPN models are superior to GF and MRA. 2. As for MAPE, FBPN has the smallest MAPE and it is only 3.09%. Therefore, FBPN model by combining FDM and BPN model is a very powerful and effective forecasting tool that can be further applied in other field of applications since expert s opinion can be incorporated into the model. References 1. Chang, P. T., Huang, L. C., Lin, H. J.: The Fuzzy Delphi Method via fuzzy statistics and membership function fitting and an application to the human resources, Fuzzy Sets and Systems, Vol. 112 (2000) Chen, T.: A Fuzzy Back Propagation Network for Output Time Prediction in a Wafer Fab, Applied Soft Computing Journal (2003) Chen, T., Wang, M. J. J.: Forecasting Methods using Fuzzy Concepts, Fuzzy Sets and Systems, Vol. 105 (1999) Dash, P. K., Liew, A. C., Rahman, S.: Peak load forecasting using a fuzzy neural network, Electric Power Systems Research, Vol. 32 (1995) Hsu, C. C., Chen, C. Y.: Applications of improved grey prediction model for power demand forecasting, Energy Conversion and Management, Vol. 44 (2003) Huarng, K.: Heuristic models of fuzzy time series for forecasting, Fuzzy Sets and Systems, Vol. 123 (2001) Hwang, J. R., Chen, S. M., Lee, C. H.: Handling forecasting problems using fuzzy time series, Fuzzy Sets and Systems, Vol. 100 (1998) Kuo, R. J.: A Sales Forecasting System Based on Fuzzy Neural Network with Initial Weights Generated by Genetic Algorithm, European Journal of Operational Research, Vol. 129 (2001)

10 Fuzzy Back-Propagation Network for PCB Sales Forecasting Kuo, R. J., Xue, K. C.: A decision support system for sales forecasting through fuzzy neural networks with asymmetric fuzzy weights, Decisions Support Systems, Vol. 24 (1998) Lin, C. T., Lee, C. S. G.: Neural-Network-Based Fuzzy Inference Systems, IEEE Trans. On Computer, Vol. 40, No. 12 (1991) Lin, C. T., Yang, S. Y.: Forecast of the output value of Taiwan s opto-electronics industry using the Grey forecasting model, Technological Forecasting & Social Change, Vol. 70 (2003) Luxh, J. T., Riis, J. O., Stensballe, B.: A hybrid econometric-neural network modeling approach for sales forecasting, The International Journal of Production Economics, Vol. 43 (1996) Mills, T. C.: Time series techniques for economists, Cambridge University Press (1990). 14. Tang, J. W.: Application of neural network in cause and effect model of time series data, Chung-Huwa University, Civil Engineering, Unpublished master thesis, Taiwan (2003) 15. Xue, K. Q.: An Intelligent Sales Forecasting System through Artificial Neural Networks and Fuzzy Neural Network, I-Shou University, Department of Management, Unpublished master thesis, Taiwan (1994).