Supply Chain Data Mining

Supply Chain Data Mining Applied Advanced Non-Linear Data Mining Techniques for Forecasting Distorted Demand of the Extended Supply Chain Réal Carbonneau Email: contact@realcarbonneau.com For: Dr. Kevin Laframboise John Molson School of Business Concordia University, Montréal, Québec, H3G 1M8 Fall 23

Abstract The objective of this research is to study the feasibility of forecasting the distorted demand signals in the extended supply chain with advanced non-linear machine learning data-mining techniques. The source of data for forecasting comes both from a simulation and real data. The first source is a simulation of an extended supply chain that is developed to generate distorted demand for the manufacturer. In this simulation, the demand distortion is a result of demand signal processing by all downstream parties. The second source of data is the foundries (primary metal manufacturing) monthly industry average demand. The forecasting techniques researched, in order of forecasting accuracy on the foundries testing data set, are; Recurrent Neural Networks (RNN), Least Squares Support Vector Machine (LS-SVM), Neural Networks (NN), Multiple Linear Regression (MLR), Average, Trend (Simple Linear Regression) and Naïve forecasts. Specifically in the area of forecasting distorted demand in the extended supply chain, advanced non-linear data mining techniques are shown to be superior to other forecasting techniques. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 1

Table of Contents ABSTRACT...1 TABLE OF CONTENTS...2 INTRODUCTION...4 PURPOSE...4 PROBLEM...4 Demand Distortion...4 Non Cooperation...4 SCOPE...5 CONTEXT...6 THEORY...6 RESEARCH MODEL...7 METHODOLOGY...8 SUPPLY CHAIN SIMULATION...9 MACRO VIEW OF THE SUPPLY CHAIN...9 FINAL CUSTOMER DEMAND...9 MICRO VIEW OF A PARTNER...9 DEMAND FORECASTING...1 PURCHASE ORDER...1 ORDER PROCESSING...11 SIMULATION RESULTS...11 Overview of Orders...11 Effects on deliveries, inventory and backlogs...11 DATA MINING EXPERIMENTS...13 DATA PREPARATION...13 Simulation Data...13 Foundries Monthly Sales Data...15 RESULTS OVERVIEW...16 NEURAL NETWORKS...16 Simulation Distorted Demand Forecasting...16 Foundries Demand Forecasting...17 RECURRENT NEURAL NETWORKS...18 Simulation Distorted Demand Forecasting...18 Foundries Demand Forecasting...19 SUPPORT VECTOR MACHINE...19 Simulation Distorted Demand Forecasting...2 Foundries Demand Forecasting...21 MULTIPLE LINEAR REGRESSION...21 Simulation Distorted Demand Forecasting...22 Foundries Demand Forecasting...22 TREND FORECAST...23 Simulation Distorted Demand Forecasting...23 Foundries Demand Forecasting...23 CONCLUSION...25 FUTURE RESEARCH...25 REFERENCES...26 APPENDIX A ADDITIONAL SIMULATION COMPONENTS...28 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 2

MACRO VIEW OF THE SUPPLY CHAIN...28 ORDER PROCESSING...28 LEAST SQUARES POLYNOMIAL FIT...28 OUTPUT DATA...29 APPENDIX B ADDITIONAL SIMULATION RESULTS...3 WHOLESALER...3 DISTRIBUTOR...3 MANUFACTURER...3 APPENDIX C ADDITIONAL FORECASTS TIME SERIES...31 NAÏVE FORECAST...31 Simulation Distorted Demand Forecasting...31 Foundries Demand Forecasting...31 AVERAGE FORECAST...32 Simulation Distorted Demand Forecasting...32 Foundries Demand Forecasting...33 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 3

Introduction Purpose Problem The objective of this research is to study the feasibility of forecasting the distorted demand signals in the extended supply chain with advanced non-linear machine learning data-mining techniques. There will be two sources of distorted demand for the forecasting experiments. The first source will be a simulation of the extended supply chain and the second will be the estimated value of new orders received by Canadian Foundries (Statistics Canada, 23). Demand Distortion This research is focused on forecasting the demand at the end of the extended supply chain because as the signal travels up the extended supply chain, it gets increasingly distorted. Even though the final customer s demand may have a pattern, these increasing distortions cause the manufacturer s demand to appear to change by a random amount. Forecasting the manufacturer s demand becomes very difficult because there does not appear to be any rules which describe the change. Non Cooperation While there is much focus on collaborative extended supply chain systems for reducing and eliminating demand distortions, there are many reasons why collaboration may be possible. Below are some issues that must be addressed to permit successful supply chain collaboration (Premkumar, 2): Alignment of business interests Long-term relationship management Reluctance to share information Complexity of large-scale supply chain management Competence of personnel supporting supply chain management Performance measurement and incentive systems to support supply chain management In many companies these issues have not been addressed yet and it will take much time before they have been addressed successfully to permit working extended supply chain collaboration. There is also the issue of power that has been researched. In the extended supply chain there power regimes and power sub-regimes that can prevent extended supply chain optimization (Watson 21, Cox 21). Hence, even if it is technically feasible to integrate systems and chare information, organizationally it may not be feasible because it may cause major upheavals in the power structure. (Premkumar, 2) Even in cases where there are collaborative system in place, there can still be inaccurate information that is introduced into the system. This was seen in a study of the telecom industry demand chain where some partners were double forecasting and ration gaming (Heikkila, 22), this occurred despite the fact that there was a collaborative system in place and there was a push for the correct use of this system. The above reasons will impede extended supply chain collaboration, or even worst, cause inaccurate results in information sharing systems. Also, the current realities of REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 4

businesses today are that most extended supply chains are not collaborative all the way upstream to the manufacturer. In these cases, accurate forecasting of distorted demand is important. Scope Simulations have shown that genetic algorithm based artificial agents working as a team to minimize the total extended supply chain cost can achieve lower costs than human players. They even minimize costs lower than the 1-1 policy (Kimbrough 2, Kimbrough 21) without information sharing. But the objective of each member of a real supply chain is not to minimize the total supply chain s cost, but to maximize it s own profits. Because of these objectives, supply chain collaboration and global cost minimization is difficult to achieve. Since it is not always possible to have the members of a supply chain work as a team, it is important to study the feasibility of forecasting the distorted demand information in the extended supply chain without information from other partners. Therefore minimizing the entire extended supply chain s costs are not within the scope of this research. The focus is on more accurate forecasting to help reduce the supply chains cost. A large part of the purpose of supply chain collaboration is to increase the accuracy of forecasts (Raghunathan 1999), so we know that better forecasts are very important. But the benefits of information sharing for the purpose of improving forecast accuracy will not be examined in this research. It can easily be added in future reasearch by providing more information from down the supply chain to the data mining tools. The data mining tools considered will be Neural Networks (NN), Support Vector Machines (SVM) and Recurrent Neural Networks which are non-linear machine learning data mining tools. They will be compared with linear regression for the same model to determine if non-linearity has added to the model s accuracy. Additional Trend, Naïve and Average forecast will also be used as comparison. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 5

Context Theory The source of the demand distortion (Forrester 1961) in the extended supply chain simulation will be demand signal processing (Lee 1997) by all members in the supply chain. In this simulation, the demand signal processing is a trend estimation. As the final customer s demand signal moves up the supply chain, it gets increasingly distorted because of demand signal processing. This occurs even if the demand signal processing function is identical in all parties of the extend supply chain. This distortion is explained by the chaos theory, where small variation result in large seemingly random behavior (Kullback 1959, Donahue 1997, Julia 1918) even in the presence of identical functions. Basic time series analysis (Box 197) will be used in this research. Because the manufacturer s demand is considered to be a chaotic time-series, Recurrent Neural Networks (RNN) will be one of the machine learning tools used. Recurrent Neural Networks do back-propagation of error through time which permits them to learn patterns through an arbitrary depth in the time series. Neural Networks and Recurrent Neural Networks are frequently used to predict time series (Lawrence 1996, Herbrich 2, Landt 1997, Dorffner 1996). A more recent learning algorithm that has been developed from statistical learning theory is Support Vector Machines (Vapnik 1995, Vapnik 1997). They have a very strong mathematical foundation and their objective is structural risk minimization as opposed to empirical risk minimization used in regression and Neural Networks. Some time series analysis have previously been done with Support Vector Machines (Rüping 23, Mukherjee 1997). REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 6

$ $ Research Model Considering certain constraints within a supply chain that do not permit collaborative information exchange, demand still needs to be forecasted with as much accuracy as possible. Below is a model of the extended supply chain with increasing demand distortion that includes a collaboration barrier. Using only past manufacturer s order, we can forecast future demand far enough in advance to compensate for our lead time, and the forecasts will be more accurate than other less sophisticated techniques. This increased forecasting accuracy will result in lower costs because of reduced inventory and higher revenues because of increased customer satisfaction that will result from an increase in on-time deliveries. Below is a model of the Distorted Demand Forecasting. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 7

Methodology To determine if it is feasible to use non-linear data mining techniques for forecasting distorted demand of the extended supply chain, a simplified but representative simulation is first used. This simulation includes four parties that make up the supply chain which delivers a product to the final customer. Based on this extended supply chain model, as indicated previously, the source of the demand distortion will be demand signal processing. This demand signal processing is represented by a real-time simple linear regression that uses the past demand to determine the trend to forecast future demand. This results in enough demand signal processing to cause significant distortion at the end of the extended supply chain. With this distorted demand signal, that we know has been generated from a pattern, we can attempt forecasting using various machine learning techniques. The distorted demand signal is divided into two parts, a training part that is used by the various machine learning techniques and the testing part that is used for model validation. Using the testing set, the demand will be forecasted and compared with the actual demand. The Mean Average Error (MAE) is used to determine how close the forecasting model is to the actual demand. The following data mining and forecasting techniques are compared for both the simulation distorted demand and the actual foundries demand. Neural Networks (NN) Recurrent Neural Networks (RNN) Support Vector Machine (SVM) Multiple Linear Regression (MLR) Naïve Forecast Average Forecast Trend Forecast REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 8

Supply Chain Simulation Experiment The Extended Supply Chain simulation is developed in MATLAB Simulink (MathWorks 2). All parts of the supply chain simulation model that are not presented below can bee seen in Appendix A Macro View of the Supply Chain The first part of the simulation is the extended supply chain overview (see Appendix A). This defines the macro structure of the supply chain, which includes the one day delay for the order to move to the next party, the two day delay for the goods to be delivered, as well as all the display points required for monitoring the simulation and the data collection points that collect data for the machine learning tools that forecast the manufacturer s distorted demand. Final Customer Demand The final customer (also called end customer) demand is generated using a sin wave pattern that varies between 8 and 12 with additional white noise. The pattern repeats itself approximately every month. Micro View of a Partner Each partner in the extended supply chain has the same structure and behavior, although this is a simplification of reality, it still provides demand distortion as can be observed in real supply chains. A partner is divided into the following functions; demand forecasting, purchase order calculation, and order processing which includes goods receipt and goods delivery. The mechanism for demand signal processing is visually clear here since we can diagrammatically see the demand signal that has been processed by the demand forecasting function feed into the calculation purchase order function. The reason that the Delivery, Inventory and Backlog signals are part of the partner functions interface is to facilitate simulation monitoring and data extraction. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 9

Demand Forecasting Demand forecasting is the business function that is the source of the demand signal distortion. It is a simple linear regression of the past 1 days of demand which is used to estimate a trend that is the source of the estimation of demand in 3 days. This permits the ordering today of the estimated quantity required in 3 days. The combined effect of small random variations in end-customer demand and demand signal processing distorts the initial demand which has a clear pattern, into chaotic demand once it reaches the manufacturer. The effects of the simple linear regression are a trend estimation which is used to forecast the demand in 3 days. Purchase Order Once the demand signal has been processed by the demand forecasting function, it is then used to calculate the purchase order that will be submitted to the next partner. We assume that the ordering party keeps track of the current total un-received quantities so that he does not keep ordering more for the items that are in backlog. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 1

Order Processing Simulation Results Deliveries received from the supplier plus the quantity that is currently in inventory is used to fulfill backlog order, then regular orders and the remaining quantity goes into the inventory (see Appendix A). An initial simulation run of 2 days is used to review the results of the extended supply chain. The final customer demand history was shown previously. Overview of Orders Below we can see the order history for all partners and including the Production Orders of the manufacturer, assuming that his Production Order also have a 1 day processing delay because of non-integrated intra-company information systems or other reasons. From here and comparing with the initial final customer demand, we can clearly see the simple demand signal processing, which is a trend estimation, cause significant distortions in the manufacturer s orders and production orders. Effects on deliveries, inventory and backlogs Looking at the distorted demand, it is obvious that the whole extended supply chain will suffer from these information distortions, as we can see below in the deliveries, inventories and backlogs for the retailer. All other details about the deliveries, inventories and backlogs for all the other extended supply chain parties can be found in Appendix B. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 11

REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 12

Data Mining Experiments Now that we have generated distorted demand base on demand signal processing in the extended supply chain, we can use modern non-linear data mining techniques to build a model that will forecast this distorted demand. We also use the same data mining techniques to build forecasting models for the real foundries demand. Lastly we construct the remaining comparative forecasting models. Here is the list of models we will build: Neural Networks (NN) Recurrent Neural Networks (RNN) Support Vector Machine (SVM) Multiple Linear Regression (MLR) Naïve Forecast Average Forecast Trend Forecast Data Preparation The Naïve forecast simply uses the last observed change (or percentage change). The average forecast uses the average change (or percentage change) calculated on the training set, this should be around. The trend estimation is a Simple Linear Regression to estimate the trend in the change (or percentage change) for the past 1 days. The trend estimation can only start after the 1 th observation, so in the final data sets, this is after the 5 th observation. The forecasting time series for the Naïve and Trend forecast will not be presented in this section since they are trivial, they can be examined in Appendix C. The data is first prepared before running the machine learning tools. The exact same training and testing data set are used for all of the forecasting models. Simulation Data The daily manufacturer orders obtained in the previous simulation are first used as a source of data for building forecasting models. The first 3 days of this data can be seen in the chart below. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 13

Simulation Manufacturer Daily Orders 3 25 2 Quantity 15 Orders 1 5 1 16 31 46 61 76 91 16 121 136 151 166 181 196 211 226 241 256 271 286 Day The independent variables or input variables for all the models is the change in demand for each of the past four days plus today and the dependent variable or output variable is the total change over the next 3 days. The exceptions are the Naïve, Average and Trend forecast. Percentage change could not be used because the days where the demand was would cause a divide by zero error, and we would have to remove many observations. The data pre-processing creates null values at the beginning and end of the prepared data set, these observations are dropped. A sample of the first 15 observations and the prepared data set can be seen in the table below. Day Demand Over the Next 3 Days Lag 4 Days Lag 3 Days Lag 2 Days Lag 1 Day Today 1 1 37.8 2 1 426.7 3 1 537.1 4 137.8 979.3 37.8 5 1426.7 595.2 37.8 388.9 6 1537.1 567.4 37.8 388.9 11.4 7 217.1-244.4 37.8 388.9 11.4 48 8 221.9-489.8 37.8 388.9 11.4 48 4.8 9 214.5-99.8 388.9 11.4 48 4.8 82.6 1 1772.7-937.8 11.4 48 4.8 82.6-331.8 11 1532.1-1145.77 48 4.8 82.6-331.8-24.6 12 1194.7-184.94 4.8 82.6-331.8-24.6-337.4 13 834.9-834.9 82.6-331.8-24.6-337.4-359.8 14 386.33-386.33-331.8-24.6-337.4-359.8-448.57 15 19.76-19.76-24.6-337.4-359.8-448.57-276.57 We separate the simulation demand into two sets, a training and testing set. The training set has 6 days and the testing set has 6 days. The training set of 6 observations may be then subdivided into subsets as required for each data mining techniques. For example, for Neural Networks, the training set is further subdivided into a training and cross validation data set. Support Vector Machines do the same thing but repeatedly to do a more in depth cross validation. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 14

Foundries Monthly Sales Data The foundries monthly sales data time series data set was obtained from the Statistics Canada table 34-14, which is defined as Manufacturers' shipments, inventories, orders and inventory to shipment ratios, by North American Industry Classification System (NAICS), Canada, monthly (Dollars unless otherwise noted) (Statistics Canada 23). From this table, the monthly sales for all foundries are used. The classification of foundries is as Durable Goods Industries, Primary Metal Manufacturing, and this Canadian industry comprises establishments primarily engaged in pouring molten steel into investment moulds or full moulds to manufacture steel castings. (Statistics Canada 23) Therefore it is far upstream from the final customer and should therefore be subject to much demand signal distortion. The chart of the foundries monthly order value is show below. Foundries Monthly Orders Foundries Monthly Orders 35 3 25 Total (Cnd $) 2 15 1 5 May-9 Sep-91 Jan-93 Jun-94 Oct-95 Mar-97 Jul-98 Dec-99 Apr-1 Sep-2 Jan-4 Month There are 136 months of estimated sales data for Canadian Foundries. The observations start on January 1992 and end on April 23. Because of the data preparation which causes null values, 5 observations must be dropped from the beginning of the data set and 1 observation must be dropped from the end of the data set. This gives a final total of 13 months of observations. The independent variables or input variables for all the models is the percentage change in demand for each of the past four days plus today and the dependent variable or output variable is the percentage change over the next day. The exceptions are the Naïve, Average and Trend forecast. Rel Over Next 1 Days Lag 4 Days Lag 3 Days Lag 2 Days Lag 1 Days Month Demand Today Jan-92 127712.55124 Feb-92 134752.14154.55124 Mar-92 148787.2971.55124.14154 Apr-92 149229.54661.55124.14154.2971 May-92 157386.3724.55124.14154.2971.54661 Jun-92 163213 -.188392.55124.14154.2971.54661.3724 Jul-92 132465.18678.14154.2971.54661.3724 -.18839 Aug-92 146861.94797.2971.54661.3724 -.18839.18678 Sep-92 16783 -.16289.54661.3724 -.18839.18678.94797 Oct-92 158164 -.27579.3724 -.18839.18678.94797 -.1629 Nov-92 15382 -.134933 -.18839.18678.94797 -.1629 -.2758 Dec-92 13349.15929.18678.94797 -.1629 -.2758 -.13493 Jan-93 15313.3363.94797 -.1629 -.2758 -.13493.15929 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 15

Feb-93 158193.128356 -.1629 -.2758 -.13493.15929.3363 Mar-93 178498 -.72415 -.2758 -.13493.15929.3363.128356 Results Overview We separate the foundries demand data into two sets, a training and testing set. The training set has 1 months (77%) and the testing set has 3 months (23%). The training set of 1 observations may be then subdivided into subsets as required for each data mining techniques. The results of the various forecasts are presented below. They have been sorted in ascending order based on the test set Mean Average Error (MAE). The Recurrent Neural Network and the Least-Squares Support Vector Machine have the best results, but their superior accuracy is relatively larger for the Foundries demand data series than the simulation demand data series. In both cases the SVM has the best training set accuracy with the least impact on it generalization ability seen in the testing set. We can also see that the trend estimation and the naïve forecast are the worst types of demand signal processing since they have the highest level of error. Simulation Test Set MAE Train Set MAE RNN 449.9597 46.7788 LS-SVM 453.2889 448.432 MLR 454.1278 463.724 NN 456.199 47.769 Average 521.8643 535.698 Naïve 75.7452 786.2997 Trend 861.794 91.6183 Foundries Test Set MAE Train Set MAE RNN 8.33% 7.765% LS-SVM 8.451% 1.727% NN 1.63% 6.152% MLR 13.364% 9.96% Average 13.848% 9.938% Trend 17.591% 12.561% Naïve 24.517% 16.846% Neural Networks A Multi-Layer Perceptron (MLP) Feed Forward (FF) Artificial Neural Network (ANN) with back-propagation of error (BP) with one hidden layer is used to build a forecasting model (Demuth 1998). This NN will learn any relationship between the five inputs and the one output, given enough time and neurons. We use a 2% cross validation set to stop the training once the error on this set start increasing, this attempts to find the point of best generalization by not permitting the network to be over-trained. Simulation Distorted Demand Forecasting A heuristic 8 training samples to one neural network weight was chosen to determine that 1 neurons for the hidden layer will be used ((Training Samples * Training Set REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 16

Size) / ((Inputs * Hidden Layer Neurons) + (Hidden Layer Neurons * Output)) <=> (6*.8)/(((5*1)+(1*1))= 8). NN Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 NN Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Foundries Demand Forecasting The number of neurons in the hidden layer were set to 3 with a ratio of samples to weights of 5.7. NN Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 17

NN Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 Recurrent Neural Networks The next forecasting model is a Recurrent Neural Network. It has all of the same properties are the Neural Network described above, but additionally it has a recurrent connections for every neuron in the hidden layer that feeds back into itself at the next step. This permits the Recurrent Neural Network to learn patterns with an arbitrary depth through time. The Recurrent Neural Networks outperform all other techniques on the two test sets. This is probably related to the fact that the problem is a time series and that the RNN can learn patterns through time. This may also help it generalize since the patterns it learns must make sense simultaneously in both the time and input dimensions. Simulation Distorted Demand Forecasting For the Recurrent Neural Network, 6 neurons in the hidden layer were used. With the 6 2 recurrent connections, the final ratio of samples to network weights is 6.7 ((Training Samples * Training Set Size) / ((Inputs * Hidden Layer Neurons) + (Hidden Layer Neurons^2) + (Hidden Layer Neurons * Output)) <=> (6*.8)/(((5*6)+(6^2)+(6*1))= 6.7). RNN Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 18

RNN Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Foundries Demand Forecasting The number of neurons in the hidden layer were set to 2, so with the recurrent connection, the ratio of samples to weights of 6.5. RNN Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 RNN Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 Support Vector Machine A Least Squares Support Vector Machine (LS-SVM) with a Radial Basis Function (RBF) kernel is used to forecast the distorted demand. The specific tool used is the LS-SVMLab1.5 for MATLAB (Suykens 22, Pelckmans 22). For choosing the best RBF kernel parameters, LS-SVM s automated 1-fold cross validation based hyper parameter selection function was used. In the sections below there will be two ridge diagrams that show the search space for the optimal SVM hyper parameters. The vertical axis is the log of the RBF kernel parameter and the horizontal axis is the log of the regularization parameter. Ridges help illustrate the search space s high and lows. Blue to Red ridges are better to REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 19

worse and lower to higher error. The optimization function does two passes to zero in on the best parameters. Once the best parameters have been selected, then the final LS-SVM model is trained. This is where it will attempt to build a model while minimizing it s structural risk. Simulation Distorted Demand Forecasting The hyper parameter optimization function decided on a regularization parameter of.626 and a RBF kernel parameter of 26.374. Hyper parameter search space ridge diagram SVM Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 SVM Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 2

Foundries Demand Forecasting The hyper parameter optimization function decided on a regularization parameters of 16.397 and a RBF kernel parameter of 2.151. With the LS-SVM we can see that it learn the training set extremely well without sacrificing it s generalization ability Hyper parameter search space ridge diagram SVM Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 SVM Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 Multiple Linear Regression Multiple Linear Regression learns the linear trends between the past changes and changes in the future. Performance of the MLR model is quite good, indicating that many of the relationships between the past and future data is linear. The restriction to linear relationships also helps it to avoid over specializing on the training data. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 21

Simulation Distorted Demand Forecasting MLR Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 MLR Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Foundries Demand Forecasting MLR Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 MLR Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 22

Trend Forecast The trend estimation is a Simple Linear Regression to estimate the trend in the change (or percentage change) for the past 1 days. It disregards any data that is beyond the 1 days and the model is a linear relationship between the current change and the next change, it is therefore simple. The trend estimation is also the forecasting model that can create the most distortion in the extended supply chain since it can be the one that takes the longest to recognize a change in the trend, depending on the depths of the depth of the trend estimation. Simulation Distorted Demand Forecasting Here we see the most demand distortion because within the 1 day window for trend estimation, a trend becomes evident and it forecasts for 3 days in advance, therefore we see amplitude magnification. Trend Training Set (First 2) 4 3 2 1-1 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-2 -3-4 Trend Testing Set (First 2) 3 2 1-1 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-2 -3-4 Foundries Demand Forecasting In the case of the foundries data we do not see extreme amplitude magnification because the 1 month trend estimation spans over many inverse trends, therefore canceling them out. Also the forecast is for the next month (1 step) as opposed to the above model that forecast in 3 days (3 steps). REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 23

Trend Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 Trend Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 24

Conclusion Future Research The objective of this research is to study the feasibility of forecasting the distorted demand signals in the extended supply chain with advanced non-linear machine learning data-mining techniques. The advanced data mining techniques results, as shown in the Data Mining Experiments Results Overview section, are not a large improvement over traditional non-linear techniques (as represented by the MLR model) for the simulation data set. But for the real foundries data, the more advanced data mining techniques (RNN and SVM) provide larger improvements. Overall the Recurrent Neural Networks (RNN) and Support Vector Machine (SVM) provide the best results on the foundries test set. Either way, data mining techniques that learn from all the data, as opposed to other techniques such as trend estimation that does not look at all the data, results in better forecasting accuracy We can also see that the trend estimation and naïve forecast are the worst types of demand signal processing since they have the highest level of error. Therefore we can conclude that data mining for forecasting distorted demand signals in the extended supply chain is feasible and provides more accurate forecasts than other forecasting techniques which do not look at all the data. These results are important because there are situations where parties in the supply chain cannot collaborate and the ability to increase forecasting accuracy in these situations will result in lower costs and higher customer satisfaction because of more on-time deliveries, therefore increasing sales. For the overall supply chain, more accurate forecasts by each individual will result in a reduction in the demand distortion. For further examination of the impacts of these techniques on the extend supply chain, the results found in this research should be incorporated into the supply chain simulation. Since we have determined various data mining tools that provide more accurate forecasts than trend estimation, we can use the forecasting models that they have created and incorporate them in the supply chain simulation as an automated real-time forecasting and procurement agent. A simulation run of the supply chain with one or more intelligent forecasting and procurement agents will yield information on the impacts of these technologies on the complete supply chains. It is clear that without collaboration, there is information missing. With additional information from various downstream partners, we can study the impacts of information sharing on forecasting accuracy. To study the impact of collaborative forecasting, the current simulations, models and techniques can be used while providing more information from down the supply chain to the data mining tools. This additional information would probably increase the models accuracy, provided that there are enough training sets to offset the added dimensionality. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 25

References Box G.E, G.M Jenkins. (197) Time Series Analysis, Holden-Day, San Francisco. Cox, A., J. Sanderson, et al. (21). Supply chains and power regimes: Towards an analytic framework for managing extended networks of buyer and supplier relationships. Journal of Supply Chain Management 37(2): 28-35. Demuth, H. and M. Beale, (1998) Neural Network Toolbox for Use with MATLAB. The MathWorks, Inc., Natick, Massachusetts. User's Guide (version 3.). Donahue, Manus J. III. (1997) An Introduction to Chaos Theory and Fractal Geometry. http://www.duke.edu/~mjd/chaos/chaosp.html Dorffner, G. (1996), Neural Networks for Time Series Processing. Neural Network World 96(4), 447-468. http://citeseer.nj.nec.com/17978.html Forrester, J. (1961) Industrial Dynamics. Productivity Press, Cambridge, MA. Herbrich, R., M. Keilbach, T. Graepel, P. Bollmann-Sdorra, and K. Obermayer. (2) Neural networks in economics: Background, applications and new developments. Advances in Computational Economics: Computational Techniques for Modeling Learning in Economics 11:169-196. http://citeseer.nj.nec.com/23132.html Heikkila, J. (22). From supply to demand chain management: efficiency and customer satisfaction. Journal of Operations Management 2(6): 747-767. Julia, G. (1918) Mémoire sur l'itération des fonctions rationnelles, J. Math. Pures Appl. 8: 47-245 Kimbrough, S., D.-J. Wu, and Zong Fang. (2). Artificial Agents Play the Beer Game Eliminate the Bullwhip Effect and Whip the MBAs. http://opimsun.wharton.upenn.edu/~sok/sokpapers/2-1/beergameslides.ppt Kimbrough, S., D.-J. Wu, and Zong Fang. (21). Computers play the beer game: Can artificial agents manage supply chains? Decision Support Systems 33(3): 323-333. Kullback, S. (1968) Information Theory and Statistics, 2nd edition, New York: Dover Books, (1st ed. New York: John Wiley, 1959). Lawrence S, A.C. Tsoi, C.L. Giles (1996). Noisy time series prediction using symbolic representation and recurrent neural network grammatical inference. University of Maryland, Institute for Advanced Computer Studies, Technical Report UMIACS-TR- 96-27. http://citeseer.nj.nec.com/lawrence97noisy.html. Landt, F.W. (1997). Stock Price Prediction using Neural Networks, Master Thesis, Laiden University. http://citeseer.nj.nec.com/landt97stock.html Lee, H. L., Padmanabhan, V and Whang, Seungjin. (1997). Information Distortion in a Supply Chain: The Bullwhip Effect. Management Science 43(4): 546-558. MathWorks, Inc. (2) Using MATLAB, Version 6. Mukherjee S., E. Osuna, and F. Girosi (1997) Nonlinear prediction of chaotic time series using support vector machines. Proc. of IEEE NNSP 97. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 26

Pelckmans K., Suykens J.A.K., Van Gestel T., De Brabanter J., Lukas L., Hamers B., De Moor B., Vandewalle J., (22). LS-SVMlab : a Matlab/C toolbox for Least Squares Support Vector Machines, Internal Report 2-44, ESAT-SISTA, K.U.Leuven (Leuven, Belgium), (presented at NIPS22 Vancouver in the demo track), Premkumar, G. P. (2). Interorganization Systems and Supply Chain Management: An Information Processing Perspective. Information Systems Management 17(3): 56-59. Raghunathan, S. (1999). Interorganizational Collaborative Forecasting and Replenishment Systems and Supply Chain Implications. Decision Sciences 3(4): 153-171. Statistics Canada. (23) table 34-14, 23 Rüping, S., K. Morik (23) Support Vector Machines and Learning about Time Proceedings of ICASSP 23. http://citeseer.nj.nec.com/566731.html Suykens, J.A.K., T. Van Gestel, J. De Brabanter, B. De Moor, J. Vandewalle, (22) Least Squares Support Vector Machines, World Scientific, (ISBN 981-238-151-1) Vapnik, V. The Nature of Statistical Learning Theory. Springer, New York,1995. Vapnik, V. S.E. Golowich, and A. Smola (1997). Support vector method for function approximation, regression estimation, and signal processing. Advances in Neural information processings systems 9:287-281. Watson, G. (21). Subregimes of power and integrated supply chain management. Journal of Supply Chain Management 37(2): 36-41. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 27

Appendix A Additional Simulation Components Macro View of the Supply Chain Order Processing Least Squares Polynomial Fit REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 28

Output Data REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 29

Appendix B Additional Simulation Results Wholesaler Below are the wholesaler s deliveries, inventory and backlog. Distributor Below are the distributor s deliveries, inventory and backlog. Manufacturer Below are the manufacturer s deliveries, inventory and backlog. REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 3

Appendix C Additional Forecasts Time Series Naïve Forecast Simulation Distorted Demand Forecasting Naive Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Naive Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Foundries Demand Forecasting Naive Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 31

Naive Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 Average Forecast Simulation Distorted Demand Forecasting Average Training Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 Average Testing Set (First 2) 25 2 15 1 5 1 9 17 25 33 41 49 57 65 73 81 89 97 15 113 121 129 137 145 153 161 169 177 185 193-5 -1-15 -2 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 32

Foundries Demand Forecasting Average Training Set.5.4.3.2.1 -.1 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 -.2 -.3 -.4 Average Testing Set.5.4.3.2.1 -.1 1 2 3 4 5 6 7 8 9 1 11 12 13 14 15 16 17 18 19 2 21 22 23 24 25 26 27 28 29 3 -.2 -.3 -.4 REAL CARBONNEAU SUPPLY CHAIN DATA MINING PAGE 33