David Coufal 2 Institute of Computer Science Academy of Sciences of the Czech Republic Pod Vodarenskou vezi 2, Prague 8, Czech Republic

Transcription

1 V.012 SHORT TERM PREDICTION OF HIGHWAY TRAVEL TIME USING DATA MINING AND NEURO-FUZZY METHODS 1 David Coufal 2 Institute of Computer Science Academy of Sciences of the Czech Republic Pod Vodarenskou vezi 2, Prague 8, Czech Republic Esko Turunen 34 Tampere University of Technology P.O. Box 692, Tampere, Finland Abstract We show that prediction of travel time on a 28-km long highway section based on on-line travel time measurements with video is practicable by data mining and neuro-fuzzy methods. We introduce two new prediction models. The first one is a result of GUHA style data mining analysis and Total Fuzzy Similarity method, and the second one is a hierarchical model based on neuro-fuzzy modelling. Comparing results with the existing Traficon model, both new models improve the travel time prediction. The results obtained by the new methods are comparable to MLP neural network model, too. Key words: fuzzy logic, neural networks, data mining. 1. Introduction The aim of this study is to show that short-term travel time prediction presented in [3] can be carried out by data mining and neuro-fuzzy methods, too, and that results are comparable. Research [3] was carried out on main road 4 between points A (Lahti) and D (Heinola) in Southern Finland. According to [3], the average daily summertime traffic on this 28-kilometer section is about vehicles per day, in particular, the traffic volumes are high during summer weekends. The study section AD is divided into three sub-sections AB, BC and CD with camera stations approximately equally distributed over link AD length and equipped with an automatic travel time monitoring system. The system is based on an artificial vision and neural network application, which automatically reads license plates. Moreover, there is an inductive loop detector on station C gathering information on traffic volumes and point speeds. A variable message sign (VMS) at point A gives upper and lower bounds of a estimations about the travel to the point D. In an unpublished preliminary study of the problem done by Laura Lanne, the estimation categories are below 25 min, min, min, min and above 50 min. In [3], travel time from point A to point D is regarded as congested if it is above 25 min. In [3], travel time prediction p is regarded as acceptable if the real travel time lies in the interval [0.9*p, 1.1*p], i.e., it is used ±10% marginal error based correctness of results. 1 This research is part of research project COST Action 274 [TARSKI] 2 Supported by grant OC of Ministry of Education, Youth and Sports of the Czech Republic 3 Supported by grant of Finnish Academy 4 Correspondence via esko.turunen@tut.fi

2 The erroneous predictions can be divided into two categories: the travel time prediction can be too pessimistic (too long travel times, i.e. road users arriving to their destinations earlier than expected) or too optimistic (too short travel times, i.e. road users arriving to their destinations later than expected). The too optimistic travel time prediction is worse for the road user than the too pessimistic travel time prediction. In this study we used two data sets we received in autumn 2001 and in summer 2002 from Helsinki University of Technology, Transportation Engineering. The data was given in form of the following table. Input Output traffic flow at C avg. speed at C avg. tt AB avg. tt BC avg. tt CD avg. tt real travel time AD [min] [vehicles/5min] [km/hour] [min] [min] [min] AD [min] the value to be predicted avg. tt = average travel time There are 4541 rows (cases) in the first set and 9333 rows (cases) it the second set. Notice that average travel time AD is not the sum AB+BC+CD but it is the average travel time of the vehicles that passed point D during the last 5 minutes spent in the whole section. In other words, the sum AB+BC+CD contains travel time information of at least three different vehicles while AD can be based on travel time information of one single car. The structures of the data sets, in a sense that all predictions are correct, are presented in following Table1 and Table 2. Predicted travel time Travel time <= 25 (25, 30] (30, 40] (40, 50] > 50 row sum <= (25, 30] (30, 40] [40-50) > column sum Table 1. The structure of the first data set. From Table 1, we see that in first data set there are 4426 (97%) cases of non-congested conditions and 115 (3%) cases of congested conditions. In congested conditions we have distribution into particular categories given a s 16.5% - (25, 30]; 37.4% - (30,40]; 34% - (40,50]; 12.1% - >50.

3 Predicted travel time Travel time <= 25 (25, 40] (30, 40] (40, 50] > 50 row sum <= (25, 30] (30, 40] (40, 50) > Column sum Table 2. The structure of the second data set. According to Table 2, in the second data set there are 8693 (93%) non-congested cases and 640 (7%) congested cases. With respect to congested cases, the distribution into particular categories is 26.9% - (25,30]; 60% - (30,40], 13.1% - (40,50], 0.02% - >50. In the present application, called Traficon model, the estimated value of the travel time, which is around 20 minutes in normal non-congested circumstances, is based on the last measurements on each sub-link AB, BC and CD. The estimation is normally updated every 5 minutes. However, a problem is that the estimation is always more or less outdated. Based on a similar kind of data and presented in [3], the Traficon model estimates the congested travel times right within 10% error marginal in 32,9%.. Therefore, one of the objectives in [3] was to develop a model that would improve the travel time prediction, in particular, in the congested cases. This was done by MLP-neural networks and, indeed, the prediction results improved. In the new model the travel times are predicted right in 98,4% of all cases, while 0,5% of all cases are predicted too high and 1,1% of all cases are predicted too low. Among the congested cases the improvement is significant: 66,5% of the congested cases are predicted right, while 5,3% are predicted too high and 28,2% too low. In this paper we show that comparable results can be attained by other methods, too. 2. A Travel time prediction by GUHA analysis and Total Fuzzy Similarity method The aim in the first new approach is to develop a prediction method that would increase correctness of forecasts and, simultaneously, be as simple and reasonable as possible. The underlying consumption is that input values similar enough imply relatively similar output values. Indeed, the idea in Total Fuzzy Similarity method introduced in [6] is to present an experts knowledge by a finite set of fuzzy IF - THEN inference rules and, in each particular case, to search for the most similar IF - part in the role base and, finally, to fire the corresponding THEN - part. In the travel time prediction problem there was, however, no expert to tell which kinds of traffic situations lead congested travel times and which ones to normal travel times. Thus, we used GUHA method [2] to create the rule base. GUHA - General Unary Hypotheses Automation - is a logically and statistically founded data mining method to find all interesting facts following from a data matrix containing n columns and m rows. In the core of GUHA analysis there are frequency tables of a form

4 succedent non(succedent) antecedent a B non(antecedent) c D In practice, the GUHA analysis was carried out by means of a computer program LISp-Miner [4]. A results of such an analysis is presented in Appendix 1. To have as up-to-date information as possible, we added two new columns to the data, the sums AB+BC and AB+BC+CD. The main difference in values AB+BC and AB+BC+CD with respect to values AC and AD, respectively, is that e.g. travel times of vehicles calculated into the value AD have already left the whole section, while most vehicles calculated into the values AB+BC+CD have not jet reached destination D and, if they are in a congested situation, then this congestion has an effect on travel time of the vehicles just starting from point A. We used the first data set of 4541 cases as a model teaching material. First we searched for simple conditions that should divide the data into two subsets X = {predicted travel time and real travel time are both above 30 min} and Y = {predicted travel time and real travel time are both below 30 min} such that union of X and Y would be as large as possible. It turned out that such a condition really exists. Indeed, if the data is divided by a condition (A) AD is at least 23 min and AB+BC is at least 17,5 min and AB is at least 5,58 min then only 9 cases out of 4541 are not in X nor in Y. Moreover, in the set X, predicted travel time is a more or less linear function of the value AB. However, as we wanted to keep things simple, we did not include the function into the model but let the output be a prediction class. Second, we searched for simple conditions that would divide the set Y into suitable subsets Y1 = {predicted travel time and real travel time are both above 25 min} and Y2 = {predicted travel time and real travel time are both below 25 min}. We found such a condition. Indeed, by a condition (B) (AB+BC+CD is at least 21,25 min or AD is at least 35 min) and CD is at least 6,3 min the set Y can be divided such that 30 cases out of 4409 are not in Y1 nor in Y2. Third, we searched by GUHA method for simple conditions that would divide the set Y2 into suitable subsets Y21 = {predicted travel time and real travel time are both above 20 min} and Y22 = {predicted travel time and real travel time are both below 20 min}. It turned out that such rules exist, however, they are no more simple nor reasonable. This is, after all, not restrictive as in the VMS the cases 'Travel time < 20 min' and 'Travel time min' are not distinguished from each other. Another result of GUHA analysis is that the input values 'Traffic flow at point C' and 'Average Speed at point C' do not seem have any significant importance with respect to the travel time prediction.

5 Based on these kinds of GUHA analyses, the rule base of a Total Fuzzy Similarity - inference system is the following TÄHÄN ASTI KOPOITU IFSA 2003 ARTIKKELIINl IF AD >= 23 AND AB+BC >= 17.5 AND 23 <= AB THEN PREDICTION is ' > 50 min' IF AD >= 23 AND AB+BC >= 17.5 AND 12 =< AB < 23 THEN PREDICTION is '(40,50] min' IF AD >= 23 AND AB+BC >= 17.5 AND 5.58 <= AB < 12 THEN PREDICTION is '(30,40] min' IF AB+BC+CD >= AND CD>=6.3 THEN PREDICTION is '(25,30] min' IF AD >= 35 AND CD>=6.3 THEN PREDICTION is '(25,30] min' ELSE PREDICTION is '(20,25] min' The corresponding fuzzy sets reduce to crisp ones. If the output would not be unique i.e. there are several IF - parts possessing the maximal total similarity degree, then - corresponding to the 'pessimistic prediction principle' - the prediction should be the longest one. In Table 3, we see the rate of correct predictions among the teaching data set obtained by the above IF-THEN inference system. Indeed, 98,9% among all data fall into the proper prediction class and 80,9% among the congested cases. Predictions travel time < 25] (25, 30] (30, 40] (40, 50] >50 row sum <= (25, 30] (30, 40] (40, 50] > Column sum Table 3. Rate of correct predictions among teaching data obtained by Total Fuzzy Similarity Since the rules of the above IF-THEN inference system are tuned with respect to the first data set, a real and relevant test is to see correctness of predictions among the second data set of 9333 cases. Real Prediction in test data Travel time < 25] [25-30) [30-40) [40-50) > 50 Row sum < 25] [25-30) [30-40) [40-50) > column sum Table 4. Rate of correct predictions among test data obtained by Total Fuzzy Similarity According to the results in Table 4, in all test data, prediction is right in 96,5 % of all 9333 cases, too low in 1,6 % of these cases and too high in 1,9 % of cases. In congested situations, and there are 640 such cases, the figures are 60,5 %, 23,1 % and 16,4 %, respectively. Note that due to fact

6 that output of the Total Similarity model is a class and not an exact value, thus, we cannot use here ±10% marginal error, but we form the table without this marginal test so our results are stronger than if we would use ±10% marginal error. For example, a real travel time 50,3 min is predicted to fall into the class (40,50]. This prediction is regarded as an error. It is worth emphasising that the present Total Fuzzy Similarity model is extremely simple, indeed, it contains only six rules. Adding more rules and counting the output e.g. by a linear function would improve correctness of predictions. This we have, however, not done as our main purpose is to show that hidden in the data, there is a reasonable structure that can be found by GUHA data mining method and then implemented by an IF-THEN rule base. 3. Hierarchical neuro-fuzzy model of traffic data The above prediction model based on GUHA analysis and Total Fuzzy Similarity method gives better results than the one introduced in [3]. However, the Total Fuzzy Similarity model does not predict exact travel times but only prediction classes. This disadvantage could be overcome by adding linear functions to count the exact output, however, the model would no more be simple. From this reason and to evaluate other approach as well, we created a hierarchical neuro-fuzzy model of the data. is AD<=20 YES neuro-fuzzy model for non-congested conditions NO neuro-fuzzy model for congested conditions Figure 1. Hierarchical neuro-fuzzy model. The hierarchical structure of the model is presented in Figure1. Actually, it is very simple. On the first level, data are divided into two classes according to travel time AD. Consequently, data in each class are treated by special neuro-fuzzy model. More precisely, if travel time AD is less or equal to 20 min the case is treated by a neuro-fuzzy model regarding non-congested conditions and, if travel time AD is greater than 20 min, the case is treated by neuro-fuzzy model regarding congested conditions. In the following section we aim to describe the architecture of employed neuro-fuzzy model Neuro-fuzzy model o ( x ) = n j = 1 The model is actually a Wang fuzzy inference system (FIS) designed in neural network fashion to be capable to learn its parameters [7]. Wang FIS computes according to formula c j A j ( x )

7 (1) Note that, to avoid division by zero, the denominator is omitted from a more general equation in [7]. In formula (1), the Aj(x) are multidimensional Gaussians, given by A j (x) n = exp i= 1 (x i a 2b ji 2 ji ) 2 (2) where x=(x 1,, x n ) is a n-dimensional input, a j,=( a j1,, a jn ), b j,=( b j1,, b jn ), b ji 0 are parameters. The Gaussians are combined by linear combination, formula (1), employing coefficients c j. From fuzzy inference system's point of view A j (x) are the antecedents of the system's particular rules and c j are the centroids of their succedents fuzzy sets. Having the architecture of fuzzy model given, its parameters are centers of Gaussians a j, their widths b j and centroids c j. To be able to learn the parameters of the model, we interpret it in a form of neural network of architecture given in Figure 2. h 1 x 1 u 1 w 1 x i u i h 2 w 2 o o x w m1 x n h m1 w m u n h m Figure 2. Neural network representation of Wang FIS. Neurons of input layers are the transmitting ones, i.e., u i (x)= x i. The hidden neurons compute according to (2), i.e., h j (x)= A j (x) and output neuron computes linear combination of hj via weights. Weights corresponds to parameters of linear combination, i.e., w j =c j. Apparently, computation of whole network is given just by formula (1). That is, we have neural network s representation of Wang FIS, i.e., Wang neuro-fuzzy system. To let the network to learn its parameters, we have to apply on it an appropriate structure and parameters learning algorithm. In this study, we used for structure learning incremental learning algorithm [1]. For parameter learning we used Levenberg-Marquard method [5] with respect to error function given by E = i ( t o( )) 2 i x i (3)

8 In (3), it is considered that pairs (x i, t i,) are elements of training set. In the next section we present results of four experiments regarding both traffic data sets. Because traffic flow and average speed variables were observed to do not have an effect on the real travel time, input to the neuro-fuzzy model was only four-dimensional, i.e., x=(x 1, x 2, x 3, x 4 ). - travel time AB, travel time BC, travel time CD and travel time AD Results obtained by neuro-fuzzy model In the first experiment we split the first data set (4541 cases) on two halves. The first half we used for learning of model and the second for its testing, the results of the experiment with respect to ±10% marginal error are presented in Table 5. Note that due to fact that output of neuro-fuzzy system are exact values we can use the ±10% marginal error based correctness to compare results with these given in [3]. Predictions Travel time (0,25] (25,30] (30,40] (40,50] > 50 row sum < (25, 30] (30, 40] (40, 50] > col sum Table 5. Results of the first experiment with neuro-fuzzy model. In the second experiment we take the same strategy is in the first one, but we used the second data set (9333 cases). That is, we split the data on two halves, the first one was used for learning and the second for testing. Results with respect to ±10% marginal are in Table 6. Predictions Travel time < 25 (25, 30] (30, 40] (40, 50] > 50 row sum < (25, 30] (30, 40] (40, 50] > col sum Table 6. Results of the second experiment with neuro-fuzzy model Regarding the correctness figures for both models we have in the first case figures as 99,2 % of all predictions on whole testing data set correct, 0.5% too low and 0.3% to high. Aiming only at predictions in congested conditions the figures are 70.2% correct, 21% too low and 8.8% too high. In the case of second model (for the second data set) with respect to whole testing set we have 98.1% predictions correct, 1.1% too low and 0.9% too high. In congested conditions figures are as 77.5% correct, 15.6 too low and 6.9% too high. We see that both models have almost the same behaviour with respect to their correctness. The correctness is comparable well with results

9 in [3] and of Total Fuzzy Similarity method presented above. Note that neuro-fuzzy models are proper predictive ones, i.e., different data were used for building (learning) the models from data used for their testing. In the third experiment we use for learning the whole first data set (4541 cases) and created model was tested on data of the second set (9333 cases). The results with respect to ±10% marginal are presented in Table 7. Predictions Travel time < 25 (25, 30] (30, 40] (40, 50] >50 row sum < (25, 30] (30, 40] (40, 50] > col sum Table 7. The results of the third experiment with neuro-fuzzy model. Finally, reverse experiment to the third one was performed. That is, we learned the model on the second data set (9333 cases) and tested on the first one (4541 cases). The results with respect to ±10% marginal are in Table 8. Predictions Travel time < 25 (25, 30] (30, 40] (40, 50] >50 row sum < (25, 30] (30, 40] (40, 50] > col sum Table 8. The results of the fourth experiment with neuro-fuzzy model. When analysing Table 8 the figures are really disgusting, especially for congested conditions. For all 9333 testing cases we have 95.4% correctness of predictions, 3.8% are too low and 0.8% too high, which is still quite good but for congested conditions we have only 34.7% predictions correct, 55.8% are too low and 9.5% are too high. The reasons for such a bad performance lies in different distributions of congested cases in both data sets as presented in section % - (25,30]; 37.4% - (30,40]; 34% - (40,50]; 12.1% - >50 for the first data set and 26.9% - (25,30], 60% - (30,40], 13.1% - (40,50], 0.02% - >50 for the second set. We see that when learning on the first data set then the cases from (30,40] and (40,50] classes have almost equal distribution but in the second set the class (30,40] is much more preferred. The other reason is that in the first data set there are only 115 cases of congested conditions but in the second set we have 640 of them so there is more information about congested cases in the second set. This is also reflected in figures for fourth experiment where for all predictions we have 98.6% correct, 0.8% too low and 0.5% too high. and in congested conditions 60.0% correct predictions, 33% too low and 7% too high which is comparable with results of the first two experiments and of [3], especially noting the predictive character of our models.

10 4. Concluding remarks We have presented two new prediction models for a travel time prediction problem. The models are based on two data sets one containing 4541 cases and the second 9333 cases. The first model utilises GUHA data mining analysis and Total Fuzzy Similarity method, the second model is based on hierarchical neuro-fuzzy inference system. The introduced new models improve the results obtained in [3]. Both new prediction models are simple, too. The first experiments we have introduced strengthen an impression that similar but more complicated models would rise the prediction above 90% exactness level, a target imposed in [3]. Acknowledgements The authors wish to thank Satu Innamaa, Laura Lanne, Matti Pursula and the staff at Helsinki University of Technology, Transportation Engineering, for fruitful scientific comments and ideas in preparing the manuscript. In particular, thanks are due to Laura Lanne for her kind assistance. References [1] Coufal, D. Incremental Structure Learning of Three-Layered Gaussian RBF Networks. International Conference on Computational Science - ICCS 2002, Amsterdam, The Netherlands, April 21-24, 2002, Proceedings, Part III, Springer-Verlag [2] Hajek, P., Havranek, T. Mechanizing Hypothesis Formation - Mathematical Foundations for a General Theory. Berlin - Heidelberg -New York, Springer -Verlag, 1978, 396 p. Internet version (free) [3] Innamaa, S. Short-term prediction of highway travel time using MLP-neural networks. 8.th World Congress on Intelligent Transportation Systems. Sydney, Australia, 30 Sept. - 4 Oct [4] Rauch, J.: Mining for Scientific Hypotheses. In Meij, J.(Editor): Dealing with the data flood. Mining Data, Text and Multimedia. STT/Beweton, The Hague pp (An internet version of software available at [5] Press W.H., Teukolsky S.A., Vetterling W.T, Flannery B.P., Numerical Recipes in C, The Art of Scientific Computing, Second Edition, Cambridge University Press, 1992; internet version available at [6] Turunen, E. Mathematics Behind Fuzzy Sets. Springer-Verlag, Heidelberg [7] Wang L.X., Mendel J.M. Fuzzy basis function, universal approximation, and orthogonal least-squares learning. IEEE Trans. on Neural Networks, vol.3, no.5, pp APPENDIX 1. A GUHA ANALYSIS RESULT Antecedent Travel time AB from 11 to 12 min and Travel time CD from 6 to 7 min Succedent Prediction from 30 to 40 min Frequency table

11 Succeden NOT Succedent Row Sum Antecedent NOT Antecedent Column Sum