Web Traffic Mining Using a Concurrent Neuro-Fuzzy Approach

Transcription

1 Web Traffic Mining Using a Concurrent Neuro-Fuzzy Approach Xiaozhe Wang, Ajith Abraham and Kate A. Smith School of Business Systems, Faculty of Information Technology, Monash University, Clayton, Victoria 3800, Australia {catherine.wang,ajith.abraham,kate.smith}@infotech.monash.edu.au Abstract: Web servers play a crucial role to convey knowledge and information to the end users. With the popularity of the WWW, discovering the hidden information about the users and usage pattern is critical to determine effective marketing strategies and to optimise the server usage and accommodate future growth. Many of the currently available server analysis tools could provide only statistical data without much useful information. Mining useful information becomes a challenging task when the user traffic volume is enormous and keeps on growing. In this paper, we propose a concurrent neuro-fuzzy model to analyse useful information from the available statistical/text data from the Web log analyser. We made use of the cluster information generated by Self Organising Map (SOM) for data analysis and a Fuzzy Inference System (FIS) to forecast the daily and hourly traffic volume. Empirical results clearly demonstrate that the proposed hybrid technique is efficient and could be extended to other Web environments. Keywords: Web mining, clustering, self organising map, hybrid, neuro-fuzzy 1. Introduction and Motivation for Research The World Wide Web (WWW) is continuously growing with rapid increase of the information transaction volume and number of requests from Web users around the world. For Web administrators and managers, discovering the hidden information about the users access or usage patterns has become a necessity to improve the quality of the Web information service performances. From the business point of view, knowledge obtained from the usage or access patterns of Web users could be applied directly for marketing and management of E-business, E-services, E-searching, E- education and so on. However, the statistical data available from the normal Web log data files or even the information provided by Web trackers could only provide the information explicitly because of the nature and limitations of the methodology itself. Generally, one could say that the analysis relies on three general sets of information given a current focus of attention: (1) past usage patterns (2) degree of shared content and (3) inter-memory associative link structures [7]. Computational Web Intelligence (CWI) [1], a recently coined paradigm, is aimed to improve the quality of intelligence in the Web technology [8]. The pattern discovery of Web usage mining consists of several steps including statistical analysis, clustering, classification and so on [6][9]. Most of the existing research is focused on finding the patterns; with little efforts on the detailed pattern analysis. We propose SOM [12] to cluster and discover patterns from the data. These clustered data were further used for different statistical analysis. In order to make the analysis more intelligent we also used the clustered data to forecast the daily traffic volume and the hourly page requests. Using a Takagi Sugeno Fuzzy Inference System (TSFIS) [], we explored the prediction of average daily traffic 1

2 volume (1 to days ahead) and the hourly page requests traffic in a day (1,12 and 24 hours ahead). As a case study, we explored the Web user access patterns of Monash University s Web site located at We made use of the statistical data provided by Analog [16] Web access log file analyser, which is embedded at the University s Web server. Analog generated data and text information covers different aspects of the users access log records, weekly-based reports include traffic, types of files accessed, domain summary, operating system used, navigation summary and so on. We illustrate the typical Web traffic patterns of Monash University in Figures 1 and 2 showing the daily and hourly volume of traffic (number of requests from different domains and the number of page requests) for the week starting 14-Jul- 2002, 00:13 to 20-Jul-2002, 12:22. For more user access data logs please refer [10] Figure 1. Daily Web traffic of Monash University Figure 2. Hourly Web traffic of Monash University In a week, over 7 million visitors access data from the University s Web site [11] and since the data cover different aspects (domains, files accessed, no of daily and hourly access, page requests etc.), it is a real challenge to find some hidden information or to extract usage patterns. The mere complexity of the data volume paves way for the requirement of hybrid intelligent systems for information analysis and trend forecast. 2

3 In the subsequent section, we present the structure of proposed concurrent neurofuzzy model for mining Web access patterns. In section 3, we present the analysis of the clustered Web data using SOM followed by modeling the TSFIS to forecast the usage pattern trends in Section 4. Finally, some conclusions and future works are given. 2. Hybrid Neuro-Fuzzy Approach for Web Traffic Mining The hybrid framework combines SOM and a FIS operating in a concurrent environment as illustrated in Figure 3. In a concurrent model, neural network assists the fuzzy system continuously to determine the required parameters especially when certain input variables cannot be measured directly. Such combinations do not optimise the fuzzy system but only aids to improve the performance of the overall system [18]. Learning takes place only in the neural network and the fuzzy system remains unchanged during this phase. The pre-processed data (after cleaning and scaling) is fed to the SOM to identify the data clusters. The clustering phase is based on SOM; an unsupervised learning algorithm [12], which can accept input objects described by their features and place them on a two dimensional (2D) map in such a way that similar objects are placed close together. Referring to Figure 4, data X, Y and Z may be segregated into three clusters according to the SOM algorithm. The clustered data is then used by the Web Usage Data Analyser (WUDA) for discovering different patterns and providing useful information to the Web analyst. Web Log File Data Data preprocessing Fuzzy Inference System Web Usage Pattern Forecast Self Organising Map Web Usage Data Clusters Web Usage Data Analyser Figure 3. Architecture of the concurrent neuro-fuzzy Model for Web pattern analysis FIS is used to forecast the Web traffic patterns on an hourly and daily basis. FIS is a popular computing framework based on the concepts of fuzzy set theory, fuzzy if-then rules, and fuzzy reasoning. The basic structure of the FIS consists of three conceptual components: a rule base, which contains a selection of fuzzy rules; a database, which 3

4 defines the membership functions used in the fuzzy rule and a reasoning mechanism, which performs the inference procedure upon the rules and given facts to derive a reasonable output or conclusion. As shown in Figure 4, data X is associated with Cluster 3 strongly but data Y and Z have weak associations with other clusters. Example: Data Z is associated with cluster 1 but can be also considered to have some weak association with Clusters 2 and 3. The degree of association of the data with a particular cluster is modelled as an additional fuzzy variable. We used the Takagi- Sugeno FIS to derive the rule conclusions []. The FIS could forecast the hourly and daily Web traffic. An important advantage of the FIS is the interpretability of the developed model in the form of simple if-then rules. Cluster 3 x Z Y Cluster 1 Cluster 2 Figure 4. Data association with clusters 3. Data Clustering and Experimental Analysis Using SOM The data source selected for our approach is the single-site [11] Web traffic data generated by the Analog Web access log file analyser [16]. It is a usual practice to embed some Web trackers to analyse the user access logs. After browsing through some of the features of the best trackers available [14][16][3][2] it is easy to conclude that rather than generating statistical data and texts they really don t help to find much meaningful information. Data Pre-processing In this research, we used the data from 17 February 2002 to 07 July Selecting the useful data is an important task in the data pre-processing block. After some preliminary analysis, we selected the statistical data comprising of domain byte requests, hourly page requests and daily page requests to generate the cluster models for finding Web users usage patterns. It is also important to remove irrelevant noisy data in order to build a precise model. Further, the datasets were scaled to 0-1. Besides the two inputs, volume of requests and volume of pages (bytes), we also included an additional input index number to distinguish the time sequence of the data. The most recently accessed data were indexed higher while the least recently accessed data were placed at the bottom [17]. Data Clustering Using SOM During working days, there are millions of user requests with different interests from different countries all around the world. The huge volume and the dynamic nature are two important features of the Web data, which invokes technology challenge to create appropriate Web mining models. SOM allows different data to be grouped together based on some similar characteristics. 4

5 The SOM algorithm for forming the Web usage or access patterns map is given below: Initialise the weight w, neighbourhood size N m (0) and parameter functions ij α(t) and σ 2 ( t). Select the training term vector x i at random for the input layer and calculate the similarity (distance) d of this input to the weight w of each node j. d j = x w j = n i= 1 2 ( x w ) (1) i ij Select the node with the minimum distance as the winner vector m Update the weights connecting the input layer to the winning node and its neighbouring nodes by the learning rule w ( t + 1) = w ( t) + c x w ( t) (2) ij ij [ ] i ij where c = α( t)exp( r r / σ 2 ( t)) for all nodes j in N m (t) i m Repeat steps 2-4 by increasing t by 1 at a time and decreasing the neighbourhood size, α(t) and σ 2 ( t) until the weights are stabilised. Map each term to a node on the SOM. Label each winning nodes with an associated text term. A 2D map of Web usage patterns with different clusters are formed after the training process. The related transaction entries are grouped into the same cluster and the relationship between different clusters is explicitly shown on the map. We used the Viscovery SOMine [4] to simulate the SOM. All the records were processed using SOM and the clustering results were obtained after the unsupervised learning process. We adopted a trial and error approach by comparing the normalised distortion and quantization error to decide the various parameter settings of the SOM algorithm. We finally decided the parameter setting, which could minimise both normalised distortion and quantization errors. The obtained 2D cluster map showing five different clusters according to the country of origin (domain) is shown in Figure. Cluster Cluster 4 Cluster 3 Cluster 1 Cluster 2 Figure. Clustering results of daily number of requests according to the domains

6 3.1 WUDA to Discover Domain Patterns Figure 6 depicts the details of the number of requests allocated to each cluster (total clusters) according to country (domain) of origin. For clarity purpose, we have used logarithmic scale in the Y-axis. Figure 6. WUDA: Daily number of requests according to the domain of origin As evident from Figure 6, clusters 4 and separated out from the rest of the others (maximum number of requests) with a very few nodes compared to others and spread along all the time. For clusters 1, 2 and 3, the patterns are very similar with a bit of difficulty to identify each other. The domain analysis reveals that cluster contains only Australian domains and cluster 4 accounts only *.com and *.net users. This is because majority of the requests originated from Australian domains (60%) followed by *.com and *.net users (8%)users. The remaining 3 clusters were shared by users from different domains depending on the volume of requests and volume of pages (bytes). As depicted in Figure 7, more useful information is available from clusters 1, 2 and 3 with reference to the time of accessing the Web site. Figure 7. WUDA: Daily number of requests with reference to the time of access Even though clusters 1, 2 and 3 have very similar patterns for the requests, the time of access is separated very clearly as shown in Figure 7. Cluster 2 accounts for the most recent visitors and cluster 3 represents the least recent visitors. Cluster 1 accounts for the users, which were not covered by Clusters 2 and 3. 6

7 3.2 WUDA to Discover Request Patterns The training process generated 4 clusters for the hourly number of requests. The developed clusters are illustrated in Figure 8. The hour of day the request was made is indicated in each cluster Cluster Cluster 10 2 Cluste 21 r Cluster Cluster Figure 8. Hourly requests clusters using SOM From the developed clusters as depicted in Figure 8, it is very difficult to tell the difference between each cluster, as the requests (according to the hour) are scattered. From Figures 9 (a) and (b) it may be concluded that clusters 2 and 3 have much higher requests and pages (nearly double) than clusters 1 and 4. (a) Number of hourly page requests (b) Number of hourly requests Value Figure 9. WUDA: Comparison of hourly page traffic volume and page requests Figure 10 depicts the clustering results of the data focusing on the hourly trends of the traffic. Clusters 2 and 3 are mainly responsible for the traffic during the office hours (00:09 18:00) and clusters 1 and 4 account for the traffic during the University off peak hours. It is interesting to note that the access patterns for each hour could be analysed from the cluster results with reasonable classification features. 7

8 Figure 10. WUDA: Hourly traffic patterns 3.3 WUDA to Discover Daily Requests Clusters Due to the dynamic nature of the WWW it is difficult to understand the daily traffic pattern using conventional Web log analysers. We attempted to cluster the data depending on the total activity for each day of the week using volume of daily requests, pages and index value as input features. The training process using SOM generated 7 clusters and the developed 2D map is shown in Figure 11. Wed Thu Fri Mon Tue Wed Mon Sat Sat Sun Sun Sun Sun Tue on Wed Wed Sat Su Sun Sat Wed Thu Tue Tue Thu Fri Thu ue Thu Sat Sa Cluster 1 Tue Cluster 2 Mon Wed Tue Thu Fri Mon Fri Sat on Thu Cluster 6 Mon Sun Tue Wed Wed Mon Su Mon Tue Fri Thu Sun Wed Fri Sat Fri Su Fri Thu Mon Wed Thu Sat Sat Mon Fri Thu Sun Tue Tue Fri Wed Fri Wed Sun Sa Thu Mon Tue Sat Cluster 4 Fri Mon Wed Thu ri Sun Su Thu Tue Wed Thu Sat Mon Mon Fri Tue Tue Wed Sun Cluster 3 F Mon Tue Fri Sun Cluster Thu Sat Mon hu Wed Fri Tue Fri Sun Sun F Cluster 7 Mon Sat Tue Wed Thu Sat Wed Wed on MonTue Thu Thu Fri Tue Wed Sat Sun Sat Sun Sat Sa Figure 11. Developed daily request clusters showing the days WUDA reveals that the clusters are separated according to the time of access. Each cluster accounted the requests / access during a certain period as shown in Figure 12. The ranking of the clusters are ordered as 2, 6, 1, 4, 3, 7 and according to the descending order of the access time. Further analysis of the daily records in each cluster, also reveals some interesting patterns as illustrated in Figure 13. Clusters 3 and 6 accounts for access records, which happened during Saturday and Sunday. Cluster 1 was separated as it only covered the first few weekdays (mostly from Monday to Thursday). While, the biggest group of clusters consist of 2, 4, and 7 accounted for the transactions during Monday to Friday. 8

9 Figure 12. WUDA: Cluster ranking depending on the time of access Figure 13. WUDA: Cluster ranking depending on the day of the week 4. Fuzzy Inference Systems The world of information is surrounded by uncertainty and imprecision. The human reasoning process can handle inexact, uncertain and vague concepts in an appropriate manner. Usually, the human thinking, reasoning and perception process cannot be expressed precisely. These types of experiences can rarely be expressed or measured using statistical or probability theory. Fuzzy logic provides a framework to model uncertainty, human way of thinking, reasoning and the perception process. Fuzzy ifthen rules and fuzzy reasoning are the backbone of fuzzy inference systems, which are the most important modelling tools based on fuzzy set theory. Fuzzy modelling can be pursued using the following steps. Select relevant input and output variables. Determine the number of linguistic terms associated with each input/output variables. Also choose the appropriate family of parameterized membership functions, fuzzy operators, reasoning mechanism etc. Choose a specific type of fuzzy inference system Design a collection of fuzzy if-then rules (knowledge base) 9

10 Figure 14. Takagi Sugeno fuzzy inference system using a min or product as T-norm operator We made use of the Takagi Sugeno fuzzy inference scheme in which the conclusion of a fuzzy rule is constituted by a weighted linear combination of the crisp inputs rather than a fuzzy set []. A basic Takagi-Sugeno fuzzy inference system is illustrated in Figure 14 and the if-then rules has the following structure if xis A1 and y is B1, then z1 = p1x+ q1y + r1 (3) where p 1, q 1 and r 1 are linear parameters. A conventional FIS makes use of a model of the expert who is in a position to specify the most important properties of the process. Expert knowledge is often the main source to design FIS. According to the performance measure of the problem environment, the membership functions, rule bases and the inference mechanism are to be adapted. Evolutionary computation [19] and neural network learning techniques are used to adapt the various fuzzy parameters. Recently, a combination of evolutionary computation and neural network learning has also been investigated [18]. In this research, we used the Adaptive Neuro-Fuzzy Inference System (ANFIS) [13] framework based on neural network learning to fine tune the rule antecedent parameters and a least mean squares estimation to adapt the rule consequent parameters of the TSFIS. A step in the learning procedure has two parts: In the first part the input patterns are propagated, and the optimal conclusion parameters are estimated by an iterative least mean square procedure, while the antecedent parameters (membership functions) are assumed to be fixed for the current cycle through the training set. In the second part the patterns are propagated again, and in this epoch, back propagation is used to modify the antecedent parameters, while the conclusion parameters remain fixed. Please refer to [13] for more details. Design and Experimentation Results We used the popular grid partitioning method (clustering) to generate the initial rule base. This partition strategy requires only a small number of membership functions for each input. The technique is illustrated in Figure 1. 10

11 Figure 1. Example showing how the 2 dimensional spaces are partitioned using 3 trapezoidal membership functions per input dimension. A simple if-then rule will appear as If input-1 is medium and input 2 is large then rule R 8 is fired. Besides the inputs, volume of requests and volume of pages (bytes) and index number, we also used the cluster location information provided by the SOM output. The data was re-indexed based on the cluster information. We attempted to develop fuzzy inference models to predict (few time steps ahead) the Web traffic volume on a hourly and daily basis. We used the data from 17 February 2002 to 30 June 2002 for training and the data from 01 July 2002 to 06 July 2002 for testing and validation purposes. Daily Traffic Prediction We used the MATLAB environment to simulate the various experiments. Given the daily traffic volume of a particular day the developed model could predict the traffic volume up to five days ahead. Three 3 membership functions were assigned to each input variable. 81 fuzzy if-then rules were generated using the grid based partitioning method and the rule antecedent/consequent parameters were learned after 0 epochs. We also investigated the daily web traffic prediction performance without the cluster information input variable. Table 1 summarizes the performance of the fuzzy inference system for training and test data. Table 1. Training and test performance for Web traffic volume forecast Root Mean Squared Error (RMSE) Forecast period Fuzzy Inference System (with cluster information) Fuzzy Inference System (without cluster information) Training Test Training Test One day Two days Three days Four days Five days Figures 16 (a), (b), (c), (d) and (e) depicts the test results for one day, two days, three days, four days and five days ahead forecast of daily Web traffic volume. 11

12 (a) (b) (c) (d) (e) Figure 16 (a) (e). Test results of daily forecast of Web traffic volume Hourly Page Request Forecast Three membership functions were assigned to each input variable. 81 fuzzy if-then rules were generated using the grid based partitioning method and the rule antecedent/consequent parameters were learned after 40 epochs. We also investigated the volume of hourly page requests prediction performance without the cluster information input variable. Table 2 summarizes the performance of the FIS for training and test data. Figures 17 (a), (b) and (c) illustrates the test results for 1 hour, 12 hours and 24 hours ahead forecast of the volume of hourly page requests. 12

13 Table 2. Training and test performance for volume of hourly page requests forecast Root Mean Squared Error (RMSE) Forecast period Fuzzy Inference System (with cluster information) Fuzzy Inference System (without cluster information) Training Test Training Test 1 hour hours hours (a) (b) (c) Figure 17 (a) (c). Test results of hourly forecast of volume of page requests. Conclusions and Future Work The discovery of useful knowledge, user information and access patterns allows Web based organisations to predict user access patterns and helps in future developments, maintenance planning and also to target more rigorous advertising campaigns aimed at groups of users [1]. This case study on Monash University s Web access patterns reveals the necessity to incorporate computational intelligence techniques for mining useful information. WUDA of the SOM data clusters provided several useful information related to the user access patterns. The developed FIS could predict the daily Web traffic and hourly page requests within reasonable error limits. Our experiment results also reveal the importance of the cluster information to improve the forecast accuracy of the FIS. These techniques might be useful to the website tracker software vendors to provide more useful information to the users. We relied on the statistical/text data provided by the Analog [16] software embedded at the University s Web server [10]. Analog generates the statistical data 13

14 by analysing the access data logs. Due to incomplete details, we had to analyse the usage patterns for different aspects separately, preventing us to link some common information between the different aspects, trends, patterns etc. For example, the domain requests and the daily or hourly requests are all stand-alone information and are not interconnected. Therefore, a direct analysis of the Web access logs might be more helpful. We believe that if the detailed access information could cover different interlinked features, then the usage patterns would be more comprehensive and useful. Even the access or usage patterns for particular domain within a particular time period can be analysed and predicted for marketing segment analysis and so on. In this research, we considered only the Web traffic data during the University s peak working time. Our future research will also incorporate off-peak months (summer semesters) and so on. We also plan to incorporate more data mining techniques and improve the functional aspects of the concurrent neuro-fuzzy approach. References [1] Zhang Y.Q. and Lin T.Y., Computational Web Intelligence (CWI): Synergy of Computational Intelligence and Web Technology, 2002 World Congress in Computational Intelligence, IEEE Press, pp.72-7, [2] WebSTAT Web Traffic Analyser, < (accessed on 27 July 2002) [3] Website Tracker, < (accessed on 27 July 2002) [4] Viscovery SOMine, (accessed on 27 July 2002) [] Sugeno M., Industrial Applications of Fuzzy Control, Elsevier Science Pub Co., 198. [6] Srivastava, J., Cooley R., Deshpande M.and Tan P.N., Web Usage Mining: Discovery and Applications of Usage Patterns from Web Data, SIGKDD Explorations, 1(2): pp , [7] Pirolli P, Pitkow J, and Rao R., Silk From a Sow s Ear: Extracting Usable Structures from the Web, In Proceeding on Human Factors in Computing Systems (CHI-96), [8] Pal S.K., Talwar V., and Mitra P., Web Mining in Soft Computing Framework: Relevance, State of the Art and Future Directions. IEEE Transactions on Neural Networks, [9] Ng A. and Smith K. A., Web usage mining by a self-organizing map, C. Dagli et al. (Eds.), Smart Engineering System Design: Neural Networks, Fuzzy Logic, Evolutionary Programming, Data Mining and Complex Systems, ASME Press, vol. 10, pp , [10] Monash University Weekly Website User Access Statistics < (accessed on 27 July 2002) [11] Monash University Server Usage Statistics. < (accessed on 27 July 2002) [12] Kohenen T., Self-Organizing Maps, Springer Verlag Germany, 199. [13] Jang R., Neuro-Fuzzy Modeling: Architectures, Analyses and Applications, PhD Thesis, University of California, Berkeley, [14] Hitbox Central Web Traffic Analysis, < (accessed on 27 July 2002) [1] Cooley R., Srivastava J., and Mobasher B., Web Mining: Information and Pattern Discovery on the World Wide Web. In Proceedings of the 9th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'97) [16] Analog Website Tracker, < (accessed on 27 July 2002) [17] Aggarwal C., Wolf J.L., and Yu P.S., Caching on the World Wide Web, IEEE Trans. On Knowledge and Data Eng., vol. 11, no. 1, pp , Feb [18] Abraham A., Neuro-Fuzzy Systems: State-of-the-Art Modeling Techniques, Connectionist Models of Neurons, Learning Processes, and Artificial Intelligence, Springer-Verlag Germany, Jose Mira and Alberto Prieto (Eds.), Granada, Spain, pp , [19] Abraham A. and Nath B., Evolutionary Design of Fuzzy Control Systems - An Hybrid Approach, In Proceedings of The Sixth International Conference on Control, Automation, Robotics and Vision, (CD ROM Proceeding), Wang J.L. (Ed.), ISBN , Singapore,