Time Series Predicion of Web Domain Visis by IF-Inference Sysem VLADIMÍR OLEJ, JANA FILIPOVÁ, PETR HÁJEK Insiue of Sysem Engineering and Informaics Faculy of Economics and Adminisraion Universiy of Pardubice, Sudenská 84, 53210 Pardubice CZECH REPUBLIC vladimir.olej@upce.cz, jana.filipova@upce.cz, per.hajek@upce.cz Absrac: - This paper presens basic noions of web mining and fuzzy inference sysems based on he Takagi-Sugeno fuzzy model. On he basis of his fuzzy inference sysem and IF-ses inroduced by K.T. Aanassov novel IF-inference sysems can be designed. Thus, an IF-inference sysem is developed for ime series predicion. In he nex par of he paper we describe web domain visis predicion by IF-inference sysems and he analysis of he resuls. Key-Words: - Web mining, fuzzy inference sysem, IF-inference sysem, web domain visis, predicion, and ime series. 1 Inroducion Web serves, wih developmen of e-business and e- governmen, as a source of informaion for web mining. Web mining [1] includes far more han common saisic overviews displaying for example insananeous number of visiors or mos ofen visied web pages. Web mining analyses: Where he visiors come from. How do various visiors behave. Wha are ypical sequences of pages walkhroughs. Which sequence leads o a purchase or reservaion. How long visiors say a pages. How and from where hey leave pages. These are daa mainly abou user behaviour on he Inerne. They can be creaed as a rack of user on web page or applicaion. Tha means ha all seps and aribues of he user are recorded. These daa are kep in log files. Based on informaion saed we can idenify and filer auo-generaed visis of full-ex seekers, which are quie common and disor user behaviour saisics. Currenly here are a number of mehods which serve for modelling of daa obained by web mining. These mehods can be divided o mehods of modelling wih learning wihou a eacher and learning wih a eacher. The mos common mehod of daa modelling wih learning wihou a eacher for exploring user profiles from log files is clusering. Cluser algorihms used in he area of web mining are K-means, Rough K-means or C-means, Fuzzy c rimmed mehods and Fuzzy-c medians for fuzzy clusering, or An-Based Clusering (ABC). Oher mehods of learning wihou eacher are associaive and fuzzy associaive rules. Wihin daa modelling mehods wih learning wih a eacher, fuzzy logic, neural neworks and machine learning are used for classificaion and daa predicion from log files. Semi-supervised learning in combinaion wih Rough ses and Kohonen s self-organizing feaure maps are used for classificaion of web pages. Exracion of redundan aribues is solved by ransformaion of Singular Value Decomposiion. For cluser analysis we seleced ABC and o deermine rend we used geneic linear programming. For predicion in he area of web mining we use fuzzy inference sysem (FIS) Takagi- Sugeno, Suppor Vecore Machines and feed-forward neural neworks. In general, classificaion and predicion can be realized by FISs. Based on general FIS srucure, we can design wo basic ypes - Mamdani ype and Takagi- Sugeno ype. Boh he FISs [2] ypes differ by means of obaining he oupu. Differen oupu formulaion resuls in differen if-hen rules consrucion. These rules can be designed by user (based on his experience), or he user can obain hem hrough exracion from hisorical daa. Fuzzificaion of inpu variables and applicaion of operaors in if-hen rules are he same in boh FISs ypes. A his ime here are several generalizaions of fuzzy se heory for various objecives. Inuiionisic fuzzy ses (IF-ses) heory represens one of he generalizaions, he noion inroduced by K.T. Aanassov [3,4]. The concep of IF-ses can be viewed as an alernaive approach o define a fuzzy se in cases where available informaion is no sufficien for he definiion of an imprecise concep by means of a convenional fuzzy se. In his paper we will presen IF-ses as a ool for reasoning in he presence of imperfec fac and imprecise knowledge. The IF-ses are for example also suiable for he ime series predicion of web domain visis as hey provide a good descripion of objec aribues by means of membership ISSN: 1792-4251 156 ISBN: 978-960-474-201-1
funcions µ and non-membership funcions ν. They also presen a srong possibiliy o express uncerainy. In he paper we presen problem formulaion wih he aim o describe he ime series of he Universiy of Pardubice web presenaion visis and possibiliies of is pre-processing and basic noions of Takagi-Sugeno ype FIS for ime series predicion. Based on he FIS defined his way and he basic noions of IF-ses, we define a novel IF-inference sysem. Furher, he paper includes he comparison of he predicion resuls obained by he FIS characerized by membership funcions µ, by he FIS characerized by non-membership funcions ν, and by he IF-inference sysem. The comparison of designed sysems is realized for he saed ime series of he Universiy of Pardubice, he Czech Republic (web upce.cz) web presenaion visis from Sepember 2008 o June 2009. 2 Problem Formulaion The daa for predicion of he ime series of he Universiy of Pardubice, he Czech Republic (web upce.cz) web presenaion visis over given ime period was obained from Google Analyics. This web mining ool, by means of Java Scrip code implemened o web presenaion, offers a wide specrum of operaion characerisics (web merics). Merics provided by Google Analyics can be divided o following secions: Visis; Sources of access; Conens; Conversion. In secion Visis we can monior for example number of visiors, number of visis and number of pages viewed, raio of new and reurned visiors. Indicaor geolocaion, i.e. which counry visiors are mos ofen from, is needed o be known for example because of language muaions. In order o predic visi rae of he Universiy of Pardubice, he Czech Republic (web upce.cz) web presenaion we need o monior indicaor number of visis. The number of visis is a basic indicaor which displays he number of visis wihin given ime period. A Visi is comprehended as an unrepeaed combinaion of IP address and cookies. A sub-merics is absoluely unique visi, which is defined by unrepeaable IP address and cookies in given ime period. Basic informaion obainable from Google Analyics abou web upce.cz during May 2009, is following: Toal visi rae during given monh drops. Clear rend is obvious here, when Monday has he highes visi rae which keeps dropping unil he end of week; Saurday has he lowes visi rae. Average number of pages visied is more han hree. Visior says on cerain page five and half a minue in average. Bounce rae is approximaely 60 %. Visiors come mainly direcly, which is good. Among he mos favourie pages here is he main page, hen he pages of Faculy of Economic and Adminisraion and Faculy of Philosophy. Measuring of visi rae of he Universiy of Pardubice web presenaion, web.upce.cz, akes place in regular equidisan ime periods. I represens ime series which is given by succession of maerial and area comparable observaions, which are ordered by ime. Pre-processing of daa is realized by means of simple mahemaicsaisic mehods [5]. The generally saed mehod is represened by a funcion, which for each period of ime, akes cerain real value dependan on empirically deermined values. Primary and pre-processed ime series of visi rae of he Universiy of Pardubice web presenaion, for example simple moving average (SMA), is presened in Fig. 1. Fig. 1 The pre-processing of web upce.cz visis by SMA The general formulaion of he model of predicion of upce.cz web visi rae can be saed in his manner y=f(x 1,x 2,,x m ), m=5, where y is daily web upce.cz visis in +1, x 1 is SMA, x 2 is cenral moving average (CMA), x 3 is moving median (MM), x 4 is simple exponenial smoohing (SES) and x 5 is double exponenial smoohing (DES) a ime. 3 Basic Noions of he Fuzzy Inference Sysems General srucure of FIS [2,5] conains a fuzzificaion process of inpu variables by membership funcions µ, he design of he base of if-hen rules or auomaic ifhen rules exracion from inpu daa, operaors (AND,OR,NOT) applicaion in if-hen rules, implicaion and aggregaion wihin hese rules, and he process of defuzzificaion of gained values o crisp values. In he process of defuzzificaion, sandardizaion of inpus and heir ransformaion o he domain of he values of membership funcions µ akes place. There is no general mehod for designing shape, number and parameers of inpu and oupu membership funcions µ. The inpu o fuzzificaion process is a crisp value ISSN: 1792-4251 157 ISBN: 978-960-474-201-1
given by he universum (reference se). The oupu of fuzzificaion process is he membership funcion µ value. The design of he base of if-hen rules can be realized by exracion of if-hen rules from hisorical daa, provided ha hey are available. In [2,5] here are menioned opimizaion mehods of he number of ifhen rules. The bes resuls are obained by he mehod based on he opimizaion of he form of oupu membership funcions µ based on inpu daa. The membership funcions µ of oupu variables are assigned o inpu daa so ha he membership funcions µ of he i- h inpu and oupu are equal. If inpu daa do no form a convex funcion, a new membership funcion µ is creaed for he daa. Each addiional membership funcion µ causes addiion of a new if-hen rule o base rules. Advanage of his mehod is ha i generaes only a small number of if-hen rules. Their number reduces he compuing inensiy of FIS, which enables us o design FIS capable of working in real ime. Anoher opion of deriving base rules from hisorical daa is he so-called Adapive Neuro-Fuzzy Inference Sysem mehod [2,5]. The principle of his mehod is a neuro-adapive learning process, based on which i is possible o derive membership funcion µ parameers from hisorical daa and o exrac base rules. This mehod helps o assign inpu daa o oupu daa, while in he process of learning he parameers of individual membership funcions µ are gradually changed in order o characerize he relaions beween he range of inpu variables and he range of oupu variables in he bes way. In he process of learning, i is possible o use he mehod of regressive error spread, he combinaion of he mehod of regressive error spread and minimal squares mehod, or evoluionary sochasic opimizaion algorihms. The base of rules consiss of if-hen rules. These rules are used for creaing predicae clauses represening he base of FIS. Le x 1,x 2,...,x j,...,x m be inpu variables defined on reference ses X 1,X 2,...,X j,...,x m and le y be an oupu variable defined on reference se Y. Then FIS has m inpu variables and one oupu variable. Furher, each se X j, j=1,2,...,m, can be divided ino i=1,2,...,n fuzzy ses µ j,1 (x),µ j,2(x),...,µ j,i(x),...,µ j,n(x). (1) Individual fuzzy ses represen assignmen of linguisic variables values, which are relaed o ses X j. Then he k- h if-hen rule R k in FIS Takagi-Sugeno ype can be wrien down in he following form R k : if x 1 is A 1,i(1,k) AND x 2 is A 2,i(2,k) AND... AND x j is A j,i(j,k) AND... AND x m is A m,i(m,k) hen y=h, (2) j=1,2,...,m; i=1,2,...,n, where A 1,i(1,k),A 2,i(2,k),...,A j,i(j,k),...,a m,i(m,k), represen he values of linguisic variable and h is consan. Fuzzy inference sysem composed of if-hen rules defined by relaion (2) is called a zero order Takagi-Sugeno ype FIS. If he k-h if-hen rule R k in Takagi-Sugeno ype FIS is in form R k : if x 1 is A 1,i(1,k) AND x 2 is A 2,i(2,k) AND... AND x j is A j,i(j,k) AND... AND x m is A m,i(m,k) hen y=f(x 1,x 2,...,x m ), (3) where f(x 1,x 2,...,x m ) is a linear funcion, hen he FIS consising of if-hen rules R k, k=1,2,,n, defined by relaion (3) is called a firs order Takagi-Sugeno ype FIS. In case ha f(x 1,x 2,...,x m ) is a polynomial funcion, i is called a second order Takagi-Sugeno ype FIS. Operaor AND beween elemens of wo fuzzy ses (A 1 AND A 2 ) can be generalized by -norm [2,5]. Analogous o operaor AND, operaor OR beween elemens of wo fuzzy ses (A 1 OR A 2 ) can be generalized by s-norm [2,5]. The Takagi-Sugeno ype FIS was designed in order o achieve higher compuaional effeciveness. This is possible as he defuzzificaion of oupus is no necessary. Is advanage lies also in involving he funcional dependencies of oupu variable on inpu variables. The oupu level y k of each he k-h if-hen rule R k is weighed by he w k =µ(x 1 ) AND µ(x 2 ) AND AND µ(x m ). The final oupu y of he Takagi-Sugeno ype FIS is he weighed average of all N if-hen rule R k oupus y k, k=1,2,,n, compued as follows N y k w k k = 1. y = N w k k = 1 (4) 4 IF-Inference Sysems Design The concep of IF-ses is he generalizaion of he concep of fuzzy ses, he noion inroduced by L.A. Zadeh. The heory of IF-ses is well suied o deal wih vagueness. Recenly, in his conex, IF-ses have been used for inuiionisic classificaion [6] and predicion [7] models which can accommodae imprecise informaion. Le a se X be a non-empy fixed se. An IF-se A in X is an objec having he form [3,4] A = { x, µ Α (x), ν Α (x) x X}, (5) where he funcion µ Α :X [0,1] defines he degree of membership funcion µ Α (x) and he funcion ν Α :X [0,1] defines he degree of non-membership funcion ν Α (x), respecively, of he elemen x X o he se A, which is a subse of X, and A X, respecively; ISSN: 1792-4251 158 ISBN: 978-960-474-201-1
moreover for every x X, 0 µ Α (x) + ν Α (x) 1, x X mus hold. The amoun π Α (x) = 1 (µ Α (x) + ν Α (x)) (6) is called he hesiaion par, which may caer o eiher membership value or non-membership value, or boh. For each IF-se in X, we will call π Α (x) = =1 (µ Α (x)+ν Α (x)) as he inuiionisic index of he elemen x in se A. I is a hesiancy degree of x o A. I is obvious ha 0 π Α (x) 1 for each x X. The value denoes a measure of non-deerminancy. The inuiionisic indices π Α (x) are such ha he larger π Α (x) he higher a hesiaion margin of he decision maker. Inuiionisic indices allow us o calculae he bes final resuls (and he wors one) we can expec in a process leading o a final opimal decision. Nex we define an accuracy funcion H o evaluae he degree of accuracy of IF-se by he form H(A) = µ Α (x) + ν Α (x), where H(A) [0,1]. From he definiion H, i can be also expressed as follows H(A) = µ Α (x) + ν Α (x) = 1 π Α (x). The larger value of H(A), he more he degree of accuracy of he IF-se A. Le here exiss a general IF-sysem defined in [8]. Then i is possible o define is oupu y η as y η = (1 π Α (x)) y µ + π Α (x) y ν, (7) where y µ is he oupu of he FIS µ using he membership funcion µ Α (x), y ν is he oupu of he FIS ν using he nonmembership funcion ν Α (x). In he process of defuzzificaion, sandardizaion of inpus and heir ransformaion o he domain of he values of membership funcions µ akes place. There is no general mehod for designing shape, number and parameers of inpu and oupu membership funcions µ. Then, based on equaion (7), i is possible o design he IF-inference sysem of Takagi-Sugeno ype is presened in Fig. 2. (1 π Α (x)) y µ FIS µ x 1 x 2 x m x 1 x 2 x m Fig. 2 IF-inference sysem For he IF-inference sysem designed in his way, i holds: Σ y η FIS ν π Α (x) y ν If inuiionisic index π Α (x)=0, hen he oupu of IF-inference sysem y η =(1 π Α (x)) y µ (Takagi- Sugeno ype FIS characerized by membership funcion µ). If inuiionisic index π Α (x)=1, hen he oupu of IF-inference sysem y η =π Α (x) y ν (Takagi-Sugeno ype FIS, characerized by non-membership funcion ν). If inuiionisic index 0< π Α (x) <1, hen he oupu of IF-inference sysem y η = (1 π Α (x)) y µ + π Α (x) y ν, and i is characerized by membership funcion µ and non-membership funcion ν. Le x 1,x 2,...,x j,...,x m be inpu variables FIS η defined on reference ses X 1,X 2,...,X j,...,x m and le y η be an oupu variable defined on reference se Y. Then FIS η has m inpu variables x 1,x 2,...,x j,...,x m and one oupu variable y η, where η=µ are membership funcions (η=ν are non-membership funcions). Furher, each se X j, j=1,2,...,m, can be divided ino i=1,2,...,n fuzzy ses which are represened by following way η j,1 (x),η j,2 (x),...,η j,i (x),...,η j,n (x). (8) Individual fuzzy ses, where η=µ are membership funcions (η=ν are non-membership funcions) represen mapping of linguisic variables values, which are relaed o ses X j. Then he k-h if-hen rule R k in FIS η can be defined as follows R k : if x 1 is A 1,i(1,k) η AND x 2 is A 2,i(2,k) η AND... AND x j is A j,i(j,k) η AND... AND x m is A m,i(m,k) η hen y η =h, or y η =f(x 1,x 2,...,x m ), j=1,2,...,m; i=1,2,...,n, (9) where A 1,i(1,k) η,a 2,i(2,k) η,...,a j,i(j,k) η,...,a m,i(m,k) η represen he values of linguisic variable for FIS µ and FIS ν, h is consan, f(x 1,x 2,...,x m ) is a linear or polynomial funcion. The oupu y µ of FIS µ (he oupu y ν of FIS ν ) is defined in he same way as presened in equaion (4). 5 Modelling and Analysis of he Resuls Based on he fac ha he model for predicion of upce.cz web visi rae is formulaed as follows y=f(x 1,x 2,,x m ), m=5, where y is daily web upce.cz visis in +1, x 1 is SMA, x 2 is CMA, x 3 is MM, x 4 is SES and x 5 is DES a ime. I is possible o design an inpu membership funcion µ for FIS µ and inpu nonmembership funcions ν for FIS ν as follows. Inpu language variable SMA is represened by means of four membership funcions. They are bell membership funcions. Individual membership funcions are described by means of language variable value: low SMA, med low SMA, med high SMA and high SMA. Membership funcions of language variable SMA for ISSN: 1792-4251 159 ISBN: 978-960-474-201-1
model of upce.cz web visi rae predicion are presened in Fig. 3 and non-membership funcions are presened in Fig. 4. Oher membership and non-membership language variable funcions are designed analogically (CMA, MM, SES, DES). Membership funcion µ and nonmembership funcion ν, and if-hen rules were designed using subsracive clusering algorihm [2,5]. To be specific, wo if-hen rules are designed for FIS µ and FIS ν respecively. The oupu level y k of each of he k-h if-hen rule R k is weighed. The final oupus y µ and y ν of he FIS µ and FIS ν are he weighed averages of all he if-hen rule R k oupus y k, k=1,2,,n. The oupu of IF-inference sysem is represened by he prediced value y η in ime +1. Fig. 5 The resuls of web upce.cz visis predicion for esing daa O es Fig. 6 The resuls of web upce.cz visis predicion for esing daa O es Fig. 3 Inpu membership funcions µ for SMA of FIS µ Fig. 4 Inpu non-membership funcions ν for SMA of FIS ν The resuls of visi rae of he web upce.cz visis on esing daa O es for µ max =0.6 for inuiionisic index π=0.3 are presened in Fig. 5 and Fig. 6, where µ max represens he maximum value of inpu membership funcions µ. The Table 1 and Table 2 (on esing daa O es ) shows he qualiy of web upce.cz visis predicion represened by Roo Mean Squared Error (RMSE) for differen values of µ max and differen values of inuiionisic index π. The resuls show ha he RMSE µ is for FIS µ consan. The size of µ max does no affec he resuling error of FIS µ. This resuls from he fac ha he oupu y µ is a weighed average of oupus y k from he single if-hen rules R k. Relaive weighs w k remains he same for differen values of µ max. Maximum RMSE is obained for he FIS ν and he FIS η, for which ν min =0 holds, i.e. µ max +π=1. Therefore, non-membership funcions ν limied in his way are no suiable for he used daa. Table 1 RMSE on esing daa O es for differen values of µ max and inuiionisic index π π=0.1 µ max 0.5 0.6 0.7 0.8 0.9 RMSE µ 0.237 0.237 0.237 0.237 0.237 RMSE ν 0.297 0.309 0.325 0.352 0.407 RMSE η 0.239 0.240 0.240 0.242 0.245 π=0.2 µ max 0.5 0.6 0.7 0.8 RMSE µ 0.237 0.237 0.237 0.237 RMSE ν 0.304 0.320 0.348 0.407 RMSE η 0.243 0.245 0.248 0.256 π=0.3 µ max 0.5 0.6 0.7 RMSE µ 0.237 0.237 0.237 RMSE ν 0.314 0.343 0.407 RMSE η 0.250 0.255 0.269 π=0.4 µ max 0.5 0.6 RMSE µ 0.237 0.237 RMSE ν 0.337 0.407 RMSE η 0.263 0.285 ISSN: 1792-4251 160 ISBN: 978-960-474-201-1
Table 2 RMSE on esing daa O es for differen values of µ max and inuiionisic index π π=0.5 µ max 0.1 0.2 0.3 0.4 0.5 RMSE µ 0.237 0.237 0.237 0.237 0.237 RMSE ν 0.278 0.286 0.301 0.329 0.407 RMSE η 0.250 0.253 0.259 0.270 0.302 π=0.6 µ max 0.1 0.2 0.3 0.4 RMSE µ 0.237 0.237 0.237 0.237 RMSE ν 0.279 0.292 0.320 0.407 RMSE η 0.255 0.262 0.275 0.321 π=0.7 µ max 0.1 0.2 0.3 RMSE µ 0.237 0.237 0.237 RMSE ν 0.282 0.308 0.407 RMSE η 0.262 0.278 0.341 π=0.8 µ max 0.1 0.2 RMSE µ 0.237 0.237 RMSE ν 0.292 0.407 RMSE η 0.276 0.362 π=0.9 µ max 0.1 RMSE µ 0.237 RMSE ν 0.407 RMSE η 0.384 The RMSE for FIS ν and FIS η increases wih a higher inuiionisic index π, i.e. wih a higher uncerainy. The resuls, however, show ha i is possible o achieve a relaively low level of RMSE (on raining and esing daa) even in he cases where i is no possible o deermine he membership funcions µ and nonmembership funcions ν unambiguously (i.e. for high inuiionisic index π). 6 Conclusion The model based on IF-ses as a model for web mining is designed in he paper as hey allow processing uncerainy and he exper knowledge. IF-ses can be viewed in he conex as a proper ool for represening hesiancy concerning boh membership and nonmembership of an elemen o a se. The IF-inference sysem FIS η defined his way works more effecive han he sandard of Takagi-Sugeno ype FIS µ as i provides sronger possibiliy o accommodae imprecise informaion and beer model imperfec fac and imprecise knowledge. In his sudy we presen a novel approach o imes series predicion based on he exension of Takagi- Sugeno ype FIS µ which is characerized by membership funcion µ wih Takagi-Sugeno ype FIS ν which is characerized by non-membership funcion ν. The cenral poin in he design of IF-inference sysem lies in he inuiionisic index π expressing he level of uncerainy. The designed IF-inference sysem represens an efficien ool for modelling of ime series, which is demonsraed on he predicion of he Universiy of Pardubice visi rae predicion. Daa for web mining needs were obained from log files of upce.cz web. The model design was carried ou in Malab in MS Windows XP operaion sysem. ACKNOWLEDGMENTS This work was suppored by he scienific research projecs Minisry of Environmen, he Czech Republic under Gran No: SP/4i2/60/07 wih ile Indicaors for Valuaion and Modelling of Ineracions among Environmen, Economics and Social Relaions, Czech Science Foundaion under Gran No: 402/09/P090 wih ile Modelling of Municipal Finance by Compuaional Inelligence Mehods, and Czech Science Foundaion under Gran No: 402/08/0849 wih ile Model of Susainable Regional Developmen Managemen. References: [1] R. Cooley, B. Mobasher, J. Srivisava, Web Mining: Informaion and Paern Discovery on he World Wide Web, Proc. of he 9 h IEEE Inernaional Conference on Tools Wih Arificial Inelligence, (ICTAI 97), Newpor Beach, CA, 1997. [2] L. I. Kuncheva, Fuzzy Classifier Design, A Springer Verlag Company, Germany, 2000. [3] K. T. Aanassov, Inuiionisic Fuzzy Ses, Fuzzy Ses and Sysems, Vol.20,1986, pp.87-96. [4] K. T. Aanassov, Inuiionisic Fuzzy Ses, Springer, Berlin Heidelberg, Germany, 1999. [5] V. Olej, Modelling of Economics Processes by Compuaional Inelligence, Hradec Králové, 2003, (in Slovak). [6] V. Olej, P. Hájek, Inuiionisic Hierarchical Fuzzy Inference Sysems Design for Air Qualiy Modelling, Proc. of he 5 h Inernaional Conference on Energy, Environmen, Ecosysems and Susainable Developmen, (EEESD'09), Masorakis, N. and all. (Eds.), Vouliagmeni Beach, Greece, 2009, pp.89-94. [7] V. Olej, P. Hájek, IF-Inference Sysems Design for Predicion of Ozone Time Series, Proc of he 20 h Inernaional Conference on Arificial Neural Neworks, ICANN 10, Thessaloniki, Greece, 2010, (in Press). [8] O. Moniel, e al., Mediaive Fuzzy Logic: A new Approach for Conradicory Knowledge Managemen, Sof Compuing, Vol.20, No.3, 2008, pp.251-256. ISSN: 1792-4251 161 ISBN: 978-960-474-201-1