MUNICIPAL CREDITWORTHINESS MODELLING BY NEURAL NETWORKS

0 Acta Electrotechnica et Informatica Vol. 8, No. 4, 008, 0 5 MUNICIPAL CREDITWORTHINESS MODELLING BY NEURAL NETWORKS Petr HÁJEK, Vladimír OLEJ Institte of System Engineering and Informatics, Faclty of Economics and Administration, University of Pardbice, Stdentsá 95, 53 0 Pardbice, tel. +4 466036074, e-mail: petr.haje@pce.cz, vladimir.olej@pce.cz ABSTRACT The paper presents the design of mnicipal creditworthiness parameters. Frther, the design of model for mnicipal creditworthiness classification is presented. The realized data pre-processing maes the sitable economic interpretation of reslts possible. Mnicipalities are assigned to clsters by nspervised methods. The combination of Kohonen s self-organizing featre maps and K-means algorithm is a sitable method for mnicipal creditworthiness modelling. The nmber of classes in this model is determined by indees evalating the qality of clstering. The model is composed of Kohonen s self-organizing featre maps and fzzy logic neral networs, where the otpt of Kohonen s self-organizing featre maps represents the inpt of fzzy logic neral networs. The sitability of the designed model is compared with other fzzy logic based classifiers. The highest classification accracy is achieved by the fzzy logic neral networ. The fzzy logic neral networ s model is the best in terms of generalization ability de to this fact. By the designed model high classification accracy, low average classification error and sitable interpretation of reslts are achieved.. Keywords: mnicipal creditworthiness, Kohonen s self-organizing featre maps, fzzy logic neral networs. INTRODUCTION Mnicipal creditworthiness [8] is the ability of a mnicipality to meet its short-term and long-term financial obligations. It is determined by factors (parameters) relevant to the assessed object. High mnicipal creditworthiness shows a low credit ris, while the low one shows a high credit ris. Mnicipal creditworthiness evalation is crrently being realized by methods combining mathematical-statistical methods and epert opinion [8]. Their otpt is represented either by a score evalating the mnicipal creditworthiness (scoring systems) or by an assignment of the i-th object o i O, O={o,o,,o i,,o n } to the j-th class i,j Ω, Ω={,j,,j,, i,j,, n,j } according to their creditworthiness (rating, nspervised methods). Rating is an independent epert evalation based on comple analysis of all nown ris parameters of mnicipal creditworthiness, however, it is considered to be rather sbjective. Mnicipalities are classified into classes i,j Ω by rating-based models. The classes are assigned to the mnicipalities by rating agencies. Therefore, the methods capable of processing and learning the epert nowledge, enabling their ser to generalize and properly interpret, have proved to be most sitable. For eample, fzzy inference systems [7] and neral networs [3] are sitable for mnicipal creditworthiness evalation. Neral networs are appropriate for mnicipal creditworthiness modelling de to their ability to learn, generalize and model non-linear relations. Nevertheless, the comptational speed and robstness are retained. Mnicipal creditworthiness evalation is considered to be a problem of classification, that is, it can be realized by varios models of neral networs. They differ in their topology and learning process. Advantages of both methods can be gained by nero-fzzy systems [6]. The paper presents the design of mnicipal creditworthiness parameters. Only those parameters were selected which show low correlation dependences. Therefore, data matri P is designed where vectors p i characterize mnicipalities o i O. Frther, the paper presents the basic concepts of the Kohonen s selforganizing featre maps (SOFMs) and fzzy logic neral networs (FLNNs). The contribtion of the paper lies in the model design for mnicipal creditworthiness evalation. The model realizes the advantages of both the nspervised methods (combination of the SOFM and K- means algorithm) and FLNN. The final part of the paper incldes the analysis of the reslts.. MUNICIPAL CREDITWORTHINESS PARAMETERS DESIGN In [] common categories of parameters there are mentioned namely economic, debt, financial and administrative categories. The economic, debt and financial parameters are pivotal. Economic parameters affect long-term credit ris. The mnicipalities with more diversified economy and more favorable social and economic conditions are better prepared for the economic recession. Debt parameters inclde the size and strctre of the debt. Financial parameters inform abot the bdget implementation. Their vales are etracted from the mnicipality bdget. The design of parameters, based on previos correlation analysis [] and recommendations of notable eperts, can be realized as presented in Table. Based on the presented facts, the following data matri P can be designed P = o o i o n, i, n, where o i O, O={o,o,,o i,,o n } are objects (mnicipalities), is the -th parameter, i, is the vale, i, n, m,m i,m n,m, j i, j n, j, ISSN 335-843 008 FEI TUKE

Acta Electrotechnica et Informatica Vol. 8, No. 4, 008 of the parameter for the i-th object o i O, i,j is the j-th class assigned to the i-th object o i O, p i =( i,, i,,, i,,, i,m ) is the i-th pattern, =(,,, m ) is the parameters vector. Table Mnicipal creditworthiness parameters Economic Debt Financial Parameters = PO, PO r r is poplation in the r-th year. PO r /PO r s =, PO r-s is poplation in the year r-s, and s is the selected time period. 3 = U, U is the nemployment rate in a mnicipality. = ) 4 i i= (PZO /PZ, PZO i is the employed poplation of the mnicipality in the i-th economic sector, i=,,,, PZ is the total nmber of employed inhabitants, is the nmber of the economic sector. 5 = DS/OP, 5 <0,>, DS is debt service, OP are periodical revenes. 6 = CD/PO, CD is a total debt. 7 = KD/CD, 7 <0,>, KD is short-term debt. 8 = OP/BV, 8 R +, BV are crrent ependitres. 9 = VP/CP, 9 <0,>, VP are own revenes, CP are total revenes. 0 = KV/CV, 0 <0,>, KV are capital ependitres, CV are total ependitres. = IP/CP, <0,>, IP are capital revenes. = LM/PO, [Czech Crowns], LM is the size of the mnicipal liqid assets. 3. DESIGN OF MODEL BASED ON NEURO- FUZZY SYSTEMS Mnicipal creditworthiness modelling is realized by a design of the model. Data pre-processing maes the sitable economic interpretation of reslts possible. Mnicipalities are assigned to clsters by nspervised methods. The otpt of SOFMs are sed as the inpts of FLNN. Classification of mnicipalities into classes is realized this way. Mnicipal creditworthiness modelling represents a classification problem. It is generally possible to define it this way: Let F() be a fnction defined on a set A, which assigns pictre ˆ (the vale of the fnction from a set B) to each element A, ˆ =F() B, F : A B. The problem defined this way is possible to model by spervised methods (if classes i,j Ω of the objects are nown) or by nspervised methods (if classes i,j Ω are not nown). 3.. Model design and data pre-processing The classes i,j Ω of mnicipal creditworthiness are not nown a priori. Therefore, it is sitable to realize the modelling of mnicipal creditworthiness by nspervised methods. Based on the analysis presented in [8], the combination of SOFMs and K-means algorithm is a sitable method for mnicipal creditworthiness modelling. Its reslts are sed as the inpts of the FLNN [6] in the model presented in Fig.. Spervised learning is realized by the FLNN, which maes the sitable economic interpretation of created clsters possible. Data pre-processing is carried ot by means of data standardization. Thereby, the dependency on nits is eliminated. z i, z i, z i, z i,m SOFM Fig. Model design, where p i =( i,, i,,, i,,, i,m ), m= is the i-th pattern, z i,,z i,,,z i,,,z i,m are standardized vales of parameters,,, m for the i-th object o i O, (p i,t i ) are patterns p i assigned to classes i,j Ω, i,j Ω is the otpt of the FLNN. 3.. Mnicipal creditworthiness modelling by Kohonen s self-organizing featre maps Kohonen s self-organizing featre maps [3,4] are based on competitive learning strategy. The inpt layer serves the distribtion of the inpt patterns p i, i=,,,n. The nerons in the competitive layer serve as the representatives (Codeboo Vectors), and they are organized into topological strctre (most often a twodimensional grid), which designates the neighboring networ nerons. First, the distances d j are compted between pattern p i and weights of all nerons w i,j in the competitive layer according to the relation n j i i, j) i = w w w j w s K-means algorithm (p i,t i ) FLNN d = ( p w, () where j goes over s nerons of competitive layer, j=,,,s, p i is the i-th pattern, i=,,,n, w i,j are synapse weights. The winning neron j* (Best Matching Unit, BMU) is chosen, for which the distance d j from the given pattern p i is minimm. The otpt of this neron is active, while the otpts of other nerons are inactive. The aim of the SOFM learning is to approimate the probability density of the real inpt vectors p i R n by the finite nmber of representatives w j R n, where j=,,,s. When the representatives w j are identified, the i,j ISSN 335-843 008 FEI TUKE

Mnicipal Creditworthiness Modelling by Neral Networs representative w j* of the BMU is assigned to each vector p i. In the learning process of the SOFM, it is necessary to define the concept of neighborhood fnction, which determines the range of cooperation among the nerons, i.e. how many representatives w j in the neighborhood of the BMU will be adapted, and to what degree. Gassian neighborhood fnction is in common se, which is defined as h(j*, j) d (j*, j) ( E ) λ (t) e =, () where h(j*,j) is neighborhood fnction, d E(j*,j) is Eclidean distance of nerons j* and j in the grid, λ(t) is the size of the neighborhood in time t. After the BMUs are fond, the adaptation of synapse weights w i,j follows. The principle of the seqential learning algorithm [4] is the fact, that the representatives w j* of the BMU and its topological neighbors move towards the actal inpt vector p i according to the relation w (t + ) = w i, j(t) + η(t)h(j*, j)( pi (t) i, j(t)), (3) i, j w where η(t) (0,) is the learning rate. The batch learning algorithm of the SOFM [4] is a variant of the seqential algorithm. The difference consists in the fact that the whole training set O train passes throgh the SOFM only once, and only then the synapse weights w i,j are adapted. The adaptation is realized by replacing the representative w j with the weighted average of the inpt vectors p i [4]. Using the SOFM as sch can detect the strctre in the data. The K-means algorithm can be applied to the adapted SOFM in order to find clsters as it is presented in Fig.. This way, the topology of the data is preserved and the comptational load decreases. Interpretation of clsters is realized by the vales of parameters =(,,, m ), m= for individal representatives w j (Fig. 3). The K-means algorithm belongs to non-hierarchical algorithms of clster analysis, where patterns p,p,,p i,,p n are assigned to clsters c,c,,c r,,c q. The nmber of clsters q=6 is determined by indees evalating the qality of clstering []. Clstering process is realized in two levels. In the first level, n objects are redced to representatives w,w,,w s by the SOFM, the s representatives are clstered into q=6 clsters. Fig. Clstering of the SOFM by K-means algorithm c 6 c c Fig. 3 Vales of parameters,,, m, m= for individal representatives w j 3.3. Mnicipal creditworthiness modelling by fzzy logic neral networs It is necessary to interpret clsters c,c,,c q, q=6 and to label them by classes i,j Ω, Ω={,j,,j,, i,j,, n,j }, j=,,,6 so that the class i, represents the best mnicipal creditworthiness, while class i,6 represents the worst one. The reslts of nspervised learning are sed as inpts of the FLNN sitable for classification problem realization [6]. The FLNN parameters mae the appropriate interpretation possible. It is a perceptron-type FLNN with three layers. The inpt layer represents the inpt parameters =(,,, m ), m= the hidden layer represents rles R,R,,,,R N and the otpt layer denotes classes i,j Ω. Standard nerons are replaced by fzzy nerons in the FLNN. Contrary to standard neron, the synapse weights among fzzy nerons assign the vale of membership fnction to the inpt vale. The otpt vale y can be defined as ISSN 335-843 008 FEI TUKE

Acta Electrotechnica et Informatica Vol. 8, No. 4, 008 3 y = ( ) ( ) ( ), (4) where operator represents one of the fzzy operations (MIN, MAX) [9]. The membership fnction y is defined as a continos fnction, whose parameters are adapted in the learning process. The FLNN model for classification problem realization is presented in Fig. 4. The aim of the designed classifier is to achieve a minimm total classification error E T and maimm classification accracy S. At the same time, sitable interpretation of the designed model is demanded (i.e. low nmber N of rles with low nmber of variables in antecedent, low nmber of membership fnctions, etc.). The classifier is designed in two steps. First, the topology of the FLNN is determined. w(, j ) w(, ) Fig. 4 FLNN model design Nerons in the hidden layer se t-norms as activation fnctions, nerons in the otpt layers se t-conorms [7]. The vales of lingistic variable A (i) appropriate for the corresponding fzzy sets are represented by synapse weights w(, ). Synapse weights w(, i,j ) represent rles weights. If w(, i,j ) [0,], then every neron in the hidden layer is connected with the only one neron in the otpt layer. Then, parameters of the system are identified in the process of learning. Rles and fzzy sets represent the parameters. Rles are determined by the following algorithm. Let FLNN has m inpts,,,,, m, N<N ma nerons representing rles R,R,,,,R N and q otpt nerons representing classes i,j Ω. Net, let O={(p,t ),(p,t ),,(p i,t i ),,(p n,t n )} is a set of n patterns, where p i R n is the vector of parameters vales and t {0,} q is the vector of patterns p i assignments to classes i,j,. Then, the learning algorithm of rles is as follows:. Select an object (p i,t i ) O. q q () j. inpts find a membership fnction ( ) as () j ( j= {,,,v } i, i, i,6 () j ) = ma { ( )}, where v is the nmber of membership fnctions corresponding to inpt parameter. 3. If rle with weights w(, ) = () j,,w( m, ) = () j,w(, ) = (m) j, then create the neron by m rle and connect it with otpt neron (class), for which t j =. 4. If there are objects in set O, which did not pass throgh the learning algorithm, contine with. Else, determine the best N rles and remove all the others rles. Fzzy sets are determined by the following algorithm:. Select an object (p i,t i ) O, let it pass throgh the FLNN and determine i,j Ω.. classes i,j determine Δ y i,j is otpt of neron i,j. = t y, where i, j j i,j 3. rles with yr > 0 determine Δ = y ( y ) w(r, i, j Ω i, j)δi,j y is otpt of neron. 4. Find sch as weights w(, R )y = min{w(, R )y }, where otpt of neron., where y 5. Determine the parameters of membership fnctions (e.g. trianglar) for fzzy set represented by synapse weight w(, ) as follows Δ Δ Δ b a c = ηδ (c a) sgn(y b), (5) R ' = ηδ (c a) + Δ, (6) b b = ηδ (c a) + Δ, (7) where η > 0 is learning rate. 6. If the target criterion is reached (total classification error E T or nmber of iterations), finish the learning process, else contine with. 4. ANALYSIS OF THE RESULTS Mnicipal creditworthiness modelling is realized for different nmbers and shapes of membership fnctions and for different nmbers N of rles. The set o i O, O={o,o,,o i,,o n } (n=45 mnicipalities of Pardbice region, the Czech repblic) is divided into training set O train and testing set O test. The qality of classification is measred by average classification error E and classification accracy S for training and testing data. The reslts of classification for bell membership fnctions are presented in Table. If the nmber of membership fnctions increases, then the classification accracy S increases too. is ISSN 335-843 008 FEI TUKE

4 Mnicipal Creditworthiness Modelling by Neral Networs Table The reslts of classification for bell membership fnctions, where is the nmber of membership fnctions; S train, S test are the classification accracies for sets O train, O test, E test is the mean classification error for the set O test, σ test is the standard deviation of the classification accracy S test. v S train S test E test σ test N [%] [%] [%] 8.4 80.30 0.39 3.60 3 3 85.6 84.30 0.33 3.46 65 4 87.7 86.74 0.3 3.37 0 5 9.48 88. 0.7. 74 6 9.48 90.73 0.6.86 7 95.3 93.30 0.4.4 63 A sample of the rles for the nmber v =5 of membership fnctions is as follows R : IF is VS AND is VS AND 3 is VS AND 4 is VS AND 5 is VS AND 6 is VS AND 7 is VS AND 8 is VS AND 9 is VS AND 0 is VS AND is VL AND is VS THEN c, R : IF is VL AND is VS AND 3 is VS AND 4 is VS AND 5 is VS AND 6 is VS AND 7 is M AND 8 is VS AND 9 is VS AND 0 is VS AND is VL AND is VS THEN c, R65 : IF is VS AND is VS AND 3 is VS AND 4 is VS AND 5 is VS AND 6 is VL AND 7 is VS AND 8 is M AND 9 is S AND 0 is VS AND is VS AND is VL THEN c, 6 where VS is a very small, S is a small, M is a medim, L is a large and VL is a very large vale of parameter, =,,,. Based on the created rles R,R,,R N, N=65, clsters c,c,,c q, q=6, can be labelled by classes i,j Ω, where the class i,j Ω, j= represents the best mnicipal creditworthiness, while class i,j Ω, j=q, represents the worst one. This process has to be realized by an epert from the field of mnicipal creditworthiness evalation. Then, the classes i,j Ω, j=,,,6, can be characterized as presented in Table 3. The sitability of the designed model can be compared with other fzzy logic based classifiers as presented in Table 4. Table 4 Comparison of the FLNN model with the maimm spport method (MSM) and fzzy decision trees (FDTs) classifiers in terms of classification accracy S. MSM FDT FLNS v S train S test S train S test S train S test 67.6 54.4 66.8 58.4 8.4 80.30 3 88.50 55.04 75.66 66.37 85.6 84.30 4 96.46 56.64 84.07 67.70 87.7 86.74 5 98.3 45.3 85.84 65.49 9.48 88. 6 99.56 34.07 85.40 65.49 9.48 90.73 7 99. 5.49 9.5 64.60 95.3 93.30 Fig. 5 represents the classification of mnicipalities into classes i,j Ω. The left parts of the histogram are the reslts of the SOFM and the right parts are the reslts of the FLNN. The reslts stand for absolte freqencies of mnicipalities f m in the j-th class i,j Ω. Fzzy IF-THEN classifiers [5] MSM and FDT are selected. The inpt-otpt membership fnctions for fzzy sets and their niverses are designed for classifiers MSM and FDT by the K-means algorithm. The MSM method is better than the FLNN model in terms of classification accracy S train. The highest classification accracy S test is achieved by the FLNN. The FLNN model is the best in terms of generalization ability de to this fact. f m 40 0 00 80 60 40 0 Classification by the SOFM Classification by the FLNN Class i,j j=,,,6 i, i, i,3 i,4 i,5 i,6 Table 3 Descriptions of classes i,j Ω Description High ability of a mnicipality to meet its financial obligation. Very favorable economic conditions, low debt and ecellent bdget implementation. Very good ability of a mnicipality to meet its financial obligation. Good ability of a mnicipality to meet its financial obligation. A mnicipality with stable economy, medim debt and good bdget implementation. Mnicipality meets its financial obligation only nder favorable economic conditions. A mnicipality meets its financial obligations with difficlty, the mnicipality is highly indebted. 0 i, i, i,3 i,4 i,5 i,6 classes Fig. 5 Classification of mnicipalities into classes (v =5). 5. CONCLUSION 3 4 5 6 The paper presents the design of mnicipal creditworthiness parameters. Frther, the model is designed whereby mnicipal creditworthiness evalation is realized. First, the data are pre-processed. Net, mnicipal creditworthiness modelling is realized by the SOFM. The otpts of the SOFM are sed as the inpts of the FLNN. Its aim is the classification of mnicipalities into classes so that high classification accracy, low average classification error and sitable interpretation of reslts are achieved. The reslts are compared with other fzzy logic based classifiers. ISSN 335-843 008 FEI TUKE

Acta Electrotechnica et Informatica Vol. 8, No. 4, 008 5 The designed model was carried ot in NefClass (FLNN) and Matlab (SOFM) in MS Windows XP operation system. ACKNOWLEDGEMENTS This wor was spported by the scientific research of Czech Science Fondation, nder Grant No: 40/08/0849 with title Model of Sstainable Regional Development Management. REFERENCES [] Dnn, J. C.: Well separated clsters and optimal fzzy partitions, Jornal of Cybernetics, no.4, pp. 95 04, 974. [] Háje, P.: Mnicipal creditworthiness modelling by comptational intelligence methods, Ph.D. Thesis, University of Pardbice, 006. [3] Hayin, S. S.: Neral networs: A comprehensive fondation, Prentice Hall, Upper Saddle River, 999. [4] Kohonen, T.: Self organizing maps, Springer Verlag, New Yor, 00. [5] Kncheva, L. I.: Fzzy classifier design, Springer Verlag, Berlin, 000. [6] Nac, D. Krse, R.: A nero fzzy method to learn fzzy classification rles from data, Fzzy Sets and Systems, no.89, pp. 77 88, 997. [7] Olej, V.: Modelling of economic processes based on comptational intelligence, M&V, Hradec Králové, 003. (in Slova) [8] Olej, V. Háje, P.: Modelling of mnicipal rating by nspervised methods, WSEAS Transactions on Systems, vol.6, no.7, pp. 679 686, 006. [9] Zadeh, L. A.: Fzzy sets, Information and Control, no.8, pp. 338 353, 965. Received Jly 9, 007, accepted October 3, 008 BIOGRAPHIES Petr Háje (MSc., PhD.) was born in 980. In 003 he gradated (MSc.) at the Faclty of Economics and Administration at University of Pardbice. He defended his PhD. in the branch of System Engineering and Informatics in 006. His scientific research is focsed on modelling economic processes by nero-fzzy systems. Vladimír Olej (prof. MSc. Ph.D.) was born in Poprad, Slovaia. Since 00 he is woring as a professor at the University of Pardbice. He has pblished a nmber of papers concerning fzzy logic, neral networs, and genetic algorithms. ISSN 335-843 008 FEI TUKE