Improved Mining of Software Complexity Data on Evolutionary Filtered Training Sets

Transcription

1 Improved Mnng of Software Complexty Data on Evolutonary Fltered Tranng Sets VILI PODGORELEC Insttute of Informatcs, FERI Unversty of Marbor Smetanova ulca 17, SI-2000 Marbor SLOVENIA Abstract: - Wth the evoluton of nformaton technology and software systems, software relablty has become one of the most mportant topcs of software engneerng. As the dependency of socety on software systems ncrease, so ncreases also the mportance of effcent software fault predcton. In ths paper we present a new approach to mprovng the classfcaton of faulty software modules. The proposed approach s based on flterng tranng sets wth the ntroducton of data outlers dentfcaton and removal method. The method uses an ensemble of evolutonary nduced decson trees to dentfy the outlers. We argue that a classfer traned by a fltered dataset captures a more general knowledge model and should therefore perform better also on unseen cases. The proposed method s appled on a real-world software relablty analyss dataset and the obtaned results are dscussed. Key-Words: - data mnng, classfcaton, evolutonary decson trees, flterng tranng sets, software fault predcton, search-based software engneerng 1 Introducton The early and accurate dentfcaton of potentally dangerous (faulty) software modules s of vtal mportance for better software relablty. Relablty s one of the most mportant aspects of software systems of any knd (nformaton systems, embedded systems, etc.). The sze and complexty of software s growng dramatcally durng last decades. The demand for hghly complex software systems s ncreasng more rapdly than the ablty to desgn, mplement and mantan them. When the requrements for and dependences of computers ncrease, the possblty of crses from falures also ncreases. The mpact of these falures ranges from nconvenence to economc damages to loss of lves therefore t s clear that software relablty s becomng a major concern not only for software engneers and computer scentsts, but also for the socety as a whole. Therefore, the employment of an effcent fault predctve technque to foresee dangerous software modules s essental. Wth the creaton of large emprcal databases of software projects, as a result of stmulated research on estmaton models, metrcs and methods for measurng and mprovng processes and products, ntellgent mnng of these datasets can largely add to the mprovement of software relablty [1]. In order to help the software engneers n predctng the faulty software modules, computerzed data mnng and decson support tools can be used whch are able to help software engneers to process a huge amount of data avalable from prevous software projects and suggest the probable predcton based on the values of several mportant attrbutes. Black-box classfcaton methods (neural networks for example) are not very approprate for ths knd of task, because the software experts want to evaluate and valdate the decson makng process nduced by those tools, before there s enough trust to use the tools n practce. On the other hand, the evaluaton of the nduced classfcaton rules produced by the computerzed tools by a software expert can be an mportant source of new knowledge on the assocatons of the avalable attrbutes and new laws of software relablty engneerng. In order to acheve ths goal, the classfcaton process should be easly understandable, nterpretable and straghtforward. One of the most popular and proven-useful approach are decson trees. However, t has been shown that decson trees are a weak classfer, prone to produce very dfferent solutons based on an even small change n nput (tranng) data. Therefore, naccuraces and nose n tranng data can easly lead to an naccurate result. ISSN: Issue 11, Volume 6, November 2009

2 The dea that we present n ths paper s to construct an outler predcton method that flters out so-called data outlers,.e. data tems that fall outsde the boundares that enclose most other data tems n the data set [1, 2]. When a fltered dataset s used to tran a classfer (a decson tree) t should produce a better and more relable classfcaton result. Furthermore, as a consequence of ncreased homogenety of the data, the results should be more general and thus smpler, less complex and easer to apply n general. Smlar method has already been appled to data mnng n medcne wth some success [3]. Our proposton for the predcton of outlers s the followng: f a sngle (known) data case s classfed dfferently by dfferent accurate classfers, t potentally contans contradctory nformaton (Fgure 1). Although ths contradcton s not necessarly an error n data, a usual decson tree classfer s not able to correctly construct a general model based on such data. Therefore, our proposton s that the general knowledge model bult by a decson tree (or some other nducton method n that matter) would be more effcent when a classfer would be traned wthout such msleadng data. 1.1 The am and scope of the paper The man objectve of ths paper s to nvestgate whether t s possble to mprove the classfcaton performance of several machne learnng algorthms when the outlers are fltered out of the tranng set. For the outler predcton method a set of evolutonary nduced decson trees s used. The proposed approach s appled to a software engneerng problem of predctng potentally dangerous (contanng errors and faults) software modules. The data mnng results (both quanttatve classfcaton results and qualtatve assessment of nduced models) of dfferent machne learnng algorthms are presented and dscussed. n data set 2 Flterng the dataset by outler predcton method The basc dea of outler predcton s to defne a crteron or crtera, upon whch for each data case from a dataset can be determned whether t belongs to the majorty of the cases or not. If the specfc data case regardng the defned crtera does not belong to the majorty, then t s called an outler (regardng specfc crtera). How the crtera are defned determnes the outler predcton method. In general, defnng outler crtera s not trval, whereas the dentfcaton of outlers based on these crtera s. In our prevous work we presented how to use evolutonary algorthm for the nducton of decson trees [4, 5, 6]. One of the greatest advantage of evolutonary constructon of decson trees, besde the proven effcency n classfyng, s the ablty to produce several comparably accurate classfers for the same dataset. Havng ths n mnd, t s possble to get a decson for the same data case based on dfferent classfers (usng dfferent attrbutes and/or dfferent relatons). class 1 classfers class 2 Fg 1. When a sngle data record s classfed dfferently by dfferent classfers, t potentally contans contradctory nformaton. In our method, we frst defne an approach to the dentfcaton of outlers. For ths purpose the algorthm for the constructon of decson trees ISSN: Issue 11, Volume 6, November 2009

3 gentrees s used (descrbed n the next secton). Wth gentrees a set of classfers (decson trees) are nduced. For each tranng object (data case from the tranng set) classfcaton cc (x) s calculated for all decson classes (Eq. 1) the resultng value represents the number of classfers that classfed object x wth the decson class. Then classfcaton confuson score CCS(x) s calculated for each tranng object x (Eq. 2) the result represents the confuson score of a set of classfers when classfyng object x; f all DTs gve the same classfcaton, then the result s 0; hgher numbers represent less homogeneous objects for classfcaton possble outlers. The hgher s the number CCS(x), more probably les the object x outsde the majorty area. Based on the CCS (Eq. 2) t s determned whch objects should be fltered out from the dataset for the classfcaton process. For ths purpose a tolerance threshold tt s defned for a dataset; f CCS(x)>tt then object x s fltered out. survval of the fttest, frst lad down by Charles Darwn, so by smulatng ths process, genetc algorthms are able to evolve solutons to real-world problems, f they have been sutably encoded [9]. They are often capable of fndng optmal solutons even n the most complex of search spaces or at least they offer sgnfcant benefts over other search and optmsaton technques. The basc mechansm of evolutonary algorthms s presented n Fgure 2. cc ( x) ;, num _ DTs j 1 1; class ( DT 0; otherwse ( 1... num _ classes ) j, x) (1) CCS 2 num _ classes cc ( x) ( x) 1 (2) 1 cc max ( x) Fg 2. The basc mechansm of evolutonary algorthms: selecton, crossover and mutaton. 3 Backend technologes: evolutonary algorthms and decson trees The central pont of our approach, as has been sad above, s the use of evolutonary nduced decson trees whch are beng used to construct a set of classfers for dentfyng outlers n a tranng set. Both backend technologes enablng our approach are brefly presented frst, followng by the descrpton of our algorthm for nducng decson trees. 3.1 Evolutonary algorthms Genetc algorthms are adaptve heurstc search methods whch may be used to solve all knds of complex search and optmsaton problems [7, 8]. They are based on the evolutonary deas of natural selecton and genetc processes of bologcal organsms. Lke natural populatons evolve accordng to the prncples of natural selecton and A typcal genetc algorthm operates on a set of solutons (populaton) wthn the search space. The search space represents all the possble solutons whch can be obtaned for the gven problem, and s usually very complex or even nfnte. Every pont of the search space s one of the possble solutons and therefore the am of the genetc algorthm s to fnd an optmal pont or at least come as close to t as possble. The genetc algorthm conssts of three genetc operators: selecton, crossover (recombnaton), and mutaton. Selecton s the survval of the fttest ndvduals wthn the genetc algorthm wth the am of gvng the preference to the best ones. For ths purpose all solutons have to be evaluated, whch s done wth the use of the evaluaton functon. Selecton determnes ndvduals to be used for the second genetc operator - crossover or recombnaton, where from two good ndvduals a new, even better one s constructed. The crossover process s repeated untl the whole new populaton s completed wth the offsprng produced by the ISSN: Issue 11, Volume 6, November 2009

4 crossover. All constructed ndvduals have to preserve the feasblty regardng the gven problem; n ths manner t s mportant to coordnate nternal representaton of ndvduals wth genetc operators. The last genetc operator s mutaton, whch s an occasonal (low probablty) alteraton of an ndvdual that helps to fnd an optmal soluton to the gven problem faster and more relably. DTs have been already used n a large amount of dfferent applcatons: n medcne [5], fnance [11], envronmental studes [12], etc. Ther greatest advantage s comparatvely hgh classfcaton accuracy whle preservng the transparent knowledge model whch can be easly nterpreted and valdated by doman experts. Ths has made DTs one of the most popular classfers. 3.2 Decson trees Inductve nference s the process of movng from concrete examples to general models, where the goal s to learn how to classfy objects by analysng a set of nstances (already solved cases) whose classes are known. Instances are typcally represented as attrbute-value vectors. Tranng nput conssts of a set of such vectors, each belongng to a known class, and the output conssts of a mappng from attrbute values to classes. Ths mappng should accurately classfy both the gven nstances and other unseen nstances. A decson tree (DT) [5, 9] s a formalsm for expressng such mappngs and conssts of tests or attrbute nodes lnked to two or more sub-trees and leafs or decson nodes labeled wth a class whch means the decson. A test node computes some outcome based on the attrbute values of an nstance, where each possble outcome s assocated wth one of the sub-trees. An nstance s classfed by startng at the root node of the tree. If ths node s a test, the outcome for the nstance s determned and the process contnues usng the approprate subtree. When a leaf s eventually encountered, ts label gves the predcted class of the nstance. An example of a DT s presented n Fgure 3. Fg 3. An example of a decson tree. 3.3 Evolutonary nducton of decson trees: the gentrees algorthm Evolutonary algorthms (EAs) are generally used for very complex optmzaton tasks [8], for whch no effcent determnstc or heurstc method s developed. Constructon of DTs s a complex task, but an exact method exsts, that n general works effcently and relably [5, 9]. At a frst glance there s no reason to use EAs. Nevertheless, there are some objectve reasons that justfy our evolutonary approach. Frst, EAs provde a very general concept that can be used n all knds of decson makng problems. Because of ther robustness they can be used also on ncomplete, nosy data (whch often happens because of measurement errors, unavalablty of proper nstruments, etc.). Ths s not very successfully solvable by tradtonal technques of DT constructon. Furthermore, EAs use evolutonary prncples to evolve solutons, therefore solutons can be found that can be easly overlooked otherwse. Another mportant advantage of EAs s the possblty of optmzng the decson tree's topology and adaptng class ntervals for numerc attrbutes, wthn the evoluton process. And the most mportant advantage of the evolutonary approach n our case s that not only one but several equally qualtatve solutons are obtaned for the same dataset. In ths way the same decson can be made based on dfferent attrbutes and/or combnaton of the attrbutes. By dfferent settngs of parameters n EA runs, searchng can be drected to dfferent stuatons that gve us dfferent solutons for our fnal decson forest. Of course there are also some drawbacks of our method regardng the heurstc nducton, the most obvous one beng the hgher nducton tme complexty. When defnng the nternal representaton of ndvduals wthn the genetc populaton, together wth the approprate genetc operators that wll work upon the populaton, t s mportant to assure the feasblty of all solutons durng the whole evoluton process. We decded to present an ndvdual drectly as a DT. In ths manner all ntermedate solutons are feasble, no nformaton s ISSN: Issue 11, Volume 6, November 2009

5 lost because of the converson between nternal representaton and the DT, the ftness functon (FF) can be straghtforward, etc. The problem wth drect codng of soluton may brng some problems n defnng of genetc operators. As DTs may be seen as a knd of smple computer programs (wth attrbute nodes beng condtonal clauses and decson nodes beng assgnments) we decded to defne genetc operators smlar to those used n genetc programmng where ndvduals are computer program trees [10]. The nducton of DT-lke classfers wth the use of evolutonary algorthms has been already used n several cases [5, 14, 15], also appled to software engneerng [6] Inducton of DTs for the ntal populaton Frst step of the genetc algorthm s the nducton of the ntal populaton. A random decson tree s constructed based on the followng algorthm: 1. nput: number of attrbute nodes V that wll be n the tree 2. select an attrbute A from the set of all possble K attrbutes and defne t a root node tn 3. n accordance wth the selected attrbute's A type (dscrete or contnuous) defne a test for ths node tn: a. for contnuous attrbutes n a form of tn(a ) <, where tn(a ) s the attrbute value for a data object and s a splt constant b. for dscrete attrbutes two dsjunctve sets of all possble attrbute values are randomly defned 4. connect empty leaves to both new branches from node tn 5. randomly select an empty leaf node tn (note below) 6. randomly select an attrbute A from the set of all possble K attrbutes (note below) 7. replace the selected leaf node tn wth the attrbute A and go to step 3 8. fnsh when V attrbute nodes has been created In the above algorthm: - the probablty of selectng an empty leaf s decreased wth the depth of the leaf n a growng tree, and - the probablty of choosng an attrbute depends on a number of prevous uses of that attrbute n a tree n ths manner unused attrbutes have better chances to be selected. A smplfed pseudo-code of the above algorthm s presented at Fgure 4. // ntal decson trees Select number of nodes N repeat Select an attrbute X Defne f t (X)< or f t (X ) V Connect empty leaves to node Randomly select an empty leaf node untl N nodes are created Fg 4. The basc algorthm for constructng random DTs. For each empty leaf the followng algorthm determnes the approprate decson class: let S be the tranng set of all tranng objects N wth M possble decson classes 1,.., M and N s the number of objects wthn S of a class. Let S tn be the sample set at node tn (an empty leaf for whch we are tryng to select a decson class) wth N tn objects; N tn s the number of objects wthn S tn of a decson class. Now we can defne a functon that measures a potental percentage of correctly classfed objects of a class : F tn, ) N N tn ( (3) Decson tn for the leaf node tn s then marked as the decson, for whch F(tn,) s maxmal. The rankng of an ndvdual DT wthn a populaton s based on the FF: FF M V w ( 1 acc ) c( tn ) wu nu (4) 1 1 where M s the number of decson classes, V s the number of attrbute nodes n a tree, acc s the accuracy of classfcaton of objects of a specfc decson class, w s the mportance weght for classfyng the objects of the decson class, c(tn ) s the cost of usng the attrbute n a node tn, nu s number of unused decson (leaf) nodes,.e. where no object from the tranng set fall nto, and w u s the weght of the presence of unused decson nodes n a tree. Accordng to FF the best trees (the most ft ones) have the lowest functon values the am of the evolutonary process s to mnmze the value of FF for the best tree. A near optmal DT would: 1) classfy all tranng objects wth accuracy n ISSN: Issue 11, Volume 6, November 2009

6 accordance wth mportance weghts w (some decson classes may be more mportant than the others), 2) have very lttle unused decson nodes (there s no evaluaton possble for ths knd of decson nodes regardng the tranng set), and 3) consst of low-cost attrbute nodes (n ths manner the desrable/undesrable attrbutes can be prortzed). qualtatve solutons are obtaned regardng the chosen FF. The evoluton stops when an optmal or at least an acceptable soluton s found or f the ftness score of the best ndvdual does not change for a predefned (large) number of generatons. A smplfed pseudo-code algorthm for the evolutonary process of mprovng the qualty of a DT (n accordance wth FF) s presented at Fgure Crossover and mutaton Crossover works on two selected ndvduals as an exchange of two randomly selected sub-trees. In ths manner a randomly selected tranng object s selected frst whch s used to determne paths (by fndng a decson through the tree) n both selected trees. Then an attrbute node s randomly selected on a path n the frst tree and an attrbute s randomly selected on a path n the second tree. Fnally, the sub-tree from a selected attrbute node n the frst tree s replaced wth the sub-tree from a selected attrbute node n the second tree and n ths manner an offsprng s created whch s put nto a new populaton. Mutaton conssts of several parts: 1) one randomly selected attrbute node s replaced wth another attrbute, randomly chosen from the set of all attrbutes; 2) a test n a randomly selected attrbute node s changed,.e. the splt constant s mutated; 3) a randomly selected decson (leaf) node s replaced by an attrbute node; 4) a randomly selected attrbute node s replaced by a decson node. // evolvng DTs repeat tournament selecton crossover mutaton evaluate all DTs wth FF untl termnaton crtera reached Fg 5. The basc algorthm for evolvng DTs. 4 Applcaton of the method The descrbed tranng set flterng method has been appled to a real-world software relablty analyss dataset, composed at the Unversty of Udne, Italy, from the software development project of a hosptal nformaton system the whole medcal software system conssts of 904 modules n C programmng language representng more than lnes of code. Frst the modules have been dentfed ether as OK or DANGEROUS by applyng the model developed by Pghn [11] the modules that contan less than 5 errors were set as OK and the others as DANGEROUS. A set of 168 attrbutes, contanng varous software complexty measures, has been determned for each software module. From all 904 modules 804 have been randomly selected for the tranng set, and the remanng 100 modules have been selected for the testng set. A set of classfers were nduced wth gentrees based on the tranng set. Then outlers were dentfed usng the class confuson score metrcs (Eq. 2), whch were removed from the orgnal tranng set n order to get a fltered tranng set. Fnally, some well-known classfcaton algorthms were used on both orgnal and fltered tranng set n order to compare classfcaton results. The followng classfcaton algorthms have been used: AREX [12], ID3, C4.5 [13, 14], Naïve-Bayes (N-B), nstance-based classfer (IB,.e. k-nearest neghbors), and logstc regresson (LogReg) [15, 16]. All the results are the averages of 10-fold crossvaldaton. Wth the combnaton of presented crossover that works as a constructve operator towards local optmums, and mutaton that works as a destructve operator n order to keep the needed genetc dversty, the searchng for the soluton tends to be drected toward the globally optmal soluton. In ths manner the optmal soluton s the most approprate DT regardng our specfc needs (expressed n the form of FF). As the evoluton repeats, more 4.1 A Dataset The used software complexty measures dataset (SCM dataset) has been carefully composed from a well-prepared protocol [17]. The startng pont for the analyss was the defnton and measurement of a set of expermental attrbutes connected to the structure of software products after the code phase. Such parameters may, for example, be the total number of lnes of code and lnes of comments, the ISSN: Issue 11, Volume 6, November 2009

7 occurrence of varous types of nstructons, the operators and the types of data used. For each software module, moreover, the fault sgnals up to the moment when measurement started were consdered. By faults all the malfunctons encountered durng the nternal test phase and after the release of the software are meant. In our expermental envronment the code phase ncluded a prelmnary test of modules. After ths phase the modules went on to the real test sesson, n whch faults were sgnaled and measurement started. What had to be dealt wth was whether the chosen set of parameters would be suffcently large to dentfy the structure of a program. Accordngly t has been started wth a very large set of parameters. These parameters were affected by the persstence of multcollnearty. It was necessary to reduce the total number of parameters to a smaller set of ndependent parameters. Ths was acheved by statstcal procedures elmnatng those whch were heavly dependent on other parameters n explanng the presence of faults n a code, or whch were completely rrelevant. A subset of 168 structural parameters was defned whch the factoral analyss dentfed as beng reasonably free from multcollnearty, plus the dependent varable, the number of faults. The basc propertes of the SCM dataset are presented n Table 1. Table 1. The basc propertes of the SCM dataset. number of cases 904 number of attrbutes nomnal 1 - contnuous 167 number of decson classes 2 decson classes dstrbuton 76.4%; 23.6% 4.2 Quanttatve results As descrbed above, after flterng out the outlers by the proposed outler predcton method, the reduced data set has been used by some well-known classfcaton methods. When usng the same tranng set and the same parameter settng, the algorthms produce the same determnstc results (for example C4.5 algorthm uses entropy measures for the nducton of decson trees, etc). In ths manner, the dfference n acheved classfcaton effectveness on a testng set can be objectvely compared between the nduced classfers traned by ether the orgnal, non-fltered tranng set or the fltered tranng set. The classfcaton results are presented n tables 2 and 3 and on fgures 4 and 5. For the SCM dataset fve classfers were nduced and the tolerance threshold selected tt=0.2; f none or only one classfer (out of fve) msclassfed an object, then the object was not dentfed as an outler. Altogether 29 objects (from 804 n the orgnal tranng set) were removed from the tranng set (3.6% removal). Table 2. Average classfcaton accuraces on the SCM tranng dataset. classfcaton algorthm accuracy on the tranng set [%] orgnal set fltered set AREX C ID IB Naïve-Bayes LogReg Table 3. Average classfcaton accuraces on the SCM testng dataset. classfcaton accuracy on the tranng set [%] algorthm orgnal set fltered set AREX C ID IB Naïve-Bayes LogReg ,00 100,00 95,00 90,00 85,00 80,00 75,00 70,00 Accuracy on tranng set for SCM dataset. AREX C4.5 ID3 IB Nave- Bayes orgnal fltered LogReg Fg. 4. Classfcaton results on the SCM tranng dataset. ISSN: Issue 11, Volume 6, November 2009

8 85,00 Accuracy on testng set for SCM dataset. rules, whereas the classfers nduced on the orgnal tranng set are much more demandng to nterpret. 82,00 79,00 76,00 73,00 70,00 AREX C4.5 ID3 IB Nave- Bayes orgnal fltered LogReg Fg. 5. Classfcaton results on the SCM testng dataset All the results presented are based on the 10-fold cross valdaton for each algorthm on a tranng set and a testng set. The classfcaton results of the method AREX are based on 10 ndependent evolutonary runs for each fold. 4.3 Qualtatve results The possblty of accurate predctons of the potentally dangerous software modules based on the software complexty attrbutes s very mportant for software engneers n order to mprove the software relablty. The proposed classfcaton method proves to be effectve n performng ths task. However, the knowledge (.e. combnatons of the used attrbutes) used to make the predctons would be of an mmense mportance n order to decrease the possblty of the problems to arse n the frst place. Therefore, the bult knowledge models (lke decson trees or decson rules) should be studed to fnd ths knowledge. An nterestng phenomenon that arose wth the flterng of the dentfed outlers from the orgnal tranng set s the fact, that the bult knowledge models based upon the fltered learnng set were much less complex than those bult on the orgnal, non-fltered learnng set. Ths means that less attrbutes were used to predct the faulty modules and the overall models were smpler, smaller and less complex easer to nterpret. For the comparson on Fgure 6 there are two decson trees bult on the orgnal, non-fltered learnng set, and on Fgure 7 there are a few decson trees bult on the fltered learnng set; the accuracy of all the decson models are pretty much the same. We can see that n the case of fltered tranng set, the resultng classfers are almost as smple as smple alpha --[< ] OK --[>= ] sgnf_of_comments --[< ] StrCtrl_lnes --[< ] OK --[>= ] DANGEROUS --[>= ] selecton_nstr --[<3.799] DANGEROUS --[>=3.799] formal_functon_params --[<19.162] DANGEROUS --[>=19.162] sgnf_of_comments --[< ] OK --[>= ] DANGEROUS words_of_comments --[< ] break --[<12.282] functon_calls_to_funcs --[<11.985] vect_functon_args --[<25.088] const_wth_#defne --[<41.584] formal_func_pars --[<8.134] OK --[>=8.134] DANGEROUS --[>=41.584] DANGEROUS --[>=25.088] fletype --[c,r,h] OK --[s] DANGEROUS --[>=11.985] StrCtrl_lnes --[<10.611] DANGEROUS --[>=10.611] OK --[>=12.282] DANGEROUS --[>= ] DANGEROUS Fg. 6. Two of the nduced decson trees for predctng dangerous software modules on the orgnal, non-fltered learnng set. tot_of_comments --[< ] break --[< ] OK --[>= ] DANGER --[>= ] DANGER declared_vect --[< ] solo_comment_lnes --[< ] OK --[>= ] DANGER --[>= ] DANGER StrCtrl_lnes --[< ] sgnf_of_comments --[< ] OK --[>= ] DANGER --[>= ] DANGER ISSN: Issue 11, Volume 6, November 2009

9 case --[< ] words_of_comments --[< ] OK --[>= ] DANGER --[>= ] DANGER Fg. 7. A few of the nduced decson trees for predctng dangerous software modules on the fltered learnng set. 4.4 Dscusson The classfcaton results on the SCM dataset show that all the decson tree based classfers (AREX, ID3, C4.5) mproved on the tranng set (as expected) and also mproved consderably on the testng set. Whereas the mprovement was expected on the tranng set, as the outlers are more dffcult to place n the general model, the mprovement on the testng set was only hoped for. The Naïve-Bayes classfer mproved on the tranng set, but scored the same on the testng set. The IB and LogReg classfers scored practcally the same results on the tranng set and show some mprovements on the testng set. From the above results t can be concluded that the flterng of the tranng dataset (by removng the outlers) has only a mnor effect on IB classfers; as they search for the most smlar objects n the dataset practcally the outlers have no effect on the search. The decson tree classfers beneft from flterng the removal of outlers helps to reduce the uncertanty that s ntroduced by outlers, even as the percentage of the removed objects s rather small (3.6%). As DTs are very weak classfers, where even a change of a sngle tranng case can result n a completely dfferent soluton, the removal of dentfed outlers (whch are selected based on ther complexty of beng correctly classfed) substantally mproves the classfcaton results. The Naïve-Bayes and logstc regresson classfers do not beneft consderably from flterng; however, the classfcaton results are not worse ether. We must know that statstcs, upon whch these classfers are based, normally works towards generalzaton, neglectng mnor (boundary) cases. For ths reason, n our opnon, the removal of outlers from the tranng set does not nfluence the results consderably. Generally speakng for all the used classfcaton methods: when usng the fltered tranng set, the quanttatve classfcaton results on the testng set are at least as good as when traned on the orgnal tranng set, and n many cases better. 5 Concluson A fault predctve technque to foresee dangerous software modules has been presented n the paper. It s based on a new outler predcton and removal technque, that s used to flter an orgnal tranng set, whch s then used to tran varous classfers. Our proposton was that the fltered tranng datasets used to tran the classfers can mprove the classfcaton results regardng the orgnal, nonfltered datasets. The obtaned results show an mprovement of the classfcaton performance wth the decson tree classfers (whch are weak classfers), where outlers have rather negatve effect on the tranng process. The proposed approach has only a mnor effect on the nstance based, Naïve-Bayes and logstc regresson classfers n these cases the outlers do not represent a bg ssue as statstcal methods already work prmarly n a generalzaton manner, neglectng mnor (boundary) cases. However, all the classfcaton results are at least as good as wth the orgnal, non-fltered dataset. Ths fact speaks n favor of usng the proposed outler predcton and removal method when nducng a fault predctve knowledge model. In partcular, because traned knowledge models (n the case of nducng DT classfers), at least as t turned out n our experment, are substantally less complex when traned upon fltered tranng sets. In ths manner, an expert can draw some general knowledge out of the nduced model. As even the smallest mprovement s of vtal mportance when mnng the software relablty measures data, we plan to further explore the possbltes that the proposed approach offers. References: [1] P.K. Kapur, O. Shatnaw, A.G. Aggarwal, R. Kumar, Unfed framework for developng testng effort dependent software relablty growth models, WSEAS Transactons on Systems, vol. 8, num. 4, 2009, pp [2] M.M. Breung, H.-P. Kregel, R.T. Ng, J. Sander, LOF: Identfyng densty-based local outlers, Proceedngs of ACM SIGMOD, [3] E.M. Knorr, R.T. Ng, Algorthms for mnng dstance-based outlers n large datasets, Proceedngs of the 24th VLDB Conference, ISSN: Issue 11, Volume 6, November 2009

10 [4] V. Podgorelec, M. Hercko, I. Rozman, Improvng mnng of medcal data by outlers predcton, Proceedngs of the 18th IEEE Symposum on Computer-based Medcal Systems CBMS'2005, [5] V. Podgorelec, P. Kokol, Towards more optmal medcal dagnosng wth evolutonary algorthms, Journal of Medcal Systems, Kluwer Academc/Plenum Press, vol. 25, num. 3, 2001, pp [6] V. Podgorelec, P. Kokol, B. Stglc, I. Rozman, Decson trees: an overvew and ther use n medcne, Journal of Medcal Systems, Kluwer Academc/Plenum Press, vol. 26, num. 5, 2002, pp [7] V. Podgorelec, P. Kokol, Evolutonary nduced decson trees for dangerous software modules predcton, Informaton Processng Letters, vol. 82, num. 1, 2002, pp [8] T. Baeck, Evolutonary Algorthms n Theory and Practce, Oxford Unversty Press, Inc., [9] D.E. Goldberg, Genetc Algorthms n Search, Optmzaton, and Machne Learnng, Readng MA: Addson Wesley, [10] J.H. Holland, Adaptaton n natural and artfcal systems, Cambrdge, MA: MIT Press, [11] A. Rotshten, M. Posner, H. Rakytyanska, Fuzzy IF-THEN Rules Extracton for Medcal Dagnoss Usng Genetc Algorthm, WSEAS Transactons on Systems, vol. 3, num. 2, 2004, pp [12] C.-S. Huang, Y.-J. Ln, C. Ln, Implementaton of classfers for choosng nsurance polcy usng decson trees: a case study, WSEAS Transactons on Computers, vol. 7, num. 10, 2008, pp [13] J.R. Koza, Genetc Programmng: On the Programmng of Computers by Natural Selecton, Cambrdge, MA: MIT Press, [14] M. Kasparova, J. Krupka, P. Jrava, Applcaton of decson trees n problem of ar qualty modellng n the Czech Republc localty, WSEAS Transactons on Systems, vol. 7, num. 10, 2008, pp [15] M. Zorman, B. Cernohorsk, G. Gorsek, M. Ojstersek, Hybrd evolutonary bult decson trees for predcton of perspectve cross-country skers, WSEAS Transactons on Informaton Scence and Applcatons, vol. 2, num. 1, 2005, pp [16] M. Pghn, P. Kokol, RPSM: A rsk-predctve structural expermental metrc, Proceedngs of FESMA'99, pp , [17] V. Podgorelec, P. Kokol, I. Rozman, AREX - Classfcaton rules extractng algorthm based on automatc programmng, Proceedngs of the 15th European Conference on Artfcal Intellgence ECAI 2002, pp , [18] J.R. Qunlan, C4.5: Programs for Machne Learnng, Morgan Kaufmann, [19], RuleQuest Research Data Mnng Tools, [20], Machne Learnng n C++, MLC++ lbrary, [21] J. Demsar, B. Zupan, Orange: From Expermental Machne Learnng to Interactve Data Mnng, Whte Paper ( Faculty of Computer and Informaton Scence, Unversty of Ljubljana, [22] V. Podgorelec, P. Kokol, M. Zorman, M. Sprogar, M. Pghn, The Operatve Constrants of Software Relablty Predcton Methods, Proceedngs of the World Multconference SCI'2001, ISSN: Issue 11, Volume 6, November 2009