Developing Software Bug Prediction Models Using Various Software Metrics as the Bug Indicators

Transcription

1 Dvloping Softwar Bug Prdiction Modls Using Various Softwar Mtrics as th Bug Indicators Varuna Gupta Rsarch Scholar, Christ Univrsity, Bangalor Dr. N. Ganshan Dirctor, RICM, Bangalor Dr. Tarun K. Singhal Dan-Acadmic, INMANTEC, Gzb Abstract Th bug prdiction ffctivnss rasonably contributs towards nhancing quality of softwar. Bug indicators contribut significantly in dtrmining th bug prdiction approachs and hlp in achiving softwar rliability. Various comparativ rsarch studis hav indicatd that Dpth of Inhritanc (DIT), Wightd Mthod pr Class (WMC), Coupling btwn Objcts (CBO) and Lins of Cod (LoC) hav significantly stablishd thmslvs as rliabl bug indicators for comprhnsiv bug prdictions. Th rsarchrs hav carrid out a quantitativ rsarch and hav dvlopd prdiction modls using abov bug indicators as modls input and hav applid ths modls on opn sourc projcts (Caml and Ant). During this rsarch, th rsults dmonstrats that thr is significant corrlation btwn siz orintd mtrics (bug indicators) such as DIT, WMC, CBO, LoC and bugs. Ovrall, DIT tas dominanc in achiving bttr impact on prdicting bugs than WMC, CBO and LoC. Th outcoms of th prsnt rsarch study would b of significanc to softwar quality practitionrs worldwid and would hlp thm in prioritizing th fforts involvd in bug prdiction. Kywords Bug Prdiction; DIT; WMC; CBO; LoC; SRGM I. INTRODUCTION Softwar rliability is considrd critical and important aspct of softwar quality. Organizations pay du mphasis in dtcting th quality of softwar product at an arly stag to avoid lat mbarrassmnts arising du to lat dtction culminating in poor quality product ultimatly. This approach nsurs that organizations ar abl to rdsign whrvr possibl and nsur consistnt quality throughout. Organizations aim to nsur savings towards costs of dvlopmnt, rduction in tim to dvlop and high rliability of softwar products. Various attributs such as pronnss to faults, tsting fforts, maintnanc fforts tc govrn th quality of softwar products. Through this rsarch, w hav considrd pronnss to bugs as bug prdictor utilizing DIT, WMC, CBO and LoC indicators within th ralm of this rsarch. Various bug indicators proposd during last fw dcads hav mad th slction of right bug indicator a dmanding tas considring th complxity and natur of varying softwar dvlopmnt procsss. In th wa, a numbr of rsarchrs hav prdominantly proposd product orintd bug indicators. Th tstrs across many organizations ddicat tim and rsourcs by allocating sam prioritis across all componnts of a projct, which is not considrd as an optimal approach. Parts of th softwar systms don t hav uniformity in bug distribution. This calls for comprhnsiv idntification of fils containing bugs throughout th projct. Th tstrs with such nowldg would b abl to idntify and prioritiz th appropriat tsts whil achiving fficincy in tsting procss. In ordr to achiv th said, it is ssntial to nsur availability of appropriat softwar bug prdiction modls. Th main objctiv of this rsarch is to construct softwar bug prdiction modls using four bug indicators as th modl input. Th mtrics collctd by promis rpository ar usd as th modl input. Thrfor, th modl construction procss allows assssmnt of appropriatnss of th collctd mtrics as usabl bug prdictors. Th prdictd numbr of bugs for th fils is th modl output. Th prsnt rsarch has bn organizd into six sctions. Sction I introducs th concpts and practics bing adoptd in softwar bug prdiction. Sction II contains dtaild rviw of litratur. Sction III dmonstrats th procss map adoptd by th rsarchrs. Sction IV proposs modling framwor. Sction V & VI contain analysis, conclusion and futur rsarch wor. Nd of th Study Th gnric ralization is that softwar practitionrs nd to focus arly on bug prdiction approachs to nsur rasonabl quality in softwar products. Thrfor, a comprhnsiv rsarch was ndd to widn th scop of bug prdiction approachs and idntify bug indicators causing significant impact on softwar quality. Objctivs 1) To assss th corrlation of bug indicators (DIT, WMC, CBO, LoC) with softwar bugs. 2) To dvlop softwar bug prdiction modls using bug indicators (DIT, WMC, CBO, LoC) as modl inputs. 3) To compar th rlativ ffctivnss of DIT, WMC, CBO and LoC towards prdiction of bugs in Caml and Ant projcts. II. RELATED WORK A significant amount of wor has bn citd using product mtrics to prdict bug pron fils. Though major wor has utilizd Chidambr and Kmrr (CK) mtrics suit [18] to 60 P a g

2 prdict accuratly pr and post rlas bugs in commrcial and opn sourc systms [23, 10, 8, 20, 13, 22]. Furthr, though CK mtrics suit, mpirical justification has also bn mad rgarding usfulnss in bug prdiction [3, 6, 14]. Parto analysis has also bn usd for valuating th ability of modls for idntification of fault-pron classs, moduls and fils. As substantiatd with prsnc of 80% of bugs in 20% of fils [15, 26, 24, 25]. Linar rgrssion has bn widly considrd as a common tchniqu for bug prdiction. Also DIT has bn dmonstratd to carry a linar rlationship with bugs [16]. Furthr, our data was linar in natur advocating application of linar rgrssion. Still, ping with [1], which suggstd application of nonlinar rgrssion as bttr indicator for this typ of data, so dcidd to go ahad with non linar rgrssion. Logistic rgrssion modls hav also bn usd to idntify fault-pron moduls [4]. CK mtrics suit was also usd to find fault-pron classs [19]. This wor involvd invstigation of two C++ writtn projcts and followd with outcom involving analysis of 43-48% of classs to covr for 80% of th bugs Bug prdiction modls wr cratd basd on th modul siz rprsnting Lin of Cod (LoC). Th modls producd outputs in strong corrlation with actual data [12]. Ths modls suggstd considring LoC in th bug prdiction modls. A majority of CK mtrics wr found to b ffctiv prdictors for fault-pronnss of class. In addition, DIT and Rspons for a Class (RFC) wr found to b carrying mor influnc on th dpndnt variabl [2]. A study on data from an industrial systm comprising of mor than 200 C++ subsystms addd diffrnt mtrics than CK mtrics and applid logistic rgrssion to valuat thos mtrics. Th outcoms suggstd WMC and DIT as significant indicators for finding fault-pron classs [21]. Anothr rsarch applying logistic rgrssion on data from a tlcommunication systm having 174 C++ classs dmonstratd clos association of WMC, RFC and Coupling btwn Objcts (CBO) with softwar bugs [5]. Anothr rsarch using univariat logistic rgrssion also idntifid WMC and SLOC as significant prdictors [11]. Anothr rsarch using data from two commrcial applications, on having 150 classs and 23 KSLOC whil othr having 144 classs and 25 KSLOC valuatd th influnc of six CK mtrics on th numbr of bugs and idntifid RFC and DIT as most significant variabls [19]. As pr rcnt citations of th rsarch wors carrid out, no significant amount of wor has bn don on th us of logistic rliability growth modl for bug prdiction. Pronnss to Bugs Softwar failing to fulfill th spcifid rquirmnt nds to b fixd. Signifying that th mista has bn committd btwn th initial rquirmnt and th final opration of th softwar systm. Sinc sourc cod mattrs th most corrsponding to th ralization of th softwar systm, th rrors in sourc cod ar calld bugs. Thr ar changs that rror may not bcom a bug. Howvr, w nd to fix it if it ultimatly bcoms a bug causing a failur. Th pronnss of bugs dpnds on rasons li DIT, WMC, CBO, LoC., DIT (Dpth of Inhritanc Tr): Th maximum lngth from th root to a givn class in th inhritanc hirarchy. DIT is dfind as th maximum lngth inhritanc path from th class to th root class [19]. WMC (Wightd Mthods pr Class): WMC is dfind as th sum of th complxity of th mthods of th class. It is qual to th numbr of mthods whn all mthods ar of th complxity qual to UNITY. Th sum of normalizd complxity of vry mthod in a givn class. CBO (Coupling Btwn Objcts): Th CBO mtric rprsnts th numbr of classs coupld to a givn class. Ths couplings can occur through mthod calls, fild accsss, inhritanc, mthod argumnts, rturn typs and xcptions [18]. LOC (Lin of Cod): th LOC mtric basd on Java binary cod rprsnts sum of numbr of filds, numbr of mthods and numbr of instructions in vry mthod of th invstigatd class. Extractio n of data from Promis Rpositor y Fig. 1. Procss Map III. Extractio n of buggy fils and slction of bug indicators PROCESS MAP Assss mnt of corrlati on btwn bug indicato rs and numbr of bugs Constru ction of bug prdicti on modls Prdicti on of bugs In this papr, th proposd procss map is using th mixd mthod combining qualitativ and quantitativ rsarch mthods. Th rsarch wor is dtaild in fiv phass as shown in Figur 1. A. Extraction of Data: Rsarchrs hav usd PROMISE rpository to xtract th bug indicators (DIT, WMC, CBO and LoC) and bug data. Th rason for slcting th opn sourc projcts from PROMISE rpository was that it is a trustworthy softwar foundation having positiv fdbac from softwar usrs. It is also wll-rcognizd in th softwar community. 61 P a g

3 B. Extraction of buggy fils and slction of bug indicators: Two opn sourc projcts (Caml and Ant) wr prfrrd to xtract bug data from and slction of bug indicators for th analysis. Propr litratur rviw was prformd to slct suitabl bug indicators (DIT, WMC, CBO and LoC) for this rsarch. C. Assssmnt of corrlation btwn bug indicators and bugs: Parson corrlation analysis was prformd to assss th corrlation btwn th various bug indicators (DIT, WMC, CBO, LoC) and numbr of bugs. D. Construction of prdiction modls: Aftr significant corrlation btwn bug indicators and bugs, rsarchrs hav constructd prdiction modls using logistic softwar rliability growth modl on xtractd data from PROMISE bug databas. E. Prdiction: Aftr succssful conclusion of th abov four sub procsss, finally prdictd bugs was givn as th modl output. IV. MODELLING FRAMWORK A. Softwar Rliability Growth Modls (SRGM) Softwar rliability growth modls ar a statistical xclamation of dtctd bug s data using various mathmatical functions. To prdict th numbr of bugs in th cod ths mathmatical functions ar usd. Thr ar many typs of softwar rliability growth modls as to prdict futur bugs or failur rats. B. Modls Assumptions Som of th gnral assumptions (apart from som spcial ons for spcific modls discussd) for th abov modl ar as follows: a) Softwar systm is subjct to failur during xcution causd by bugs rmaining in th systm. b) Failur rat of th softwar is qually affctd by bugs rmaining in th softwar. c) Th numbr of bugs prdictd at any tim instant is proportional to th actual numbr of bugs in th softwar. d) Bug indicators rfrring th softwar siz and its proportional impact on bugs hav th capabilitis of crtain prdiction. ) All bugs ar mutually indpndnt from bug prdiction point of viw. f) Bug prdiction rat/bug dtction rat is a logistic larning function as it is xpctd th larning procss will grow with tim. g) Th bug prdiction phnomnon is modld by Non Homognous Poisson Procss (NHPP). C. Modls Notations a- initial fault-contnt of th softwar. - A constant paramtr in th logistic larning function b 1 - bug prdiction rat/dtction rat pr unit tim. M (t) - xpctd numbr of bugs prdictd. Bug prdiction modls using SRGM ar givn by: 1 m( t) a / 1. b. t Prdiction modl-1 (4.1) DIT is considrd as a first modl input rfrring to th blow mntiond proposd modl: m( t) a / 1. Prdiction modl-2 b1. dit (4.2) WMC is dfind as a scond modl input rfrring to th blow mntiond proposd modl: m( t) a / 1. Prdiction modl -3 b1. wmc (4.3) CBO is dfind as a third modl input rfrring to th blow mntiond proposd modl: m( t) a / 1. Prdiction modl -4 b1. cbo (4.4) LoC is dfind as a fourth modl input rfrring to th blow mntiond proposd modl: m( t) a / 1. b1. loc (4.5) D. Goodnss of Fit Critria Th prformanc of a bug prdiction modl is judgd by its ability to fit th past softwar rliability data and to prdict satisfactorily th futur bhavior from prsnt and past data bhavior. Th following critria dfind as: 2 1) Cofficint of Multipl Dtrminations ( R ) 2) Bias 3) Variation 4) Th Root Man Squar Prdiction Error (RMSPE) 5) Man Squar Error (MSE) Bug Prdiction Paramtr Estimation To xamin th ffctivnss of softwar bug prdiction modls using four indicators as modl input, a st of comparison critria is usd to compar modls quantitativly. Th diffrnt comparison critrions usd in our papr ar as follows: 1) Cofficint of Multipl Dtrmination (R 2 ): This Goodnss-of-fit masur has bn usd to invstigat significanc in trnd xisting in prdiction of bugs. This cofficint was usd as th ratio of th Sum of Squars (SS) drivd from th trnd modl to that from a constant modl subtractd from 1, that is R 2 1 rsidual SS corrctd SS 62 P a g

4 R 2 masurs th prcntag of th total variation about th man accountd for by th fittd curv. It rangs in valu from 0 to 1. Small valus indicat that th modl dos not fit th data wll. With movmnt of valu towards 1, th modl significantly xplains th variation in th data [7]. 2) Bias: Th diffrnc btwn th actual and prdictd numbr of bugs at any instant of tim i is nown as Prdiction Error (PE i ). Th avrag of PEs is nown as bias. With movmnt of valu towards 0, th modl significantly xplains low prsnc of prdiction rror. Th bias is dfind mathmatically as [9]: Bias Whr m i indicats actual bugs, m(t) indicats prdictd bugs and is th numbr of obsrvations in th data st. 3) Varianc: Th varianc is dfind as [9]. i1 m ti ^ mi ^ 1 Varianc m mt Bias i i 1 i1 4) Root Man Squar Prdiction Error (RMSPE): It masurs th closnss with which th modl prdicts th bugs and mathmatical rprsntation of this charactristic is givn as [9]. 2 2 RMSPE Varianc Bias 5) Man Squar Error (MSE): MSE masurs th diffrnc btwn th prdictd and actual valus of bugs, and is givn mathmatically as [17]. ^ 2 mi m ti i1 MSE p Whr is th numbr of obsrvations in th data st and p is th numbr of paramtrs. E. Data Sts Th data about bug indicators and bugs has bn collctd from PROMISE rpositoris. Th following data sts hav bn usd with xplanations mard in: Data St 1(Caml) Apach Caml is a powrful opn sourc intgration framwor basd on nown Entrpris Intgration Pattrns with powrful Ban Intgration. Data St 2 (Ant) Ant is a wll nown Java-basd, shll indpndnt build tool. V. ANALYSIS AND CONCLUSION Whil chcing th accuracy of diffrnt proposd modls of bug prdiction using diffrnt bug indicators, rsarchrs hav first stimatd th unnown paramtrs of bug data for final softwar product on bug cumulativ consumption data. Thn, to judg th fitting of various proposd modls of prdiction givn by quations (4.2), (4.3) (4.4) and (4.5) R 2, bias, variation, RMSPE and MSE hav bn calculatd as th prformanc masurs. Tabl I and Tabl II dpict th stimatd valus for th paramtrs whil Tabl III provids th corrlation critria and finally Tabl IV and Tabl V 2 summarizs th stimatd and optimizd valus of attributs of proposd modls. TABLE I. ESTIMATED PARAMETERS OF PROPOSED MODELS USING DS-1 S. Paramtr Estimatd paramtrs valus No. s DIT WMC LOC CBO 1 a K b TABLE II. S. No. ESTIMATED PARAMETERS OF PROPOSED MODELS USING DS-2 Paramtrs Estimatd paramtrs valus DIT WMC LOC CBO 1 a K b In our rsarch, rsarchrs obsrvd significant corrlations of WMC, DIT, CBO and LOC with bugs. In this rsarch only highly corrlatd four mtrics hav shown from ach data st that ar listd in Tabl III. Th intrsting part of this rsult is that all four indicators ar corrlatd significantly with softwar bugs. TABLE IV. TABLE III. CORRELATION TABLE Projct Mtrics Corrlation with Bugs DIT.976 Caml WMC.987 Ant LOC.984 CBO.992 DIT.997 WMC.989 LOC.992 CBO.991 ESTIMATED AND OPTIMAL VALUES OF ATTRIBUTES FOR FOUR PREDICTION MODELS FOR DS-1 Projct Mtrics R2 Bias Varianc RMSE MSE Caml TABLE V. DIT WMC LOC CBO ESTIMATED AND OPTIMAL VALUES OF ATTRIBUTES FOR FOUR PREDICTION MODELS FOR DS-2 Projct Mtrics R2 Bias Varianc RMSE MSE Ant DIT WMC LOC CBO P a g

5 Prdictd Bugs Prdictd Bugs (IJACSA) Intrnational Journal of Advancd Computr Scinc and Applications, In tabl III rsarchrs obsrvd significant corrlations of WMC, DIT, CBO and LOC with bugs. Tabl IV dpictd that in cas of DS-1 using prdiction modl 4.2 th prdictiv modl cofficint of dtrmination is it mans 99.5% of th variation in bugs is associatd with numbr of prdictor. Whras using modl 4.3, modl 4.4 and modl 4.5 th variation in bugs is 98.9%, 98.6% and 98.9% rspctivly. Tabl V dpictd that in cas of DS-2 using prdiction modl 4.2 th prdictiv modl cofficint of dtrmination is it mans 99.1% of th variation in bugs is associatd with numbr of prdictor. Whras using modl 4.3 modl 4.4 and modl 4.5 th variation in bugs is 98.3%, 98.3% and 98.9% rspctivly Fig. 2. Graph for Pattrn of Actual and Prdictd Softwar Bugs of DS Actual Bugs Actual Bugs Fig. 3. Graph for Pattrn of Actual and Prdictd Softwar Bugs of DS-2 As shown abov graphs in Figur 2 and Figur 3, th prdictd numbr of bugs is significantly highr than actual numbr of bugs. Th rsarch has comprhnsivly dsignd and tstd four modls using DIT, WMC, CBO and LoC as modl inputs. Ths modls producd significant rsults on all four modl inputs. Howvr, modl using DIT as input was shown to b bttr prforming than th othr thr modls. This conclusion can srv as strong motivation for softwar practitionrs to prioritiz and allocat sufficint rsourcs towards DIT bcaus of its bttr prformanc in comparison to WMC, CBO and LoC. VI. FUTURE WORK p_bugs(dit) p_cbo p_bugs(loc) p_bugs(wmc) Actual_Bugs Prd_bugs (DIT) prd_bugs (CBO) prd_bugs (wmc) prd_bugs (loc) Mor product mtrics as bug indicators can b includd in futur rsarch wor. Mor opn sourc data sts can also b includd to bring highr rliability in bug prdiction. An ffort can b mad of applying diffrnt non linar rgrssion modls on sam two data sts alrady considrd in prsnt rsarch wor. REFERENCES [1] A. Brnstin, J. Eanaya, and M. Pinzgr. Improving dfct prdiction using tmporal faturs and non linar modls. In Proc. Int l Worshop on Principls of Softw. Evolution, pags 11 18, [2] B asili, V.R., L.C. Briand and W.L. Mlo, A validation of objctorintd dsign mtrics as quality indicators. IEEE Trans. Softwar Eng., 22: DOI / [3] CATAL C., DIRI B. and OZUMUT B., An Artificial Immun Systm Approach for Fault Prdiction in Objct-Orintd Softwar. Proc. of Dpndability of Computr Systms, 2007, [4] DENARO G. and PEZZE M., An Empirical Evaluation of Fault- Pronnss Modls. Proc. of Intrnational Confrnc on Softwar Enginring (ICSE), [5] El Emam, K., S. Bnlarbi, N. Gol and S.N. Rai, Th confounding ffct of class siz on th validity of objct-orintd mtrics. IEEE Trans. Softwar Eng., 27: DOI: / [6] JURECZKO M., Us of softwar mtrics for finding wa points of objct orintd projcts. Proc. Of Mtody i narzędzia wytwarzania oprogramowania , 2007 (in Polish). [7] K. C. Chiu, Y. S. Huang, and T. Z. L, A study of softwar rliability growth from th prspctiv of larning ffcts, Rliability Enginring and Systm Safty, pp , [8] K. E. Emam, W. Mlo, and J. C. Machado. Th prdiction of faulty classs using objct-orintd dsign mtrics. J. Syst. Softw., 56(1):63 75, [9] K. Pillai and V. S. S. Nair, A modl for softwar dvlopmnt ffort and cost stimation, IEEE Trans. Softw. Enginring, vol. 23, no. 8, pp , 1997 [10] L. C. Briand, J. W. Daly, and J. K. W ust. A unifid framwor for coupling masurmnt in objct-orintd systms. IEEE Trans. Softw. Eng., 25(1):91 121, [11] Malhotra R., and Jain A., Fault prdiction using statistical and machin larning mthods for improving softwar quality, Journal of Information Procssing Systms, Vol.8, No.2, Jun 2012 [12] MENDE T., KOSCHKE R., Rvisiting th Evaluation of Dfct Prdiction Modls. Proc. of PROMISE, [13] N. Nagappan, T. Ball, and A. Zllr. Mining mtrics to prdict componnt failurs. In Proc. Int l Conf. on Softw. Eng. (ICSE 06), pags , [14] OLAGUE H. M., ETZKORN L. H., GHOLSTON S. and QUATTLEBAUM S., Empirical Validation of Thr Softwar Mtrics Suits to Prdict Fault-Pronnss of Objct-Orintd Class Dvlopd Using Highly Itrativ or Agil Softwar Dvlopmnt Procsss. IEEE Trans. on Softwar Enginring, 33(6), 2007, [15] OSTRAND T. J., WEYUKER E. J. and BELL R. M., Prdicting th Location and Numbr of Faults in Larg Softwar Systms. IEEE Trans. on Softwar Enginring, 31(4), 2005, [8] [16] S. Dowdy, S. Wardon, and D. Chilo. Statistics for Rsarch. Probability and Statistics. JohnWily and Sons, Hobon, Nw Jrsy, third dition, [17] S. Hwang and H. Pham, Quasi-rnwal tim-dlay fault-rmoval considration in softwar rliability modlling, IEEE Trans. Systms, Man and Cybrntics-Part A: Systms and Humans, vol. 39, no. 1, January [18] S. R. Chidambr and C. F. Kmrr. A mtrics suit for objct orintd dsign. IEEE Trans. Softw. Eng., 20(6): , 1994.[1] [19] SUCCI G., PEDRYCZ W., STEFANOVIC M. and MILLER J., Practical assssmnt of th modls for idntification of dfct-pron classs in objct-orintd commrcial systms using dsign mtrics. Journal of Systms and Softwar 65(1), 2003, [20] T. Gyimothy, R. Frnc, and I. Sit. Empirical validation of objctorintd mtrics on opn sourc softwar for fault prdiction. IEEE Trans. Softw. Eng., 31(10): , [21] Tang, M.H., M.H. Kao and M.H. Chn, An mpirical study on objct-orintd mtrics. Procdings of th 6th Intrnational Symposium 64 P a g

6 on Softwar Mtrics, Oct , IEEE Computr Socity, Boca Raton, FL., USA., pp: DOI: /METRIC [22] T. Zimmrmann, R. Prmraj, and A. Zllr. Prdicting dfcts for Eclips. In Proc. Int l Worshop on Prdictor Modls in Softwar Enginring (PROMISE 07), pags 1 7, [23] V. R. Basili, L. C. Briand, and W. L. Mlo. A validation of objctorintd dsign mtrics as quality indicators. IEEE Trans. Softw. Eng., 22(10): , [24] WEYUKER E. J., OSTRAND T. J. and BELL R. M., Adapting a Fault Prdiction Modl to Allow Widsprad Usag. Proc. of PROMISE, [25] WEYUKER E. J., OSTRAND T. J. and BELL R. M., Do too many coos spoil th broth? Using th numbr of dvloprs to nhanc dfct prdiction modls. Empirical Softwar Enginring, 13(5), 2008, [26] WEYUKER E. J., OSTRAND T. J. and BELL R. M., Comparing Ngativ Binomial and Rcursiv Partitioning Modls for Fault Prdiction. Proc. of PROMISE, P a g