Dynamic Two-phase Truncated Rayleigh Model for Release Date Prediction of Software

Transcription

1 J. Software Egeerg & Applcatos do:.436/jsea..367 Publshed Ole Jue ( Dyamc Two-phase Trucated Raylegh Model for Release Date Predcto of Software Lafe Qa Qgchua Yao Tagh M. Khoshgoftaar Departmet of Mathematcal Sceces Florda Atlatc Uversty Boca Rato USA; Departmet of Computer Scece ad Egeerg Florda Atlatc Uversty Boca Rato USA. Emal: [email protected] [email protected] [email protected] Receved October 3 rd 9; revsed November 3 th 9; accepted November 5 th 9. ABSTRACT Software relablty modelg ad predcto are mportat ssues durg software developmet especally whe oe has to reach a desred relablty pror to software release. Varous techques both statc ad dyamc are used for relablty modelg ad predcto the cotext of software rsk maagemet. The sgle-phase Raylegh model s a dyamc relablty model; however t s ot sutable for software release date predcto. We propose a ew mult-phase trucated Raylegh model ad obta parameter estmato usg the olear least squares method. The proposed model has bee successfully tested a large software compay for several software projects. It s show that the two-phase trucated Raylegh model outperforms the tradtoal sgle-phase Raylegh model modelg weekly software defect arrval data. The model s useful for project maagemet plag release tmes ad defect maagemet. Keywords: Software Testg Weekly Defect Arrval Data Sgle-Phase Raylegh Model Two-Phase Trucated Raylegh Model Software Relablty. Itroducto Software relablty s a key attrbute of software qualty. Varous models have bee developed for software relablty egeerg []. The rsg complexty sze ad fuctoalty of software systems make software relablty predcto dffcult. The problem s compouded wth short developmet tmes ad strct release deadles. Cosequetly predctg the release date for achevg pre-specfed system relablty has become a very mportat ssue software project developmet. Relablty modelg ca ot oly assst fulfllg commtmets ad project deadles but also ad effcet resource maagemet ad plag. Software relablty s the probablty of falure-free software operato for a gve perod of tme a gve operatg evromet. The key attrbute software relablty egeerg s the umber of defects observed specfed tme tervals (e.g. weeks). Software relablty predcto models assess a software product s relablty or estmate the umber of latet defects whe t s released to the customers. Such a estmate s mportat for two reasos: ) as a objectve statemet of the qualty of the product ad ) for resource plag the software mateace phase. There are two categores of software relablty models: statc ad dyamc models. Amog the statc models Bayesa belef etworks [] ad utlzg software process metrcs are relatvely popular. Related lterature also proposes varous models for software defect predcto whch ca be used to drectly gauge software relablty [34]. The prmary drawback amog statc models ca ot effectvely capture the software process ad ts varatos durg the course of software project developmet. O the other had a dyamc software relablty model s reflectve of the software testg phase ad s geerally applcable before product release. Amog dyamc models the (sgle-phase) Raylegh model has bee show sutable to ft software defect arrval patters [56]. A sgle-phase Raylegh model dvdes the whole software developmet lfe cycle to sx stages that are chroologcal order: Hgh Level Desg (HLD) Low Level Desg (LLD) CODING Ut Testg (UT) Itegrato Testg (IT) ad System Testg (ST). The sx stages are assged to a sequece of umercal scales. That s: HLD =.5 LLD =.5 CODING =.5 UT = 3.5 IT = 4.5 ad ST = 5.5 [5]. Those umercal assgmets seem rather ad hoc. Istead we Copyrght ScRes.

2 64 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software could assg the sx stages to for stace {t t t 3 t 4 t 5 6 t t 6 } as log as satsfes t < t < t 3 < t 4 < t 5 < t 6. Wth dfferet umercal assgmets for the sx stages the ftted sgle-phase Raylegh models could show a much dfferet accuracy patter as show Fgure. The defects/kloc Fgure s recostructed from the work of Thagaraja et al. [5]. The quadratc ft s show to llustrate that the small pars of data could be ftted well by a arbtrary model such as quadratc model rather tha just sgle-phase Raylegh model. Also by assgg oe umercal umber to each stage the data set ow cota oly sx pars at most. For predcto purposes most lkely 3 4 ad 5 pars avalable such a small sample sze offers o cofdece the relablty predcto. Durg the software developmet lfe cycle collectg oe sgle represetatve umber for each stage results a very small sample sze. Furthermore t s more lkely that the data of major software defects are followed weekly hece allowg project maagemet to motor the dyamc progress of the software developmet process. Our motvated weekly software developmet defects data set Fgure shows the serous adequacy of the sgle-phase Raylegh model. Ths leads to our research o developg a better dyamc software relablty model to estmate the umber of major defects hece predct software release date. The exstg orgazatoal relablty predcto model for software release date predcto at a large software compay where the weekly data Fgure were collected s the dyamc sgle-phase Raylegh model [56]. The software process the orgazato cossted of two or more developmet phases. Ths s due to the software producto cycles avalablty of supportg hardware (e.g. wgboard/test phoes) the earler software developmet stages ma-power maagemet (e.g. testers rearragemet) durg the software developmet phases ad other dyamc ssues durg developmet. Fgure for stace shows that the scatter plot overlad wth the sgle ad the ewly proposed two-phase trucated (pecewse for short) Raylegh models for the data set from the large software compay. It s clear that the two-phase trucated Raylegh model fts the data much better tha the sgle-phase Raylegh model. Motvated by the example we propose a multplephase trucated Raylegh model ths paper. Such a model s better suted to ft the weekly defect arrval patters durg software developmet process. For smplcty reasos we focus o the two-phase trucated (pecewse) Raylegh model. The model ca be exteded to clude addtoal phases reflectg the developmet process. It s show through emprcal modelg that the model accuracy s sgfcatly mproved. Furthermore usg the two-phase trucated Raylegh model the release date s predcted wth a much hgher cofdece level. The paper s orgazed as follows: Secto summarzes the sgle-phase Raylegh model ad proposes the mult-phase model wth a focus o the two-phase trucated Raylegh model. Secto 3 presets the algorthms of olear least squares estmators of the model parameters ad flowcharts of the dyamc process. Secto 4 apples the proposed two-phase trucated Raylegh model to defect arrval data of a large real-world software project from the large software orgazato. Fally Secto 5 cocludes the paper ad provdes suggestos for future work.. Mult-Phase Trucated Raylegh Models for Software Relablty Predcto The dyamc sgle-phase Raylegh model s a stadard techque for software relablty modelg ad has bee wdely used for the software project ad qualty maagemet the software dustry. The software orgazato from whch our case study data s obtaed has utlzed the dyamc sgle-phase Raylegh model for several of ther prevous software project developmets. The sgle-phase Raylegh model s a parametrc regresso model wth the regresso fucto specfed by the Raylegh dstrbuto wth a multpler coeffcet. Whe the parameters of the Raylegh dstrbuto are estmated based o the updated data from a software project dyamc projectos about the umber of defects for the software ca be made based o the model over the software developmet lfe cycle. The Raylegh dstrbuto s a specal case of Webull dstrbuto ad has varous applcatos cludg relablty estmato ad lfe cycle patter modelg [78] developg software projects lfe testg expermets clcal studes dealg wth cacer patets [9]. We ow summarze the Raylegh dstrbuto. Deote t m be the tme at whch the sgle-phase Raylegh desty curve reaches ts peak. The cumulatve dstrbuto fucto of Raylegh dstrbuto wth the costat multpler K (the total umber of latet defects) s t F (; t K ) K e where =/(t m ) s the scale parameter. The sgle-phase Raylegh model has a regresso fucto parameterzed as f (; t K ) Kte t () where both K ad are the two parameters that eed to be estmated usg the data. The sgle-phase Raylegh model () does ot ft the Copyrght ScRes.

3 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software 65 Fgure. Sgle-Raylegh model vs. quadratc model for two ad hoc umercal assgmets for the ordal stages the software developmet lfe cycle. Sold le s for the sgle-phase Raylegh model whle the dashed le s for the quadratc model. (a) (HLDLLD CODING UT IT ST) = ( ); (b) (HLDLLD CODING UT IT ST) = ( ) Fgure. Major defects vs. developmet tme weeks case study data set well. Actually t s a very poor ft as see Fgure ad makes the case for a much eeded mprovemet modelg software defect arrval patters. We propose a ew mult-phase trucated Raylegh model Copyrght ScRes.

4 66 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software defed as below: f(; t K ) t f( t; K ) t gt (; ) f( td ; Kd d) d d t where d s the umber of phases ad T =( d K K d d- d- ) s the model parameter vector. For smplcty we wll dscuss the case wth d = the two-phase trucated (pecewse) Raylegh model wth regresso fucto parameterzed as follows: f( t; K ) t gt (; ) f ( t; K ) t where s the locato of the phase chage s the startg locato for the secod phase. Due to the ature of the software defect data we suggest to use the left trucated Raylegh model for the secod phase. The T = ( K K ) s the parameter vector eed to be estmated. 3. Algorthms for Pecewse Raylegh Models I ths secto we descrbe the olear least squares estmator of the model parameters. Let ( t ) d be the defect arrval data collected over tme where t / s the tme dex for the th week d s the total umber of software defects detected durg the th week ad s the umber of weeks observed. Let S( ) d g( t; ). The the olear least squares estmator s the mmzato of S(). Notce that S() s ot dfferetable the locato of phase chage pot ad the startg pot of secod phase. I cojucto wth olear least squares method ad Gauss-Newto algorthm we utlze a four-step techque (descrbed below) to obta the estmators of the parameter vector. The package ls R laguage s used to obta the estmates of the model parameters. Step : For ay gve locato of phase chage ( ) fx a such as < < we compute the olear least squares estmators [] ( ) for the smooth T parameters ) ( K K by mmzg S() over. Step : Substtute ( ) to S() to obta the profle objectve fucto S( ). The we mmze S ( ) over < < for the gve to obta ( ). Notce that the mmzer ( ) s a fucto of. () Step 3: Substtute ( ) to S ( ) to get S ( ). The mmzer of S ( ) over ( ) s called the chage pot estmator deoted by ˆ. Step 4: Substtute ˆ to ( ) to get ˆ ad ˆ ˆ to get ˆ. Put them together we obta the olear least T ˆT ˆ ˆ ˆ of. squares estmator Fgures 3 ad 4 llustrate the flow charts of the dyamc process of the algorthm for sgle-phase ad mult-phase trucated Raylegh models respectvely. We provde the flowchart for the sgle-phase Raylegh model for comparso purpose. 4. Applcato to a Real Software Defect Data Set The data set motvated our research were collected from Feb-5-6 to Aug-4-7 at a large software compay. There are 76 weeks software defects arrval data. Number of major defects durg a week s reported. 4. Sgle-phase vs. Pecewse Raylegh Models We llustrate the two-phase trucated Raylegh model by fttg the software defect arrval data set. From Fgure t s observed that usg two-phase trucated Raylegh model mproves the model fttg sgfcatly compared to the sgle-phase Raylegh model wth respect to model accuracy ad model goodess-of-ft. For comparso Fgure 3. Algorthm for sgle-phase Raylegh model Copyrght ScRes.

5 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software 67 Fgure 4. Algorthm for mult-phase Raylegh model purpose the estmated sgle-phase Raylegh regresso fucto s gve by ˆ gt ˆ( ) Kˆ ˆ te t where Kˆ 3.7 ad ˆ.597. For the two-phase trucated Raylegh model () the estmated chage s at the ˆ = 33rd week wth the startg pot estmated at ˆ = 3st week for the secod phase. Hece phase oe s from the frst week to 33rd week ad phase two s from the 34th week to the 76th week wth estmated startg pot at ˆ = 3st week. The estmated frst phase (rght trucated) of the regresso fucto s estmated as ˆ ˆ ˆ ( t gt ˆ( ) K 3/76) ( t 3/76) e f t 33 / 76 wth K ˆ 4.3 ˆ 5.79 ad the secod phase (left trucated) of the regresso fucto s estmated as ˆ ˆ ˆ ( t gt ˆ( ) K 3/76) ( t 3/76) e f t 33/ 76 wth K ˆ ˆ.445. Fgure shows the scatter plot overlad wth the two ftted curves usg the sgle-phase ad two-phase trucated Raylegh models respectvely. From the ftted model oe ca predct the future week s umber of software defects ad establsh the qualty assurace crtero ad maagemet for predctg the release date. Ths proposed mult-phase trucated Raylegh model ca be utlzed for modelg ay future software developmet projects to obta better predcto ad provde more effcet estmato of the release date of the software product. 4. Qualty Assurace Crtero for Release Date Predcto I ths secto we establsh the qualty assurace crtero for software release. The qualty assurace crtero s determed by 95% ad 99.9% cofdece levels. That s based o the ftted model f the model shows that 95% or 99.9% of the total expected software defects has bee detected the we suggest that the software s ready for release. For the sgle-phase Raylegh model we estmate the release date wth 95% cofdece level. We set F t; Kˆ ˆ.95Kˆ ad solve for t or equvaletly ˆ e t.95. Ths mples that the release date equals to the celg of l(.95) ˆ 7 weeks where ˆ=.597. Hece wth 95% cofdece the software project wll Copyrght ScRes.

6 68 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software eed 7 76 = 3 weeks of further testg before releasg the software product. That s the predcted release date usg the 76 weeks of data s Feb-9-8 based o the sgle-phase Raylegh Model. Wth 99% cofdece t wll requre eve much loger testg tme. Alteratvely utlzg the two-phase trucated Raylegh model () we set ˆ/ / gˆ () tdt gtdt ˆ ().999 gtdt ˆ () ˆ / ad solve for to get the estmated release week wth 99.9% cofdece level. Equvaletly the estmated release week umber satsfes that e ˆ / ˆ ˆ ˆ ˆ ˆ gˆ tdt e.999a A A Kˆ where A A A e.999 ( ) gtdt ˆ( ) Kˆ (3) ˆ ˆ ˆ ˆ /.99 gˆ ( t) dt.6967 gˆ ( t) dt.586. Solvg Equato (3) to obta the estmated release week umber: ˆ.999A A ˆ / l A 76 weeks Kˆ where {x} s the smallest teger greater or equal to x. Ths dcates that wth 99.9% cofdece that the estmated release week s the ed of 76th week. That s the software s ready for release wth almost % cofdece based o the two-phase trucated Raylegh model. We ote that the large software orgazato has adopted our ew two-phase trucated Raylegh model ad s usg t to predct the umber of software defects dyamcally ad release dates for ogog software projects. Our ew two-phase trucated Raylegh model has mproved the software release lfe cycle a great deal ad has saved a lot of ma-powered resource for the large software orgazato. 4.3 Model Performace Check We utlze three measures of goodess-of-ft to assess the performace of the models: root mea square error (RMSE) magtude of relatve error (MRE) ad ad- justed coeffcet of determato R. The root mea adj square error measures the model accuracy defed as the square root of mea squared resduals. That s RMSE d dˆ 5 where d s the umber of defects detected durg the th week dˆ s the ftted (predcted) value of d. The smaller the RMSE the better the model fts. The secod crtero for assessmet of the performace of model fttg used the relablty lterature s the mea magtude of relatve error defed as d ˆ d Id ( ) d MRE. Id ( ) The mplct assumpto ths summary measure s that the serousess of the absolute error s proportoal to the sze of the observatos. The smaller the MRE the better the model fts. The thrd measure of goodess-of-ft used s the adjusted determato of coeffcet whch s the adjusted percetage of varato the umber of defects per week explaed by the model. That s SSE /( 5) Radj SSTO /( ) where SSE = (-5)(RMSE) ad d wth SSTO d. d d The hgher of the the better the model fts. Table summarzes the three performace crtera for the real-world weekly software defects data set usg both sgle-phase ad two-phase trucated (pecewse) Raylegh models. Based o the reported RMSE MRE ad values the two-phase trucated Raylegh mo- del s much better tha the sgle-phase Raylegh model. The MRE s reduced by about 5% whle the goodess-of-ft measure s roughly doubled for the two- phase trucated compared to the sgle-phase Raylegh models. The two-phase trucated Raylegh model explas the almost doubled varato the umber of defects tha the sgle-phase Raylegh model does. Thus based o the gve data we coclude that the two-phase trucated Raylegh model s a attractve model for predctg weekly software defects ad release date of software projects. Copyrght ScRes.

7 Dyamc Two-Phase Trucated Raylegh Model for Release Date Predcto of Software 69 Table. Model comparsos usg RMSE MRE ad Crtero Model RMSE MRE R adj Sgle-phase % Two-phase % 5. Coclusos The research was motvated by a real-world software defect arrval data over may weeks from a large software orgazato. The paper proposes a ew mult-phase trucated (focusg o a two-phase trucated model) Raylegh model fttg weekly defect arrval data. It s show that the proposed model s much more accurate tha the exstg sgle-phase Raylegh model. The sgle-phase model was prevously used by the orgazato durg software developmet. Usg both MRE ad performace measures the proposed model almost doubled the predcto accuracy hece shorteg the release date predcto wth a hgher cofdece level. From a software relablty perspectve our proposed two-phase trucated Raylegh predcto model wll help the maagemet ad plag of project resources toward betterg the software release cycle tme. The two-phase trucated Raylegh model ca be easly exteded to a mult-phase trucated Raylegh model. Hece t ca be used to predct release date for future software projects wth a hgher cofdece level. A geeral mult-phase Raylegh software release predcto model ca be developed to automatcally detect ad reflect all the chage locatos ad the startg pots of the software developmet phases so that the multple-phase trucated Raylegh software predcto model ca be geerated to automatcally forecast the software release tme. REFERENCES [] M. R. Lyu Software Relablty: To Use or ot to Use? Proceedgs of 5th Iteratoal Symposum o Software Relablty Egeerg November 994. [] Y. Wag ad M. Smth Release Date Predcto for Telecommucato Software Usg Bayesa Belef Networks Proceedgs of the IEEE Caada Coferece o Electrcal ad Computer Egeerg pp [3] T. M. Khoshgoftaar ad N. Selya Fault Predcto Modelg for Software Qualty Estmato: Comparg Commoly Used Techques Emprcal Software Egeerg Joural Vol. 8 No. 3 3 pp [4] T. M. Khoshgoftaar ad N. Selya Comparatve Assessmet of Software Qualty Classfcato Techques: A Emprcal Case Study Emprcal Software Egeerg Joural Vol. 9 No. 3 4 pp [5] M. Thagaraja ad B. Bswas Mathematcal Model for Defect Predcto across Software Developmet Lfe Cycle The SEPG (Software Egeerg Process Group) Coferece Ida. com/cofereces/sepg/dex.html [6] S. H. Ka Metrc ad Models Software Qualty Egeerg d Edto Addso Wesley Massachusetts 3. [7] P. V. Norde Useful Tools for Project Maagemet Operatos Research Research ad Developmet B. V. Dea Ed. Joh Wley & Sos New York 963. [8] L. H. Putma A Geeral Emprcal Soluto to the Macro Software Szg ad Estmatg Problem IEEE Trasacto o Software Egeerg Vol. SE pp [9] S. K. Bhattacharya ad R. K. Tyag Bayesa Survval Aalyss Based o the Raylegh Model Trabajos de Estadstca Vol. 5 No. 99 pp [] D. M. Bates ad J. M. Chambers Nolear Models Chapter of Statstcal Models S. J. M. Chambers ad T. J. Haste Eds. Wadsworth & Brooks/Cole 99. Copyrght ScRes.