Omar Shatnawi. Eks p l o a t a c j a i Ni e z a w o d n o s c Ma in t e n a n c e a n d Reliability Vo l.16, No. 4,

Transcription

1 Arcle caon nfo: Shanaw O. Measurng commercal sofware operaonal relably: an nerdscplnary modellng approach. Esploaacja Nezawodnosc Manenance and Relably 014; 16 (4): Omar Shanaw Measurng commercal sofware operaonal relably: an nerdscplnary modellng approach Pomar nezawodnośc esploaacyjnej oprogramowana omercyjnego: nerdyscyplnarne podejśce do modelowana In he sofware relably engneerng (SRE) leraure, few aemps have been made o model he falure phenomenon of commercal sofware durng s operaonal use. One of he reasons can be arbued o he nably of sofware engneers o measure he growh n usage of commercal sofware whle s n he mare. I s unle he esng phase where resources follow a defne paern. In hs paper, an aemp has been made o model he sofware relably growh lnng o he number of users. Snce he number of nsrucons execued depends on he number of users. The number of users s esmaed hrough an nnovaon dffuson model of mareng. Once he esmaed value s nown, he rae a whch nsrucons are execued can be found. The nensy wh whch falures would be repored depends upon hs value. To model he falure observaon or defec removal phenomena, a non-homogenous Posson process (NHPP) based sofware relably models developed n he leraure have been employed. Sofware relably models are mos ofen used for relably projecon when developmen wor s complee and before he sofware s shpped o cusomers. They can also be used o model he falure paern or he defec arrval paern n he feld and hereby provde valuable npu o manenance plannng. Numercal example wh real sofware feld relably daa s presened o llusrae descrpve and predcve performance as well as o show praccal applcaons of he proposed models. Keywords: sofware relably engneerng, sofware relably models, non-homogenous Posson process, mperfec debuggng, commercal sofware usage, nnovaon dffuson. Leraura doycząca nżyner nezawodnośc oprogramowana, podejmuje zaledwe nelczne próby modelowana zjawsa uszodzena oprogramowana omercyjnego w race jego esploaacj. Jednym z powodów może być o, ż programśc ne są w sane zmerzyć wzrosu użyowana oprogramowana omercyjnego w race obrou handlowego ego ypu oprogramowanem. Eap en różn sę bowem od fazy esowana, gdze zasoby funcjonują według oreślonego wzorca. W nnejszej pracy podjęo próbę sworzena modelu wzrosu nezawodnośc oprogramowana łącząc o pojęce z pojęcem lczby użyownów, jao że lczba wyonywanych poleceń zależy właśne od lczby użyownów. Lczbę użyownów szacuje sę na podsawe modelu marengu oparego na dyfuzj nnowacj. Gdy szacowana warość jes już znana, można oreślć częsość wyonywana poleceń. Inensywność zgłaszana uszodzeń zależy od ej warośc. Do modelowana zjaws zaobserwowana uszodzena lub usunęca user zasosowano opracowane wcześnej w leraurze modele nezawodnośc oprogramowana opare na nejednorodnym procese Possona (NHPP). Modele nezawodnośc oprogramowana są najczęścej wyorzysywane do projeowana nezawodnośc już po zaończenu prac rozwojowych, ale zanm jeszcze oprogramowane dorze do lenów. Mogą być równeż sosowane do modelowana wzorców uszodzeń lub wzorców wysępowana usere w race esploaacj, sanowąc ym samym cenny wład do planowana czynnośc onserwacyjnych. Przyład lczbowy uwzględnający dane z esploaacj rzeczywsego oprogramowana lusruje opsowe predycyjne możlwośc proponowanych model, ja równeż poazuje, ja można je sosować w prayce. Słowa luczowe: nżynera nezawodnośc oprogramowana, modele nezawodnośc oprogramowana, nejednorodny proces Possona, nedosonałe debugowane, użyowane oprogramowana omercyjnego, dyfuzja nnowacj. 1. Inroducon Sofware relably s defned as he probably of falure free sofware operaon for a specfed perod of me (Amercan Naonal Sandards Insue ANSI). I quanfes he falures of sofware sysems and s he ey facor n sofware qualy [19]. I s also a major subjec of Sofware Relably Engneerng (SRE) a dscplne whch quanavely sudes he operaonal behavor of sofware sysems wh respec o he relably requremens of he user. The quanave sudy of sofware sysems concernng relably nvolves sofware relably measuremens. Measuremen of sofware relably ncludes wo acves,.e., sofware relably esmaon and sofware relably predcon. Sofware relably modelsare used o measure a sofware produc's relably or o esmae he number of laen defecs when s avalable o he cusomers. Such an esmae s mporan for wo reasons: 1) as an objecve saemen of he qualy of he produc and ) for resource plannng for he sofware manenance phase [9]. Research has been conduced n sofware relably engneerng over he pas hree decades and many sofware relably models have been proposed [4, 1, 19, 0, 3, 9, 30]. The poneerng aemp n non-homogenous Posson process (NHPP) based on sofware relably model was he exponenal model [7]. The model descrbes he falure/removal phenomenon by an exponenal curve. There are also sofware relably models ha descrbe eher S-shaped curves or a mxure of exponenal and S-shaped curves (.e., flexble). Some of he mporan conrbuons of hese ype of models are due o[11, 1, 3] ec. In mos of hese models s assumed ha whenever an aemp s made o remove a defec, s removed wh cerany.e., a case of perfec debuggng. Bu he debuggng acvy s no always Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4,

2 Scence and Technology perfec because of number of facors le eser s sll/experse ec. In praccal sofware developmen scenaro, he number of falures observed/deeced may no be necessarly same as he number of defec removed/correced. Kapur and Garg [11] have dscussed n her defec removal phenomenon model ha as esng grows and esng and debuggng eam gans experence, addonal numbers of defecs are removed whou hem causng any falure. The esng and debuggng, however, may no be able o remove/correc defec perfecly on observaon/deecon of a falure and he orgnal defec may reman leadng o a phenomenon nown as mperfec debuggng, or replaced by anoher defec resulng n defec generaon. In case of mperfec debuggng he defec-conen of he sofware s no changed, bu because of ncomplee undersandng of he sofware, he orgnal deeced defec s no removed perfecly. Bu n case of defec generaon, he oal defec-conen ncreases as he esng and debuggng progresses because new defecs are nroduced n he sysem whle removng he old orgnal defecs [5, 14, 15, 6].Models due o [, 33] are defec generaon models appled on he exponenal model [7] and have been also named as mperfec debuggng models. Kapur e al. [1] nroduced he mperfec defec debuggng n exponenal model [7]. They assumed ha he defec deecon rae per remanng defecs s reduced due o mperfec defec debuggng. Thus he number of falures observed/deeced by me nfny s more han he nal defec-conen. Alhough hese wo models descrbe he mperfec debuggng phenomenon ye he sofware relably growh curve of hese models s always exponenal. Moreover, hey assume ha he probably of mperfec debuggng s ndependen of he esng me. Thus, hey gnore he role of he learnng process durng he esng phase by no accounng for he experence ganed wh he progress of sofware esng. Zhang e al. [34] proposed a esng effcency model whch ncludes boh mperfec defec debuggng and defec generaon, modelng on he number of falures experenced, however boh mperfec debuggng and defec generaon are acually seen durng defec removal. Recenly, Kapur e al. [15] proposed a flexble sofware relably model wh mperfec defec debuggng and defec generaon usng a logsc funcon for defec deecon rae, whch reflecs he effcency of he esng and debuggng eam. Very few aemps have been made o model he falure phenomenon of sofware produc durng s operaonal use. One of he reasons for hs can be arbued o he nably of sofware engneers o measure he growh n usage of sofware whle s n he mare. I s unle he esng phase where esng-effor follows a defne paern. Kenney [17] developed a calendar-me model for a mul-release produc usng Trachenberg s [8] general heory of sofware relably. He has assumed a power funcon o represen he usage rae of he sofware. Though he argues ha he rae a whch he sofware produc s used s dependen upon he number of s users, he model proposed by hm fals o capure he growh n number of users of he sofware. To capure he growh n number of end-users of he sofware, Kapur e al. [13] ncorporaesa model from mareng o accoun for usage n he operaonal phase as for he commercally used sofware, number of nsrucons execued depends on he number of users. The res of hs paper s srucured as follows. Secon provdes an nerdscplnary modelng approach ha combnes he subjec sofware relably engneerng and mareng. Secons 3 and 4presenparameer esmaon echnquesand fled sofware relably daa. In Secons 5 and 6, daa analyses echnques,and model valdaon and comparson crera are dscussed. Fled sofware relably daa analyses and model comparsons dscussed n Secon 7 and Secon 8 concludes he paper wh some general remars.. Inerdscplnary modellng approach Sudy of a sysem n solaon can be easer bu may no provde he opmal resuls. On he oher hand, he heory of a sngle dscplne may prove o be nadequae n explanng he dynamc neracons wh oher sysems. Hence here s a need for nerdscplnary approach. The dsngushng feaure of modern scence has been he ncreasng nerweavng of formally separaed dscplnes. Mahemacal modelng s a collecon of ools ha canno be pu under one sngle dscplne. They have been faclang nerdscplnary sudes of many complex suaons. Mahemacal modelng n mareng sared n 1950s. They have been appled o measure he effecveness of promoonal campagn, brand swchng behavour of consumers, success of a new produc, mare rs analyss, ec. Mahemacal models have been proposed for esng-effor [1, 18, 5, 31] bu hey are no suable for measurng usage of sofware n he mare. The nensy wh whch falures would manfes durng he operaonal use s dependen upon he number of mes he sofware s used and no much has been done n he leraure for he suaon [3]. Many nerdscplnary sudes as producon managemen and fnancal managemen have also been carred ou. Bu few aemps have been made a ncludng relably models, hough qualy s a very mporan arbue of a successful produc.in he sofware relably engneerng leraure, few aemps have been made o nclude mareng parameers for evaluang he operaonal relably. One aemp has used a modfed verson of nnovaon dffuson model [] o esmae he number of lcensed users as well as users of praed copes of he sofware [6]. For a relable esmae of he growh wh me n number of users who use a parcular sofware release produc durng operaonal phase, we have employed a mahemacal model developed n he dscplne of mareng managemen []. The employed model can be used o accoun for usage n he operaonal phase as for he commercally used sofware, number of nsrucons execued depends on he number of users. The usage funcon so defned can also ae care of ncreasng, lnearly decreasng rends as a funcon of me, whch mples slow sar bu gan n growh rae. A bg begnnng and alng off n he usage growh.the usage funcon s esmaed hrough nnovaon dffuson model of mareng. Such an nerdscplnary modelng approach ha combnes he subjecssofware engneerng and mareng has been aemped for he frs me [16]. 1. Sofware s subjec o falures durng execuon caused by defecs remanng n he sofware. Sofware falure occurrence or defec removal phenomena follows an NHPP wh. 3. Sofware usage s used as a bass for falure rae. 4. The number of falures experenced durng operaon s dependen upon he number of nsrucons execued. 5. The number of nsrucons execued s a funcon of he number of users. 6. The number of users s a funcon of me. The followng noaons are used for he mahemacal formulaon purpose: m Expeced number of falures experenced n he me nerval (0, ] e Expeced number of nsrucons execued on he sofware n (0, ] W Expeced number of sofware users n (0, ] and W/ = w a Expeced number of defecs lyng dorman n sofware b Proporonaly consan denoes he rae a whch remanng defecs cause falures p Probably of defec removal on a falure α Rae a whch he defec may be nroduced durng he debuggng process σ Rae a whch nsrucons are execued 586 Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4, 014

3 Scence and Technology β Consan represenng learnng parameer n logsc funcon τ Expeced number of poenal users n he populaon µ Coeffcen of exernal (mass meda) nfluence (.e., nnovaon rae) η Coeffcen of nernal (ner-personal) nfluence (.e., maon rae),γ Consans..1. Developmen We model he number of repored falures me as a pure brh counng process (N ) 0, or more specfcally, a NHPP. A pure brh counng process (N ) 0 s a NHPP wh nensy funcon λ, for all 0, f sasfes he followng properes: 1) N=0 = 0 ) ( N ) 0 has ndependen ncremens. Ths mples ha for any > j > > l he random varables N N and N N are ndependen. 3) The random varable N N has a Posson dsrbuon wh mean m j j m, for all 0 < j. Ths mples ha: ( m m j ) m m N N e j = j =! j l, for all =0,1,, (1) where m = λ x dx s he mean value funcon of he NHPP [N, 0]. 0 The quany m descrbes he expeced number of falures or defec removal up o me. Because of he underlyng assumpons abou he falures and number of defecs n he sofware, we assume m o be a bounded, srcly ncreasng funcon sasfyng he boundary condons m =0 =0. Usng he above assumpons he falure occurrence or he defec removal phenomena can be descrbed wh respec o me as follows: m m e W λ = = e W Each componen on he rgh hand sde of he dfferenal Eq. () s ndvdually dscussed below Modelng he number of falures repored per nsrucons The frs componen of Eq. () relaes he number of falures experenced durng operaon wh he number of nsrucons execued. In oher words, durng esng nsrucons are execued on he sofware and he oupu s mached wh he expeced resuls. If here are any dscrepancy a falure s sad o have occurred. Effor s made o denfy and laer remove he cause of he falure. The earler models due o Kapur e al. [13] and Shanaw [7] have he employed flexble model [11] and he exponenal model [7] respecvely, for he purpose. However, n he exponenal model [7] defecs are removed mmedaely afer a sofware falure s observed,.e. he me o remove a defec s neglgble. Bu n realy, each observed falure s repored, dagnosed, correced, and hen verfed. The me from observaon o removal should no be negleced n a praccal sofware esng process. Besdes, n he flexble model [11] defecs are removed wh cerany and no new defec nroduced durng esng and debuggng process. In realy hs may no be always rue. The correcons may hemselves nroduce new fauls or hey may nadverenly creae condons, no prevously experenced, ha enable oher fauls o cause falures. Ths resuls n suaons where he acual faul removals are less han he () removal aemps. To address hese ssues he models due o Yamada e al. [3] and Kapur e al. [15] are seleced accordngly. The frs seleced sofware relably model s he delayed S-shaped model [3] ha descrbes he esng and debuggng process as a wo-sage process falure observaon and he correspondng defec removal phenomenon. Ths model can be derved alernavely n one sage as follows: mw = bw( a mw ) e, (3) b W where b w =. 1 + bw The second seleced sofware relably model s he esng effcency model [15] ha negraes he effec of mperfec defec debuggng and defec generaon usng a logsc funcon for he defec deecon rae, whch reflecs he effcency of he esng and removal eam. In hs model, he falure nensy sasfes he followng dfferenal equaon: b where bw = + bw 1 β e mw = pb e w a w m, (4) w and aw = a+ α mw..1.. Modelng he number of nsrucons execued per users The second componen of Eq. () relaes he number of nsrucons execued wh he esng effor or he number of users of he sofware. For he sae of smplcy we assume o be consan: e = W σ, (5).1.3. Modelng he number of users per un me The hrd componen of Eq. () relaes he growh n number of users wh respec o me. Kenny [17] has used he power funcon o descrbe he growh n user populaon who use a parcular sofware release [16]: W +1 =, (6) + 1 W here s he number of users of he sofware n he operaonal phase a me. The funcon can correcly descrbe he users growh n erms of a slow sar bu gan n growh rae, a consan addon of users, or a bg begnnng and al off n he usage rae. In he mareng leraure, power funcon s rarely used for he purpose as descrbed above. One of he reasons can be ha he parameers of he funcon are no amenable o nerpreaons. In models proposed by [13, 7], he growh n number of users (or adopers) wh respec o me usng s descrbed by he Bass [] new produc dffuson model. In mareng, he dffuson of nnovaons occurs wh every launch of a new ype of produc, and s wdely hough o be nfluenced by boh ner-personal and mass meda communcaon. Bass labelled hose who adop due o exernal nfluences nnovaors, and hose who adop due o nernal nfluences maors. Mahemacally he relaonshp s expressed as follows: W W = µ ( τ W )+ η ( τ W ), (7) a Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4,

4 Scence and Technology Solvng Eq. (7) wh he boundary condon W =0 =0, we have:.. Formulaon ( µ + η) 1 e W = τ. (8) η + + ( µ η) 1 e Subsung Eq. (3) and Eq. (5) n Eq. (), we have: µ m = b W p ( ) + bw a m W σ, (9) 1 Solvng Eq. (9) under boundary condon m =0 =0, we have he frs proposed model as: W ( ) γ m = a + b bw γ 1 1 e here p σ=γ. The falure nensy s gven as: γ γ γ, (10) m a b W λ b W = bw e = ( 1 + ), (11) 1+ bw Subsung Eq. (4) and Eq. (5) n Eq. (), we have: m = b ( ( ) ) W p a 1 α m σ bw, (1) 1 + β e mnmze W Wˆ =, 1 subjec o Wˆ = W, (15) where Ŵ =W mples ha he esmaed value s equal o he acual value. Usng hese esmaed parameers values, we esmae he parameers n he proposed models gven n Eq. (10) and Eq. (13)and he models under comparson gven Table 1 by he mehod of maxmum lelhood esmaon (MLE). The Lelhood funcon L for he unnown parameers wh he mean value funcon m aes on he form: x x 1 m m 1 m m L parameers W x e ( (, ) ( 1) )=, (16) = 1 ( x x 1 )! Tang naural logarhm of (16) we ge: ln L = x x ln m m m m ln x x ( ) ( ) ( ) ( = = 1 1 ) (17) The MLE of he unnown parameers can be obaned by maxmzng he lelhood funcon subjec o he parameers consrans. For faser and accurae calculaons, he sascal pacage for socal scences (SPSS) based on he nonlnear regresson echnque has been ulzed for he esmaon of he parameers of he proposed models and he models under comparson. Non-lnear regresson s a echnque of fndng a nonlnear model of he relaonshp beween he dependen varable and a se of ndependen varables. Unle radonal lnear regresson, whch s resrced o esmang lnear models, non-lnear regresson can esmae models wh arbrary relaonshps beween ndependen and dependen varables. Solvng Eq. (1) under boundary condon m =0 =0, we have he second proposed model as: Table 1. Models under comparson. Reference Sofware Relably Model a bw γ( 1 α) + e m = 1 β 1 1 bw + e, (13) α 1 β here p σ=γ. The falure nensy s gven as: m a γ b λ = = 1+ β e bw ( 1+ β ) e 1+ β e bw γ ( 1 α ), (14) bw I should be poned ou here ha he second proposed model gven n Eq. (13) s more general han ha of he frs proposed model gven n Eq. (10) snce ncludes he effec of defec generaon, mperfec defec debuggng, and has he models due o [13, 7]as specal cases. 3. Parameer esmaon echnques Parameers esmaon s of prmary concern n sofware relably measuremen. Sofware fled relably daa can be colleced durng operaonal from he acual operaonal ses where sofware s used by s nended users durng feld ess n he form of falures x ( <x 1 <x < <x ) repored by ses W (W 1 <W <W 3 < <W ) n he me nerval (0, ] where =1,,,. Daa usage colleced durng operaonal use s esmaed by he mehod of leas square as follow: Model due o [7] a e b σ 1 W Model due o [13] Proposed gven n Eq. (10) Proposed gven n Eq. (13) 4. Fled sofware relably daa bw σ e ( + ) a 1 β 1 bw + e 1 β a ( + b b W ) γ γ 1 1 e An acual feld daa from a larger release of a elecommuncaons swch sofware gven n Table. Ths daa s avalable n he form (,w,x )(=1,,3,,140)where w_s he number of ses reporng falures x per me ( 1 <_< 3 < < 140 ), 140 =93.5, 140 W140 = w 1 =8109 =, and x 140 =100. Noe ha he daa has been normalzed o proec propreary nformaon. The man effec of normalzaon on he analyss s one of scalng. Therefore, he analyss of he non-normalzed daa would be essenally he same [8] and furher suded n [7]. Therefore, he daa se allows drec comparson wh he wor of ohers. W bw 1 a 1 e 1 1 γ( α) ( + ) β α bw 1+ e β 588 Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4, 014

5 Scence and Technology Table. Feld daa from a larger release of a elecommuncaons swch sofware. Daa s calendar-me, %Cum SW Falures s he percenage of he oal number of sofware falure experenced n he calendar nerval repored n he able, %Cum Usage Tme s he percenage of he oal n servce me accumulaed over he calendar nerval repored, and %Ses s he percenage of ses ha have hs verson of he sofware release loaded on a gven dae. Noe ha he daa has been normalzed o proec propreary nformaon [8]. Dae SW Falures Usage Falures %Se Dae SW Falures Usage Falures %Se Dae SW Falures Usage Falures %Se Dae SW Falures Usage Falures %Se 5. Daa analyss echnques Before applyng any sofware relably model o a se of falure daa s advsable o deermne wheher he falure daa does, n fac, exhb relably growh. If a se of falure daa does no exhb ncreasng relably as esng progresses, here s no pon n aempng o esmae and predc he sysem s relably. Snce he proposed models are falure coun models, he es may only be appled o daa n whch he es nervals are of equal lengh. Therefore, we dvded he me nerval (0,] no uns of me of equal lengh. The wo rend ess ha are commonly carred ou are [10]: Arhmec mean es. Ths es consss of compung he arhmec mean τ of he observed mes n, =1,,,. 1 τ = n = 1, (18) An ncreasng sequence of τ_ ndcaes relably growh and a decreasng sequence ndcaes relably decay. Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4,

6 Scence and Technology Laplace Tes. Ths es s superor from an opmaly pon of vew and s recommended for use when he NHPP assumpon s made. In erms of n_, he number of falures durng un of me, he expresson of he Laplace facor s: u = n n ( ) 1 1 = 1 = 1, (19) 1 n = 1 In pracce, n he conex of relably growh, negave values ndcae a decreasng falure nensy and hus a relably ncrease, posve values sugges an ncreasng falure nensy and hus a relably decrease, and values oscllang beween and + ndcae sable relably. In oher words, n order o deermne wheher he sofware underwen a relably growh or no, we apply boh he arhmec mean and Laplace rend es o he falure daa. 6. Model valdaon and comparson crera The performance of a sofware relably model judged by s ably o f he pas sofware falure daa (.e., goodness of f crera) and o predc sasfacorly he fuure behavor from presen and pas daa behavor (.e., predcve valdy creron). We evaluae he performance of he models under comparson gven n Table 1 usng MSE, Bas, Varaon, and RMSPE mercs. The smaller he merc value he beer [15, 4]. The mean square fng error (MSE). The models under comparson are used o smulae he falure daa, he dfference beween he expeced values, and he observed daa x s measured by MSE as follows: m 1 MSE = m x = 1 (0) where s he number of observaons. Bas. The dfference beween he observaon and predcon of number of falures a any nsan of me s nown as PE (predcon error). The average of PE s s nown as bas: Assume ha we have observed x falures by he end of operang me. We use he falure daa up o me e ( ) o esmae he parameers of m. Subsung he esmaes of he parameers n he mean value funcon yelds he esmae of he number of falures m by me. The esmae s compared wh he acually observed number x. Ths procedure s repeaed for varous values of e. We can vsually chec he predcve valdy by plong he relave error agans he normalzed me: m x Relave Error =,and Normalzed Tme = e x (4) The error wll approach zero as e approaches. If he pons are posve (negave), he model ends o overesmae (underesmae) he fuure falure phenomenon. Numbers closer o zero mply more accurae predcon. The relave error s sad o be accepable f s whn ±10 percen [1]. 7. Fled sofwarerelably daa analyses and model comparsons 7.1. Trend analyss Fgures 1 and race he arhmec mean and Laplace rend ess respecvely. Boh rend ess ndcae relably decay whch s expeced and consdered normal a he sar of a new acvy, such as a new lfe cycle phase, changng es ses whn he same phase, addng new users, acvang he sysem wh a dfferen user profle, or may also resul from regresson defecs. Snce he decay las for shor perod we should no pay aenon o. The relably decay followed by relably growh s usually welcome because ndcaes ha, afer removal of he frs defec, he correspondng acvy reveals fewer and fewer defecs [10]. Snce he falure daa exhbs ncreasng relably as esng progresses, here s a pon n aempng o esmae and predc he sysem s relably. As boh rend ess show relably decay followed by relably growh, whch sugges he use of S-shaped models. Therefore, he proposed models wh S-shaped mean value funcons can be appled o falure daa dsplayng a rend ha behaves accordng o her assumpons. 1 Bas = PE = 1 (1) where PE =Acual(observed) Predced(esmaed), Varaon. The sandard devaon of predcon error s nown as varaon: Varaon = 1 PE Bas 1 = 1 () Roo Mean Square Predcon Error (RMSPE). I s a measure of closeness wh whch a model predcs he observaon: RMSPE = Bas + Varaon Predcve valdy s defned as he capably of he sofware relably model o deermne he fuure falure behavor from presen and pas falure behavour. Ths capably s sgnfcan only when falure behavor s changng [0]. Fg. 1. Arhmec mean rend es 590 Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4, 014

7 Scence and Technology Fg.. Laplace rend es Fg. 4. Non-cumulave usage curves 7.. Goodness of f crera analyss The resulan parameers esmaon and he performance of he usage funcons under comparson are abulaed n Table 3. From hs able, we can see ha he nnovaon dffuson model [] has lower MSE, bas, varaon, and RMSPE merc values. Therefore, he nnovaon dffuson model s good enough o gve a more accurae descrpon of sofware usage n he operaonal phase. Table 3. Parameer esmaon and comparson crera mercs resuls Usage Funcons Under Comparson Parameer Esmaon κ τ μ η MSE Bas Varaon RMSPE Power Funcon [17] Innovaon Dffuson [] he parameer s no par of he correspondng funcon Fg. 3. Cumulave usage curves The fng of he usage funcons under comparson o he acual cumulave and noncumulave usage are graphcally llusraed n Fgures 3 and 4 respecvely. The power funcon [17] shows a poor fng whle nnovaon dffuson model [] fs he daa excellenly as seen n Fgure 3. I s clearly seen from Fgure 4 ha he number of ses/users who adoped hs parcular sofware produc s ncreasng a a rapd rae and here s a sably afer whch decreases n he presence of compeors or oher reasons (e.g., nex release becomes avalable). From hese fgures, we can observe ha he nnovaon dffuson model [] provdes more accurae descrpon of usage han Comparson Crera he power funcon [17]. The parameer esmaon and comparson crera resuls of he models under comparson can be vewed n Table 4. The parameers of he nnovaon dffuson [] were esmaed by he leas squares esmaon mehod and gven n Table 3. Usng hese esmaed values, he MLE mehod s hen appled o esmae he remanng parameers of he models under comparson. If we loo a he esmaon resuls, we noce ha he value of parameer β,.e., s greaer han zero, whch mples he S-shaped naure defned by he faul deecon mean value funcon for hs model. Besdes, he value of parameer α, s zero, whch mples he debuggng process s perfec. I s worh nong ha he second proposed model gven n Eq. (13) reduces o he model [13] when appled due o s bul-n flexbly. In addon, we can see ha boh of he proposed models provde mproved resuls because of lower MSE, bas, varaon, and RMSPE merc values. The fng of he models under comparson o he acual fled daa are graphcally llusraedn Fgures 5 and 6. From Fgure 5 we noce he behavor of falure daa and observe ha s S-shaped n naure. Ths furher jusfed by he use of he proposed models o deec he defecs n he sofware. I s clearly seen from Fgure 6 ha he evoluon of he falure nensy s no monoonous decreasng bu S-shaped,.e., frs ncreasng-hen-decreasng. Falure nensy has been proven o be very useful for allocang resources and deermnng when o sop esng n commercal sysems. The dsrbuon of falure occurrence durng operaon as depced n Fgure 6 shows he number of falure occurrence durng an nerval has a hgher rae n he nal sages, reaches a maxmum number per nerval and hen Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4,

8 Scence and Technology Table 4. Parameer esmaon and goodness of f merc resuls Models under Comparson Parameers Esmaon Comparson Crera a b γ α β MSE Bas Varaon RMSPE Model due o [7] Model due o [13] Proposed n Eq.(10) Proposed n Eq.(13) ndcaes he parameer s no par of he correspondng model Fg. 5. Cumulave falure removal curves Fg. 6. Non-Cumulave falure removal curves exponenally reduces over me oward zero. In oher words, we may conclude ha as he cumulave falure coun ncreases, he falure nensy decreases Predcve Valdy Analyss The fled daa s runcaed no dfferen proporons and used o esmae he parameers of he proposed models. For each runcaon, one relave defec s obaned. Fgure 7 graphcally llusraes Fg. 7. Predcve valdy Fg. 8. Rerodcve and predcve ably 59 Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4, 014

9 Scence and Technology he resul of he predcve valdy. I s observed ha he predcve valdy of he model vares from one runcaon o anoher. The error relave of he proposed models underesmaes he falure observaon process. Fgure 8 graphcally llusraes he rerodcve and predcve ably of he proposed models. The daa s runcaed a e (50% approx.) o esmae he proposed model parameers. The proposed models are hen used o esmae he whole daa. The pons before e (mared by a doed arrow) demonsrae he rerodcve ably whle he pons afere demonsrae he predcve ably of he proposed models. I s clearly seen ha 50% of he normalzed me s suffcen o predc he fuure behavor of he falure process reasonably, whch enable proper plannng and of he manenance effor. Ths, n urn, mnmzes he manenance cos whou decreasng cusomer sasfacon. 8. Concludng remars Sofware relably model s a mahemacal expresson ha specfes he general form of he sofware falure process as a funcon of facors such as faul nroducon, faul removal, and he operaonal envronmen. NHPP based Sofware relably models have been que successful ools n praccal sofware relably engneerng. These models consder he debuggng process as a counng process characerzed by s mean value funcons. Sofware relably, can be esmaed once he mean value funcon s deermned. Model parameers are usually esmaed usng eher he maxmum lelhood mehod or leas squared esmae. They have been wdely used o esmae he relably of sofware durng esng. Many auhors have even red o exend hem o represen he falure phenomenon durng he operaonal phase, ypcally used n release me problem of sofware. Bu hs approach s no correc when usage of sofware s dfferen from ha durng esng, whch s acually he case for mos of he commercal sofware. Commercal sofware brngs many benefs o socey, and plays a val role n he developmen and manenance of a dverse and vbran nformaon and communcaon echnology secor. A commercal sofware developer endeavors o mae s sofware produc popular n he mare by sellng more and more copes of s produc. Apar from sasfyng cusomers by meeng all her requremens and aachng addonal feaures, he developer a he same me maes consan effors o buld he sofware defec free. For measurng he operaonal relably of a commercal sofware produc, he man ssue s he avalably of sofware fled relably daa ha s needed for deermnng relably. Sofware developmen companes le Mcrosof employees cusomer experence mprovemen program (CEIP) echnology, o record boh falure daa and usage daa. Snce CEIP s avalable o a user by subscrpon only, he oal populaon sze of he observed group s nown. In hs paper, an aemp has been made o model he sofware relably growh lnng o he number of users who use a parcular sofware release. Because he number of nsrucons execued depends on he number of users. The number of users s esmaed hrough an nnovaon dffuson model of mareng. Once he esmaed value s nown, he rae a whch nsrucons are execued can be found. The nensy wh whch falures would be repored depends upon hs value. The sofware relably models developed n he leraure can now be used o model he faul exposure phenomenon. Followng hs he proposed models can help sofware companes le Mcrosof o mprove he qualy, relably and performance of s commercal sofware producs. The proposed models have been evaluaed by how good hey can f he fled daa and how predcve hey are. The resuls obaned from he proposed models dscussed n hs paper are que encouragng, as can be vewed hrough he numercal llusraons shown n he ables and fgures obaned afer we performed he esmaon on real fled relably daa ses. The numercal example concludes ha he consderaon of he effec of learnng wh wo ypes of mperfec debuggng n sofware relably growh modelng assumpons can mprove he descrpve performance of he models and he predcve performance as well. There s a rse of neres n ncreasng nerdscplnary sudes. I s essenal o be able o predc he fuure scenaro more accuraely. We feel hs sudy s an mporan sep n ha drecon. The emphass of he sudy s o show how one feld of acvy can enrch he oher and vce-versa. Furher sudes are needed o examne he performance of he proposed models more by usng many oher repored fled daa. Fnally, we beleve ha he approach followed n hs paper wll help o a grea exen and provdes a large scope for furher exenson and generalzaon. Acnowledgmen: Much of he research ha has found s way no hs manuscrp was carred ou durng my sabbacal leave of I am ruly graeful o Al al-bay Unversy for s suppor of my wor and o he revewers for her consrucve commens. References 1. Ahmad N, Khan MG, Raf LS. A sudy of esng-effor dependen nflecon S-shaped sofware relably growh models wh mperfec debuggng. Inernaonal Journal of Qualy & Relably Managemen 010; 7: Bass F. A new produc growh model for consumer durables. Journal Mareng 1969; 15: Bardhan AK. Modellng n Sofware Relably and s Inerdscplnary Naure. PhD hess, New Delh: Unversy of Delh, Chu K-C, Huang Y-S, Lee, T-Z. A sudy of Sofware relably growh from he perspecve of learnng effecs. Relably Engneerng and Sysems Safey 008; 93: Edrs K, Shanaw O. The Pham Nordmann Zhang (PNZ) sofware relably model revsed. Proc. of he Tenh IASTED Inernaonal Conference on Sofware Engneerng, Innsbruc, Ausra,15-17 February011: Gvon M, Mahajan V, Muller E. Sofware pracy: esmaon of los sales and he mpac on sofware dffuson. Journal Mareng 1995; 59: Goel AL, Oumoo K Tme dependen error deecon rae model forsofware relably and oher performance measures. IEEE Trans. Relably 1979: Jones WD, Vou MA. Feld daa analyss. n: Handboo of Sofware Relably Engneerng, Lyu M. (ed.), McGraw Hll, Kan SH. Mercs and Models n Sofware Qualy Engneerng, Second Edon. USA: Addson-Wesley Professonal, Kanoun K, Kaanche M, Lapre J-C. Qualave and quanave relably assessmen. IEEE Sofware 1997; 14: Kapur PK, Garg RB. A sofware relably growh model for an error removal phenomenon. Sofware Engneerng Journal 199; 7: Kapur PK, Garg RB, Kumar S. Conrbuons o Hardware and SofwareRelably.Sngapore: World Scenfc, Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4,

10 Scence and Technology 13. Kapur PK, Jha PC, Goswam DN, Shanaw O, Bardhan AK. General framewor for modelng sofware relably growh durng esng and operaonal use. Recen Developmens n Qualy, Relably and Informaon Technology 003: Kapur PK, Sngh O, Shanaw O, Gupa A. A dscree NHPP model for sofware relably growh wh mperfec faul debuggng and faul generaon. Inernaonal Journal of Performably Engneerng 006; : Kapur PK, Pham H, Anand S, Yadav K. A unfed approach for developng sofware relably growh models n he presence of mperfec debuggng and error generaon. IEEE Transacons on Relably 011; 60: Kapur PK, Sngh O, Mal R. Sofware relably growh and nnovaon dffuson models: an nerface. Inernaonal Journal of Relably, Qualy and Safey Engneerng 004; 11: Kenny GQ. Esmang defecs n commercal sofware durng operaonal use. IEEE Transacons on Relably 1993; 4: Kuo SY, Huan CY, Lyu MR. Framewor for modellng sofware relably usng varous esng-effor and faul-deecon raes. IEEE Transacons on Relably 011; 50: Lyu M. (Ed.). Handboo of Sofware Relably Engneerng, New Yor: McGraw-Hll Musa JD, Iannno A, Oumoo K. Sofware Relably: Measuremen Predcon Applcaon. McGraw-Hll, Ohba M. Sofware relably analyss models. IBM Journal of Research and Developmen 1984; 8: Ohba M, Chou XM. Does mperfec debuggng effec sofware relably growh. Proceedngs of 11h Inernaonal Conference of Sofware Engneerng, Psburgh, PA, USA, May 1989: Pham H. Sofware relably. London: Sprnger, Plla K, Nar VSS. A model for sofware developmen effor and cos esmaon. IEEE Transacons on Sofware Engneerng 1997; 3: Shanaw O. Tesng-effor dependen sofware relably model for dsrbued sysems. Inernaonal Journal of Dsrbued Sysems and Technologes 013; 4: Shanaw O. Dscree me NHPP models for sofware relably growh phenomenon. Inernaonal Arab Journal of Informaon Technology 009; 6: Shanaw O. Measurng sofware-operaonal relably: an nerdscplnary modellng approa ch.proc. of he IFIP 18h World Compuer Congress - Suden Forum, Toulouse, France, -7 Augus 004: Trachenberg M. A general heory of sofware relably modelng. IEEE Transacons on Relably 1990; 39: Wang S, Wu Y, Lu M, L H. Dscree nonhomogeneous Posson process sofware relably growh Models based on es coverage. Qualy Relably Engneerng Inernaonal 013; 9: Xe M. Sofware relably modellng. Sngapore: World Scenfc, Yamada S, Ohera H, Narhsa H. Sofware relably growh models wh esng-effor. IEEE Transacons on Relably 1986; R-35, Yamada S, Ohba M, Osa S. S-shaped relably growh modellng for sofware error deecon. IEEE Transacons on Relably 1983; 3: Yamada S, Touno K, Osa S. Imperfec debuggng models wh faul nroducon rae for sofware relably assessmen. In'l Journal of Sysem Scence 199; 3: Zhang X, Teng X, Pham H. Consderng faul removal effcency n sofware relably assessmen. IEEE Transacons on Sysems, Man and Cybernecs 003; 33: Omar Shanaw Deparmen of Compuer Scence Al al-bay Unversy Mafraq 5113, Jordan E-mal: dromal@lycos.com 594 Es p l o a a c j a N e z a w o d n o s c Ma n e n a n c e a n d Relably Vo l.16, No. 4, 014