Log-Logistic Software Reliability Growth Model

Transcription

1 Log-Logistic Software Reliability Growth Model Swapa S. Gokhale ad Kishor S. Trivedi 2y Bours College of Egg. CACC, Dept. of ECE Uiversity of Califoria Duke Uiversity Riverside, CA 9252 Durham, NC Abstract Fiite failure NHPP models proposed i the literature exhibit either costat, mootoic icreasig or mootoic decreasig failure occurrece rates per fault, ad are iadequate to describe the failure process uderlyig certai failure data sets. I this paper, we propose the loglogistic reliability growth model, which ca capture the icreasig=decreasig ature of the failure occurrece rate per fault. Equatios to estimate the parameters of the existig fiite failure NHPP models, as well as the log-logistic model, based o failure data collected i the form of iterfailure times are developed. We also preset a aalysis of two data sets, where the uderlyig failure process could ot be adequately described by the existig models, which motivated the developmet of the log-logistic model. Itroductio The past 2 years have see the formulatio of several software reliability growth models to predict the reliability ad error cotet of software systems. These models are cocered with forecastig future system operability from the failure data collected durig the testig phase of a software product. A plethora of reliability models have appeared i the literature, however, a extesive validatio of This work was doe whe the author was a graduate studet at Duke Uiversity y This work was supported i part by Bellcore as a core project i the Ceter for Advaced Computig ad Commuicatio, by the Natioal Sciece Foudatio grat umber EEC , ad Charles Stark Draper Laboratory Grat # DL-H these models seems to be lackig. The accuracy of these models whe validated usig the very few available data sets varies sigificatly, ad thus despite the existece of umerous models, oe of them ca be recommeded ureservedly to potetial users. This paper presets a Log-logistic software reliability growth model, the developmet of which was primarily motivated due to the iadequacy of the existig models to describe the ature of failure process uderlyig some of the previously reported as well as ew data sets. The layout of the paper is as follows: Sectio 2 describes the fiite failure NHPP class of software reliability growth models, ad offers a ew decompositio of the mea value fuctio of the fiite failure NHPP models, which eables us to attribute the ature of the failure itesity of the software to the hazard fuctio. Sectio 3 describes some of the limitatios of the existig fiite failure NHPP models. Sectio 4 describes the log-logistic software reliability growth model. Sectio 5 discusses parameter estimatio of the existig fiite failure NHPP models as well as the log-logistic model based o the times betwee failures data. Sectio 6 describes the techiques used for software failure data aalysis. Sectio 7 presets the aalysis of two failure data sets which led us to the developmet of the log-logistic model ad Sectio 8 cocludes the paper. 2 Fiite failure NHPP models This is a class of time-domai [3, 7, 2] software reliability models which assume that software failures display the behavior of a o-homogeeous Poisso process (NHPP).

2 The parameter of the stochastic process, (t), which deotes the failure itesity of the software at time t, is timedepedet. Let N (t) deote the cumulative umber of faults detected by time t, adm(t) deote its expectatio. The m(t) = E[N (t)], ad the failure itesity (t) are Z t m(t) = related as follows: (s)ds () ad, dm(t) = (t) (2) dt N (t) is kow to have a Poisso pmf with parameter m(t), thatis: P fn (t) =g = [m(t)] e,m(t) ; =; ; 2; :::! Various time domai models have appeared i the literature which describe the stochastic failure process by a NHPP. These models differ i their failure itesity fuctio (t), ad hece m(t). The NHPP models ca be further classified ito fiite failure ad ifiite failure categories. Fiite failure NHPP models assume that the expected umber of faults detected give ifiite amout of testig time will be fiite, whereas the ifiite failures models assume that a ifiite umber of faults would be detected i ifiite testig time [3]. Let a deote the expected umber of faults that would be detected give ifiite testig time i case of fiite failure NHPP models. The, the mea value fuctio of the fiite failure NHPP models ca also be writte as: m(t) =af (t) (4) wheref (t) is a distributio fuctio. From Equatio (4), the istataeous failure itesity (t) i case of the fiite failure NHPP models is give by: (t) =af (t) (5) which ca be re-writte as: F (t) (t) =[a, m(t)] =[a, m(t)]h(t) (6), F (t) where h(t) is the failure occurrece rate per fault of the software, or the rate at which the idividual faults maifest themselves as failures durig testig. The quatity [a, m(t)] deotes the expected umber of faults remaiig i the software at time t. Sice [a, m(t)] is a mootoically o-icreasig fuctio of time (actually, [a, m(t)] should decrease as more ad more faults are detected ad (3) removed), the ature of the overall failure itesity, (t), is govered by the ature of failure occurrece rate per fault h(t), from Equatio (6). 3 Limitatios of the NHPP models The failure occurrece rate per fault h(t) ca be a costat, icreasig, decreasig, or icreasig=decreasig. I this sectio, we describe some of the existig fiite failure NHPP models alog with their hazard fuctios. 3. Goel-Okumoto (GO) model The Goel-Okumoto (GO) model [6] has had a strog ifluece o software reliability modelig. The failure occurrece rate per fault is costat i case of the Goel-Okumoto model. Musa-Okumoto model [] is similar to the Goel- Okumoto model, the primary differece beig that it is based o executio time data, whereas the Goel-Okumoto model uses caledar time failure data. Table gives the expressios for m(t), (t), adh(t) for the Goel-Okumoto model. 3.2 Geeralized Goel-Okumoto (GO) model I case of Goel-Okumoto model, the failure occurrece rate per fault is time idepedet; however sice the expected umber of remaiig faults decreases with time, the overall software failure itesity decreases with time. The software quality cotiues to improve as testig progresses. However, i most real-life testig scearios, the software failure itesity icreases iitially ad the decreases. The geeralized Goel-Okumoto(GO) model [5] was proposed to capture this icreasig/decreasig ature of the failure itesity. The ature of the failure occurrece rate per fault is determied by the parameter, ad is icreasig for < ad decreasig for >. Refer to Table for expressios for m(t), (t),adh(t) for the Geeralized Goel-Okumoto model. 3.3 S-shaped model The S-shaped reliability growth model [4] captures the software error removal pheomeo i which there is a time

3 Table. NHPP models - m(t), (t), ad h(t) Coverage Fuctio m(t) (t) h(t) Expoetial a(, e,gt ) age,gt g Weibull a(, e,gt ) age,gt t, gt, S-shaped a[, ( + gt)e,gt ] ag 2 te,gt g2 t +gt Failure occurrece rate per fault Failure occurrece rate per fault vs. time.6 Failure occurrece rate per fault vs. time Failure Occurrece Rate per Fault GO S shaped Ge. GO (gamma < ) Ge. GO (gamma > ) Figure. Hazard for existig NHPP models delay betwee the actual detectio of the fault ad its reportig. The testig process i this case ca be see as cosistig of two phases: fault detectio ad fault isolatio. The S-shaped model has a icreasig failure occurrece rate per fault. The expressios for m(t), (t), adh(t) for the S-shaped model are preseted i Table. The hazard rates for the Goel-Okumoto model, Geeralized Goel-Okumoto model ad S-shaped model are show i Figure. The parameters of the models are set up i a such a way that there are about 26 failures i 25 time uits, for the sake of illustratio. 4 Log-logistic reliability growth model Figure shows that the existig fiite failure NHPP models ca capture costat, mootoically icreasig or mootoically decreasig failure occurrece rates per fault. I some of the failure data sets, the rate at which idividual faults maifest themselves as testig progresses, ca also exhibit a icreasig/decreasig behavior. A icreasig=decreasig tred exhibited by the failure occurrece rate per fault caot be captured by the existig NHPP models. I this sectio, we describe the loglogistic reliability growth model, which ca capture the icreasig=decreasig ature of the hazard fuctio Figure 2. Hazard for Log-logistic model The icreasig/decreasig behavior of the failure occurrece rate per fault ca be captured by the hazard of the loglogistic distributio []. The mea value fuctio, m(t), failure itesity fuctio (t), ad the hazard fuctio, h(t) of the log-logistic software reliability model are give i Equatios (7), (8) ad (9) respectively. m(t) =a (t) +(t) (7) (t) =a (t), [+(t) ] 2 (8) h(t) = c (t), c(t) = (t), +(t) (9) Figure 2 shows the hazard of the log-logistic model. The parameters of the model are set up i such a way that there are 26 failures i 25 time uits, as i case of Figure, merely for the sake of illustratio. Figure 3 shows the coditioal reliability vs. time for the log-logistic model. I Figure 3 failure detectios are assumed to be at fixed itervals. Just after the poit of detectio ad subsequet istataeous repair, reliability jumps up to., ad the decays slowly util the ext detectio. The lowest reliability i each iterval decreases iitially i the log-logistic model, ad the icreases. 5 Estimatio of parameters I this sectio we develop expressios to estimate the parameters of the fiite failure NHPP models preseted i Sectio 2 ad the Log-logistic model based o time betwee failures data. Equatios to estimate the parameters

4 Coditioal reliability Reliability growth for log logistic model The log likelihood i case of Geeralized Goel-Okumoto model ca be writte as: L(a; g; js) =,a(, e,gs )+ log a + log g + log, g P i= s i +(, ) P i= log s i (5) For a fixed value of, Equatio(5), ca be maximized with respect to a ad g to give: Figure 3. Reliability growth of the Goel-Okumoto model have bee developed i the literature [6], ad are preseted here for the sake of completeess. Let ft k ;k = ; 2;:::g deote the sequece of times betwee successive software failures. The t k deotes the time betwee (k, ) st ad k th failure. Let s k deote the time to failure k,sothat: s k = kx i= t k : () The joit desity or the likelihood fuctio of S ;S 2 ;:::;S ca be writte as [3]: f S;S 2;:::;S (s ;s 2 ;:::;s )=e,m(s) Y i= (s i ): () For a give sequece of software failure times s = (s ;s 2 ;:::;s ), that are realizatios of the radom variables S ;S 2 ;:::;S, the parameters of the software reliability growth models are estimated usig the maximum likelihood method. The log likelihood i case of Goel-Okumoto model is giveby[6]: L(a; gjs) = log a + log g, a(, e,gs ),g P i= s i (2) Maximizig Equatio (2) with respect to a ad g, we have: ad g = a =, e,gs (3) X i= s i + as e,gs (4) Solvig these two simultaeous o-liear equatios, we obtai the poit estimates of a ad g. ad g = =, e,gs a X i= (6) s i + as e,gs (7) Simultaeously solvig the above two equatios for a fixed value of, gives the poit estimates of a ad g. Differet values of give a family of Geeralized Goel-Okumoto models, ad we choose the best amog these. Similarly, the log likelihood i case of S-shaped model ca be writte as: L(a; gjs) =,a(, ( + gs )e,gs )+2 log g + log a + P i= log s i, g P i= s i (8) Maximizig Equatio (8) with respect to a ad g, we have: ad a =, ( + gs )e,gs (9) 2 g = ags 2 e,gs + X i= s i (2) Solvig the above two coupled o-liear equatios, we obtai the poit estimates of parameters a ad g. The log likelihood i case of log-logistic model is: L(a; ; js) =,a (s) +(s ) + log a + log + log +(, ) P i= log s i, 2 P i= log ( + (s i) ) (2) Maximizig Equatio (2) with respect to a,,ad gives: a = (+(s ) ) (s ) (22)

5 s = 2( + s X (23) s i ) +s i= i P log s k =, [+(t) ] 2 log, i= log s P i (24) (s +2 i) k log(s i) i= +(s i) Solvig the above three equatios simultaeously, yields the poit estimates of the parameters a, ad. 6 Data aalysis techiques I this sectio we describe the techiques used for the aalysis of the software failure data sets. 6. Tred aalysis Software reliability studies are usually based o the applicatio of reliability growth models to obtai various measures of iterest. I order to determie which reliability growth model to use, tred tests are very useful. I this sectio we discuss the most frequetly used tred tests as precursors to reliability growth modelig for failure data collected i the form of iterfailure times. The two tred tests that are commoly carried out are [8]: Arithmetic Mea Test: This test cosists of computig the arithmetic mea (i) of the observed iterfailure times t j ;j =; 2;:::;i: (i) = i ix j= t j (25) A icreasig sequece of (i) idicates reliability growth ad a decreasig sequece idicates reliability decay. Laplace Test: Laplace test is superior from a optimality poit of view ad is recommeded for use whe the NHPP assumptio is made [4]. Let N (t) deote the cumulative umber of faults detected over the period (;t). The failure itesity (t) determies reliability growth=reliability decay=stable reliability. A decreasig (t) implies reliability growth, icreasig (t) implies reliability decay, ad a costat (t) implies stable reliability. The test procedure is to compute the Table 2. Tred ad correspodig models Tred Coverage Fuctio Reliability growth Go ad Geeralized GO ( <) Reliability decay followed by growth Log-logistic / S-shaped Stable reliability Homogeeous Poisso process Laplace factor l(t) giveby[2]: l(t) = N(t) N(t) X X = j= t q 2N(t) t j, t 2 (26) The Laplace factor is evaluated step by step, after every failure occurrece. Here t is the equal to the time of occurrece of the i th failure, ad the failure at time t is excluded. Equatio (26) ca the be modified as: ix t Xi, X j j= t i, j, 2 = j= l(t) = q t 2(i,) Laplace factor ca be iterpreted as follows: (27) Negative values idicate a decreasig failure itesity, ad thus reliability growth. Positive values idicate a icreasig failure itesity, ad thus a decrease i the reliability. Values betwee,2 ad +2 idicate stable reliability. Tred aalysis ca sigificatly help i choosig the appropriate model for a give sequece of iterfailure times, so that they ca be applied to data displayig treds i accordace with their assumptios rather tha blidly. Usig a model for the aalysis of a failure data set, without takig ito cosideratio the tred displayed by the data ca lead to urealistic results, whe the tred displayed is differet tha that assumed by the model [9]. The classificatio of the failure data accordig to the tred ad the correspodig model are summarized i Table 2.

6 6.2 Goodess of fit Arithemtic mea vs. i The ability of a model to reproduce the observed failure behavior of the software, also kow as its retrodictive capability [8], is determied by the goodess-of-fit test. The observed failure data is used to estimate the parameters of the model. The estimated mea value fuctio is computed ad plotted alog with the observed mea value fuctio. The error sum of squares is the calculated to evaluate the goodess-of-fit. The lower the error sum of squares the better is the fit. Arithmetic Mea i Figure 4. Arithmetic mea test - DS I 6.3 Model bias 2 Laplace factor vs. i Model bias determies whether the predictios are o a average close to the true distributios. The estimated probability of failure for each failure iterval is used to determie the extet to which a model itroduces bias ito its predictios. If the model is optimistically biased, the estimates of time to ext failure are higher tha what is actually observed, whereas i case of a pessimistic bias the estimate of times to ext failure are lower tha the observed oes []. Model bias is determied by u-plot. 6.4 Model bias tred Model bias tred measures the o-statioarity i the predictios, i other words whether model bias chages over time. The aalysis is similar to that of model bias, except that the estimated failure probabilities are trasformed i a maer that preserves temporal iformatio []. Model bias tred is determied by y-plot. 7 Software failure data aalysis I this sectio we preset the aalysis of the two data sets which led us to the developmet of the log-logistic software reliability model. Data Set I (referred to hereafter as DS) is from the U. S. Navy Feet Computer Programmig Ceter, ad cosists of errors i the developmet of software for the real-time, multicomputer complex which forms the core of the Naval Tactical Data System (NTDS). The time (days) betwee software failures are reported for oe of the larger modules Laplace Factor i Figure 5. Laplace test - DS I comprisig the NTDS software [6]. The data set cosists of 26 failures i 25 days. Data Set II (referred to hereafter as DS2) is collected durig the test phase of a software developmet project at the Charles Stark Draper Laboratories. DS II cosists of 6 failures i approximately 5 days. Figures 4 ad 5 show the arithmetic mea ad Laplace tred tests for DS II respectively. Tred tests idicate reliability decay followed by reliability growth which suggests the use of either S-shaped or log-logistic model. Loglogistic model provides the best retrodictive capability as see i the Figure 6. Bias ad bias tred tests are ot carried out i case of DS I due to a small umber of failures i the data set. Figures 7 ad 8 show the arithmetic mea ad Laplace tred tests for DS II respectively. As see from the figures, Tred tests idicate reliability decay followed by reliability growth which suggests the use of either S-shaped or loglogistic model. Log-logistic model provides the best retrodictive capability as see i the Figure 9, ad the best predictive capability as see i the Figure. S-shaped as well

7 3 25 Expected umber of faults durig testig Field Data GO Ge. GO (.2) 8 Laplace factor vs. i Expected Number of Faults 2 5 Laplace Factor Figure 6. Goodess of fit test - DS I i Figure 8. Laplace test - DS II 2 Arithmetic mea vs. i 8 6 Expected Number of Faults vs. Field data Ge. GO (.2) 8 4 Arithmetic Mea Expected Number of Faults i Figure 7. Arithmetic mea test - DS II as log-logistic models exhibits the least o-statioarity i the predictios. Kolomogorov-Smirov distace for the goodess of fit test i case of DS I ad DS II, ad Kolmogorov distace for bias ad bias tred i case of DS II are summarized i Tables 3 ad 4 respectively. 8 Coclusios I this paper, we have preseted the log-logistic software reliability growth model which was motivated by the fact that the existig fiite failure NHPP models were i Figure 9. Goodess of fit test - DS II u plot S shaped Weibull (.5) Figure. Bias - DS II Table 3. KS distace - Goodess of fit Model DS I DS II GO 29: S-shaped 7: Log-logistic Ge. GO ( =:95) 4: Ge. GO ( =:5) 2:9 648 Ge. GO ( =:) : Table 4. Kolmogorov distace - DS II Model Bias Bias tred GO :377 :2645 S-shaped Log-logistic.377 :2673 Ge. GO ( =:95) :377 :258

8 y plot S shaped Weibull (.5) [4] O. Gauodi. Optimal Properties of the Laplace Tred Test for Software-Reliability Models. IEEE Tras. o Reliability, 4(4): , Dec [5] A. L. Goel. Software Reliability Models: Assumptios, Limitatios ad Applicability. IEEE Tras. o Software Egieerig, SE-(2):4 423, Dec Figure. Bias tred - DS II adequate to describe the failure process uderlyig some of the data sets. We offer a ew decompositio of the mea value fuctio of fiite failure NHPP models which eables us to attribute the ature of the failure itesity to the failure occurrece rate per fault. The existig fiite failure NHPP models ca capture costat, mootoic icreasig, ad mootoic decreasig failure occurrece rate per fault, whereas the log-logistic model proposed here ca capture a icreasig=decreasig ature of the failure occurrece rate per fault. Equatios to obtai the maximum likelihood estimates of the parameters of the existig fiite failure NHPP models, as well as the log-logistic model based o times betwee failures data are developed. Aalysis of two failure data sets which led us to the log-logistic model, usig arithmetic ad Laplace tred tests, goodess-of-fit test, bias ad bias tred tests is preseted. Refereces [] A. A. Abdel-Ghally, P. Y. Cha, ad B. Littlewood. Evaluatio of Competig Software Reliability Predictios. IEEE Tras. o Software Egieerig,SE- 2(9): , Sept [2] D.R.CoxadP.A.W.Lewis.The Statistical Aalysis of a Series of Evets. Lodo: Chapma ad Hall, Lodo, 978. [3] W. Farr. Hadbook of Software Reliability Egieerig, M. R. Lyu, Editor, chapter Software Reliability Modelig Survey, pages 7 7. McGraw-Hill, New York, NY, 996. [6] A. L. Goel ad K. Okumoto. -Depedet Error- Detectio Rate Models for Software Reliability ad Other Performace Measures. IEEE Tras. o Reliability, R-28(3):26 2, Aug [7] S. Gokhale, P. N. Marios, ad K. S. Trivedi. Importat Milestoes i Software Reliability Modelig. I Proc. 8th Itl. Coferece o Software Egieerig ad Kowledge Egieerig (SEKE 96), pages , Lake Tahoe, Jue 996. [8] K. Kaou ad J. C. Laprie. Software Reliability Tred Aalysis from Theoretical to Practical Cosideratios. IEEE Tras. o Software Egieerig, 2(9):74 747, Sept [9] K. Kaou ad J. C. Laprie. Hadbook of Software Reliability Egieerig, M. R. Lyu, Editor, chapter Tred Aalysis, pages McGraw-Hill, New York, NY, 996. [] L. M. Leemis. Reliability - Probalistic Models ad Statistical Methods. Pretice-Hall, Eglewood Cliffs, New Jersey, 995. [] J. D. Musa. A Theory of Software Reliability ad its Applicatio. IEEE Tras. o Software Egieerig, SE-():32 327, Sept [2] M. Ohba. Software Reliability Aalysis Models. IBM Joural Res. Develop., 28(4): , July 984. [3] K. S. Trivedi. Probability ad Statistics with Reliability, Queuig ad Computer Sciece Applicatios. Pretice-Hall, Eglewood Cliffs, New Jersey, 982. [4] S. Yamada, M. Ohba, ad S. Osaki. S-Shaped Reliability Growth Modelig for Software Error Detectio. IEEE Tras. o Reliability, R-32(5): , Dec. 983.