A Simplified Model for Evaluating Software Reliability at the Developmental Stage

Transcription

1 A Simplified Model for Evaluaing Sofware Reliabiliy a he Developmenal Sage Shelbi Joseph Division of Informaion Technology School of Engineering Cochin Universiy of Science and Technology Cochin, India Shouri P.V Deparmen of Mechanical Engineering Model Engineering College Cochin, India Jagahy Raj V. P School of Managemen Sudies, Cochin Universiy of Science and Technology Cochin, India achayanshelbil@gmail.com pvshouri@gmail.com jagahy@cusa.ac.in Absrac The use of open source sofware is becoming more and more predominan and i is imporan ha he reliabiliy of his sofware are evaluaed. Even hough a lo of researchers have ried o esablish he failure paern of differen packages a deerminisic model for evaluaing reliabiliy is no ye developed. The presen work deails a simplified model for evaluaing he reliabiliy of he open source sofware based on he available failure daa. The mehodology involves idenifying a fixed number of packages a he sar of he ime and defining he failure rae based on he failure daa for hese prese number of packages. The defined funcion of he failure rae is used o arrive a he reliabiliy model. The reliabiliy values obained using he developed model are also compared wih he exac reliabiliy values. Key words: Bugs, Failure Densiy, Failure Rae, Open Source Sofware, Reliabiliy 1. INTRODUCTION Open Source Sofware (OSS) has araced significan aenion in recen years [1]. I is being acceped as a viable alernaive o commercial sofware []. OSS in general refers o any sofware whose source code is freely available for disribuion [3]. However he OSS developmen approach is sill no fully undersood [4]. Reliabiliy esimaion plays a vial role during he developmenal phase of he open source sofware. In fac, once he package has sabilized (or developed) hen chances of furher failure are relaively low and package will be more or less reliable. However, during he developmenal sage failures or bug arrival are more frequen and i is imporan ha a model has o be developed o evaluae he reliabiliy during his period. The bug arrivals usually peak a he code inspecion phase and ge raher sabilized in he sysem es phase [5]. Sofware reliabiliy evaluaion is an increasingly imporan aspec of sofware developmen process [6]. Reliabiliy can be defined as he probabiliy of failure free operaion of a compuer program in a specified environmen for a specified period of ime [4,5]. I is eviden from he definiion ha here are four key elemens associaed wih he reliabiliy namely elemen of probabiliy, funcion of he produc, environmenal condiions, and ime. Reliabiliy is nohing bu he probabiliy of success. As success and failure are complemenary, a measure of he failure is essenial o arrive a he reliabiliy. Tha is, Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 15

2 Reliabiliy = Probabiliy of success = 1- Probabiliy of failure (1) From equaion (1), i is eviden ha he firs sep in reliabiliy analysis is failure daa analysis. This involves fixing up a ime inerval and noing down he failures a differen ime inervals. The number of packages a he sar of he analysis is defined as he iniial populaion and he survivors a any poin of ime is he difference of iniial populaion and he failures ha have occurred ill his poin. Failure rae associaed wih a ime inerval can be defined as he raio of number of bugs repored during he NOMENCLATURE f d () N R() Z() λ failure densiy iniial populaion reliabiliy ime failure rae consan failure rae given ime inerval o he average populaion associaed wih he ime inerval. Once he variaion of failure rae wih respec o ime can be esablished an equaion can be used o fi he variaion which will be he failure model for reliabiliy esimaion. Typical reliabiliy models include Jelinski-Moranda [6], Lilewood [7], Goel- Okumoo [8], Nelson model [9], Mills model [1],Basin model [1],Halsead model [11] and Musa-Okumoo [4]. For sofware projecs ha have no been in operaion long enough, he failure daa colleced may no be sufficien o provide a decen picure of sofware qualiy, which may lead o anomalous reliabiliy esimaes [1, 13] Weibull funcion is also used for reliabiliy analysis and he funcion has been paricularly valuable for siuaions for which he daa samples are relaively small [14]. Concern abou sofware reliabiliy has been around for a long ime [15,16 ] and as open source is a relaively novel sofware developmen approach differing significanly from proprieary sofware waerfall model, we do no ye have any maure or sable echnique o assess open source sofware reliabiliy [17]. I is clear from he above discussions ha even hough a variey of models are available for reliabiliy predicion, a deerminisic model is presenly no available. Or in oher words, none of hese models quanifies reliabiliy. The presen work focuses on developmen of an algorihm and here by a simplified mehod of quanifying reliabiliy of a sofware.. MODEL DEVELOPMENT AND ALGORITHM An open source program ypically consiss of muliple modules [18]. Aribues of he reliabiliy models have been usually defined wih respec o ime wih four general ways o characerize [19, ] reliabiliy, ime of failure, ime inerval beween failures, cumulaive number of fauls upo a period of ime and failure found in a ime inerval. The presen mehodology involves defining an equaion for he paern of failure based on he available bug arrival rae and developing a generalized model for he reliabiliy of he sofware. The following are he assumpions involved in he analysis. 1. The sofware analyzed is an open source.. As he open source sofware is made up of a very large communiy he environmenal changes are no considered. 3. The oal number of packages a he beginning of he analysis is assumed o remain consan and is aken as he iniial populaion. 4. The failures of various packages are assumed o be independen of each oher. 5. The model is developed for evaluaion of he sofware reliabiliy a he developmenal sage and he packages ha fail during his period are no furher considered. I is furher assumed ha by he end of Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 16

3 developmenal sage he bug associaed wih he failed packages would be eliminaed and will be sable furher. 6. The reliabiliy of he sofware is inversely proporional o he number of bugs repored a any poin of ime. 7. The beginning of he ime period afer which he bug arrival or failure rae remains consan marks he culminaion of he developmenal sage and he sofware will be sabilize. Based on he above assumpions a 6- sep algorihm is developed for he analysis as deailed below. 1. Indenify he oal iniial populaion. This corresponds o he oal number of packages exising a he beginning of he ime period. Tha is, a he sar of analysis.. Define a ime period and find ou he bugs repored during his ime inerval. As he failure would have occurred anywhere beween he ime inerval, he repored failures are indicaed in beween he ime inerval. 3. Calculae he cumulaive failures and hereby he survivors and differen poins in ime. 4. Esimae he failure rae associaed wih he ime inervals by dividing he number of failures associaed wih he given uni ime inerval by average populaion associaed wih he ime inerval. Average populaion associaed wih a given ime inerval is he average of survivors a he beginning and end of he ime period. 5. Plo he graphs defining he relaion beween failure rae and ime and obain he equaion defining he relaion beween failure rae and ime. 6. Obain he expression for reliabiliy of he sofware by subsiuing he equaion of failure rae in equaion(1) given as Z ( ) d R( ) (1) 3. RESULTS AND DISCUSSION A oal of 188 packages were available a he sar of he analysis as per he deails available from he official websie of Debian [1]. This is aken as he iniial populaion. A ime inerval of 1 monh is fixed and he bug arrival rae during his inerval is noed. The repored errors a differen ime inervals are given in he Table 1. The observaions are aken for 1 year afer which he bug arrival is negligible indicaing ha he sofware has more or less sabilized. TABLE 1: Failure Daa Analysis Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 17

4 .5. Failure rae F() = Time (monhs) FIGURE 1. Variaion of failure rae wih ime The variaion of failure rae wih respec o ime is shown in Fig. 1. I can be seen ha afer he 8 h monh onwards he sofware has somewha sabilized indicaing he compleion of developmenal phase. The failure model corresponding o he failure rae can be expressed by he equaion () Z ( ) = () The corresponding reliabiliy can be expressed by he equaion (3) as R( ) Tha is, ( ) d.4.78 R( ) (3) Failure densiy associaed wih he ime inervals is he raio of number of failures associaed wih he given uni ime inerval o he iniial populaion. Failure densiy can be relaed wih Reliabiliy and failure rae using he equaion (4) as f d ( ) = R( ) Z( ) (4) Therefore, based on he developed model failure densiy can be expressed as.4.78 f d ( ) ( ) (5) The reliabiliy of he sofware a differen poins in ime is calculaed using he equaion (3). The acual values of reliabiliy obained by dividing he survivors a he given poin in ime by he iniial populaion are also calculaed. The Musa model assumes a consan value for he failure rae and by considering his as he average value of failure raes he reliabiliy values are calculaed using he equaion λ R( ) (6) Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 18

5 The reliabiliy values calculaed using he hree differen mehods and he failure densiy values are shown in able. TABLE. Reliabiliy and failure densiy Fig. shows a comparison of reliabiliy obained using he developed simplified model and Musa model wih he acual reliabiliy values. I can be seen ha he simplified model and he Musa model nearly provides he same resuls. Furher, hese wo models very closely approximae he real siuaion. The variaion of failure densiy wih ime is also shown in Fig. 3 Reliabiliy Reliabiliy (Model) Reliabiliy (Acual) Reliabiliy (Musa) Time (monhs) FIGURE. Comparison of reliabiliy obained using differen models Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 19

6 Failure densiy Time (monhs) FIGURE 3. Variaion of Failure densiy wih ime Fig 4 compares he reliabiliy value obained using he model wih he heoreical value. I can be seen ha he percenage error is always wihin 1% of he acual value which is a reasonably good resul for all engineering problems. 1 5 % Error Time (monhs) FIGURE 4. Error Analysis 4. CONCLUSION A simplified model for evaluaion of sofware reliabiliy was presened. This is a relaively simplified and a oally new mehod of analysis of sofware reliabiliy as he iniial populaion is assumed o remain consan. The mehod provides fairly good resuls and he relaed errors are negligible. I is hoped ha his model will prove o be a powerful ool for sofware reliabiliy analysis. 5. REFERENCES 1. Ying ZHOU,Joseph Davis. Open Source Sofware reliabiliy model: an empirical approach, ACM 5. Sharifah Mashia Syed-Mohamad,Tom McBride. A comparison of he Reliabiliy Growh of Open Source and In-House Sofware. 8 IEEE 15h Asia-Pacific Sofware Engineering Conference Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 13

7 3. Cobra Rahmani,Harvey Siy, Azad Azadmanesh. An Experimenal Analysis of Open Source Sofware Reliabiliy. Deparmen of Defense/Air Force Office of Scienific Research 4. Lars M.Karg, Michael Groke, Arne Beckhausa. Conformance Qualiy and Failure Coss in he Sofware Indusry: An Empirical Analysis of Open Source Sofware. 9 IEEE 5. Kan H.S. Merics and models in sofware qualiy engineering, nd ediion, Addison-Wesley(3) 6. S.P. Leblanc, P.A.Roman Reliabiliy Esimaion of Hierarchical Sofware Sysems. Proceedings annual reliabiliy and mainainabiliy symposium 7. J D Musa and K. Okumoo. A logarihamic Poisson execuion ime model for sofware reliabiliy measuremen 7h inernaional conference on Sofware Engineering(ICSE),1984, pp H. Pham, Sofware Reliabiliy Spriner-Veriag, 9. E.C. Nelson, A Sasisical Basis for Sofware Reliabiliy Assessmen, TRW-SS ShaoPing Wang, Sofware Engineering. BEIJING BUAA PRESS. 11. M.H.Halsead, Elemens of Sofware Science, Norh Holand, Z. Jelinski and P.B. Moranda, Sofware Reliabiliy research, in Sasisical Compuer Performance Evaluaion, W. Freiberger, Ed. New York: Academic Press, 197, pp B. Lilewood and J.L. Verrall, A Bayesian reliabiliy growh model for compuer sofware, Applied Sasisics, Vol, 1973, pp A. L. Goel and K. Okumoo, A ime-dependen error-deecion rae model for sofware reliabiliy and oher performance measure, IEEE Transacions on Reliabiliy, vol. R-8, 1979, pp Adalbero Nobiao Crespo, Albero Pasquini, Applying Code Coverage Approach o an Infinie Failure Sofware Reliabili Model, 9 XXIII Brazilian Symposium on Sofware Engineering, 9 IEEE. 16. Hudson, A, Program Error as a Briish and Deah Process, Technical Repor SP- 311, Sana Monica, Cal,:Sysems developmen Corporaion, Fenghong Zou, Joseph Davis Analysing and Modeling Open Source Sofware Bug Repor Daa, 19h Ausralian Confeence on Sofware Engineering.,8 IEEE. 18. Fenghong Zou, Joseph Davis A Model of Bug Dynamics for Open Source, The second Inernaional Conference on Secure Sysem Inegraion and Reliabiliy Improvemen., 8 IEEE. 19. Sharifah Mashia Syed-Mohamad, Tom McBride, Reliabiliy Growh of Open Source Sofware using Defec Analysis, 8 Inernaional conference on Compuer Science and Sofware Engineering, 8 IEEE.. Musa, J.D., Iannino, A. and Okumoo, K. (1987), Sofware Reliabiliy: Measuremen, Predicion, Applicaion, pp hp:// Inernaional Journal Sofware Engineering (IJSE), Volume (1): Issue (5) 131