Testing and Debugging Resource Allocation for Fault Detection and Removal Process

Transcription

1 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) Testng and Debuggng Resource Allocaton for Fault Detecton and Removal Process Md. Nasar and Prashant Johr School of Computng Scence and Engneerng Galgotas Unversty, Gr. Noda, INDIA ABSTRACT In software development lfe cycle (SDLC) testng s very mportant step. One of the key elements of software qualty s testng. Fault detecton and removal process s also very mportant when we are dong testng. In the last 30 years numerous software relablty growth models where developed for fault detecton and correcton process. Majorty of models where developed under statc condton. The man goal of ths artcle s to examne the resource allocaton plan for fault detecton and correcton process of the software to control the cost durng testng and operatonal phase. For achevng ths we developed a model for fault detecton and correcton process. For solvng ths model we use Pontryagan s Maxmum prncple. For optmally resource allocaton we use Dfferental Evoluton (DE). Dfferental Evoluton (DE) s an optmzaton algorthm. A numercal example s also explaned for resource allocaton for fault detecton and removal process. KEYWORDS SRGM, Testng Effort Allocaton, Correcton- Removal Process, Optmal Control Theory, Dfferental Evoluton (DE) INTRODUCTION For the last few decades t has been perceved that computer has been wdely used n dffcult knd of problem solvng. Lot of complcated and lfe crtcal work s done by software systems. So for ths knd of software we have to develop qualty software. A fault n the software system may produce huge loss of money as well as tme. There has been numerous software falures occurred n last three decades resultng n huge fnancal and other knd of losses. ence, t s essental for the frm to develop software product that should be error free, relable and should be sutable for market condton. Software testng s the process of exercsng a system or program wth the ntent of fndng bugs. It s the method of ncreasng the confdence that the software s free of flaws. But the man problem n testng software s that t cannot be made 00% bugs free. Ths s not because programmers are rresponsble or careless, but because the nature of software code s complex and humans have only lmted ablty to manage complexty. Therefore, although testng helps n assessng and mprovng software qualty, but t cannot be performed ndefntely. ence, testng tme s one of the major factors that have to be consdered durng the testng phase. Another reason whch makes testng phase mportant s that t consumes major part of the resources avalable for the software development. Bascally, testng actvtes account for 30 to 50 percent of labor expended to produce a workng program []. Therefore, t s very mportant to devote the lmted testng-resource effcently. For statc allocaton testng and debuggng effort for all the tme nterval s fxed but n realty t s not true because some error s smple error and some error s complex error. So for smple error t wll take less effort and for complex error t wll take more effort. For achevng ths we developed mathematcal model for dynamc allocaton of effort allocaton we use Pontryagan s Maxmum prncple for solvng the model and Dfferental Evoluton (DE) for optmally allocaton of resources. DE s a smple powerful populaton based stochastc search technque for solvng global optmzaton problems. DE has only a few control varables whch reman fxed throughout 93

2 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) the optmzaton process, whch makes t easy to mplement. A numercal example s also explan for resource allocaton. The paper s subdvded nto the followng sectons. Secton two descrbe the related work of ths research, three and four descrbe the model for fault detecton and correcton processes, and the cost optmzaton modelng. In secton fve, we dscuss the soluton Procedure. Secton sx dscusses the basc parameter for dfferental evaluaton. A numercal example for fault detecton and correcton process Vs tme usng dfferental evaluaton s gven n secton seven. Fnally, n secton eght, we conclude our paper wth a dscusson on results and fndngs. RELATED WORKS Wth the speedy growth n software sze and complexty, most software systems today are composed of a number of dfferent modules. Durng the testng phase, all the testng actvtes of dfferent stage are challengng for the lmted testng-resource. Thus, a dffcult problem s how to allocate the total avalable testng-resource among software modules n an optmal way so that we can fnsh testng on specfed testng lmt. [] ave dscussed an overvew of the methods that have been developed snce 977 for solvng numerous relablty optmzaton problems ncludng testng-resource allocaton problems. [3] Formulated and solved two resource allocaton problems for software-module testng, consderng the mean number of remanng faults n the software modules. When software testng s fnshed, the exact number of remanng faults may turn out to be much larger than the mean, and hence [4] studed a dynamc resource allocaton strategy for software module testng. Ths approach takes nto account the varatons of the number of detected faults throughout testng, reestmates the model parameters usng all the avalable fault detecton data and dynamcally allocates the testng resources to the software modules. [5] Dscussed optmzaton problem for testng resource allocatons and for the software systems havng modular structure and assumed that testng effort allocaton s depends upon the sze and severty of fault. Numerous SRGM has been proposed n last 4 decades mostly fault detecton models based on the assumptons that detecton and correcton of faults s done concurrently. But, n realty there s always been a tme gap ([6], [7], [8], [9] and [0]). [] Dscussed the assocaton between the number of faults deleted wth respect to testng effort and/or tme. The authors proposed that throughout the testng phase of an SDLC, faults are removed n two stages. Frst a fault occurred and then the fault causng that falure s corrected; hence the testng effort must be consumed on two dstnct processes; falure detecton and falure correcton. In ther artcle, the authors developed an SRGM ncorporatng tme delay not only between the two phases but also through the segregaton of resources between them and proposed two alternate methods for controllng the testng effort for achevng the preferred relablty or error detecton level. [] Solved testng resource allocaton problem maxmzed the number of faults removed from each module under constrant on budget. [3] Investgated an optmal resource allocaton problem n modular software systems durng testng phase. The man goal s to mnmze the cost of software development when the number of remanng faults s to mnmze and a desred relablty objectve s gven. Authors analyzed the senstvty of parameters of proposed software relablty growth models. In addton, they also see the mpact on the resource allocaton problem f some parameters are ether overestmated or underestmated. Authors evaluated the optmal resource allocaton problems for varous condtons by examnng the behavor of the parameters wth the most sgnfcant nfluence whch have been stated by solved numercal example. [4] Consders two categores of software testng resource allocaton problems. The frst problem s to mnmze the total number of resdual faults, gven a fxed quantty of testng-effort, and a relablty objectve. The last problem s to optmze the amount of software testng effort gven the total number of remanng faults n software, and a desred relablty objectve. Author has also proposed several strateges for module testng to help software project managers 94

3 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) to solve these types of problems, and make the good decsons. Author provdes several systematc solutons based on a nonhomogeneous Posson process model, allowng effcent allocaton of a specfed amount of testng resource expendtures for each and every software module under some specfed constrants. Author also descrbes numercal examples on the optmal testng resource allocaton problems and performed senstvty analyss. [5] Dscussed optmal resource allocaton plan to mnmze the software cost throughout the testng phase and operatonal phase under dynamc condton usng genetc algorthm (GA) technque. Author has been developed mathematcal model for allocatng testng and debuggng resource. [6] Proposed optmal resource allocaton to mnmze the software cost durng testng and operatonal phase usng optmal control theory. [7] Dscussed resource allocaton for testng and debuggng n dynamc condton. For ths authors developed a mathematcal model for developng mathematcal model author assume when a fault s detected that tme the fault s corrected. Dfferental Evoluton (DE) s used for optmally allocaton of testng and debuggng resource. [8] Dscussed Resource control and resource mantenance durng the software testng. Durng the software testng many of the resources lke tme, effort and budget are consumed. In ths paper author proposed an mperfect debuggng SRGM durng testng and resource allocaton s done based on optmzng the effort and relablty. Author used two testng resource allocaton schemes one by mnmzng the number of remanng faults and allocatng the resources to attan the maxmum relablty. An expermental result also shows the proposed model well ftted for software testng. Notatons Used: T : The tme perod. a : Intal number of fault content n the software. b : Fault detecton rate. b : Fault correcton rate. w : Total resources consumed durng the software development at any pont of tme t. t w : Resources consumed durng the software development for testng purpose at any pont of tme t. w t t : Resources consumed durng the software development to fx bugs at any gven pont of tme t. f : Recognzed total number of faults tll tme t. f r t : Removed total number of faults tll tme t. f t, w t : Cost per unt at tme t for cumulatve fault detected s f d t wth detecton efforts c d 3 t t w ( ). 3 t c f r, w : Cost per unt at tme t for cumulatve fault corrected s f r t debuggng efforts w ( c 3 : Budget of testng per unt effort at tme t. 3 MODEL DEVELOPMENT Fault Detecton and Correcton Processes Modelng: For perfect deleton of faults, the predctable number of fault corrected s the same as predctable number of faults dentfed. owever, n realty f a fault s dentfed, t s not necessary to correct the fault nstantly. May be t wll take tme for understandng the type of fault. ence, after dentfyng the fault, the debuggng people ask the developer to resolve the fault. Thus, there must be a tme lag between fault dentfcaton and fault removal processes. In general, the expected number of faults detected at tme T s more than faults corrected at tme T. ence, the fault detecton and correcton processes are done n two stages. To begn wth, let s consder a s the total 95

4 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) number of fault content n the software. Therefore, t s necessary to assume the followng equaton of fault dentfcaton and deleton process; dfd t xt bw ( ( a fd ( ) and t dfr y t bw( (f d( fr( ) where f d 0 0, f 0 0 r () 4 COST OPTIMIZATION MODELLING Now assume the software company wants to mnmze the total expendture over the fnte plannng horzon. Therefore, mathematcally the model can be gven as; T 0 mn c t x t c( y( c3w ( subject to t df xt bw ( ( a f ( ) dfr t y t bw( (f ( fr( ) where f 0 0,fr 0 0, r, c ( c f t, w t,c ( 3 c f t w t and w t w t w t w 3 w t ; w t ; w t SOLUTION PROCEDURE () To solve the problem for equaton (), Pontryagan s Maxmum prncple s appled. The amltonan functon s as follows [9]: (f (,f (, (, w (, w (, w (, r t xt 3 c t x t c( y t c3w ( ( y( (3) t and are the adjont varables, whch fulflls the followng dfferental equaton. d () t f (4) And; d () t fr (5) Termnal condton for the dfferental equaton (4) and (5) are gven by T 0 and ( T) 0 respectvely. The adjonng varable t represents per unt change n the objectve functon for a small change n f (.e. t can be nterpreted as margnal value of faults detected at tme t. Smlarly, ( can be nterpreted as margnal value of fault removed at tme t. Thus, the amltonan s the sum of current cost ( cx c y) and the future cost ( x y). In short, represents the nstantaneous total cost of the frm at tme t. The followng are the essental condtons hold for an optmal soluton: w 0 c w ( x( ( c ( ( ) xw ( c 3 0 (6) w 0 c ( x( c ( y( ( c ( w w ( ) y ( 0 w Where; c c w w c c w w c cw w xt () xw w (7) 96

5 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) yw yt () w (8) The addtonal optmalty condtons for amltonan maxmzaton are; w 0 w and w w w w w 0 w (9) On solvng equatons (6) and (7), we get; * ( c ( ( ) b ( a f( ) c3 w cw ( b ( a f( (0) And; ( c * ( ( ) b (f ( fr ( ) c w ( x( w () t c ( b (f ( f ( ) w r () Usng the assumpton that the entre resource s w t w t w t w. Thus, fxed.e; 3 * * * 3 w w w w () Now, upon ntegratng equaton (4) wth the transversalty condton, we have the future cost of detectng one more fault from the software; c ( x x( c T ( ( f f ( y t c ( ( f (3) Smlarly, ntegratng equaton (5) wth the transversalty condton, we have the future cost of removng one more fault from the software; T ( t c f r y( c ( ( y f r (4) Now takng tme dervatve of equaton (6), we have; w w x() t w w w w w f y() t x wf r w Tme dervatve of equaton (7) mples; w w x( y( (5) ww ww w f w fr y Where; w ww ww ww wf r t () w t () w ( w ( t () w t () w ( f ( r (6) t () wf, for,. w( f( To solve the above optmzaton problem, we used Dfferental Evoluton. Dfferental Evoluton s a computerzed search and heurstc optmzaton method for solvng dffcult type of problem whch cannot be solved easly by general methods. Dfferental Evoluton optmzes a problem by mantanng a populaton of canddate solutons and creatng new canddate solutons by combnng exstng ones accordng to ts easy formulae, and then keepng whchever canddate soluton has the best score or ftness on the optmzaton problem at hand. 6 DIFFERENTIAL EVOLUTION Dfferental evoluton (DE) s a technque that optmzes a problem by teratvely tryng to mprove a canddate soluton wth regard to a assumed measure of qualty. Such technques are 97

6 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) commonly known as metaheurstcs as they make few or no assumptons about the problem beng optmzed and can search very large spaces of canddate solutons. owever, metaheurstcs such as Dfferental evoluton (DE) do not guarantee an optmal soluton s ever found. Dfferental evoluton s a populaton-based evolutonary algorthm. Proposed by [0], s a smple powerful populaton based stochastc search technque for solvng global optmzaton problems. Dfferental Evoluton has been successfully used n varous felds such as communcaton [], [] and mechancal engneerng [3] to optmze non convex, non-dfferentable and mult modal objectve functons. DE has many strkng propertes compared to other evolutonary algorthms such as, mplementaton ease, fast convergence rate, the small number of control parameters, and robust performance. DE has only a few control varables whch reman fxed throughout the optmzaton process, whch makes t easy to mplement. Moreover, DE can be mplemented n a parallel processng framework, whch enables t to process a large number of tranng nstances effcently. These propertes of DE make t an deal canddate for the current task of learnng a rankng functon for nformaton retreval, where we must optmze non convex objectve functons. The general structure of the Dfferental Evoluton (DE) algorthm looks lke that of most other populaton-based searches. DE keeps parallel verson of two arrays, each of whch holds populaton of NP and a D-dmensonal, realvalued vectors. The prmary array holds the current vector populaton whle the secondary array accumulates vectors that are selected for the next generaton. The man step of the Dfferental evoluton (DE) algorthm s gven below: Intalzaton Evaluaton Repeat Mutaton Recombnaton (crossover) Evaluaton Selecton Untl (termnaton crtera are me 7 NUMERICAL SOLUTION: Usng DE approach, n ths secton we dscuss the numercal soluton of the problem dscussed n secton 3 to fnd the total number of fault detected and corrected n a gven pont of tme t for solvng ths problem we used MATLAB [4] The smulaton parameter values for solvng ths problem s as follows: Parameters Values Total number of populaton Sze: 00 Total number of generaton: 00 Crossover constant (CR): 0.7 Dfferentaton constant (F): 0.8 a 00 b 0.3 b 0.3 w 0.38 w f d (0) 0 f r (0) 0 c c b Table Smulaton Parameters We used above parameters n equaton 3 for optmally allocaton of fault dentfed and corrected. The projected number of fault dentfed and corrected s as follows: 98

7 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) consdered fault detecton and removal process n two stages and estmated ths n MATLAB, we found that the result s better than our statc method and also better then Genetc Algorthm. The result descrbed Vs total number of fault detected and corrected. Ths means the tester and developer can devote ther tme and resource to fnsh off ther testng and debuggng task for well controlled expendture. REFERENCES Fgure:. Number of fault detected and corrected Vs tme. Fgure:. Number of fault detected and correted Vs tme. Above fgure Shows the fault detecton and correcton process Vs tme. In frst fgure we used W = 0.38 and W =0.40, n second fgure we used W =0.4 and W = CONCLUSION In ths paper we developed a mathematcal model for detecton and removal process of fault. In ths study we use Dfferental Evaluaton for dynamc estmaton of fault detecton and correcton process for a gven pont of tme. We have. Bezer B., Software Testng Technques. Boston: Internatonal Thomson Computer Press (990).. Kuo W. and Prasad V. R., An Annotated Overvew of System-Relablty Optmzaton, IEEE Trans. Relablty, vol. 49(), pp (000). 3. Ohtera,., Yamada, S., Optmal allocaton and control problems for software-testng resources. IEEE Trans. Relab. 39 (), (990). 4. Leung, Y.W., Dynamc resource-allocaton for software-module testng. J. Syst. Software 37 (), (997). 5. Kapur, P.K., Bardhan A.K.and Yadavall V.S.S., 'On allocaton of resources durng testng phase of a modular software', Internatonal Journal of Systems Scence, 38 (6), (007) 6. Ohba, M., 'Software relablty analyss models', IBM Journal of research and Development 8, (984). 7. Schnedewnd, N.F., 'Analyss of error processes n computer software', Sgplan Notces 0, (975). 8. Xe, M. and Zhao, M., ' The Schnedewnd software relablty model revsted.' Proceedngs of the 3rd Internatonal Symposum on Software Relablty Engneerng, 5, 84-9 (99). 9. Schnedewnd, N.F., 'Analyss of error processes n computer software', Sgplan Notces 0, (975). 0. Gokhale, S.S., Wong, W.E., Trved, K.S. and organ, J.R., 'An analytc approach to archtecture-based software relablty predcton' n Proceedngs of the Internatonal Symposum on Performance and Dependablty, September, pp. 3- (998).. Kapur, P.K., and Bardhan, A.K.., Testng Effort Control Through Software Relablty Growth Modellng, Internatonal Journal of Modellng and Smulaton,, (00).. Khan M, Ahmad N, and Raf L. "Optmal Testng Resource Allocaton for Modular Software Based on a Software Relablty Growth Model: A Dynamc Programmng Approach, " Proceedngs of the Internatonal Conference on Computer Scence and Software Engneerng (008), 99

8 Internatonal Journal of New Computer Archtectures and ther Applcatons (IJNCAA) 4(4): The Socety of Dgtal Informaton and Wreless Communcatons, 04 (ISSN: ) 3. uang, C. Y., Lo, J.., Kuo, S. Y. and M. R, Optmal Allocaton of Testng-Resource Consderng Cost, Relablty, and Testng-Effort, Dependable Computng, 004. Proceedngs. 0th IEEE Pacfc Rm Internatonal Symposum, pp. 03- (004). 4. Chn-Yu uang, and Mchael R. Lyu, Optmal Testng Resource Allocaton, and Senstvty Analyss n Software Development, IEEE Transactons on Relablty, VOL. 54, NO. 4, (005). 5. Nasar. Md., Johr Prashant, Udayan Chanda,"Dynamc Effort Allocaton Problem Usng Genetc Algorthm Approach", IJMECS, vol.6, no.6, pp.46-5, (04).DOI: 0.585/jmecs Kapur P.K., Pham oang, Chanda Udayan and Kumar Vjay: Optmal allocaton of testng effort durng testng and debuggng phases: a control theoretc approach, Internatonal Journal of Systems Scence, (0) DOI:0.080/ Nasar Md, johr prashant and chanda Udayan A Dfferental Evoluton Approach for Software Testng Effort Allocaton Journal of Industral and Intellgent Informaton (JIII), SAN JOSE, CA, USA Volume, Issue, pp. -5, ISSN (03) 8. SK.Md.Raf, and ShahedaAkthar, " Resource Allocaton to Software Modules n Software Testng wth Imperfect-debuggng SRGM, " Internatonal Journal of Computer Applcatons ( ) Volume 8 No., (0) 9. Seth, S.P., and Thompson, G.L., Optmal Control Theory - Applcatons to Management Scence and Economcs (nd ed.), New York: Sprnger (005). 0. Prce,, K. and Storn R., Dfferental evoluton a smple evoluton strategy for fast optmzaton, Dr. Dobb s Journal, vol., pp. 8 4, (997). Storn, R., Dfferental evoluton desgn for an IIR-flter wth requrements for magntude and group delay, Techncal Report TR-95-06, Internatonal Computer Scence Insttute, Berkeley, (995).. Angra, R. and Babu, B., Evolutonary computaton for global optmzaton of non-lnear chemcal engneerng processes, n Proceedngs of Internatonal Symposum on Process Systems Engneerng and Control, Mumba,, pp (003). 3. Babu, B. and Angra, R., Optmzaton of non-lnear functons usng evolutonary computaton, n Proceedngs of the th ISME Internatonal Conference on Mechancal Engneerng, Inda, pp (00). 4. Prce, K. and Storn, R., (July 003) Web ste of DE. Avalable: 00