The Use of Cuckoo Search in Estimating the Parameters of Software Reliability Growth Models Dr. Najla Akram AL-Saati and Marwa Abd-ALKareem Software Engineering Dept. College of Computer Sciences & Mathematics, Mosul, Iraq International Journal of Computer Science and Information Security (IJCSIS), 2013 2013. 8. 7 Taehyoun Kim S P I R A L Software Process Improvement and Reliability Assurance Leb.
Outlines Introduction Related Work Cuckoo Search (CS) Experiments Conclusion & Future Work Discussion
Introduction Software Reliability The probability of failure-free software operation for a specified period of time in a specified environment (ANSI/IEEE, Standard Glossary of Software Engineering Terminology, STD-729-1991, 1991) Software Reliability Growth Models (SRGMs) The theoretical models for estimating software reliability growth based on failure detection phenomenon during testing phase (S. Yamada and S. Osaki, Software Reliability Growth Modeling: Models and Applications, TSE, 1991) Actual Failure data Figure 1. Basic concept of SRGMs 1/21
The number of failure Parameter Estimation of SRGM Traditional SRGMs have some parameters which have physical interpretations. Ex) Goel-Okumoto Model Cumulative failure function m t = a 1 e rt a : the number of total failures r : failure detection rate a = 120, b= 0.04 a = 120, b= 0.02 a = 80, b= 0.04 a = 80, b= 0.02 Failure data Optimal Time (Weeks) Figure 2. cumulative failure function 2/21
Classical Estimation Techniques There are two of the most popular estimation techniques. Maximum Likelihood Estimation (MLE) Least Square Estimation (LSE) However, these techniques are not effective to estimate parameters of complicated function like non-linear function. SRGMs are non-linear functions. (L. Shanmugam and L. Florence, An Improved ACO Technique for Parameter Estimation, European Journal of Scientific Research, 2012) The interest in heuristic algorithm began to have effect in solving this problem. There are many nature inspired algorithms. 3/21
Related Work Genetic Algorithm (GA) Approach : Survival of the genetically fittest (T. Minohara et al., Parameter estimation of Hyper- Geometric distribution Software Reliability Growth model by Genetic Algorithm, ISSRE, 1995) Particle Swarm Optimization (PSO) approach : Flock migration (Zhang Ke- han et al., Estimating parameters of software reliability models using PSO Computer Engineering and Applications, 2008) Ant Colony Optimization (ACO) approach : Shortest path to food source (Changyou Zheng et al., A parameter Estimation Method for Software Reliability Models, Procedia Engineering, 2011) 4/21
Cuckoo s Inspiring Behavior (1/2) Behavior 1. They have a strange habit of laying their eggs in public nests. Behavior 2. They remove other s eggs to boost the hatching probability of their own eggs. Behavior 3.When the first cuckoo baby bird is hatched, he instinctively throw out host eggs to increase his share of food. 5/21
Cuckoo s Inspiring Behavior (2/2) 6/21
Cuckoo Search 1 2 3 5 4 (X.S. Yang and S. Deb, Cuckoo Search via Levy Flights, Nature & Biologically Inspired Computing (NaBIC) 2009) 7/21
Basic Steps of Cuckoo Search (1/5) Step1. Generate initial population of n host nests. Ex) Goel-Okumoto model m t = a 1 e rt a 1 r 1 a 2 r 2 a n r n nest 1 nest 2 nest n (a i, r i ) : a candidate for optimal parameters 8/21
Basic Steps of Cuckoo Search (2/5) Step2. Lay the egg (a k, b k ) in the k nest. K nest is randomly selected. Cuckoo s egg is very similar to host egg. a k = a k + Randomwalk(Le vy flight) ak r k = r k + Randomwalk(Le vy flight) rk Cuckoo s egg a k rk a k r k nest k 9/21
Basic Steps of Cuckoo Search (3/5) Step3. Compare the fitness of cuckoo s egg with the fitness of the host egg. Root Mean Square Error (RMSE) RMSE = 1 n n t=1 m t m t 2 Which egg is better? m t : the number of predicted failures by time t m t : the number of actual failures by time t a k rk a k r k nest k 10/21
Basic Steps of Cuckoo Search (4/5) Step4. If the fitness of cuckoo s egg is better than host egg, replace the egg in nest k by cuckoo s egg. fitness cuckoo s egg fitness(host egg) i th generation i+1 th generation b k a k rk a k r k a k r k nest k nest k a k r k fitness cuckoo s egg < fitness(host egg) nest k 11/21
Basic Steps of Cuckoo Search (5/5) Step5. If host bird notice it, the nest is abandoned and new one is built. (p <0.25) (to avoid local optimization) Cuckoo s egg b k a k bk a k bk New egg nest k new nest k Iterate steps 2 to 5 until termination criterion is met. 12/21
Experimental Set-up (1/2) Comparison with other techniques reference Parameter estimation techniques Data Set A. Sheta s [1] Particle Swarm Optimization (PSO) Data Set 1 Data Set 2 Data Set 3 L. Shanmugam et al. s [2] Ant Colony Optimization (ACO) L. Shanmugam et al. s [3] Extended Ant Colony Optimization (Ex-ACO) Musa Data 1* Musa Data 2* Musa Data 3* * John D. Musa, Software Reliability Data, Cyber Security and Information Systems Information Analysis Center, January 1980. Table1. Estimation techniques and data sets [1] A. Sheta, Reliability Growth Modeling for Software Fault Detection Using Particle Swarm Optimization, IEEE Congress on Evolutionary Computation, 2006 [2] L. Shanmugam and L.A. Florence, Comparison of Parameter Best Estimation Method for Software Reliablity Models, International Journal of Software Engineering & Applications (IJSEA), 2012 [3] L. Shanmugam and L. Florence, An Improved ACO Technique for Parameter Estimation, European Journal of Scientific Research, 2012 13/21
Experimental Set-up (2/2) SRGMs for the experiments SRGMs Mean value function (cumulative failure function) Goel-Okumoto (GO) model m t = a 1 e rt Power (POW) model m t = at r Yamada Delayed S-Shaped (DSS) model m t = a 1 (1 + rt)e rt Musa-Okumoto (M-O) model m t = aln(1 + rt) Parameter settings for the cuckoo search Parameters Table2. SRGMs for the experiments Value Lower and Upper bound for parameter a [0.00001 ~ 2000] Lower and Upper bound for parameter r [0.00001 ~ 1] Number of Nests 10 Number of iterations(generations) 100 Table3. Parameter settings for cuckoo search 14/21
The impact of training and testing data A balance point is training 70% and testing 30%. (a) (b) (c) Table4. The impact of training and testing percentage (a) data set 1, (b) data set 2, (c) data set 3 15/21
Comparison with other techniques (1/4) PSO vs. Cuckoo Search (Data set 1) (A. Sheta, Reliability Growth Modeling for Software Fault Detection Using Particle Swarm Optimization, IEEE Congress on Evolutionary Computation, 2006 ) Training 70% Testing 30% PSO CS PSO CS G-O 20.2565 34.0933 119.4374 16.8945 POW 22.2166 44.8663 152.9372 33.6623 DSS 15.9237 32.6376 26.3015 10.9945 Table5. Comparison with PSO using Data set 1 Figure3. Comparison with PSO using Data set 1 16/21
Comparison with other techniques (2/4) PSO vs. Cuckoo Search (Data set 2) Training 70% Testing 30% PSO CS PSO CS G-O 24.9899 33.2311 80.8963 14.2998 POW 32.355 47.0571 149.9684 56.6807 DSS 20.8325 27.9159 17.0638 11.8833 Table6. Comparison with PSO using Data set 2 Figure4. Comparison with PSO using Data set 2 17/21
Comparison with other techniques (3/4) PSO vs. Cuckoo Search (Data set 3) Training 70% Testing 30% PSO CS PSO CS G-O 12.8925 13.5404 13.6094 8.9523 POW 11.9446 13.0886 14.0524 13.4669 DSS 18.5807 13.6634 47.4036 15.1916 Table7. Comparison with PSO using Data set 3 Figure5. Comparison with PSO using Data set 3 18/21
Comparison with other techniques (4/4) ACO vs. Ex-ACO vs. Cuckoo Search (L. Shanmugam and L.A. Florence, Comparison of Parameter Best Estimation Method for Software Reliablity Models, International Journal of Software Engineering & Applications (IJSEA), 2012) (L. Shanmugam and L. Florence, An Improved ACO Technique for Parameter Estimation, European Journal of Scientific Research, 2012) (a) Musa Data Set 1 - Training 100% ACO EX-ACO CS G-O 60.0371 28.5891 41.7971 POW 52.8854 34.0521 45.9783 DSS 52.8854 33.0461 42.2256 M-O 26.0385 17.359 41.7732 (b) Musa Data Set 2 - Training 100% ACO EX-ACO CS G-O 71.5489 34.0709 21.7256 POW 57.5801 47.5814 15.5885 DSS 57.5801 48.4914 22.4944 M-O 36.1891 24.126 19.5448 (c) Musa Data Set 3 - Training 100% ACO EX-ACO CS G-O 71.4015 35.0007 25.7682 POW 53.2234 34.2645 28.1951 DSS 53.2234 35.2635 25.7294 M-O 33.1728 22.1152 26.4575 Table8. Comparison with ACO and Extended ACO (a) musa data set 1, (b) musa data set 2, (c) musa data set 3 19/21
Conclusion & Discussion Conclusion Cuckoo Search can be successfully applied to find acceptable solutions to problem of parameter estimation of SRGM. Future work Other swarm intelligent methods can be applied and compared. Ex) Harmony Search, Firefly algorithm, Accelerated Particle Swarm Optimization(APSO), etc.. 20/21
Discussion Cons. New heuristic algorithm was applied to estimate the parameters of SRGM. The results are obtained very quickly and accurately. Pros. Experiments were not proper to compare cuckoo search with other heuristics. 21/21
Q&A Thank you
Least Square Estimation (LSE) (1/2) Minimize the sum of square of the deviations between actual data and expected data Ex) parameter estimation of function f x E a, b = y i ax i + b 2 E(a,b) n i=1 = ax + b a b Optimal Point When d E a,b da = 0, d E a,b db = 0, The values of a and b are the best estimates of a and b. Figure 2. Basic concept of LSE
Least Square Estimation (LSE) (2/2) Ex) Goel-Okumoto Model When m t = a 1 e rt n E a, b = (y i a(1 e rt i)) 2 d E a,b da i=1 d E a,b = 0, db = 0, The values of a and b are the best estimates of a and b. E(a,b) b a
Le vy flight (1/2) A random walk in which the step-lengths have a probability distribution that is heavy-tailed (to avoid local optimization) - Normal Distribution - Heavy tail distribution Step Size * random() Step Size * random() Current point Step Size Figure6. the basic concept of random walk (Le vy flight)
Le vy flight (2/2) Normal distribution VS. Heavy tail distribution Figure7. an example of Brownian motion (Normal distribution) Figure8. an example of Le vy flight (Heavy tail distribution)