Questions & Answers Chapter 10 Software Reliability Prediction, Allocation and Demonstration Testing

Transcription

1 M13914 Questions & Answes Chapte 10 Softwae Reliability Pediction, Allocation and Demonstation Testing 1. Homewok: How to deive the fomula of failue ate estimate. λ = χ α,+ t When the failue times follow an exponential distibution, the numbe of failues in a fixed inteval of time follows a Poisson distibution with associated paamete λt. Poisson pocess is applicable when the pobability that en even occus is popotional to the length of time inteval. The pobability of exactly n failues in the time inteval (0,t] is given by Poisson pobability mass function (in case of discete andom vaiables and constant failue intensity λ): pmf{n, t} = λtn e λt The above fomula can be used to obtain the uppe bound of failue ate λ by solving the following equation: 1 CL = λtn e λt Whee is the total numbe of failues, CL is the confidence level and λ is the failue ate to be estimated at confidence level of CL. We can futhe manipulate the above equation in some steps to show the elationships with Chi- Squaed distibution. We define: Then x = λt 1 CL = xn e x Using this equation, fo a given confidence level CL, the uppe bound of andom vaiable X can be solved fo. P(X < x) = CL = 1 xn e x 1

2 M13914 This equation can be elated to the Gamma Distibution. The geneal fomulation of gamma function is defined by: Γ(α) = x α 1 e x dx Fo a Gamma Distibution Y Γ(k, λ), the cumulative distibution function (cdf) is: 0 k 1 P(Y < y) = F(y, k, λ) = 1 λyn e λy Compaing the latest two equation, one can see that x follows the Gamma Distibution X ~ Gamma(+1,). In addition, χ (+1) is a special case of the Gamma Distibution if the andom vaiable follows Gamma(+1,). Theefoe we can also say: X χ (+1) and X χ (+1) Since x = λt, we know the uppe bound of the failue ate (λ) is: λ = χ (α,+) T. Explain the meaning of each column in the 1 st case study. I have explained it in the discussion slide, theefoe hee I will explain again in biefs. The desciption of each column in the case study of foecasting failue ate afte peiod of gowth is as follows. Column Name CSCI UTIL λ s SYS τ CSCI τ β λ(τ) λ s (τ) Definition Stands fo Compute Softwae Configuation Items, softwae elements that compise the subsystem of a softwae. Utilization of each CSCI which is expessed as pecentage of total CPU time fo each CSCI. It is the atio of CSCI execution time to system opeating time. The total utilization of the host compute can be found by adding up the utilizations of the CSCIs. [% of Total CPU Seconds] Initial failue ate, with espect to system opeating seconds. [failue/cpu seconds] The numbe of CPU time used in system tests [40 hous = 144,000 seconds] The numbe of CPU time that each CSCI utilized [CPU seconds] Paamete of basic execution time model. It is the decement of failue intensity pe occuence of the failue [1/CPU Seconds]. Each CSCI s failue intensity afte one hou system test. CSCI s failue ate with espect to system opeating seconds [failue/cpu seconds]

3 3. What is the use of softwae eliability allocation? Ahmad Robiyan M13914 Softwae eliability allocation is a pocess to tanslate oveall (o global) softwae eliability equiement into eliability goals fo each existing lowe-level softwae elements. Fo example, if 10 pogams ae exists in a subsystem of softwae and the equiement of oveall subsystem failue ate is one failue pe 10,000 system opeating hous. We can allocate this oveall equiement as a popotions to each component s foecasted failue ate. By this we know the failue ate equiement of each component of the subsystem i.e. we know how eliable a component should be to achieved the specified oveall eliability equiement. 4. What data is used to do the allocation? The main infomation to be used fo softwae eliability allocation ae the foecasted CSCI s system opeating time failue ate (λ l ), and the equied failue ate (Λ REQD ). 5. What is the advantage of demonstation testing? It is impotant to quantify and detemine whethe a softwae is eady to use by demonstating its ability to pass the maximum allowable occuence of failue as specified in the contactual aangement o in the pe-detemined softwae eliability specification. Thus, it can be consideed as a tool of quality assessment fo a softwae. 6. Which one is the best among the thee methods of demonstation testing? Thee types of softwae demonstation tests ae fixed-duation test, failue-fee execution inteval test and sequential test. Each type seves paticula conditions in which they ae ecommended fo. Accoding to MIL-HDBK-781: A fixed-duation test plan must be selected when it is necessay to obtain an estimate of the tue MTBF demonstated by the test, as well as an accept-eject decision o when total test time and the amount of cost must be known in advance. A sequential test plan may be selected when it is desied to accept o eject pedetemined MTBF values (θ 0,θ 1) with pedetemined isks of eo (α,β) and when uncetainty in total test time elatively unimpotant. This test will save time, as compaed to fixed duation test plan having simila isks and discimination atios, and when the tue MTBF is much geate than θ 0 o much less than θ 1. A failue-fee execution inteval test will accept softwae that has a failue ate much lowe than λ 0 moe quickly than a fixed duation test. 7. If a softwae has been ejected by demonstation testing, what will happen next? Get though the entie cycle (again)? System eliability pogams ae often mandated though contactual equiements in which the develope must comply with. Even if it is found that the softwae contains moe failues that it is allowed by the specification, then up to the end of this demonstation testing, though the pocess of debugging and emoving fault, the develope may have been able to focus its effots towads cetain pat of the softwae which is the most susceptible to fault. Theefoe, it is not necessay fo the develope to epeat the entie cycle of softwae eliability engineeing. Instead the 3

4 M13914 develope can concentate on one o seveal unit testing and then pefom the integation and system testing to e-check fo any impovement. 8. What is the diffeence between poduce s isk and consume s isk? Poduce s isk is the isk belongs to the softwae develope. It is the pobability of ejecting softwae with a (unknown) tue failue ate when it is equals to the eliability goal of the softwae (λ 0). Simply speaking, the poduce has disadvantage in this situation since he eject a softwae which is actually has a lowe failue ate compaed with what the custome wants aleady. It means he will be equied to allocate exta esouces fo e-engineeing of the softwae which is actually unneeded. Consume s isk is the isk belongs to the custome. It is the pobability of accepting softwae with a tue failue ate equals to the eliability value specified by the custome (λ 1). Pio to the demonstation test, λ 0 and λ 1 ae specified by an ageement of the develope and custome whee the eliability goal of the softwae should be lage than custome minimum specification (λ 0< λ 1). 9. How do you calculate two decision isks, which ae specified as test paamete? The two test paamete (poduce s and consume s isks) ae not calculated by means of mathematical fomulation. It is detemined by ageement between softwae develope and its custome o it is adopted fom industy s widely applied pactices which usually ange within 10% to 30%. 10. In the failue-fee execution inteval test, what if we have a failue ate lowe than λ 0? In any of thee types of demonstation test, a good test plan should be able to accept with high pobability a softwae with tue failue ate that appoaches λ 0 moeove if it has lowe failue ate than λ In failue-fee execution inteval test, how to decide the inteval? It is calculated by using standads. Hee I give an example of a failue fee-execution inteval test which efe to MIL-HDBK-781. In a failue fee execution inteval test, the softwae is given T time units to achieve a failue-fee inteval of t-time units. Fist, the custome specifies λ 1 as failues/hou. The poduce s and consume s isks ae set at 30%. The eliability goal fo the softwae was specified as λ 0 = failues/hou. The discimination atio (d) is calculated as / =. Now, we would like to calculate the lowe test MTBF, θ 1 = 1/λ 1. This is the lowest acceptable level of MTBF fo the softwae as equied by the custome. By using the table 1 which is adopted fom MIL-HDBK-781, we see that at α=0.3 (column 1) and β=0.3 (column ). Since d=.0, it is appoximated by (column 3). It povides λ 1T = (column 4) o T = hous. This is the given test time given to the develope fo achieving failue-fee inteval. 4

5 M13914 Since t/t = 0.40 (column 6) then t = This is the failue fee inteval which has to be obtained by the softwae. Table 1 Failue Fee Execution Inteval Test Plan MIL-HKBD-781 α β d λ 1T λ 0T t/t ETT/T fo λ 0 ETT/T fo λ