Int. J. Pure Appl. Sci. Technol., 18(1) (13), pp. 26- International Journal of Pure and Applied Sciences and Technology ISSN 2229-6107 Available online at www.ijopaasat.in Research Paper Reliability Levels of Some Selected Software Produced by Nigerian Companies (Case Studies of Glomosoft, X-Bios Systems, Micro Bits ) Boukari Souley 1, * and Nnamso M. Umoh 1 1 Mathematical Science Programme, ATBU, Bauchi * Corresponding author, e-mail: (bsouley01@yahoo.com) (Received: 9-6-13; Accepted: 9-7-13) Abstract: Software reliability is a major factor in software quality and is the index by which users judge particular software. Software reliability to the users is the ability of the software to perform its perceived function without fail. It therefore must be determined in all software projects of importance. Some software reliability models were reviewed and the Jelinski-Moranda model was chosen which was considered the most appropriate for the intended software tool (i.e RelSoft). Some software houses were visited and failure data collected, formatted and analyzed. The least square method was used to determine some parameters such as E t (i.e. total error) and K, the proportionality constant. These parameters were used as input to design Relsoft tool which was then used in determining the reliability levels of the selected software produced by Nigerian software companies. The result obtained from RelSoft, showed that nine out of the ten selected software have reliability levels ranging from 0.07 to 0.82, except one which has a reliability level of 0.42. Other reliability indicators such as failure rate and Mean Time To Failure (MTTF) were also calculated using RelSoft. The software reliability is measured on a scale ranging from zero to one (i.e. 0 to 1). However, for life-critical application the reliability must be in the region of 0.999 for it to be considered to be reliable. Keywords: Software Reliability, Software Reliability Level, Failure Rate, MTTF, Total Error (E T ), Constant of Proportionality (K).
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-27 Introduction In the modern business environment in which we find ourselves today, there are a lot of competitions among business organizations to have a share of the market in their particular interest area(s) for them to remain in contention, these organizations need tools to boost their accuracy, productivity, efficiency, performance, competitiveness etc. Among these tools is the computing system. Organizations budget huge amount of money to purchase and maintain this computing system. The computing system consists of the hardware and the software. Organizations (Nigerian organizations inclusive) demand for highly reliable software and software of high quality. In [1] it was stated that not only do sponsors and end-users want more software, they also want better software. A fundamental fact of life in this 21 st century is that software is pervasive. It influences our lives in ever more complex and significant ways. Lives and well-being are increasingly dependent on computerized (software) systems. Greater number of life-critical systems such as aircraft, spacecraft, other transportation vehicle; medical treatment machines, military equipment and communication systems are controlled by computers. Therefore, it is inevitable, the software, whether embedded or otherwise must be developed with an acceptable level of reliability [1]. With the advent of the computer age, computers, as well as the software running on them are playing a vital role in our lives. We may not have noticed, but appliances such as washing machines, telephones, TVs and watches, are having their analog and mechanical parts replaced by CPUs and software. Like machinery replaced craftsmanship in the industrial revolution computers and intelligent parts are quickly pushing their mechanical counterparts out of the market [6]. The above two contributions from [1] and [6] point to the fact that computers and computer software have virtually taken over almost every facets of our lives. It therefore means that to prevent losses, which range from loss of lives (i.e. in safety critical systems) to minor inconveniences, Software Companies must design and build software with high reliability. However, to achieve high reliability in software, certain other factors are likely to suffer. These include among others effort, cost and development time. So far, mentions have been made of software reliability and reliable software. What then is software reliability? In [7], software reliability is viewed as the probability that a given system will operate without failure under given conditions for a given time interval. Software reliability is expressed on a scale from 0 to 1. A system that is highly reliable will have a measure close to 0 while a system that is unreliable will have a measure of reliability close to 1. Others view Software reliability as the probability of failure free operation for a specified time in a specified environment for a specific purpose [11]. In Nigeria, it will be observed, that some giant strides are being made in software development considering the proliferation of software development companies in the Country. But the question is, are these companies meeting the quality assurance attributes of which reliability is a very important factor? What is the Problem? Organizations spend a good percentage of their budget either to develop in-house or to contract software companies (developers) to build software for them. These software are supposed to enhance their operations and improve their businesses, but this is not the case, most especially in Nigeria, rather, what they get are unreliable software whose rate of failures are alarming. Most software fail many times after they must have been delivered, thus disrupting the operation of the system (organization). This results in huge losses in terms of money and time to the organization and at times loss of lives to the society. Considering the above, it becomes imperative that delivered software that meets the required level of reliability so as to allow for failure-free operation for a specified time, environment and purpose.
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-28 There has not been any concerted effort (that we know of) towards ascertaining the reliability levels of software produced by Nigerian developers. Review of Literature Software Engineering is defined according to [8], as the disciplined application of theories and techniques from computer science to define, develop, deliver, and maintain, on time and within budget, software products that meet customers need and expectations. However, this definition seems, inadequate as Software Engineering (SE) as a discipline rests on other intellectual foundation apart from that of computer science concepts. These according to [9] are engineering knowledge complemented by the social and economical fundamentals. Software development which is synonymous with SE is business oriented, therefore is expected to make some level of profit as developers not losing sight of the assurances or expectations of their clients, that is, users. Software Development Companies (SDC) are the producers of software. Producers are important in Information Technology (IT). This is so because the functionality of a computer system depends largely on the quality of the software installed. Like any other business, for the software companies to remain in the market, software products must of necessity be competitive. This implies that software companies must develop products that have certain quality attributes such as reliability, functionality, efficiency, portability, maintainability, reusability, testability, and some software management issues such as delivering products within schedule and within budget. Developers and users therefore will expect that the software churned out from these software companies meet certain standards and provide satisfaction. Software has many quality attributes that are subject to a series of factors compromising achieving them. Expectations of the Nigeria User Community Software acquisition and usage usually involves expenses, users therefore will expect that the software they purchase should meet certain standards and provide satisfaction. This satisfaction can be considered broadly under the following: reliability, maintainability, portability, usability, reusability, cost-effectiveness and efficiency Expectations of Software Producers Community in Nigeria Software developers also have their own expectations. As pointed out earlier, software producers are in business to make profit among other things. Loosing user loyalty is a problem that software companies take seriously, especially in this dynamic market where customer allegiance lasts only until the competition produces a more user-friendly version. Therefore, every software producer wants to satisfy the user, stay in the market and make profit. Developers also expect the product to have long life because a system with a long life span has more value. System Reliability System reliability consists of both software reliability and hardware reliability. Many definitions of reliability exist, depending upon the viewpoints of the stakeholders. However, they all have a common core, which contains the statement that, reliability R (t), is the probability that a device performs adequately over the interval [0, t], Among the attributes of computer software, the first that commands the attention of quality is the degree to which the software can be depended upon to perform its function. The extent to which software is not reliable is a function of the bugs embedded within it or the inconsistencies between it and the purposes for which it was intended [3]. Software reliability is a function of the number of failures experienced by a particular user of that software. A software failure occurs when the software is executing. It is a situation in which the
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-29 software does not deliver the service expected by the users. Software failures are not the same as software faults although these terms are often used interchangeably [11]. Software faults may be programming or design errors where by the delivered program does not conform to the system specification. Alternatively, they can be specification or documentation errors. Software faults are static. They are characteristics of the program code. They are discovered either through program inspections or by inferring their existence from software failure. Software faults cause failures when the faulty code is executed with a set of inputs, which expose the software system as a mapping of an input to an output set. The program responds to these inputs by producing an output or a set of outputs. The software reliability is related to the probability that, in a particular execution of the program, the system input will be a member of the set of inputs, which cause an erroneous output. There is a complex relationship between observed system reliability and number of latent software faults. And that not all software faults are equally likely to cause software failure [11]. Usually, there are a number of member of erroneous inputs which are more likely to be selected than others. If these inputs do not cause the faulty parts of the software to be executed, there will be no failure. The reliability of the program, therefore, mostly depends on the number of inputs causing erroneous outputs, which arise during normal, rather than exceptional use of the systems. It therefore follows that, a program may contain known faults but may still be considered to be reliable by its users. They may never select an erroneous input so program failures never arise. This means that software quality and reliability does not necessarily depend on the number of bugs present in software. A software product can be of a high quality and reliability if the bugs it contains, however, are numerous and are located in infrequently used modules. On the other hand, software quality and reliability is lower if it contains fewer bugs, located in frequently used paths, Thus, the presence of faults is a function not just of the software, but also of user, customer expectation, and set of inputs [2]. While it is tempting to draw an analogy between Software Reliability and Hardware Reliability, software and hardware have basic differences that make them different in failure mechanisms. Hardware faults are mostly physical faults while software faults are faults, which are harder to visualize, classify, detect and correct. Design faults are closely related to fuzzy human factors and design process, which, we don t have a solid understanding ([], [6]). Deviations An exact analysis and classification of error types that can appear in system is of central importance to the development of reliable system [4]. In the discussions on software reliability a number of different terms are used to indicate a deviation between the intended and actual state or behavior of a software, for example, error, fault, failure, mistake, malfunction, deficiency, flaws and so on. With each one of these concepts, some specific attributes, are commonly associated which make it difficult to use such a concept in a general context. In order to avoid such semantic problems, the listed terms will be used henceforth with the following meaning: Error: This is the deviation between intended and observed state or behavior of a system (This is the most general term) Failure: This is error during operation of the system Fault: This is error in the hardware or software Mistake: This is human error.
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26- Some Special Error and Failure Types As we know errors and failures constitute software that is unreliable. Therefore, for any system for that matter to be reliable, reliability must be built into the system right from the conception stage. This could be done by carrying out certain analyses among which are: systems analysis error, error in programming, data preparation errors, execution errors, transient hardware faults, data bank errors. Software Reliability Models Reliability can be calculated using software reliability models. Reliability models can be grouped into two namely, Macro and Micro models. Macro models are those that are based on the number of instructions, the number of error removed, and the overall details of the control structure. Constants for the model can be evaluated from data on past systems as well as analysis of test data on the software being developed [10]. Micro models on the other hand are based on detailed analysis of the statements and the control structure. However, for this paper, the macro model, that is, the Jelinski-Morada Model is adapted. Jelinski Moranda Model (MACRO) Jelinski and Moranda proposed on hazard function according to [10] of the form Z(t) = ф [N (i 1)] eq.1 Where ф = Constant of proportionality N = Total number of errors present i = Number of errors found by debugging time t In addition the eq.1 can be related to the basic reliability model below: R(t) = exp-[k Є r (τ)]= exp-[k(e T I T Є c (τ))]t eq.2 So that Then E T = N E T I T = ф Є c (τ)] = (i-1) I T R(t) = exp[-(ф [N (i 1)])t] eq.3 eq.4 eq. eq.6 Since eq. 3, eq.4 and eq. are merely notational differences Jelinski Moranda Model could be written as R(t) = exp[-(k [E T Є c ])t] eq.7 The Jelinski Moranda model assumes that each defect is removed as soon as it is detected. It also assumes that all the bugs in the program are equally likely to cause a failure in the testing environment. It further assumes that the failure rate is a Poisson process and proportional to remaining defects. Finally as do most models, it assumes no new defects will be introduced. The parameters K and E T must be calculated or estimated using any of the three parameter estimation techniques (i.e.maximum-likelihood estimation, Moment Estimation and Least-Square Estimation Methods). For this, paper we shall be using the Least-Square Estimation Method.
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-31 The reliability function is R(t) = exp[-(k [E T Є c ])t i ] Where t i = elapsed Time between (i 1)st and ith failures Є c = number of defect removed by (i 1) st interval E T = Initial or total number of errors MTTF associated With Jelinski Moranda Model As with Shooman model, this is of the form R(t) = e - αt, from which MTTF = 1/α or MTTF = 1/ф (N-n) Research Methodology This study was prompted by the outcries or complaints by software users and by the need to ascertain the reliability levels of the software developed by these developers. This study was used to put things in their right perspectives i.e. to find out if these complaints are genuine or not. During the study, relevant data were collected through interviews and record reviews, and converted to suit the input format required. The formatted data were then used as input for the designed software reliability tool. However, the tool was designed to meet the system requirement of ascertaining the reliability level of each software product under study. The reliability level gives the state of affairs for each product and in fact, the software companies that produced them. As mentioned above, a study was carried based on the used to ascertain the level of reliability of each software under investigation. The request or the problem that necessitated this study was the complaints made by the software users and / or those directly or indirectly affected. The main systems requirement in this case will be to determine the level of reliability of these software. The other requirement will be to reduce the level of complains by users and to create the awareness and the need for software developer to design and build reliable system for the growing user-population in Nigeria. With the above stated requirements in mind, it is specified that software developed for the Nigerian users must range between 0. and 1.0 on the probability scale. This specification, however depends on the application domain. For life critical or mission critical software, the reliability scale should read up to 0.999 and above for the system to be said to be reliable. Data Collection In the first phase, data were collected from several software developers and users in Nigeria. This was done through interviews and record review of program logbooks where historical data, defects data and failure data are recorded. However, most software houses visited did not have the necessary documentations in their logbooks. In such cases, we had to sit down with the designers and programmers who were involved in the design to do a reengineering of the process to estimate some of the data that were needed for this work. In few cases where the logbooks were available, they were reviewed and relevant data needed to make a success of this work were extracted. The data collected were properly formatted and presented as shown in table1.
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-32 Table 1: Failure data from the selected software under study S/No Project Time Period in Weeks (1) ProjectSW 1 Week I I (2) ProjectSW 2 Week I I (3) ProjectSW 3 Week I I (4) ProjectSW 4 Week I I () ProjectSW Week I I (6) ProjectSW 6 Week I I (7) ProjectSW 7 Week I I (8) ProjectSW 8 Week I I (9) ProjectSW 9 Week I I (10) ProjectSW 10 Week I I Test Period in Hours 2 1 0 9 0 32 0 33 12 48 2 90 18 83 73 42 32 18 39 22 22 7 48 29 26 48 2 2 Software failure 6 24 10 0 43 49 18 9 4 0 2 8 8 2 11 7 7 69 23 12 4 2 0 7 29 13 8 36 7 37 48 9 6 66 44 24 18 Failure Rate Error- Z, h -1 Corrected E c (ԏ) 1. 0.93 0.47 0.33 0.10 1.43 0.83 0. 0.6 0.4 2.0 0.91 0.0 0.4 0.27 0.92 0.42 0.26 0.17 0. 0. 0.12 0.10 1. 0.66 0. 0.22 0.10 1. 0.90 0. 0.31 0.2 2.00 1.0 0.1 0.32 0.27 1.68 0.84 0.42 0.31 0.23 1. 0.92 0.8 0.46 0. 4 26 10 4 6 24 12 12 32 13 7 2 11 7 41 38 37 17 10 33 46 18 14 0 16 37 4 63 2 13 10 31 6 12 Cumulative error corrected 4 70 7 4 110 134 146 168 2 6 72 77 2 1 8 81 119 4 37 7 62 77 94 10 43 89 107 121 0 16 3 8 63 4 108 133 146 6 86 92 104 MTBF 0.63 1.08 2.13 3.00 10.00 0.70 1. 1.67 1.79 2.22 0. 1.10 2.00 2.22 3.70 1.09 2. 3.90.0.00 6.67 8.33 10.00 0.77 1.2 3.33 4. 10.00 0.62 1.11 2.0 3.23 4.00 0. 1.19 2.38 3.23 4. 0.83 1.09 1.72 2.17 2.86 0.83 1.09 1.72 2.17 2.86 Software size (I T ) 26,00 33,000,0 16,000 18,2 24,000 21,00 16,0,00,000
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-33 Design Model The design of the software reliability tool for determining the reliability levels of the selected software is based on the Jelinski-Moranda Model. This model is stated in eq.7 and discussed in ([3], [10] and [2]). The estimation technique for E T and K (i.e. total error and constant of proportionality) used in this work is the least square method. Estimation of E T and K (Model Parameters) From table1, data were extracted to plot a graph of errors against failure rates. From the Least Square graph, K values, the constants of proportionality were estimated. The estimated values are shown in the table 2. Estimation Procedure (1) Read the values of E T from graph (2) Read the values of Z from graph (3) Set Z against K E T (i.e. Z = K E T ) (4) Then calculate K. Table 2: Estimated Values of K and E T Parameters Using Least Square Technique Software project K E T I T Project SW 1 0.016 90 26,00 Project SW 2 0.009 6 33,000 Project SW 3 0.0 93,0 Project SW 4 0.022 70 16,000 Project SW 0.012 210 18,2 Project SW 6 0.0 101 24,000 Project SW 7 0.012 136 21,00 Project SW 8 0.0 73 16,0 Project SW 9 0.008 174,00 Project SW 10 0.013 129,000 Input Design The input design screen is such that the systems prompts the operator for each of the data element needed to implement the systems by way of calculating the reliability, failure rate and mean time to failure (MTTF) for each of the software. The total error, constant of proportionality, error corrected, and operating time are as shown in table 3. Table 3: Intermediate result used as program input Software Project K E T E C T Project SW1 0.016 90 7 1.00 Project SW 2 0.009 6 168 1.0 Project SW 3 0.0 93 77 2.00 Project SW 4 0.022 70 8 2.00 Project SW 0.012 210 4 1.2 Project SW 6 0.0 101 94 3.00
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26-34 Project SW 7 0.012 136 121 1.00 Project SW 8 0.0 73 63 2.00 Project SW 9 0.013 174 6 2.00 Project SW 10 0.013 129 104 1.00 Output Design The output screen is designed in such a way that it tabulates the result under four fields. These are: project (meaning the software for which the reliability level is sought) and it related failure rate, reliability and mean time to failure (MTTF). Calculation of the Failure Rate, Reliability, MTTF With the values of E T and K estimated and using the Jelinski Moranda model stated in eq.7, the reliability for each of the 10 selected software were calculated using RelSoft. Other reliability metrics such as MTTF and failure rate were also calculated. Results A software reliability tool called RelSoft was developed. It was fully tested. Data were collected from the selected software (see table 1), based on the least square estimation method, the model parameters were estimated i.e. E T and K (see table 2). An intermediate result shown in table 3 was used as input for the RelSoft. After running the relsoft, the following results were obtained as displayed in table4 Table 4: Results from RelSoft Project Failure Rate Reliability MTTF Project SW1 0.26 0.774 3.906 Project SW2 0.1 0.91 2.849 Project SW3 0.3 0.07 2.941 Project SW4 0.286 0.64 3.497 Project SW 0.684 0.42 1.462 Project SW6 0.1 0.698 8.333 Project SW7 0.192 0.82.8 Project SW8 0.286 0.64 3.497 Project SW9 0.247 0.610 4.049 Project SW10 0.338 0.713 2.99 Discussion From table 4 it can be observed that 9 of the 10 selected software exhibited a reliability level above 0., indicating average reliability. The results indicate that project SW7 has a highest reliability of 0.82, followed by projects SW1 and SW10 with reliability levels of 0.774, and 0.713. Two software projects have reliability levels of 0.698 and 0.610. These are projects SW6 and SW9. While projects WS2, WS4, SW8 and SW3 has a reliability of 0.91, 0.64,.064 and 0.07 respectively. The only project that did not attend average reliability is project SW with a reliability of 0.42. Project SW7 is the most reliable of the 10 selected software while project SW is the least reliable. As mentioned
Int. J. Pure Appl. Sci. Technol., 18(1) (13), 26- earlier, the mean time to failure (MTTF) and failure rate are reliability metrics which are also reliability indicators. The values for these metrics for each of the software indicated in table 4. There is a price to be paid for a high reliability, therefore it is advisable to determine the optimum release time for a given software project. Conclusions Results from the reliability software tool indicate that 90% of the selected software are reliable. This work has been able to establish the fact that software development companies in Nigeria have performed adequately on the average contrary to speculations in some quarters of the poor state of affairs in the industry. A large scale study is not likely to deviate too much from this result. It will therefore be expedience to conclude that more than 0% of software produced in Nigeria are reliable and that there is a better hope for the Nigerian software industry and with concerted efforts, Nigeria could join the league of software exporting nations. And by implication could become a source of foreign exchange earner for the nation. References [1] C. Adiele, Software crisis and its social impact, In Proceeding of Computer Association of Nigeria, Conference Series, 9(1998), 94-101. [2] V. Allain, Reliability, Availability, Maintainability and Safety Assessment, John Wiley and Sons, New York, 1992. [3] R. Dunn and R. Ullman, Quality Assurance for Computer Software, Mcgraw Hill Book Company, New York, 1982. [4] H. Kopetz, Software Reliability, Macmillan Press Ltd., London, 1979. [] M.R. Lyu, Software Fault Tolerance, John Wiley and Sons Inc., Englewood Cliffs, NJ, USA, 199. [6] J. Pan, Software Reliability, Carnegie Mellon University, Carnegie Mellon, 1999. [7] S.L. Pfleeger, Software Engineering: Theory and Practice, Prentice Hall, New Jersey, 1998. [8] T.K. Ralston, E.T. Roberson and H.T. Gilb, Software Engineering Principles, Prentice Hall, New Jersey, 00. [9] S.E. Shaw, Software Engineering Theories and Practice, Mcgraw-Hill Book Company, NY, 0. [10] M.L. Shooman, Software Engineering: Design, Reliability and Management, Mcgraw Hill International Book Company, Singapore, 1983. [11] I. Sommerville, Software Engineering, Addison Wesley Longman Limited Harlow, England, 199.