SOFTWARE ENGINEERING SOFTWARE QUALITY Today we talk about software process quality and certification SOFTWARE QUALITY - QUALITY COMPONENTS Objective quality component: properties that can be measured or approximated objectively Subjective quality component: customer satisfaction ( What does the product feel like? ) Other: features which can not be (even subjectively) evaluated at the time. This is related with future events which can not be predicted - unexpected circumstances, changes, etc. 1 2 THE QUALITY SYSTEM THE QUALITY SYSTEM (cont d) Quality control - controlling the way things are done Quality assurance - making sure quality is achieved Quality policy Quality planning Quality improvement These terms come from the standard ISO 8402 The quality system of a company is simply the way the company works and it covers all areas of activity. Therefore, a quality system always exists. It may be documented or not. However, in the long run in a big company it makes a major difference, if the management takes quality seriously and knowingly emphasizes it in all activities. 3 4 1
QUALITY CONTROL QUALITY ASSURANCE Controlling the software development process Standards for the development process, e.g. - well-defined phases - checklists - reviews: what and when - organisational standards Visibility and bookkeeping of the development process Standards for software code and documentation, e.g. -namingand style - document skeletons and formats - different kinds of review and feedback forms Improving software quality by monitoring the products (software) and process Ensuring full compliance with the standards for products and process Ensuring that any inadequacies in the product and process (and standards) are brought to management s attention. 5 6 QUALITY ASSURANCE - PREREQUISITES PROCESS QUALITY METRICS Top management commitment Actual vs. estimated time and cost The quality assurance organisation must be independent from the development organisation. Number of faults detected The quality assurance organisation must be properly staffed. Phase of the project when faults were found Productivity (e.g. lines of code / man-month) The quality assurance organisation must cooperate properly with the development organisation. Quality must be seen as a common goal. 7 Quality expenses (e.g. repairs required after delivery) 8 2
QUALITY REVIEW TYPES Design or program inspections/audits. These are commonly driven by a checklist of possible errors. Management reviews. These are intended to provide information for the management about the overall progress of the software project. Quality reviews. The work of an individual or a team is reviewed by a panel made up of project members and technical management. A GENERAL SCHEME FOR REVIEWS Select the review team. Arrange the place and time. Distributed the documents. Hold the review. Note the actions and complete the review forms. 9 10 POSSIBLE ACTIONS FOR FINDINGS IN REVIEWS INSPECTIONS / AUDIT SESSIONS No action. Some kind of an anomaly was found, butitwasnotcost-effectiveto fixit. Refer for repair. The team or individual in charge of the product has to correct the fault. Reconsider the overall design. It may be more reasonable to fix other components of the system to solve the problem which was found. 11 The Fagan method - named after an IBM employee who pioneered the technique All types of defects are noted - not just logic or function errors. Inspections can be carried out by collagues at all levels except the very top. Inspections are carried out using a predefined set of steps. Inspection meetings do not last for more than two hours. The inspection is lead by a moderator who has a specific trainging in the technique. 12 3
INSPECTIONS / AUDIT SESSIONS (continued) The other participants have defined roles. For example, one person will act as a recorder and note all defects found and another will act as a reader who takes the other participants through the document under inspection. Checklists are used to assist the fault-finding process. Material should be inspected at an optimal rate of about 100 lines per hour. Statistics are maintained so that the effectiveness of the inspection process can be monitored. A CHECKLIST FOR REVIEW EVALUATIONS Is material complete (and does it meet the standards)? Was material distributed on time? Are applicable standards referenced and available? Were attendees prepared to contributed? Did evaluation start on time? Were number of defects identified? Was review conducted per standard protocols? Were entrance criteria met? Were exit criteria met? 13 14 A CHECKLIST FOR REVIEW EVALUATIONS (cont d) Are all issues scheduled for resolution? Are interface issues coordinated? Is disposition of all defects complete? Were HCI, testing, maintenance, and tools considered? Were quality attributes reported? Has trace of defects been initiated? SOFTWARE QUALITY CIRCLES A Japanese practice. A quality circle is a group of four to ten volunteers working in the same area who meet for, say, an hour a week to identify, analyse and solve their work-related problems. One of the group is a group leader and one (maybe from outside) is a facilitator giving advice. 15 Training is required, and so is full support from the management. 16 4
PROBLEM SOLVING BY SOFTWARE QUALITY CIRCLES ISO 9000 Identify a list or problems and choose one. Clarify the problem. Identify and evaluate the causes. Identify and evaluate the solutions. Decide on the solution. Develop an implementation plan. Present the plan to the management. Implement the plan. Monitor the plan. Consider wider apllicability of solution. Restart from choosing another problem. Series of standards for general use in different fields of industry. ISO 9001 is for companies with product development and production. Separate certification organisations give certification for companies which fulfill the requirements of the standards. 17 18 INGREDIENTS OF ISO 9001 INGREDIENTS OF ISO 9001 (continued) Management responsibility Quality system Contract review Design control Document control Purchasing Purchaser supplied control Product identification adn traceability Process control Inspection and testing Inspection, measuring and test equipment Inspection and test status Control of nonconforming product Corrective action Handling, storage, packaging and delivery Quality records Internal quality audits Training Servicing Statistical techniques 19 20 5
AN OVERVIEW OF ISO 9001 REQUIREMENTS The management must define and document the policy concerning quality and must ensure that this policy is communicated to all levels of the organisation. All quality control procedures must be documented. All contracts to supply goods or services should contain mutually agreed requirements that the developer is capable of delivering. There should be procedures to approve design and other documentation. 21 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) Where components of the system to be supplied to the client are obtained from third parties there must be procedures to ensure, check, and maintain the quality of these components. Individual products should be identifiable as should their components. The process by which the final product is created should be planned and monitored. Inspection and testing should take place during the development phase, at its completion and before delivery. Tests and inspections should also be carried out on components obtained from third parties. 22 AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) AN OVERVIEW OF ISO 9001 REQUIREMENTS (continued) The equipment used in the production process itself should be properly controlled with respect to quality. The testing status of all components and systems should be clearly recorded all times. Care must be taken to ensure that items which are known to be defective are not inadvertently used. When a defect is detected, measures must be undertaken to remove the defective part and to ensure that the defect does not occur again. Satisfactory procedures must be in place to deal with correct handling, storage, packaging, and Sufficient records must be maintaned to demonstrate that the quality system is working satisfactorily. The software quality management system should be audited on a regular basis. Servicing and supprt activiies must be subject to the quality management system. The developer must establish appropriate statistical techniques to verify the acceptability of the final product. 24.3.2004 delivery of the product. Software Engineering 2004 23 24 6
CERTIFICATION (for standards like ISO 9001) GENERAL CONCLUSIONS The company familiarises itself with the standards. The company applies for the certification. The certification organization inspects the company (based on documents which it requires from the company). An evaluation meeting is held with representatives from both the company and the certification organisation. If necessary, some corrective actions are performed. A certification is given to an accepted company. Re-evaluations will be performed. 25 The development of a quality system requires changes and changes create resistance. Be realistic. Quality depends a lot on attitudes. Internal and management motivation is essential. Small steps can be the best way forward. 26 7