Chap 1. Software Quality Management Part 1.1 Quality Assurance and Standards Part 1.2 Software Review and Inspection Part 1.3 Software Measurement and Metrics 1
Part 1.1 Quality Assurance and Standards 1. Introduction 2. Software Quality Attributes 3. Quality Management 4. Quality Assurance 5. Quality Planning 6. Quality Control 2
1. Introduction -Achieving a high level of product or service quality is the objective of most organizations. -It is no longer acceptable to deliver poor quality products, and fix deficiencies after they have been shipped to the customer. -However, software quality is a complex notion that cannot be defined in a simple way, unlike for other manufactured products such as cars, televisions, or computers. Quality product is expected to comply with customer requirements; Unfortunately such requirements are often incomplete for software. We do not know how to precisely specify and check certain quality requirements (e.g., usability, maintainability, reusability etc.) -In large organizations or projects, ensuring that the required level of product quality is achieved is the responsibility of the so-called quality managers. Quality management involves defining procedures and standards to be used during software development, and checking that these are followed by all engineers. 3
2. Software Quality Attributes Notions Quality: the totality of features and characteristics of a product, process or service that bear on its ability to satisfy stated or implied needs. -The expected quality features and characteristics of a software product are commonly referred to as Quality Attributes. Quality attributes are used early in the development process to identify user quality requirements. Each system has specific and unique quality needs, which are a function of the purpose of the application. Quality attributes for a given system may be conflicting; for instance, achieving security may be at the expense of performance or interoperability. Software quality attributes are assessed by using, when possible, appropriate software measurements or metrics. 4
-There are two main categories of quality attributes: functional (what a product should do) and non-functional (what a product should be like) Functional quality attributes: apply to pieces of software, from the smallest components to entire systems. Example 2.1: which quality attribute (s) can be derived from the following requirements? when the pressure sensor reading climbs through a pressure level of 3.2 bar, the relief valve control line shall be set to open all relevant data shall be secured to disc before any transaction is cleared Non-functional quality attributes: apply to any product of the development process : specifications, code, manuals, or final system. Example 2.2: which quality attribute (s) can be derived from the following requirements? the system shall be capable of operation on a computer with 64Mb of memory the system shall give uninterrupted service despite any power outages of up to 2s duration 5
Typical Software Quality Attributes Lifecycle Product Transition Product Operations Product Revision Quality Attributes Portability Reusability Interoperability Correctness Reliability Efficiency Usability Safety Security Robustness Maintainability Flexibility Testability -Correctness: ability of software products to perform their exact tasks, as defined by their specification -Robustness: ability of a program to react appropriately to abnormal conditions. -Reliability: ability of a program to achieve precisely its intended mission. -Efficiency: ability of a software to place as few demands as possible on hardware resources, such as processor time, memory space occupied, or network bandwidth, to achieve a given task. 6
Typical Quality Attributes (ctd.) -Maintainability: effort required modifying, updating, evolving, or repairing a program during its operation. -Flexibility: effort required to evolve or modify an operational program. -Testability: effort required to test a program. -Portability: effort required to transfer a program from one computing environment or platform to another. -Reusability: ability and effort required to reuse a program or part of a program in other applications. -Interoperability: effort required interconnecting or relating two different applications, running possibly in different computing environment. -Usability: effort required to learn, operate and use a program 7
Example 2.3 -Identify the quality attributes for the ATM System. The bank s motivation for developing the system is to attract new customers by offering low banking fees, and a variety of services. The bank will also be able to reduce its wage costs by processing an increased number of banking transactions automatically through the system instead of manually through cashiers. It is essential for them to lower the development cost and to minimize the product support cost. Security is critical; the system must be fully integrated into existing enterprise security infrastructure. More specifically the ATM system will reuse an existing secured database. The time for 90% of the users to learn (through supplied step-by-step instructions) how to use the first time the system must not be more than 5 minutes. When a user issues a request, the system should respond with a verification of the request within 1.0 second in 90% of the cases. The time for the verification must never exceed 10.0 s, unless the network connection is broken (in which case the user should be notified). The ATM System must have no more than 1 hour per month of down time. 8
3. Quality Management -Software quality management can be structured into three main activities: 1. Quality assurance: establishment of a framework of organizational procedures and standards which lead to high-quality software. 2. Quality planning: selection of appropriate procedures and standards from this framework and the adaptation of these for a specific project. 3. Quality control: definition and enactment of processes which ensure that the project quality procedures and standards are followed by the developers. -Quality management provides an independent check on the software development process. Quality management should be separated from project management so that quality is not compromised by management responsibilities for project budget and schedule. The deliverables from the software process are input to the quality management process and are checked to ensure that they are consistent with organizational standards and procedures. 9
Example of Quality Standard: ISO 9000 -ISO 9000 is a set of international standards that can be used in the development of a quality management system in all industries. ISO 9001 is the most general of these standards and applies to organizations concerned with the quality process in products design, development, and maintenance ISO 9000-3 is a supporting document, which interprets ISO 9000 for software development. -ISO 9001 is a generic model of a quality process that describes which standards and procedures that should exist within an organization. As it is not industry-specific, this description is high-level. Within any specific organization, a set of appropriate quality processes should be defined and documented in an organizational quality manual. The quality procedures in an organization are documented in a quality manual defining the quality process. Certification bodies exist that check regularly conformance with ISO 9001 of the quality process as expressed in the quality plan. 10
Areas covered by ISO 9001 To obtain ISO registration, a formal audit of twenty elements is involved and the outcome has to be positive. Guidelines for the application of the twenty elements to the development, supply, and maintenance of software are specified in ISO 9000-3 The twenty elements are as follows: 1. Management responsibility 2. Quality system 3. Contract review 4. Design control 5. Document control 6. Purchasing 7. Purchaser-supplied product 8. Product identification and traceability 9. Process control 10. Inspection and testing 11. Inspection, measuring, and test equipment 12. Inspection and test status 13. Control of nonconfoming product 14. Corrective action 15. Handling, storage, packaging, and delivery 16. Quality records 17. Internal quality audits 18. Training 19. Servicing 20. Statistical techniques 11
Example: ISO 9000 Requirements for Document control (section 5) Must be adequate for purpose Owner must be identified Properly approved before issued Distribution must be controlled Version identified Pages numbered Total pages indicated Promptly destroyed when obsolete 12
Quality management based on ISO 9000 ISO 9000 Quality model Instantiated as Organization quality manual documents Organization Quality process Is used to develop Instantiated as Project 1 Quality plan Project 2 Quality plan Project 3 Quality plan Project Quality management supports 13
4. Quality Assurance -Quality assurance (QA) activities define a framework for achieving quality. -QA process involves defining or selecting adequate process or product standards: Product standards apply to the software product being developed; include document standards (e.g., requirements document structure), documentation standards, coding standards etc. Process standards define the processes that should be followed during software development; include definitions of specification, design and validation processes and a description of the documents to be generated in the course of these processes. Product standards Design review form Requirements document structure Procedure header format Java programming style Project plan format Change request form Process standards Design review conduct Submission of documents to configuration Management (CM) Version release process Project plan approval process Change control process Test recording process 14
Example: ISO 9000 requirements for Product and Process Metrics (reported under Statistical techniques Section 20). Product metrics:measurements should be used for the following purposes: To collect data and report metric values on a regular basis To identify the current level of performance on each metric To take remedial action if metric levels grow worse or exceed established target level At a minimum, some metrics should be used to represent Reported fields failures Defects from customer viewpoint Process metrics: Ask if in-process quality objectives are being met Address how well development process is being carried out with checkpoints. Address how effective the development process is reducing the probability that faults are introduced or go undetected. 15
5. Quality Planning -Quality planning should begin at an early stage in the software process. The result of the quality planning process is the project quality plan. Definition: a quality plan is a document setting out the specific quality practices, resources and activities relevant to particular product, process, service, contract or project. -A quality plan should set out the desired product qualities, and define how these are to be assessed. -The quality planner should select those organizational standards that are appropriate to a particular process and development process. -The quality plan should be as short as possible. Due to the wide range of quality attributes, in general it is not possible to handle all quality attributes. Hence, the plan should identify among all the potential quality attributes, the most critical ones, and focus on how to achieve them. The quality plan should also define the quality assessment process. 16
-An outline structure for a quality plan may include the following: 1. Product introduction: description of the product, its intended market and quality expectations. 2. Product plans: critical release dates and responsibilities for the product along with plans for distribution and product servicing. 3. Process descriptions: development and service processes which should be used for product development and management. 4. Quality goals: quality goals and plans for the product, including an identification and justification of critical product quality attributes. 5. Risk and risk management: key risks which might affect product quality and the actions to address these risks. 17
6. Quality Control -Involve overseeing the software development process to ensure that QA procedures and standards are being followed. -There are two complementary approaches to quality control: 1. Quality reviews: the software, its documentation and the processes used to produce that software are reviewed by a group of people against the project standards. 2. Automated software assessment: the software and the produced documents are processed by some program and compared to the project standards. This may involve a quantitative measurement of some software attributes. Quality Review Techniques Requirements review Design and Analysis review Code review Documentation review Automated Quality Assessment Software testing Static Analysis (e.g., compiler, syntax checker, type checker, spellchecker etc.) Dynamic analysis (e.g., Run-time check for memory leaks and pointer problems etc. ) Formal verification (e.g., proof-checking, model checking) 18
Example: Quality Attributes Quality Control Activities when the pressure sensor reading climbs through a pressure level of 3.2 bar, the relief valve control line shall be set to open Analyze and validate Vs. Observation How much can we realistically check? All relevant data shall be secured to disc before any transaction is cleared Analyze the software connecting sensor reading to the valve control and verify this behavior will be its effect. Set up a situation where the pressure climbs through 3.2 bar and observe the relief valve control line setting. Analyze the design, in particular checking the locations of data updates at the point where every transaction is cleared. 19
Summary Software quality -Complex notion -Captured through quality attributes Quality management -Quality assurance: quality model quality manual -Quality planning: quality manual quality plan -Quality control: review, automated checking Text - Coursepack: Section 1: Design Qualities, by D. Budgen 20