CSE3308/DMS/2004/25 Monash University - School of Computer Science and Software Engineering Software Engineering: Analysis and Design - CSE3308 Software Quality CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.1 Lecture Outline What is Software Quality? How can we measure Software Quality? Software Quality Activities Software Quality Assurance Software Quality Planning Software Quality Control Quality Standards AS3563 and ISO9000 Beyond Quality Standards Capability Maturity Model Personal Software Process CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.2 Page 1
What is Software Quality? The problem of quality management is not what people don t know about it. The problem is what they think they do know. Quality has much in common with sex. Everybody is for it, (under certain conditions, of course). Everyone feels that they understand it, (even though they wouldn t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations (after all, we do get along somehow.) And, of course, most people feel that problems in these areas are caused by other people (if only they would take the time to do it right.) Crosby 1979 CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.3 Definitions of Quality Quality is fitness for use - Juran those product features which meet the needs of the customers and thereby provide product satisfaction freedom from deficiencies Quality is conformance to requirements and zero defects - Crosby Quality is the totality of characteristics that bear upon its ability to satisfy stated or implied needs - ISO9000 stated needs - specified as requirements by a customer implied needs - identified and defined by the company providing the product CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.4 Page 2
Defining Software Quality Software Quality is conformance to: explicitly stated functional and performance requirements explicitly documented development standards implicit characteristics that are expected of all professionally developed software Software Quality is ambiguous, subjective and multidimensional CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.5 The need for more detailed measures of quality Previous definitions are high-level and difficult to measure Requirements are difficult to specify Requirements are usually incomplete Experiment: Compare software warranties with hardware warranties CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.6 Page 3
The Hardware Warranty Company X warrants that the products it manufactures and sells are free from defects in materials and workmanship. If any product fails to operate properly, the company will repair the defective product and restore it to normal operation without charge CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.7 The Software Warranty Company X s sole obligation under this warranty will be to provide support services described in our current software support policy. The company does not warrant that the licensed software is free from defects or that the support services will correct any defects that might exist. CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.8 Page 4
McCall s Software Quality Factors Maintainability Flexibility Testability Portability Reusability Interoperability Product revision Product transition Product operations Correctness Reliability Efficiency Integrity Usability CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.9 The ilities -Software Quality Attributes - Product Operation Correctness Does it do what I want? Tool - Use Cases Reliability Does it do it accurately all of the time? Tool - Formal methods Efficiency Will it run on my hardware as well as it can? Tool Good algorithmic design, appropriate language (even Assembler in some cases) Integrity Is it secure? Tool - Java Useability Is it designed for the user? Tool - User-centred design CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.10 Page 5
The ilities -Software Quality Attributes - Product Revision Maintainability Can I fix it? Tool - Encapsulation Flexibility Can I change it? Tool - Encapsulation Testability Can I test it? Tool - Interfaces CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.11 The ilities -Software Quality Attributes - Product Transition Portability Will I be able to use it on another machine? Tool - Java, ANSI C, (and other emerging technologies) Reusability Will I be able to reuse some of the software? Tool - OO class libraries (e.g. GUI components), function libraries (e.g. Numerical Recipes in C) Interoperability Will I be able to interface it with another system? Tool - CORBA, DCOM CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.12 Page 6
Metrics for Quality Many quality factors are difficult or impossible to measure directly Need to develop indicative measures of the quality factors Unfortunately many of these measures are still subjective Two methods are: McCall s checklist approach (see download on website) Hewlett-Packard s FURPS Functionality» feature set, capabilities of program, generality of delivered functions, security of overall system Useability» human factors, overall aesthetics, consistency and documentation Reliability» frequency and severity of failure, accuracy of output, MTTF, ability to recover from failure, predictability Performance» processing speed, response time, resource consumption, throughput, efficiency Supportability» extensibility, adaptability, serviceability testability, compatibility, configurability, ease of installation, ease of problem localization CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.13 Software Quality Activities Quality Assurance The production of organisational procedures and standards which lead to high-quality software Quality Planning Choosing appropriate procedures and standards and tailoring them for a specific software project Quality Control Ensuring that procedures and standards are followed by the software development team CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.14 Page 7
Software Quality Assurance How an organisation defines the methods by which it will achieve quality Organisation must define or select standards Quality of the product is influenced by the quality of the process Standards need to be embedded in the software development process All this will be documented in a Quality Manual The relationship between software process and product quality is complex and poorly understood CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.15 Assuring Quality Need to ask four questions WHAT attributes of the product manifest quality in your context? HOW is quality to be measured? WHEN do we evaluate the product and the process? WHO is responsible for carrying out the process? Quality relies on people, but we can greatly help or hinder people from producing quality CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.16 Page 8
Quality Principles Try to prevent defects from being introduced Ensure that defects that get in are detected and corrected as early as possible Establish and eliminate the causes as well as the symptoms of defects Measure quality characteristics Independently audit work for compliance with standards and procedures CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.17 The Quality Plan Is specific to a project Produced early in the life of a project Should set out desired product qualities Should define how the quality is to be assessed Should indicate which standards are to be applied Indicate how compliance to the standards is monitored and assured May define new standards CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.18 Page 9
Contents of Software Quality Plan from ISO9000 Management responsibility Quality system Contract review Design control Quality control Purchasing Customer supplied info Configuration management Process control Inspection and testing Inspection and testing equipment Control of non-conforming product Corrective action Handling, storage, packing and delivery Quality records Internal quality audits Training Software maintenance Statistical techniques Control of the development environment CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.19 Quality Control Ensuring that all of the above gets done! Tyranny is not effective in ensuring that the work is done People must see a clear benefit to them in performing the above activities Embedding the procedures in day-to-day work is essential Reviews are one of the major tools for quality control CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.20 Page 10
Quality Standards Major standards used in Australia ISO9000 international set of quality standards ISO9000-3:1997 is a specific subset aimed at software development. It may be superceded by the latest IS9000 standard HB 90.9-2000 Provides guidelines for the software industry on quality management systems (QMS) complying with ISO9001:2000 (Quality Management Systems requirements) Specifies requirements for a QMS for a software development organization that» needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements, and» aims to enhance customer satisfaction through the effective application of the system, including processes for continual improvement of the system and the assurance of conformity to customer and applicable regulatory requirements. Governments and other large organizations often require quality certification, such as ISO9000 earlier standards superceded by this include AS 3563 and AS 3905.8 CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.21 Quality Standards (2) Organisations must be certified to be granted ISO9000 status Certification is granted by independent auditing groups (e.g. Standards Australia) Certification is not cheap (approximately $500,000 for a medium-sized company) Certification might not bring any benefits to the company in terms of quality, but could in terms of marketing Certification is an on-going process; audits are carried out annually CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.22 Page 11
Quality Standards (3) To get benefits from quality standards must use sound management techniques must aim to improve the process employees must participate actively Quality standards are of greatest value to those organisations which don t already have formal development processes They don t replace individual skills and abilities They can only be as good as the work practices which they document CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.23 Beyond Quality Standards Quality standards are only a necessary first step towards a software development environment which produces quality software We need to define what sub-processes are necessary in the overall process The Capability Maturity Model (CMM) documents this for organisations The Personal Software Process (PSP) does this for individual developers CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.24 Page 12
The Capability Maturity Model Developed by the Software Engineering Institute (SEI), based on work by Watts Humphrey 5 Levels Level 1: Initial Level 2: Repeatable Level 3: Defined Level 4: Managed Level 5: Optimising At each level Key Process Areas (KPAs) provide goals and example practices CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.25 Process Maturity Levels Optimising Managed Process Control Defined Process measurement Repeatable Process Definition Initial Basic Management Control CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.26 Page 13
Level 1: Initial Process is ad hoc and sometimes chaotic Few processes are defined Success depends upon individual effort Key Process Areas None Too many software development organisations are in this category CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.27 Level 2: Repeatable Basic project management processes defined Process discipline in place to repeat successes Change is inherently dangerous at this level Key Process Areas Configuration management Quality assurance Sub-contract management Project tracking Project planning Requirements management CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.28 Page 14
Level 3: Defined Process is documented, standardised and integrated into a standard software process All projects use this standard software process Measurement is still qualitative Key Process Areas Peer reviews Intergroup coordination Software product engineering Integrated software management Training program Software process definition and focus CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.29 Level 4: Managed Detailed measures of software process and product quality are collected Process and product are quantitatively controlled Data gathering should be automated Key Process Areas Quality management Quantitative process management CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.30 Page 15
Level 5: Optimising Continuous process improvement is enabled There is quantitative feedback from the process and from piloting innovative ideas and technologies Move from a purely product focus to focusing on process as well Key Process Areas Process change management Technology change management Defect prevention CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.31 The Personal Software Process Mirrors the CMM for an individual designed to help you become a better software engineer requires research, motivation and study to work Framework for why you make errors and how you find them determining the quality of your reviews determining the types of errors you make Developed by Watts Humphrey CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.32 Page 16
PSP0 PSP0 - Baseline Process your current process with some basic measurements and a reporting format time recording defect recording defect type standard PSP0.1 a coding standard size measurement process improvement proposal CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.33 PSP1 PSP1.0 - Personal Planning Process adds planning steps to PSP0:» test report» size and resource estimation PSP1.1 to establish a performance rate for future planning PSP1.0 enhanced by adding:» task planning» schedule planning CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.34 Page 17
PSP2 PSP2.0 Personal Quality Management Process adds review techniques to PSP1 to help you find defects early: PSP2.1» design reviews» code reviews» defect rates are typically 1 per 5-12 lines of code, do you know what yours is? establishes design completeness criteria:» design templates CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.35 PSP3 PSP3 - Cyclic Personal Process For large programs - 10,000 LOC Sub-divide into PSP2-sized modules Enhance the base module in iterative cycles In each iteration do a complete PSP2 including design, code, compile, test Effectively scale up from base module to large program, if each increment is of high quality CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.36 Page 18
Overview of CMM and PSP CMM sets out the principal practices for managing the processes in large-scale software development PSP sets out the principal practices for defining, measuring and analysing an individual s own processes CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.37 References Pressman, Roger S., Software Engineering: A Practitioner s Approach, McGraw-Hill, 2000 (Ch. 19). Capability Maturity Model for Software (SW- CMM ), The Software Engineering Institute (SEI), Carnegie Mellon University. http://www.sei.cmu.edu/cmm/cmm.html The Personal Software Process SM (PSP SM ), The Software Engineering Institute (SEI), Carnegie Mellon University. http://www.sei.cmu.edu/tsp/psp.html CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A.38 Page 19