Software Quality Assurance Software Engineering 9 1 McCall s Triangle of Quality Maintainability Flexibility Testability PRODUCT REVISION Portability Reusability Interoperability PRODUCT TRANSITION PRODUCT OPERATION Correctness Usability Reliability Efficiency Integrity 2 SOE9 1
Internal and external attributes Maintainability Number of procedure parameters Cyclomatic complexity Reliability Portability Usability Program size in lines of code Number of error messages Length of user manual 3 Why SQA Activities Pay Off? cost to find and fix a defect 60-100x log scale 1x 1.5-6x Definition Development After release 4 SOE9 2
Quality Concepts General objective: reduce the variation between samples... but how does this apply to software? Quality control: a series of inspections, reviews, tests Quality assurance: analysis, auditing and reporting activities Cost of quality Appraisal costs Failure costs External failure costs 5 Software Quality Assurance Process Definition & Standards Formal Technical Reviews Analysis & Reporting Measurement Test Planning & Review 6 SOE9 3
Reviews & Inspections... there is particular reason why your friend and colleague cant also be your sternest critic. Jerry Weinberg 7 Quality management and software development Software development process D1 D2 D3 D4 D5 Quality management process Standards and procedures Quality plan Quality review reports 8 SOE9 4
What Are Reviews? A meeting conducted by technical people for technical people A technical assessment of a work product created during the software engineering process A software quality assurance mechanism A training ground 9 What Reviews Are Not! They are t: a project budget summary a scheduling assessment an overall progress report a mechanism for reprisal or political intrigue!! 10 SOE9 5
The Players review leader standards bearer (SQA) producer maintenance oracle recorder user rep reviewer 11 1. 2. be prepared evaluate product before the review review the product, t the producer Conducting the Review 4. 5. 6. 7. 8. 3. stick to the review agenda keep your tone mild, ask questions instead of making accusations raise issues, don't resolve them avoid discussions of style stick to technical correctness schedule reviews as project tasks record and report all review results 12 SOE9 6
Reviews: An Effectiveness Scale Most Effective Inspection (FTR) Walk through (FTR) Formality Formal presentation Informal presentation Peer group review Casual conversation 13 Review Options Matrix trained leader agenda established reviewers prepare in advance producer presents product reader presents product recorder takes tes checklists used to find errors errors categorized as found issues list created team must sign-off on result IPR * maybe maybe maybe maybe WT IN RRR maybe * IPR informal peer review WT Walkthrough IN Inspection RRR round robin review 14 SOE9 7
Metrics Derived from Reviews inspection time per page of documentation inspection time per KLOC or FP inspection effort per KLOC or FP errors uncovered per reviewer hour errors uncovered per preparation hour errors uncovered per SE task (e.g., design) number of mir errors (e.g., typos) number of major errors (e.g., nconformance to req.) number of errors found during preparation 15 Statistical SQA Product & Process collect information on all defects find the causes of the defects move to provide fixes for the process measurement an understanding of how to improve quality... 16 SOE9 8
Process-based quality Define process Develop product Assess product quality Improve process No Quality OK Yes Standardize process 17 Organization of SQA Management - an example Systematic Software Engineering Dansk Standard AQAP Auditor Finance Business Process Improvement Danish Air Force R&D R&D and and Consulting Defense Systems Industrial Systems Danish Air Force Project Group Command & Control IRIS IRIS Interoperability EC EC // EDI EDI Key Key Accounts Air: OPUS CCIS Navy: RDN CCIS Navy: RDN EWare F-16 Simulator IRIS - Message Format. IMT - Data Modeling X-Post EDItoolbox EDIsec EC/EDI projects DSB Superfos Realkreditrådet Post Danmark Tele Danmark Internet Landbr. Rådgiv.center Den Danske Bank 18 SOE9 9
Guideline for SQA reporting SQA should t report to the project manager. SQA should report somewhere within the local laboratory or plant organization. There should typically be more than one management position between SQA and the senior location manager. SQA should always have a "dotted-line" relationship to a senior corporate quality executive. Whenever possible, SQA should report to someone who has a vested interest in software quality, like the staff head responsible for field service. 19 SQA Considerations SQA organizations are rarely staffed with sufficiently experienced or kwledgeable people. The SQA management team is often t capable of negotiating with development. Senior management often backs development over SQA on a large percentage of issues. Many SQA organizations operate without suitably documented and approved development standards and procedures. Software development groups rarely produce verifiable quality plans. Getting good people into SQA is one of the most difficult problems software managers face. One effective solution is to require that all new development mangers be promoted from SQA. 20 SOE9 10
Software Quality Assurance CMM Goals Goal 1 Goal 2 Goal 3 Goal 4 Software quality assurance activities are planned. Adherence of software products and activities to the applicable standards, procedures, and requirements is verified objectively. Affected groups and individuals are informed of software quality assurance activities and results. Noncompliance issues that cant be resolved within the software project are addressed by senior management. 21 Key points Software quality management is concerned with ensuring that software meets its required standards Quality assurance procedures should be documented in an organisational quality manual Software standards are an encapsulation of best practice Reviews are the most widely used approach for assessing software quality 22 SOE9 11