Lecture 9. Software Quality

Transcription

1 T Software Project Management - Spring 2004 Lecture 9. Software Quality Chapter 12 Maria Paasivaara

2 What is Software Quality? Quality (by IEEE glossary) The degree to which a system, component or process meets specified requirements The degree to which a system, component, or process meets customer or user needs or expectations All stakeholders have different viewpoints (e.g. customer, user, programmer, tester, project manager) Even a bug free product can be unacceptable to the user Validation and verification needed in all project steps

3 One Grouping of Software Qualities* Product operation qualities Correctness satisfies specifications and fulfills users needs Reliability performs intended function with required precision Efficiency computer resources required Integrity unauthorized access control Usability effort required to learn to use, operate, etc. Product revision qualities Maintainability effort required to locate and fix an error Testability effort required to test Flexibility effort required to modify Product transition qualities Portability effort required to transfer a program from one environment to another Reusability extend to which can be used in other applications Interoperability effort required to couple one system to another * By James A. McCall

4 Validation and Verification Validation are we building the right product? Implementation meets customer requirements, needs and expectations Verification are we building the product right? Program conforms to its specification

5 Quality Assurance (1) A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements. (2) A set of activities designed to evaluate the process by which products are developed or manufactured. Application of sound technical methods and tools Formal technical reviews and inspections Software testing Enforcement of standards Control of change Extensive measurement Record keeping and reporting of the process QA vs. Testing Different attitude QA is focused to build in quality and prevent defects to ever happen constructive Testing is focused to find defects and show the level of quality destructive

6 Product vs. Process Quality Product quality can be measured only when there is a product developed E.g. when choosing subcontractors assessing process quality is a possibility All software companies should also pay attention on improving the quality of their development process Capability process models (CMM) Certifications 5 levels: 1.Initial - haphazard 2.Repeatable basic project management procedures 3.Defined defined way 4.Managed measurement and control 5.Optimizing improvement according to measurements

7 Software Testing

8 Software Testing Testing is application of any possible techniques to find errors in programs or documents Testing does not create quality software or remove defects Testing can show the presence of defects but cannot show the absence of defects Testing can not be separated from rest of the software development Testing has to be involved from the beginning Testers can, and should, contribute in each phase of the software development life-cycle Testing is much more than the final acceptance testing phase

9 V-model of Testing Requirements Acceptance testing Functional specification System testing Build Architecture design Module design Unit testing Integration testing Test Coding Specifications ----> Planning ----> Testing

10 What to Test? Testing everything is impossible All bugs cannot be found All found bugs will not be fixed Prioritizing tests Time is always limited Use company s significant risks to focus testing effort what to test first what to test most How thoroughly to test each feature what not to test (at least for now) Most important tests first

11 Black-box and White-Box testing Black-box (functional, behavioral, data-driven) The software under test considered as a black-box and no knowledge of the internal structure or how the software actually works is utilized in testing Testing based on inputs and respective outputs The size of the black box can vary from one class or component to a whole system White-box (structural, logic-driven) Testing based on knowing the inner structure of the system and the program logic

12 Dynamic and Static Testing Dynamic testing executes code Testing in the traditional sense Static testing does not execute code Reviews, inspections, static analysis Some authors think that there is not such thing as static testing: Testing is dynamic and inspections, reviews and static analysis is static

13 Scripted and Non-scripted Testing In scripted testing test cases are pre-documented in detail, step-by-step descriptions Different levels of scripting possible Scripts can be manual or automated Non-scripted testing is usually manual testing without detailed test case descriptions Can be disciplined, planned, and well documented or ad-hoc testing

14 Re-testing vs. Regression testing Re-tests Regression tests New version of software Running a set of tests that have with fixed fault been run before Re-run the same test (i.e. Any level (component, re-test) integration, system, must be exactly acceptance) repeatable After software or environment same environment, changes versions (except for the For confidence that software which has everything still works been intentionally For emergency fixes (possibly changed!) a subset) same inputs and An asset (regression test preconditions suite/pack) If test now passes, fault Standard set of tests - needs has been fixed correctly designing & maintaining or has it? Well worth automating

15 Reviews and Inspections People Techniques

16 Reviews and Inspections Review - A formal meeting at which an artifact, or a set of artifacts is presented to the user, customer, or other interested parties for comments and approval. IEEE Standard Glossary Inspection - A visual examination of a software product to detect and identify software anomalies, including errors and deviations from standards and specifications. Inspections are peer examinations led by impartial facilitators who are trained in inspection techniques. Determination of remedial or investigative action for an anomaly is a mandatory element of a software inspection, although the solution should not be determined in the inspection meeting. -- IEEE Standard for Software Reviews

17 The Cost of Fixing Defects Relative cost to fix a fault Reviews Dynamic testing Requirements Design Coding Development testing Acceptance testing Operation (Boehm. Software Engineering Economics. 1981)

18 Reviews and Inspections The primary goal of reviews and inspection is to detect defects before the testing and hence strongly contribute to improve the overall quality of software Reviews should be applied early, because defects are easier and cheaper to fix when they are found early Types of reviews: Inspection, management review, technical review, walkthrough, audit In inspection, remember to Plan inspections Prepare for inspections Focus on finding defects, not solutions Criticize the product, do not blame the producer Collect metrics and use them to improve the process

19 What can be Reviewed or Inspected? Software requirements specification Software design description Source code Software test documentation Software user documentation Maintenance manual System build procedures Installation procedures Release notes Anything written down

20 Types of Reviews Review - process or meeting during which a software product is presented to project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval. Walkthroughs - May be held for the purpose of educating an audience regarding a software product. Management reviews - Ensures project progress and proper resource allocation Technical reviews - Evaluates a software product for its suitability for intended use, identifies discrepancies from specifications and standards Audits - Independent evaluation to check conformance of software products and process to prescribed standards, procedures, etc. Inspections Use walkthroughs for training, reviews for consensus, but use Inspections to improve the quality of the document and its process. (Gilb & Graham. 1993)

21 Inspection - Roles and Responsibilities The moderator responsible for leading an inspection session successfully and efficiently The author people whose work is being inspected The reviewers (or checkers) people who actually inspect the work by presenting different viewpoints (sponsor, design, test, etc.) The recorder (or scribe) records the proceedings of the session including anomalies, action items, decisions, and recommendations made by the inspection team. The reader leads the inspectors in logical and comprehensive fashion trough the document interpreting sections of work and highlighting important aspects Others: e.g. Inspection / Review coordinator

22 Summary Quality Assurance in a Software Project Define what is the required quality level for the product / customer project Design the quality assurance for the project, e.g. Reviews and inspections used Testing Determine the schedule Assign personnel Put effort on quality assurance / testing as early phase as possible, since fixing bugs is cheaper when they are found early Especially in distributed projects / with subcontractors use early design and code reviews and give coding and documentation standards

23 Approaches to Testing in Incremental Development - Scrum Example from Lecture 2 Revisited

24 Automated Regression Approach Development Heartbeat Testing Heartbeat Release Project Cycle 3 month cycle Increment Monthly cycle Daily cycle

25 Stabilizing Phase Approach Release Project Cycle Increment Stabilizing phase Stabilizing increment

26 Separated System Testing Approach Release Project Cycle Increment Testing Increment Separated, concurrent system testing increments

27 QA Time Horizons Heartbeat time horizon Activities and practices that are part of the design and implementation work of each development task Increment time horizon Activities and practices that are part of each increment What has to be done before an increment is completed Increment themes affect Release time horizon Activities and practices that are part of each release What has to be done before a release is completed Release type affects

28 Approach for Each Time Horizon Tests that are designed and documented Manual tests Automated tests Checklists of test ideas Tests that are executed and reported Automated tests Manual tests Exploratory tests Performance, usability, other qualities Other QA practices that are performed Reviews Coding conventions and other practices Static analysis Process conformance Who performs When How often Goals and purpose