LogicaCMG & Testing: TestFrame & RRBT Mike Peters Senior Test Consultant/Delivery Manager mike.peters@logicacmg.com LogicaCMG 2006. All rights reserved
About LogicaCMG > 40,000 employees worldwide 41 countries > 40 years experience in IT services Offers a wide range of IT-solutions Specialised in industry-specific solutions Aquisition of WM-data in 2006 Risk & Requirement Based Testing & TestFrame 2
LogicaCMG & Software Testing 2500 Test Professionals world-wide Test consultancy Managed testing Test outsourcing Test Research Centre in the Netherlands TestFrame method & tooling Test management: Risk & Requirement Based Testing Business Acceptance Management & - Testing ERP & Testing Embedded Software Testing STAPPIN (Standard infrastructure testing, e.g. Vista, Exchange) Risk & Requirement Based Testing & TestFrame 3
Testing credentials LogicaCMG s knowledge & experience in testing is based on more than 1,000 test projects over the world Risk & Requirement Based Testing & TestFrame 4
Agenda Everyday testing challenges Risk & Requirement Based Testing (RRBT) TestFrame Discussion Risk & Requirement Based Testing & TestFrame 5
Testing of systems The later you find faults, the more it costs to solve them Costs Definition Design Build Use Time Source: Boehm Risk & Requirement Based Testing & TestFrame 6
Testing is often under pressure Planning and specification Development Test Risk & Requirement Based Testing & TestFrame 7
Testing in parallel to the development activities Development activities Testing activities Risk & Requirement Based Testing & TestFrame 8
Agenda Everyday testing challenges Risk & Requirement Based Testing (RRBT) TestFrame Discussion Risk & Requirement Based Testing & TestFrame 9
Position various solutions Insight Control Focus Reporting Predictable Commitment Business objectives Risk Control Quality to Market Time to Market Business Alignment Budget Control Reputation Management Test Management Test analysis and execution Structure/Fitting/Tooling Standard Repeatable Reliable Maintainable Risk & Requirement Based Testing & TestFrame 10
Risk & Requirement Based Testing Risk & Requirement Based Testing & TestFrame 11
Test Management Model TEST ORGANIZATION PROGRESS MANAGEMENT PLANNING ISSUE MANAGEMENT ESTIMATION REPORTING AND IMPLEMENTATION ADVICE RISK ANALYSIS AND TEST STRATEGY EVALUATION AND TRANSFER Risk & Requirement Based Testing & TestFrame 12
How to reduce risk... And still test effectively? Risk & Requirement Based Testing & TestFrame 13
Product risks versus requirements A risk, no requirement: Add requirement (find faults earlier) Remove risk (don t test more than necessary) A requirement, no risk: Adjust risk list (better coverage of testing) Remove requirement (don t develop more than necessary, no frills ) Matching risks and requirements Requirements Product risks Matching requirements and risks Risk & Requirement Based Testing & TestFrame 14
RRBT & Quality Attributes (ISO 9126) Functionality Reliability Efficiency Suitability Maturity Time behaviour Accuracy 40% Fault 15% tolerance Resource 15% utilisation Interoperability Recoverability Efficiency compliance Security Reliability Compliance 40% 15% Functionality compliance Usability Understandability Learnability 20% Operability Attractiveness Usability compliance Maintainability Analysability Changeability 10% Stability Testability Maintainability compliance Portability Adaptability Installability 0% Co-existence Replaceability Portability compliance Risk & Requirement Based Testing & TestFrame 15
Testing under pressure OUCH!! Must test: Should test:... Could test: Won t test: Risk & Requirement Based Testing & TestFrame 16
Risk & Requirement Based Testing Added value: Product risks: use the same language as stakeholders Unambiguous method for prioritizing product risks Detect gaps in requirements Good base for managing progress of test project Risk & Requirement Based Testing & TestFrame 17
Agenda Everyday testing challenges Risk & Requirement Based Testing (RRBT) TestFrame Discussion Risk & Requirement Based Testing & TestFrame 18
TestFrame Risk & Requirement Based Testing & TestFrame 19
TestFrame LogicaCMG s method for structured testing Testing plays a role in quality Verifying test process and products Working in a structured way Testing takes effort Re-use defined products Optimal use of tools Testing is not a goal, it is used to reach a goal Use consciously and goal-oriented Risk analysis Testing has to fit with development method and organization Flexible and fitting Risk & Requirement Based Testing & TestFrame 20
TestFrame model Adjust testing to organization (not vice versa) Quality-to to-market & Time-to to-market F I T T I N G Structured process Structured products S T R U C T U R I N G Reusable Test Products T O O L I N G Use of tools for test analysis, test execution and test management Risk & Requirement Based Testing & TestFrame 21
TestFrame phases Preparation Analysis Navigation Execution Risk & Requirement Based Testing & TestFrame 22
TestFrame products Assessment Report Preparation Plan of Approach Test plan Scoping Risk analysis Technical test Navigation Clusters Test Conditions Navigation structure Engine Analysis Test cases Test report Navigation scripts Findings Evaluation report Execution Handover set Risk & Requirement Based Testing & TestFrame 23
Test specification Breakdown & automation Test clusters Automated test scripts Test conditions Test tool & Engine Test reports Test cases Actionwords System Under Test Risk & Requirement Based Testing & TestFrame 24 24
Èxample setup using Excel Risk & Requirement Based Testing & TestFrame 25
Testing with action words (1) Identify the smallest functional test step Assign an action word to every step Add parameters Use the action words when defining test cases and test scenario s When necessary: automate the action words Risk & Requirement Based Testing & TestFrame 26
Testing with action words (2) A B C D check check balance balance Taylor Taylor 0 transfer transfer Houston Houston Taylor Taylor 210 210 check check balance balance Taylor Taylor 210 210 transfer transfer Smith Smith Taylor Taylor 150 150 check check balance balance Taylor Taylor 360 360.... Test preparation and - specification Automated test execution Test cluster Test execution check balance transfer Navigation script Risk & Requirement Based Testing & TestFrame 27
TestFrame: facts & advantages Easy accessible and well maintainable test products Early start possible and therefore time saving at the end of the project Effort scarce experts is minimized Better steering and control test execution process Test automation possible without adjustments to test documentation Test automation also possible when application is not 100% correct Easy reporting on test execution status and test results Risk & Requirement Based Testing & TestFrame 28
Conclusion Risk & Requirement Based Testing: Make the right choices TestFrame: Do it structured for a list of all the ways technology has failed to improve the quality of life, please press 3. (Alice Kahn) Risk & Requirement Based Testing & TestFrame 29
Agenda Everyday testing challenges Risk & Requirement Based Testing (RRBT) TestFrame Discussion Risk & Requirement Based Testing & TestFrame 30
ne global team Risk & Requirement Based Testing & TestFrame 31