End-to-End Testing. Helping our Customers improve quality and reduce costs

Transcription

1 End-to-End Testing Helping our Customers improve quality and reduce costs

2 Introduction Concept Reply focuses on End-To-End (E2E) Testing, which means Testing, Validation and Quality Assurance (QA), specifically addressing several markets: Telco & Media Banking Automotive Extended Enterprise The areas of intervention cover the full QA chain: from test methodology and process to design and execution via testing management, automation and tools. The course of action can vary from professional services to turn-key projects, and even an innovative Test-as-a-Service (TaaS) approach. 2

3 A pragmatic approach to QA Cost per defect Field Shipment Prevention Phase A formal approach to QA does not allow for certain key factors: Cost Of Poor Quality Identifying defects early Phase Containment Effectiveness metrics Phase Screening Effectiveness metrics Maturity Level of the organisation Cost Of Quality CMM Levels Quality Non Conformance Costs Conformance Costs Cost But, based on our experience, the pursuit of Quality must be pragmatic: each organisation needs to find its own trade-off between Quality, Cost & Time-to-market. Cost Time 0% Quality 100% Metric (Perfect)

4 Concept Reply E2E Test unit QA Services Process & Tools Q.A. Business Process Consultancy Test Management & Automation Test Design Test Execution Defect Analysis Traceability Requirement Definition of Acceptance Criteria Definition of Test Architecture Manual Automated First Level Troubleshooting Defect Triage Test coverage optimisation

5 Concept E2E Test: market intervention areas Test Process Test Tools Telco & Media Automotive Banking Enterprise Telco & Multimedia Devices Test Design/Execution/Reporting Embedded System & M2M Solutions Applications & Services

6 Positioning of Concept Reply E2E Testing Professional Services Process Improvement Test Engineering Business Processes Technology Tool Integration & Development Test Factory Turn-key Projects/Services

7 Process Improvement: Methodological Approach Observe Identify client s target validation objectives, not only in the test area but also in terms of business impacts (e.g.: customer satisfaction/loyalty, liability in the event of critical incidents) Understand existing test process Analyse gathered data to identify most effective way for Reply to approach intervention (full process coverage, single/multi area improvement) Share and discuss proposed solution with the customer Improve Define an optimised test process, according to customer s needs and priorities Implement the process Deploy the process, and verify the application Improve Observe Check Monitor improvement Perform lessons learned and corrective actions to drive continual process and quality improvements Check

8 Tool integration & development Wide range of solutions: Integration of commercial tools Customisation of open source tools Development of ad-hoc solutions: E.g.: Integration of different standalone customer tools in a unique dashboard In different areas: Full test process management Requirement management Test management Fault management Configuration management Test automation

9 Test Factory Model Client Service, Turn-Key projects On-Site/Off-Site GOVERNANCE Project Management Logistic Management Vendor Management REQUIREMENTS REQUIREMENT ANALYSIS OBJECT UNDER TEST Vendor ADAPTER TEST DEVELOPMENT TEST EXECUTION REPORT MANAGEMENT REPORTS TEST RESULT & FAULT MNGMT. Commercial, proprietary, open source customisation Requirement & Test Management TEST TOOLS Test Reporting, issue tracking Test Automation

10 Test as a Service The Test-as-a-Service implements an innovative approach to Testing (TaaS) : Full Customer control over process and plan with clear and optimised interfaces (Service Manager, Project Managers) A pay-per-use model within a general framework agreement A flexible and scalable level of service with guaranteed Quality of Service, milestones and Key Performance Indicators. Workload peak management 3rd party management Client Service Governance Reply Test House Service Manager Integrated Planning & Logistic Project Managers Technical Experts \ Test Team 3rd party Project Assignment & Planning Request Project Endorsement & Planning Response Project Management & Operations Project Closure Confirmation Project Manager Test Team

11 Test Engineering: Test Design & Execution Test Specification Support to definition of requirements Risk and business impact analysis Definition of Acceptance criteria Test Design Design of test architecture Definition of test environment Traceability of requirements Development of test cases and automation scripts Test Execution Validation of test environment Scheduling of tests Test Execution (Manual or Automated) Defect/Incident management (Triage) First level troubleshooting Functional Testing Interaction Testing Regression Testing Boundary Testing Compatibility Testing Esploratory Testing Testing from Escaped Defect Analysis Stress Testing Performance Testing Negative Testing Perceived Quality Testing Comparative Testing

12 References

13 Thanks