Amdocs Testing Factory
Amdocs Testing Factory 2 Contents Introduction...3 Telco & CSP Challenges in Testing...4 The Amdocs Testing Factory Model...5 Set-up and Transition...6 Governance...8 Engagement model...10 About Amdocs Testing...11
Amdocs Testing Factory 3 Introduction The success of a new system or application is directly influenced by the diligence with which it was tested. Communication Service Providers are challenged to maintain a delicate balance between their external goals of speedily delivering upon their subscriber s demands and internal goals of reducing costs while improving software quality. This white paper will present the Amdocs Testing Factory, a new concept in Software Testing that delivers Communications Service Providers with a ready framework to help them begin testing quickly as well as ensuring high quality and efficiency while mitigating costs. This paper offers an overview of the different stages that comprise this concept, what each includes, roles and responsibilities of parties involved as well as how the concept is governed.
Amdocs Testing Factory 4 Telco and Communications Service Provider Challenges in Testing The constantly changing world of Telecom forces Communications Service Providers (CSPs) to quickly adapt to change. The market and specifically the CSP s subscribers demand quick, new business solutions and smart applications that improve their experience and fulfill the needs of their lifestyle. Regulatory bodies also demand changes and adjustments in applications at short notice. All these factors add immense pressure upon CSPs to quickly adapt their systems and applications which, today, require a significant number of changes to fulfill demands. The challenges outlined call for a more efficient way to develop and test systems and application solutions while ensuring higher quality. CSPs are hard pressed to reduce time to market and ensure their systems and applications run like clockwork, without causing severe business disruptions. To effectively support the service provider s business, Amdocs has developed the Testing Factory concept that offers integrated processes, methodology, governance and tools. This concept aims to reduce costs and efficiently bring the system or application to a state of readiness in the production environment, with lowest possible number of defects. TESTING EXPERTISE EFFICIENCY TELECOM FOCUSED TESTING INNOVATION CONSTANTLY EVOLVING TRENDS Finding: Maintaining balance Focus of testing vendors Reactive approach to trends Skilled & experienced testers between: Testing adequately spread over too many industries Keeping pace with the industry Testers that understand telecom business processes Meeting deadlines Lowering costs Lack of vendors focusing on telecom tools & techniques Automation Mobile testing Agile approach Test design Test case reuse Figure 1: CSP challenges in Testing
Amdocs Testing Factory 5 The Amdocs Testing Factory Model Amdocs perceives the Testing Factory as the industrialization of software testing. It is essentially a software product line that configures testing methodology, processes, tools, patterns, frameworks and guidance, designed to test specific kinds of systems and applications. The Amdocs Testing Factory operational model provides a production line with well-defined processes and activities to meet the needs of the Communications Service Provider. The model essentially delivers all the necessities required to begin testing quickly and effectively. It is similar to a factory or plant where the floor is populated with all the equipment required to begin performing its intended function effectively. Following is an overview of the key building blocks for successful Testing Factory engagement: Set-up and Transition Phase Governance Model Engaging Testing Factory Services Figure 2: Amdocs Testing Factory Model
Amdocs Testing Factory 6 Set-up and Transition Phase To ensure high quality integration into the Communication Service Provider s system and ensure consistency with project goals, Amdocs leverages a set-up and transition plan that draws upon our experience in this field to effectively gather knowledge from the CSP s team or thirdparty vendor. Key phases and main activities of the plan can be seen in the figure below. This generic plan is fine-tuned to suit the specific Communication Service Provider s needs during the process of transition. Program management Governance, Contract, SLA and Change Management plan and design prepare transfer handover Identify involved parties Detailed transition plan Organize gathered information Adjust / update work procedures LOI Build management and PMO teams High-level transition plan Initiate preparation activities Establish forums and committees contract LOI Interview and recruit candidates on- and off-shore Prepare onsite facilities Prepare offshore facilities Initiate knowledge transfer Establish work procedures Set SLA procedures Set up testing tools Establish an improved testware baseline Handover responsibility transition completion (tbd) Stafffing plan and initiate recruitment Create KT materials Knowledge transfer (OJT, training, documentation) Prepare infrastructure Start joint execution and shadowing Reverse shadowing Figure 3: Set-up and Transition Phase
Amdocs Testing Factory 7 The Set-up and Transition phase comprises four subphases: Plan and design Amdocs assigns a transition team during this sub-phase. A transition progress report of expectations is also defined and includes status, risks, and AI tracking. Timelines and milestones are reviewed and agreed upon at this stage. Prepare The primary focus of this phase is the creation of the transition plans that comprehensively cover all aspects of the transition. Handover The purpose of this stage is to smoothly handover all testing findings and recommendations to the permanent testing team. During this stage all activities are planned in detail according to their phase. This before takes between 1 and 3 months depending on the number of systems or applications included in the scope. An example timeline of the Set-up and Transition phase can be seen below. Set-up / Transfer This stage includes actual execution of all transition plans. Another key event that takes place here is the definition of all activities leading to a successful handover to the permanent testing team. month 1 month 2 month 3 week 1 week 2 week 3 week 4 week 1 week 2 week 3 week 4 week 1 week 2 week 3 week 4 Plan and Design Prepare Set-up / Transfer Transition complete Handover Figure 4: Set-up and TransitioN Phase Time Line
Amdocs Testing Factory 8 Testing Governance The implementation of Testing Governance needs significant management of people, process and technology with transparent communication and information. To give the Communications Service Provider full control of the engagement, Amdocs Testing Factory includes a set of well defined roles & responsibilities, a designated testing organization structure and strict KPIs and SLAs that make it possible to control, assess quality and manage progress of System Testing. Governance Roles and Responsibilities As part of our testing methodology and approach, Amdocs recognizes that the success of the Acceptance Test phases is mutually dependent on both Amdocs and the Communications Service Provider. The table below presents high-level roles and responsibilities between both parties. TASKS AMDOCS SERVICE PROVIDER Transition period Lead & perform Provide personnel to execute transition and knowledge transfer of systems involved in this program Test strategy & plan Lead & perform Review & approve Test design & detailed design Test data Test execution Quality management IT support Lead the design and detailed design Design test cases and test scenarios Lead & perform Test execution management Perform test execution Coordinate and define testing process and activities Request test environment based on the test scope Provide business requirements Allocation of business experts to participate in design and design reviews. Participate Participate Provide inputs from external systems Allocate super users and system operators to participate in the test execution Execute on satellite system onsite Review & approve Release management (planning and provide scheduled releases and drops, emergency releases) Test environment set up and maintenance. Provide production data including reference data and any other data that is defined in the test execution plan Summary & LL Lead & perform Participate SLA easurements and reporting Participate Lead & perform Remote access Participate Provide remote access to test environment for execution of test cases Table 1: Example of Governance Roles and Responsibilities The implementation of Testing Governance needs significant management of people, Processes and technology with transparent communication and information.
Amdocs Testing Factory 9 Amdocs Testing Factory Organization Structure We have defined a sample organization chart to cover the Testing Factory model as seen below. The team is divided into streams, such as Functional, Non-functional, Defect Management, Data/Environments and Automation. Test Manager TMO Functional testing Team Performance Team Defect Management Test Data and Environments Automation Team Figure 5: Amdocs Testing Factory organization part of Governance structure Key Performance Indicators (KPIs) and Service Level Agreements (SLAs) An SLA is a negotiated agreement between Amdocs and the Communications Service Provider at the beginning of the project that defines the types of service included and their levels of availability. It is also part of our implementation approach to define KPIs to measure activity performance. Tools At Amdocs, we invest significant efforts in innovation to ensure our tools and methodologies deliver real value through superior software quality, fast time to market and lower cost of ownership. Our developers and testers have created the best tools to comprehensively address all types of tests and can provide these tools according to the Communication Service Provider s request. Amdocs uses an HP Platform including Quality Center, Load Runner and other tools. As part of our testing methodology and approach, Amdocs recognizes that the success of the Acceptance Test phases is mutually dependent on both Amdocs and the Communications Service Provider.
Amdocs Testing Factory 10 Engaging Testing Factory Services Amdocs performs an initial estimation of the service required in accordance with the specifications received from the business area. The estimation process could be based either on the number of business scenarios or test cases, and is aligned with the Communications Service Provider in the first stage of the project. The most common types of requests that generate the requirement for testing services are as follows: Project (New Application): Development and implementation of a new system or a major systemic change. Evolutionary Maintenance: Development of new features or new modules in existing systems with or without changing the database. Corrective Maintenance: Addressing specific functionality in an existing systems either with or without changing the database. A generic version of the process or flow is presented in the figure 6 below. As Amdocs gains a greater understanding of the Communications Service Provider in the Set-up and Transition phase, this flow is adjusted and customized to effectively address the provider s specific needs. The diagram below shows a typical macro process framework that covers the entire Testing Factory service. In green are the many areas requesting services from the factory, such as Business, IT Development and Infrastructure. The Amdocs Testing Factory is shown in gray. Inside the factory, one can see all processes and phases supported by strong Amdocs Testing Methodology. BUSINESS TEAM DEVELOPMENT TEAM 1 DEVELOPMENT TEAM 2 GOVERNANCE QUALITY Test & Quality Management REQUEST ESTIMATION APPROVALS DEFECTS FIXED DEFECTS DAILY REPORTS KPI REPORTS Amdocs Test Management (TMO) Test Request Management Defect Management Reports Plan Design Preparation Execution Close/Lesson Learned Project Management (Working planning, monitoring, reporting, KPI s and risk management) Requirement Management (Capture and maintain scope changes during all project phases) Environment Management (Requirement definition, validation, version management) Tools service request Infra Team Figure 6: Example of Amdocs Testing Factory Macro Process Framework
Amdocs Testing Factory 11 About Amdocs Testing Amdocs Testing provides comprehensive testing services including Acceptance Test management and planning, test design and execution to confirm full system compliancy with our customers business and operational requirements. After many year of experience working on complex test projects, Amdocs Testing developed the Testing Factory service offering, combining an innovative methodology, people, processes and governance for effective and efficient test design and execution that delivers value at every stage. Amdocs Testing is a dedicated unit of Amdocs Consulting Division and offers the advantages of a highly experienced team of industry professionals leveraging a structured, fieldproven testing methodology supported by procedures, automated tools, testing scripts and libraries. Over 2,000 SI experts located across 30+ locations Up to 50% faster execution time Manage & operate 25+ concurrent Telecom focused SI projects Seamless integration between offshore, onsite, customer and partners Amdocs Testing Up to 40% reduction in testing TCO Up to 50% less defects in production Experience with world s most complex testing projects - over 25,000MM 3 patent pending applications Figure 7: Amdocs Testing in a nutshell
Amdocs Testing Factory 12 QUALITY TIME TO VALUE COST Unparalleled Quality Up to 50% less defects in production when Amdocs Testing conducts Acceptance Tests Up to 50% faster execution time for combined systems Up to 60% faster for Amdocs systems Reduce testing TCO up to 40% for combined systems Up to 60% lower for Amdocs systems (Shama & Ben Cnaan, 2013; Kenett & Ben Cnaan, 2013) (Capers Jones, 2011; Ross Report, 2012) (Capers Jones, 2011; Ross Report, 2012) Figure 8 : Amdocs Testing delivers value in terms of quality, time to value and total cost of ownership
Amdocs Testing Factory 13 What the Analysts Say Nelson Hall RANK VENDOR Est d sts revenues 2012 (sm) Est d sts market share 2012 (%) 1 Amdocs 300 13 2 Wipro 240 10 3 Mahindra IT 200 9 = Accenture 200 * Global Software Testing Services 9 Market in Telecom Sector 2012-2016 5 Capgemini 170 7 = CGI (PF) 140 6 = TCS 140 6 8 IBM 120 5 9 HP 70 3 10 AtoS 30 1 Total top 10 1,640 70 Other ~660 30 Total 2,300 100 Figure 9: Software Testing and Assessment Forecast 2013, Nelson Hall TechNavio Amdocs is the leading firm in the Global Software Testing market... it excels in providing software testing services in the Telecom sector. HP 9-11% IBM 10-12% ACCENTURE 9-11% OTHERS 52-54% AMDOCS 15-17% Figure 10: Global Software Testing Services Market in Telecom Sector 2012 2016, TechNavio
Amdocs Testing Factory 14 Amdocs Testing Offering Amdocs Testing offers a comprehensive testing portfolio that enables you to speedily launch new services of superior quality. Our offering includes: Core Testing Services Functional Non-Functional System Integration Tests Acceptance Tests End to End Tests Performance Tests Operational Tests Testing Management Services Test Environment Management Test Assessment Test Management Office Test Data Management Specialized Testing Services Bill to Bill Revenue Assurance Tests Data Warehouse Business Intelligence Tests Mobile Application Tests Test Automation Services Functional Web & e-commerce SOA & Middleware Figure 11: Amdocs Testing Product Offering
Amdocs Testing Factory 15 What Our Customers Say Amdocs Team without your help this would not be possible. Please continue with the efforts, which without doubt add value and bring benefits to our company..this was definitely the BEST upgrade we ve done in all my years and I really, really appreciate the care and attention that you took to ensure this significant work went in without any disruption to our business. Telefonica Chile QA Lead Test Factory Senior VP, Tier 1 North American Service Provider We re very happy with the level and high quality of service Amdocs Testing has delivered. They ve worked very hard to ensure deployment occurs on time while maintaining quality. No matter the deadline, we re always sure, Amdocs Testing will deliver. Bevan Williams, Executive: ITS Solution Delivery Center, Telecom South Africa
Amdocs Testing Factory 16 Contact Us For more information about Amdocs Testing Factory, please contact the Pre-sales Manager responsible for your region. Amdocs Testing Contacts: CALA Luiz Tenorio luiz.tenorio@amdocs.com +55 11 99236 9249 EMEA Hitesh Joshi hiteshj@amdocs.com +44 07920847707 APAC Manoj Jasrotia manojj@amdocs.com +91 9881391905 North America Ori Cohen oric@amdocs.com +1 647 449 7589 GLOBAL David Chernitzky davidche@amdocs.com +1 416 702 8867
About Amdocs For more than 30 years, Amdocs has ensured service providers success and embraced their biggest challenges. To win in the connected world, service providers rely on Amdocs to simplify the customer experience, harness the data explosion, stay ahead with new services and improve operational efficiency. The global company uniquely combines a market-leading BSS, OSS and network control product portfolio with valuedriven professional services and managed services operations. With revenue of $3.2 billion in fiscal 2012, Amdocs and its 20,000 employees serve customers in more than 60 countries. Amdocs: Embrace Challenge, Experience Success. For more information, visit Amdocs at www.amdocs.com. For the most up-to-date contact information for all Amdocs offices worldwide, please visit our website at www.amdocs.com/corporate.asp Copyright 2013 Amdocs. All Rights Reserved. Reproduction or distribution other than for intended purposes is prohibited, without the prior written consent of Amdocs. Amdocs owns or has rights to use trademarks or trade names in conjunction with the sale of our products and services, including, without limitation, each of the following: Amdocs, Bridgewater Systems, ChangingWorlds, Clarify, Cramer, CES, Collabrent, DST Innovis, Ensemble, Enabler, Intelecable, Intentional Customer Experience, JacobsRimell, jnetx, MX Telecom, OpenMarket, Qpass, SigValue, Streamezzo, Stibo Graphic Software, Xacct, Aging in Place, Embrace Challenge, Experience Success, and Quality Consumption.