Managing Successful Offshore QA Delivery

1 Managing Successful Offshore QA Delivery White Paper Authored for: 13th International Conference, QAI Author 1: Prasuna Potteti Date: 13-Sep-2011 Email: ppotteti@deloitte.com Deloitte Consulting India Private Limited Divyasree Technopolis, 124, Yemlur P.O Off Old Airport Road, Bengaluru 560037 Author 2: Anup Jaiswal Date: 13-Sep-2011 Email: anjaiswal@deloitte.com Deloitte Consulting India Private Limited

2 Abstract This paper synthesizes the key message and findings from 15 20 projects focusing on offshore delivery for testing. This paper will in detail discuss the key advantages, challenges and suggested solutions of testing projects that are carried out completely from offshore. The world of work is faster and farther reaching than ever before. Managers are asked to deliver better products and services in shorter timeframes with less cost. To meet this demand, many Clients are considering offshore delivery model as an opportunity. For increased technology, globalization and international business, offshore delivery model presents a viable solution in terms of cost and resources. Offshore outsourcing has established itself as a business trend for many years now. Offshore outsourcing has gained popularity mainly for two reasons, namely the flexibility it offers and the cost advantage. Each sector and company experiences flexibility in different ways, but offshore outsourcing has overall been a right step towards meeting these diverse needs. Addressed Audience: Test Practitioners and Engineers, Software and Test Managers, QA Managers and Development Managers as well as other professionals interested in building and delivering better software.

3 Contents Table of Contents 1. Benefits to Client from Outsourced, delivery model:... 4 2. Management Control... 5 3. Estimating actual efforts (revalidating the planned timelines)... 7 4. Communication Model... 7 5. Onshore coordination... 8 6. Validating offshore QA work quality & progress... 9 7. Communication Mechanism... 9 8. Other critical challenges in QA testing... 10 9. Data privacy and confidentiality... 11 10. What if client have different processed to follow & DC have different?... 11 11. Case Study... 11 Situation... 11 Problem... 11 Decision to Use Offshore Software Testing... 12 Solution... 12 12. Conclusion... 12 References... 12 Author Bio... 13 Appendix... 13

4 1. Benefits to Client from Outsourced, delivery model: Outsourcing any work (Be it QA or development) is the process of delegating a company s business process to third parties or external agencies, leveraging benefits ranging from low cost labor, improved quality to product and service innovation. When outsourcing transgresses national boundaries and is managed by companies located in other countries, outsourcing takes the form of offshoring. Given below, as reflected by companies, are the key factors to outsource: 1. Lower operational and labor costs are among the primary reasons why companies choose to outsource. Outsourcing, many a time is undertaken to save costs and provide a buffer capital fund to companies that could be leveraged in a manner that best profits the company. When properly executed it has a defining impact on a company s revenue recognition and can deliver significant savings. 2. Better focus on core business focus: Companies also choose to outsource or offshore so that they may continue focusing on their core business processes while delegating mundane time consuming processes to external agencies. Freeing up internal resources that could be put in to effective use for other purposes is also one of the primary benefits realized when companies outsource or offshore. 3. Access to greater pool of resources: Many times stranded with internal resource crunches, at times there is a need to immediately add resources with specific skills however there may not be enough in-house resources available & hiring new resource may not be give fast turnaround. Many world class enterprises outsource to gain access to resources not available internally. Certain countries have a very large supply of IT professionals, some with specific skills in niche areas, which may appeal to companies that need to get particular types of work done while saving money Outsourcing and offshoring also enable companies to tap in to and leverage a global knowledge base, having access to world class capabilities. 4. Outsourcing also enables companies to realize the benefits of re-engineering. Some companies also outsource to help them expand and gain access to new market areas, by taking the point of production or service delivery closer to their end users. 5. 24/7 availability: Because of different time zones, companies can have people working on a project or providing IT support around the clock.

5 Having listed out all the Good Stuff on what are the benefits that comes as part of offshore delivery model it appears to be the most logical way to go considering the fast paced timelines we run into whether it s about bringing a new product to market or enabling a new feature to your business to keep pace with competition. However as with any model there are some challenging situations to be handled to make it a success- 2. Management Control Whether you sign a contract to perform the function of an entire department or single task, how do you make sure that the management control functions smoothly (proper handshakes, decision making, changes in priority if any?) True, you will have a contract, but the managerial control will belong to Client Company. Same way how do we make sure that client management does not run into situation that they feel that they had lost the management control to track the overall status & taking corrective actions. Threat to Security and Confidentiality The life-blood of any business is the information that keeps it running. If you have payroll, medical records or any other confidential information that will be shared to you, it is fair the expect that there is a risk that the confidentiality may be compromised. Quality: How much is good enough? How do we ensure that quality is up to the mark in deliverables. If there is a drop in quality how is that identified in time for correction. At client end they may or may not have necessary resources & timeline to QA each & every details of delivery so in such cases what kind of quality process are needed to bring in the visibility to client so that adequate feedback is provided on time for correction.

6 In this paper we present some of the basics of must have to manage a successful QA delivery in an offshore model - Specified in Section5. Offshore Outsourcing is one of the best options when the client expects to have a series of IT projects, critical to achieve its business objectives, to be done at an affordable cost. The project is well planned in advance to its commencement and the service provider has a clear understanding as to what the client's needs are. Also the client should be very clear about what they expect and it need to be documented in clear definite terms. The level of fluctuations in the client s requirements is expected to be minimum & there need to be provision to accommodate the same. Documenting what a successful deliverable means with respective owners & timelines. State of Work should detail each and every detail about the tasks that need to be accomplished. SOW should specify who is responsible for which piece of work, tasks to be accomplished and data privacy details. It should also detail about how the work is split into meaningful chunks and which vendor is responsible for what delivery. Scope of the project: it should clearly specify what is the project scope and what is shared with which vendor and the responsibility of the vendor. High level Timelines: SOW should specify the high level timelines of the project. How the work is split into meaning chunks. Who will be owner of which chunk and what the timelines for each chunks are. What are the timelines for each chunk delivery? Ownership, pricing: SOW should detail about who is the owner for which piece of work and the pricing model followed. Data Privacy: SOW should detail about the Data privacy and also should specify the terms and conditions around data privacy. Standards: What are the standards that need to be measured? The ways to measure, track and report work progress. Also it should specify how corrective actions will be implemented.

7 3. Estimating actual efforts (revalidating the planned timelines) Estimates will be created based on the piece of work allocated to vendor. While estimating vendor should consider all the information provided in SOW and come up with estimations. Vendor should list out clearly what are the items for which estimates are provided and he should take high level agreements & assumptions into consideration. Once we get the actual work details available they need to be revalidated in terms of actual efforts & any deviations from estimated efforts, any deviations here need to be discussed with proper channels. There is a strong need to match the high-level agreements and the statements of work with the actual work estimates. Time & Material is always better than Fixed Price for offshore project as there might be hidden risk of delivering the project in the agreed upon price. For example, the contract does not match the work environment when there is a fixed price contract and set statement of work that does not allow for increases in the amount of work being accomplished. The client may need more work to be done, but the contract is set at a fixed price, with the vendor possibly doing the extra work in order to maintain the working relationship with the client. Without this work explicitly being stated in the contract or in a change order, it goes unrecorded. Thus, neither the client nor the vendor management team fully understands the level of effort needed by the vendor to accomplish a specific set of tasks. With fixed price contracting, the risk of not being able to accurately predict the amount of testing work at software release time is transferred to the vendor (QA team). 4. Communication Model In Offshore Delivery Model, the entire project is accomplished at the service provider s offshore location, which may be located in a different country. The client team(s)will be dealing directly with the offshore team. The service provider will have no face-to-face interaction with the client team(s) during the entire process once the initial interaction with the client regarding their requirements and expectations is over. Of course as the project progresses, both the parties will be communicating regularly through other means of communication so as to clear off any doubts that may arise. Since there is no presence of the service provider at the client s site, in the offshore model there is always a possibility of communication gap between the client and service provider, which may lead to some misunderstanding. Here the level of risk involved is also high as if any disaster strikes the offshore center or the offshore center faces any problem then the progress of the project would be interrupted resulting in delay of project completion. Client and Vendor have to come up with the communication mechanism and make sure that it is implemented successfully without failure throughout the project.

8 5. Onshore coordination Client & vendor need to come up with the onsite & offshore coordination model. Different designation roles need to be identified as POC at both places and they will be the primary point of contacts for escalations & coordination. They also will be responsible for the delivery of the desired work. At times for clients who are doing first time outsourcing they may not have this type of role, in such cases Client need to develop vendor management and interaction roles, there is a tendency to share this role with existing resource which may lead to varying priority which would need to be addressed (i.e. resource may have existing priorities so offshore coordination responsibilities need to be clearly prioritized). Similarly there need to be a vendor coordinator at the client organization he/she must coordinate the work with vendors and stay in touch with them on daily basis to track the progress of the work. For example, a vendor test lead should report to client test lead. There might be situations where client employee could have two or three onsite vendor representatives with multiple products and multiple releases being worked on, as long as the client employee has the bandwidth to check all the vendor outputs and deliverables. OFF SHORE ON SHORE Delivery Manager Customer Project Manager Onsite Coordinator Onsite Team Analysts Designers Developers Testers

9 6. Validating offshore QA work quality & progress Vendor representatives will be responsible for delivering quality product. Most of the clients will go with the previously identified vendors as they have worked with them in the past and have good experience. The work was done is a matter of trust between the vendor and client. If the work is not high quality they might not be given more work or be asked to work on the next software release. There have to be mechanism devised for client test lead to review the progress in QA testing (what vendor test lead says about work status). Client and Vendor team will use same test management tools so that client can review the documents created by the vendor team. As a vendor we need to come up with the review process that can be followed across the project for the deliverables, the same need to be discussed with client test lead & any concerns arising out of it to be addressed. The reporting mechanism in terms of progress & status of work need to be as transparent as possible, having common reporting tools greatly helps here instead to building custom reports*. In other works if client test lead cannot directly validate the report from test management tool then you have a problem to be addressed. Note *: highly derived reports in excel/ppt from original data does not work very well if client test lead were to validate the data. Keys areas to bring in visibility for Client to be able to monitor & access the offshore QA quality 1. Requirements coverage 2. Test Case Coverage across various cycles/waves/regression 3. Defect Density 7. Communication Mechanism In Offshore Delivery Model, the entire project is accomplished at the service provider s offshore development center, which is located in a different country. The client will be dealing directly with the offshore team. The service provider will have no face-to-face interaction with the client during the entire process once the initial interaction with the client regarding their requirements and expectations is over. Of course as the project progresses, both the parties will be communicating regularly through other means of communication so as to clear off any doubts that may arise. Since there is no presence of the service provider at the client s site, in the offshore model there is always a possibility of communication gap between the client and service provider, which may lead to some misunderstanding. Here the level of risk involved is also high as if any disaster strikes the offshore development center or the offshore center faces any problem then the progress of the project would be interrupted resulting in delay of project completion. Client and Vendor have to come up with the communication mechanism and make sure that it is implemented successfully without failure throughout the project.

10 8. Other critical challenges in QA testing The research identified 13 challenges to providers operating in offshore markets. These challenges may also be seen to constitute the major risks that apply in this form of activity. When managed effectively, the challenges become the basis of good practice examples for the system. These challenges relate to the key phases of planning and implementation as described in below Figure. Effective practice in offshore activity also involves strategies to manage providers exit from offshore markets. These strategies need to manage the strategic interests of the provider following an initial evaluation of the offshore market. Major Key Challenges are as below: 1. Access to Sensitive Customer Data 2. Knowledge Transfer & SME Availability 3. Communication 4. Challenges of Educating Outsourced team on Product 5. Increased Stress & Work load

11 9. Data privacy and confidentiality This is one of the biggest challenges for QA testing irrespective of whether done onshore or outsourced. With outsourced QA delivery it becomes more sensitive that there are no violations, fortunately there exists various means to achieve this: 1. Encrypting sensitive data 2. Masking customer PII data 3. Restricted access to master DB, config files that may have sensitive data related to customer, admin user, passwords. 4. Providing a confined & restricted area for offshore team i.e. physically isolated area for offshore team to work on (may incur cost to vendor). 10. What if client have different processed to follow & DC have different? When the process followed by the organization is different form the processes followed by the client we should explain the process followed in organization and explain advantages and disadvantages of the process followed and try to convince client. If client is not in agreement with the process proposed then we need to understand the process followed by the client and implement the same throughout the project delivery. 11. Case Study Situation An enterprise software company specializing in distributed order fulfillment was recognized as one of the premier development organizations in US. Although they had released their first product with zero customer reported defects, they quickly realized that their software testing process which consisted of a combination of inhouse testing and expensive consultants would not scale as they began the build out of their second release. Problem Organization was preparing to release a new product version to market. Although the product had been successful in production environments, the management team was worried about the changing dynamics of their quality processes. As implementation of their second product release began, it became obvious that their existing software testing processes would not be effective going forward. We had tremendous success in the quality of our first release. However, as the size and complexity of our application grew, there was just too much ground to cover. There was no way we could cost effectively staff and train the people we needed to maintain our rigorous quality standards. Not only was there a tremendous amount of code to test, but there was also a wide variety of skills needed to effectively analyze the quality of our application and diagnose problems.

12 Decision to Use Offshore Software Testing Organization decided to turn to offshore software testing in order to achieve the scalability, flexibility, and diversity of skills that were required. After interviewing and accepting bids from over a dozen offshore software testing firms, we selected one of the offshore company to continue the work. Engineered Assurance, coupled with some great overseas relationships, gave us the real-time visibility and predictability that no other firm could deliver. Solution At the end of each workday, a build is produced with a list of new features implemented and defects resolved. Offshore project manager, aided by Engineered Assurance technology, then analyzes the work and formulates a test strategy covering the next 12 hours. A team of specialists is assembled whose skills exactly match the night s challenges. By morning, when Clients development team returns to work, their issue-tracking system is fully populated with the results of the preceding day s code analysis. Client immediately knows of any problems that were introduced into the product within a day of their creation. At the same time, Clients management team has a full online report of the preceding day s activities as well as analysis and recommendations going forward, all because of offshore software testing. 12. Conclusion Offshore outsourcing for testing projects will help the clients in achieving better quality product at lower costs in the given timelines. But the challenges of offshore delivery model should be addressed in appropriate manner to achieve the results and maintain the success rate. References Google Wikipedia

13 Author Bio Prasuna Potteti System Integration Deloitte Consulting Prasuna Potteti is working as a Senior Consultant with Deloitte, Bangalore. She has around 10 years of experience in manual, Automation testing, TDM & DW testing. She worked on different domains like Banking, Manufacturing, and Healthcare. She has played different roles like team member, module test lead and release manger. She has experience in co-coordinating Offshore-onsite model and worked with people across India, US, and Germany. Prasuna has been actively involved in competency building in testing COE (Center of Excellence), Deloitte. Anup Jaiswal System Integration Deloitte Consulting Anup Jaiswal is a specialist master with core competency in software testing, test automation, and testability. Familiar with many test tools, he helps teams develop test automation strategies and architectures. He has helped develop automated tests for functional, as well as performance tests for package applications as well as custom development. Anup had been actively involved in competency building in testing COE (Center of Excellence), Deloitte. He has published white papers in both internal, as well as external presentations. Appendix DW Data Warehouse TDM- Test Data Management QA Quality Assurance COE Center of Excellence