Oliver Au Software Engineering Specialist Group HK Computer Society

Transcription

1 outsourced software development projects from Customer s Perspective Oliver Au Software Engineering Specialist Group HK Computer Society

2 Outline Risks & rewards of IT outsourcing Current state of IT outsourcing Payment models Actions required for successful projects Select good vendors Decide what to outsource Negotiate good contracts Communicate well Manage deliverables (frequently and inspected) Monitor the processes (with metrics) Deal with miscellaneous problems The ultimate test Is your vendor replaceable? 2

3 Risks of Outsourcing Unrealistic expectations dishonest vendors Security, privacy and intellectual property (IP) Geographical and cultural distances The us-and-them mentality Security, privacy and intellectual property Inadequate / deteriorating vendor capabilities Financial stability Moving capable staff to another customer s project Customers do not Monitor project progress frequently and effectively Review deliverables quickly and rigorously 3 Keep their internal staff happy

4 Rewards of Outsourcing Cost Reduction Shortened Time to market Internal staff stay focus on core competencies Offloading boring work Great learning experience for internal staff to run outsourced projects and to work with staffs of the vendor Flexible arrangement in human resources Shortened ramp up time to start a project No layoffs and fewer transfers on project termination Access to experts Is offshore outsourcing ethical? 4

5 Cost Reduction Challenges in Outsourcing A staff at the vendor may not be as productive as a staff at the customer s company Extra work to protect intellectual property Communication overhead Misrepresented turnover ratio Effort to review vendor s work For offshore outsourcing Time zone difference can be worked around In some culture, people do not say no. This makes communication more difficult. 5

6 2014 Global Outsourcing & Insourcing Survey by Deloitte Outsourcing (not limited to IT) will continue to grow at 12 to 26% a year IT outsourcing has 60% penetration Vendors are generally meeting service levels and cost targets. Thanks to vendor performance, 84% will not move work from offshore back to onshore. Outsourcing buyers and vendors have a tendency to rush through the transaction. They should spend more time on request for proposal (RFP), vendor selection and transition. When third party advisors are used, they add values to outsourcing projects. Offshore outsourcing expands from India, China, Poland and the Philippines to Romania, Mexico, Brazil and Malaysia. Other South American and Asian countries are joining. Cloud computing, big data, mobility and business process as a service are changing the game. But BYOD and gamification are not significant factors in outsourcing decisions. 6

7 Outsourced Functions Voice network Application maintenance Data network Application hosting Application development IT service desk Data center End user device suport IT architectural design

8 Perceived Weaknesses in Vendor Management Issue and dispute management Documents management Multi-service provider integration Transition and transformation management Vendor risk management Contract management and compliance Change and request management Service performance management

9 Issues with Vendors Reactive vs proactive Poor service quality Lack of innovation Unqualified resources Lack of responsiveness Ineffective issue resolution Fail to meet service levels Poor relationship Communication barriers High service provider attrition Too costly Incompatiable culture

10 Actions To Resolve Issues Escalation to vendor leadership Enhance governance processes Additional training Restructure the deal Restructure service levels Bring in competition Add vendor management capability Reduce project scope Early termination

11 Payment Model 1 T&M Under the time and materials model, the customer pays the vendor based on the negotiated rate and number of hours actually worked. It s simple for project initiation. Customers not knowing how much they will pay at the end. Vendors have no incentives to be more efficient. (In fact, the opposite is true.) Time tracking is tedious. It s not easy and require work for the customer to check if the vendor is honest with the number of hours worked. It supports the waterfall and agile development model equally well. Special consideration - Productivity suffers when vendor staff is replaced. Should the customer get a discounted rate during the ramp-up period? Put it on the contract. 11

12 Payment Model 2 Fixed Price In the fixed-price model, the vendor estimates the work and quotes a price for the entire project. It demands more work upfront to define the project scope and requirements. Customers know exactly how much they will pay at the end. Vendors have incentives to be more efficient. Vendors absorb the costs of underestimation. They may add a safety margin to cope. Requirements change causes work and trouble. Time tracking work is unnecessary as far as the customer is concerned. It supports the waterfall better than the agile model. It is the most commonly used model. 12

13 Fixed Price Per Function Point Pros Clients understand how the price is determined Pricing schedule from different suppliers can be compared more meanigfully Suppliers have incentives to improve their efficiency Cons FP counting is not completely objective A change in functionality may not affect the FP count but nevertheless affects the effort required Independent FP counter can be hired 13

14 Good Vendors (1 of 2) Vendor s stability Financial strength of the vendor Safe political environment Effective legal system to resolve disputes Protection of intellectual property Vendor s track record for quality, on-time delivery, and service Customer references Performance metrics (product or service), e.g. fault history, on-time delivery and response time for customer calls Vendor s Maturity Certification like ISO 9001 and CMMI 14 Length of time in business and providing similar services

15 Good Vendors (2 of 2) Human resources low staff turnover to ensure continuity and high productivity throughout the project Technical and methodological expertise to implement the solution effectively and productively Domain knowledge to understand client s requirements without an excessive amount of effort or help from the customer Communication are available for the clients, friendly and easy to work with speak the same language as the customer openly share the true project status, their strengths and weaknesses, Professional ethics ethical staff to protect the privacy and confidentiality of clients Process-oriented The vendor is easier to deal with if they follow agreed processes 15

16 Outsourcing Code Development? Some countries have the bad reputation of producing incomprehensible code. Code delivered by vendors must be rigorously reviewed. Use tools like Checkstyle to ensure adherence to coding standards. Use tools like Clover to measure unit test coverage. It is risky to outsource high-value IP work. Keep it home. 16

17 Outsourcing Quality Assurance? New graduates generally prefer to work on development than testing. We can keep them happy to outsource QA and do code development in house. But QA requires good domain knowledge that vendors may not have. In addition, communication skills are important for testers to write useful bug reports. Good leadership is required for QA. With the mix of pros and cons, there is no clear-cut answer whether QA should be outsourced. Use screen recording tool to record how to perform a test or reproduce a bug. The customer can recreate a reported bug or a new QA person can pick up 17 the job quickly and easily.

18 Outsourcing Both? If you only outsource code development, the QA work at home helps to ensure the vendor is delivering. If you only outsource QA, the vendor helps to ensure your code development is done properly. If both code development and QA are outsourced, the above cross-evaluation link between two parties is gone. It s more risky especially for outsourcing newbies. If you have to outsource both, and you are also inexperienced, hire a third-party intermediary. Even if it adds to your total cost, you would learn how to do it properly. consider to outsource code development and QA to different vendors. 18

19 Contracts Conducive to Successful Projects Contracts written in more details Contracts that are in shorter-term (less than 5 years) Contracts that are of higher-dollar valued may appear contradicting with the trend to use multiple vendors Align the interests of the client and the vendor Pay for legal and advisory services to negotiate a contract in your favor. In many cases, the vendor may be more experienced than you in contract negotiation. Adaptability may be built into the contract to allow changes in requirements and environments, e.g. costs to the client be based on the market price per function point In some cases of requirements change, it may be better to renegotiate the project scope instead of the project price You can choose to get out of the contract unharmed 19

20 Fundamental Laws of Outsourcing Described by Nick Krym in his book Outsource It!. 1. Nothing is as easy as it looks - Murphy s 1 st law 2. The uninspected deteriorates 34 th U.S. president Dwight David Eisenhower 3. If an order can be misinterpreted, it will be 20

21 Communication is King ( 21

22 Communication Management The plan should identify the objectives, processes, frequency, participants, media and methods of communication. Put photos of team members on an org chart especially for offshore project. Share information, e.g. with a project wiki or Google Docs There must be weekly meetings. Try 15-min daily Scrum meetings. Continuously evaluate communication to identify areas for improvement There is a cost to communicate but it costs more not to communicate well. You just pay more at a later time. Use open-ended questions when talking with someone how would never give no as an answer. Some cultures or upbringing discourage people to bring bad news to the authority. 22

23 Horror Story for Outsourcing Customers Story from P.156 Outsource It! by Nick Krym. The U.S. customer provided requirements for an outsourcing vendor in Byelorussia to do the software development. Due to the aggressive schedule and weak communication infrastructure, the offshore team would maintain documents and source control. When the product was almost done, the offshore vendor demanded an outrageous amount of money before transferring the finished product. The offshore vendor rebranded the product to sell it directly. 23

24 What Customers Can Do? Manage their own business processes well outsourcing demands more discipline in following effective processes Focus on managing the inputs you provide the vendors and the outputs you get from them. Avoid the temptation to over-manage the resources and processes of the vendors. (Interview employees of vendors and daily 15-min Scrum-style meetings are not over-management. Asking them the same status a few times a day or looking over their shoulders is over-management.) Have knowhow in technology and methods sufficient to deal with the vendor (so that you can catch a dishonest vendor). Know when they need help from an advisory firm and how to select one. 24

25 Continuous Deliverables From the Byelorussia story, we learn that we must demand deliverables on a continuous basis. They include architecture and design documents, source code complying with appropriate coding standards completed with readable and useful comments. Review/inspect/test all deliverables (of course including documents). The result of user acceptance testing or a review is either passed or failed. All failed artifacts are returned for rework. Acceptance criteria must be explicitly stated and followed. Establish metrics and collect measurements to see if desired performance is met. Use metrics wisely to avoid hostility. 25

26 Useful Agile Practices for All Projects Even for waterfall projects, some agile practices are beneficial. Use continuous integration one of the easiest agile practices to adopt regardless of your software processes. Insist to see new functionalities frequently every one or two weeks aligning with the sprint size in Scrum. (That is a good indicator if everything is going well.) Use test-driven development (TDD) to help understand the requirements. 26

27 Recommended Practices Martin Fowler wrote the classic article Using an Agile Software Process with Offshore Development The advices are applicable to waterfall projects too. Send an ambassador to the other site for several months and then switch person. Have two kinds of contact visits: Seeding visits early in the project to build relationship Maintaining visits to keep the good relationship Outsourcing reduces the amount of the face-to-face communication so it is more important to have quality documentation. (Often we want more documents than non-outsourced projects.) Code and documentation should be checked into a version control system daily. 27

28 Monitoring Metrics Progress Counting the tasks or modules completed or inspected Use work breakdown structure until tasks are no more than 3 person-days or so. (Large tasks are bad for tracking progress. Reject claims that a task is 90% complete because the last 10% may take the same effort as the first 90% to complete.) Quality transaction throughput, user response time, number of open defects, mean time to failure Fixes (Responsiveness) Are they fixed within an acceptable time frame defined? 28

29 It s Hard to Lie with Burndown Charts One Chart Per Small Team

30 Trends in Vendor Management Customers should make vendor management an integral part of their business strategy. The market is under-invested in the area of vendor management esp. in tools, methods and processes. Leading companies are moving away from dealing with a single large vendor to dealing with multiple smaller vendors to exploit their specialties and low cost structures. 30

31 Dealing with Time Zone Difference Time zone difference is often seen as an obstacle to (offshored) outsourcing. It does not need to be as long as the communication between client and vendor is in the form of input and output rather than the looking over your shoulder type of monitoring. Collaborating teams at different locations can work around the clock for increased productivity. It s like running two 8-hour shifts in a factory. It s also fine for two sites to be up to 8 or 9 hours apart. It s not too difficult for them to Skype. 31

32 Dealing with Staff Turnover Vendors with low turnover rates are good because it takes the replacement months to get up to speed. Vendors are motivated to lie about their turnover rate e.g. they may not count a particular project that will make their turnover rate looks too high. It is not meaningful to compare the turnover rates reported by two vendors as the less honest one will benefit. Vendors may assemble the best team for your project initially but reassign the best staff to a project for another customer that may be in trouble. The contract should protect you in case of staff turnover For fixed-price projects, demand an overlapping period of at least 1 month for the outgoing staff to train the new staff. For T&M projects, demand a discounted rate for the first two months of the new staff. 32

33 Dealing with Inflated Job Titles Vendors are motivated to give their employees more prestigious title to keep them happy. The time to reach the title of intermediate developer in the customer s company may be enough for someone to become a senior developer in the vendor s company. Customers should remap the titles first for a fair comparison e.g. in the calculation of savings. Customers should insist on taking part in the decision of appointing a member to the vendor s project team including the interview even if it means doing the interview on Skype. Check or write the position description even if the staff will work in the vendor s team. A well-written position description helps to find a good candidate. Validate the claims in the résumé. If the job to manage the hiring looks daunting, have a third party to manage the vendor for you. 33

34 Make your vendor replaceable Your contract must be written to allow you to unilaterally decide getting out of the contract quick and relatively unharmed. The vendor tries to make themselves irreplaceable. You have to counterbalance this attempt. All the things we have recommended so far should allow the next vendor to take over in a relatively short time including: All deliverables regularly checked into a version control system managed by the customer and immediately reviewed for acceptance. With all the precautions, the vendor knows that they cannot hold you hostage as in the horrible Byelorussia story. 34

35 Collaboration? The outsourcing industry has not yet established itself as a professionalized industry so that results are predictable. A software specification language that can allow requirements/design to be expressed more precisely than UML and yet break the language barrier (of English, Chinese or whatever natural language you speak). I am working on a research to invent such a specification language. I am Oliver Au, assistant professor at the Open University of Hong Kong. Contact me at oau@ouhk.edu.hk 35

36 Additional References Deloitte (2014) Deloitte s 2014 Global Outsourcing and Insourcing Survey. Laird, Linda M., Brennan, M. Caro (2006) Software Measurement and Estimation : A Practical Approach, Chapter 11 Outsourcing, Wiley-IEEE Computer Society Press. Lacity, Mary and Willcocks, Leslie (2012) Outsourcing Business and I.T. Services: the Evidence of Success, Robust Practices and Contractual Challenges, Legal Information Management, Vol 2, No 1, March 2012, pp 2-8, Cambridge University Press. Krym, Nick (2012) Outsource It! Pragmatic Bookshelf. Outsourcing Journal Top 10 Risks of Offshore Outsourcing (2003) Agile Outsourcing - Managing People Metrics in Outsourced Software Development Why India outsourcing is doomed (2013) 36