Software Testing Excellence Program (STEP) Project Proposal Offshore Outsourcing Management of Software Testing This project focuses on issues related to the management of software testing in a global environment that includes offshore vendors. The sections below provide a justification identifying the importance of this topic, the theoretical basis for this project, research methods to be employed in conducting this project, and expected future benefits. Additional information includes a proposed timeframe for project completion, a budget, and examples of ways by which the project team at the University of Memphis would expect to collaborate with FedEx s Global Testing Center of Excellence. Justification for Research Published reports continue to indicate a trend toward offshore outsourcing, including estimates that U.S. companies account for approximately 70% of the global offshore market and about 80% of India s offshore clientele (Amiti and Wei 2003; Greene 2006). Meanwhile, industry trends indicate software testing is becoming a larger part of software development projects. One study measuring software development processes found 41% of the project effort was spent in testing, 25% in designing, 13% in coding, and 21% in other activities (Waligora and Coon 1994). With testing consuming much of the resources used in software development, it becomes a viable candidate for delegation to vendors. Furthermore, FedEx software testing management has indicated an interest in the development of a model to assist in determining what work would be best to delegate to offshore vendors without the loss of mission critical knowledge. This project is designed to address that need. This project is expected to result in several useful outcomes, including: Development of an initial knowledge base of factors to consider when working with offshore software testing vendors, Examination of issues in maintaining a desired balance between keeping critical business knowledge in house versus delegating activities to offshore testing vendors, Determination of knowledge that is critical to retain internally, Creation of an unbiased external view of potential best practices for the delegation of work offshore (Noble 2004), Delineation of the factors in cross cultural remote management that impact the delegation of activities to offshore testing vendors, and Establishment of a baseline plan for expanding the research to examine other vendor issues related to software testing activities.
Theoretical Basis This research will be based on theories of offshore outsourcing. An initial review of the literature identified general theories as to what to perform in house and what to delegate to an outsourcing vendor within the context of software development (Barthelemy 2003; Franceschini et al. 2003). Additional theoretical sources provide recommendations related to the need to keep knowledge and control of core competencies in house (Lacity and Hirschheim 1993), as well as the methods of managing offshore collaboration (Cusick and Prasad 2006) and other considerations to achieve offshore outsourcing effectiveness (Noble 2004). This literature examines the management of activities inside and outside the outsourcing organization based on theories of 1) transaction cost economics, 2) psychological contracting, and 3) organizational culture. Below we briefly summarize each theory and highlight implications of each for the current research project. Transaction cost economics (TCE) theory suggests the goal of any company is to minimize the costs of transacting business. TCE further states that asset specificity and uncertainty (situational conditions) and opportunism and bounded rationality (human behavior conditions) make transacting business more complex (Williamson 1975; Pfeffer 1982). Outsourcing of non assetspecific activities is shown to have positive effects on outsourcing success (Grover et al. 1996). Also, TCE suggests vendors may not invest heavily in order to maintain their client operations (Aubert et al. 1996). In the context of managing offshore software testing vendors, TCE informs us that software testing activities that are performed using specific technology and knowledge assets, and that have higher levels of uncertainty tend to be outsourced less frequently and less successfully. Based on the results of this research we expect to identify such software testing activities. While TCE suggests that one party may take advantage of another by behaving opportunistically, psychological contracting theory suggests that tight contractual mechanisms can reduce opportunistic behaviors (Ang and Beath 1993; Lacity and Hirschheim 1993). A psychological contact refers to people s mental beliefs about their mutual obligations in contractual relationships (Roussear 1995). Prior studies show these mental beliefs differ between the buyer and the supplier of outsourcing services and yet it is the fulfillment of the mutual obligations that predicts outsourcing success (Koh et al. 2004). In the context of managing offshore software testing vendors, psychological contacting informs us that each party s expectations are important to a successful and productive working relationship. Through this research we expect to identify a set of such mutual obligations within the context of software testing as this will help us explain the potential outcome of different outsourcing relationships. Organizational culture is shaped by management practices and not necessarily by management values (Hofstede 1980). Culture is defined as the shared mental beliefs that distinguish one group from another while values are defined as the tendencies to prefer certain states of affairs over others (Hofstede 1980). Across countries, cultural differences tend to be based on differing values but not on different management practices. Meanwhile across companies, cultural differences tend to be based on different management practices but not on differing values (Hofstede 1980). Thus, in the context of managing offshore software testing vendors, management issues span across countries and across companies. This literature suggests both
differing values and differing management practices may influence the delegation of tasks in an outsourcing relationship with software testing vendors. Finally, models of software testing exist to inform our examination of the delegation of specific activities to offshore vendors. The V model conveys the need to test during many of the system development phases so that testing efforts in later stages can be minimized. The Software Engineering Laboratory (SEL) Testing Approach, based on quality improvement paradigms, proposes ways to deliver greater testing efficiencies (McGarry et al. 1994; Waligora and Coon 1994). Also, the literature discusses approaches to software testing education and suggests educators should develop specific skills for activities needed to effectively accomplish the goals of software testing (see Table 1) (Jones 2001). Similar to the System Development Life Cycle (SDLC), the software testing model suggests the phases of analysis, design, implementation, execution, and evaluation for managing various activities in software testing lifecycle. Phase Activities Output Skills needed Analysis Define scope and Test plan, Read and understand a specification strategy for testing. refined Identify deficiencies in a specification Refine specification. specification. Convert specification to a decision table Devise a testing strategy based Design Derive test cases. Organize test effort. Implementation Develop machinery needed to conduct tests. Execution Run test as controlled experiment. Capture test results Evaluation Verify results against expectations. Take follow up action such as reporting, debugging. Test cases, test data sets. Test procedures/ scripts, drivers. on the specification Develop an adequate set of functional test cases. Identify redundant test cases Develop boundary test cases Write a test procedure/ script Develop test drivers Create test data sets Test results. Perform testing according to test procedure Document test results Use test drivers Test log, modified software. Identify program errors from test results Debug program given description of bug Debug test drivers/ procedures Table 1 Modified Software Testing Lifecycle and Representative Skills Model (Adapted from Jones 2001) This preliminary examination of the literature provides background for our work. In this project, the literature regarding offshore outsourcing will be studied in more depth, and the literature on basic theories and models of software testing will be examined in detail. The team will use the findings to identify issues in software testing that are unique to offshore vendors. Based on the preliminary literature review, the proposed research model is illustrated in Figure 1.
Software testing activities Critical business knowledge Delegation of software testing activities Offshore culture, values, practices Remote management issues Figure 1 Proposed Research Model Research Method The data gathering phase is expected to begin with these major activities: 1. In depth review of existing literature to use as a knowledge base 2. Interviews with appropriate FedEx personnel to identify and better understand the issues related to the project topic a. Interviews will include a variety of open ended questions; b. Interviewees will include key stakeholders, such as CIO, project managers, testers, developers, and vendors; and c. Follow up interviews will be conducted to clarify initial findings or to expand the coverage, as needed. Following the initial data gathering phase, the project team activities will include: 1. Analysis of results leading to an initial model of best practices for the delegation of software testing components to offshore vendors, 2. Development and administration of a survey for software testing managers at other organizations for external data that can be used to validate and update the model of best practices, and 3. Development of reports for distribution to FedEx software testing managers as well as for academic publications. Additional details of the activities can be found in the Timeline section of this proposal. Expected Contributions to Both Theory and Practice Little, if any, research exists that is specifically related to best practices for management decisions related to offshore outsourcing of software testing activities. This research will provide a unique perspective regarding issues related to the delegation of software testing.
Few methods have been developed to measure the success of delegating software testing components to offshore testing vendors, and this project will make a significant contribution in that area. This project will provide a model of best practices tailored directly for issues of FedEx software testing management. Furthermore, this project will develop an initial understanding of software testing activities containing critical business knowledge to keep in house, which will serve as a solid basis for future research. Additional future research based on this project could come from the initial understanding of the assignment of responsibilities, monitoring mechanisms and bonding methods for offshore testing vendors. Publication outlets Numerous academic outlets are likely to be interested in the findings of this research. For example, conferences and symposiums such as ICIS, AMCIS, SIGSAND, etc., allow academicians to network with systems testing researchers and obtain feedback on research ideas at early stages. Completed research projects could be targeted for top MIS journals such as Management Information Systems Quarterly, Information Systems Research, etc. Some publications have clearly stated interests in software testing. For example, there is a current call for papers for an IEEE Software special issue on Software Testing. Researcher Bios Robin Poston is an Assistant Professor of Management Information Systems at the University of Memphis. Robin s research interests include performance evaluation models and measurement metrics for testing environments, best practices for outsourcing software testing, and intercultural and communication differences with offshore testing vendors. She has published articles in publications such as Management Information Systems Quarterly, Communications of the ACM, Information Systems Management, and the Journal of Information Systems. Robin s teaching focuses on introduction to MIS concepts, systems analysis and design, and resource management courses including undergraduate, graduate and PhD students. Dr. Poston has spent time working at the University of Melbourne, Meta Group Research, KPMG Consulting, and Convergys. She received her bachelor s degree in Computer Science from The University of Pennsylvania (1987), master s degree in Accounting from The University of Central Florida (1992), and Ph.D. in Management Information Systems from Michigan State University (2003). Judith C. Simon is a Professor of Management Information Systems and Director of the Center for Innovative Technology Management at the University of Memphis. She previously served as department chair for many years. Her bachelors and doctoral degrees were received from Oklahoma State University, and her MBA was earned at West Texas State University. She was previously employed in the production management offices of a large manufacturing organization and continues to work with a wide range of organizations on IT related issues. Her research interests include information security management, IT quality, healthcare information
systems, and global IT issues. She has written nine textbooks and published numerous articles in academic journals. She currently serves on a global research team sponsored by the Society for Information Management International, which is conducting ongoing studies of the IT workforce trends. She has taught a wide range of courses at the undergraduate, Masters, and Ph.D. levels. Radhika Jain is an Assistant Professor of Management Information Systems at the University of Memphis. Her research interests include knowledge management issues in software testing, managing contextual process knowledge, business process integration, knowledge management in health care, and ubiquitous computing. Her work has appeared in or has been accepted at the Communications of the ACM, Requirements Engineering Journal, Decision Support Systems, IEEE Computer, Information Systems Management, Journal of International Information Management, and International Journal of Mobile Computing, as well as in several leading conferences. Her teaching interests include object oriented systems analysis and design, telecommunications and mobile computing, programming languages, and management information systems. She is currently completing her doctoral work at Georgia State University. She received her Masters degree in Computer Engineering from Stevens Institute of Technology. Her Bachelor s degree was earned in Computer Engineering from Pune Institute of Computer Technology in India. Timeline The Project Plan Timing 1. Background information gathering a. Academic and practitioner literature and book review Oct Dec 2006 b. Summarizing and synthesizing the literature and books Oct Dec 2006 c. Develop initial best practices model Oct Dec 2006 2. Interviews with key informants at FedEx a. Develop a set of basic questions to guide interviews Oct Dec 2006 b. In depth interviews (approx. 2 hours) to gain understanding of Nov/Dec 2006 relationships with vendors and issues of concern Nov/Dec 2006 c. In depth interviews (approx. 2 hours) with vendors to gain understanding of relationships with FedEx and issues of concern Jan/Mar 2007 d. Contact interviewees with follow up questions Jan/Mar 2007 e. Report summarized results Mar 2007 3. Survey of project managers within and outside FedEx a. Design survey Jan/Mar 2007 b. Pilot test survey instrument Jan/Mar 2007 c. Develop random sample of testing managers Jan/Mar 2007 d. Administer survey to sample of testing managers Mar/Apr 2007 e. Enter and cleanse data Apr 2007 f. Analyze data May 2007 g. Update best practices model May 2007
h. Report summarized results Jun 2007 4. Develop manuscripts for submission to MIS journals July/Aug 2007 Budget Mileage reimbursement for travel to interviews $500 Travel to conferences to gain and share knowledge $5,000 Copies/Printing/Supplies $500 Survey administration $1,500 Purchase of journals, research reports and data if needed $500 Graduate assistant time for data gathering/transcription/coding $3,500 License for Atlas.ti software for analyzing qualitative interview data $500 Total Budget $12,000 Opportunities and Mechanism for Data Collection and Collaboration with the Global Testing Center of Excellence at FedEx Corporation Numerous opportunities and mechanisms will exist for data collection and collaboration with FedEx s Global Testing Center. Most of these activities have been described in previous sections, and a few of the major points of collaboration are identified below: Background information about current software testing processes Identification of software testing components and where each activity is performed Schedules of software testers and their work loads/ assignments Offshore software testing vendor selection guidelines Evaluations of in house and offshore testing vendor performance Testing measurement success factors and supporting data Interview access to software testing key stakeholders Identification of software testing knowledge critical to keep within the organization References Amiti, M. and Wei, S. Fear of Service Outsourcing: Is it Justified? Economic Policy, April 2005, pp. 309 347. Ang, S. and Beath, C.M. Hierarchical Elements in Software Contracts, Journal of Organizational Computing (3:3), 1993, pp. 329 361. Aubert, B., Rivard, S., and Patry, M. A Transaction Cost Approach to Outsourcing Behavior: Some Empirical Evidence, Information & Management (30), 1996, pp. 51 64.
Barthelemy, J. The Seven Deadly Sins of Outsourcing, Academy of Management Executive (17:2), 2003, pp. 87 98. Cusick, J. and Prasad, A. "A Practical Management and Engineering Approach to Offshore Collaboration," IEEE Software (23:5), 2006, pp. 20 29. Francheschini, F., Galetto, M. and Varetto, M. Outsourcing: Guidelines for a Structure Approach, Benchmarking: An International Journal (10:3), 2003, pp. 246 260. Greene, W. Growth in Services Outsourcing to India: Propellant or Drain on the US Economy? US International Trade Commission, January 2006. Grover, V., Cheon, M., and Teng, J. A Descriptive Study on the Outsourcing of Information Systems Functions, Information & Management (27), 1996, pp. 33 44. Hofstede, G. Culture s Consequences, Sage Publications, Beverly Hills, CA, 1980. Jones, E.L., An Experiential Approach to Incorporating Software Testing into the Computer Science Curriculum," 2001 Frontiers in Education Conference (FIE 2001), Reno, Nevada, October 10 13, 2001, IEEE Computer Society, F3D7 F3D11. Koh, C., Ang, S. and Straub, D.W. IT Outsourcing Success: A Psychological Contract Perspective, Information Systems Research (15:4), 2004, pp. 356 373. Lacity, M.S. and Hirschheim, R. The Information Systems Outsourcing Bandwagon, Sloan Management Review (35:1), 1993, pp. 73 86. McGarry, F., Page, G., Basili, V. An Overview of the Software Engineering Laboratory, Software Engineering Laboratory (SEL 94 005), 1994. Noble, S. Organizing for Offshore Effectiveness: How the Vendor s Organization Structure Can Affect the Success of Your Outsourcing Initiative, Global Sourcing Insights, 2004, pp. 1 3. Pfeffer, J. Organization Level Rational Action, Organizations and Organization Theory, Pitman, Boston, 1982, pp. 121 177. Rousseau, D.M. Psychological Contracts in Organizations; Understanding Written and Unwritten Agreement, Sage Publications Inc. Thousand Oaks, CA, 1995. Waligora, S. and Coon, R. Improving the Software Testing Process in NASA s Software Engineering Laboratory, working paper, 1994. Williamson, O.E. Market and Hierarchy: Analysis and Antitrust Implications, Free Press, New York, NY, 1975.