IT projects have an unenviable reputation for their high failure rate. Successful

Transcription

1 RISK MANAGEMENT AND PROBLEM RESOLUTION STRATEGIES FOR IT PROJECTS: PRESCRIPTION AND PRACTICE HAZEL TAYLOR, University of Washington ABSTRACT This study reports findings from an exploratory field study of risk management and problem resolution strategies used by 25 experienced Hong Kong project managers working for vendor firms on local and multinational IT projects. In contrast to formal prescriptions, respondents described a pragmatic approach applying general strategies, within four broad categories of control, negotiation, research, and monitoring, to all their projects, in order to both manage identified risks and be prepared for whatever problems might arise. These findings provide support for the conjecture that a practical alternative to traditional formal prescriptions may be needed if project managers are to succeed in managing highly uncertain situations with limited resources. Keywords: risk management strategies; problem resolution strategies; IT project management 2006 by the Project Management Institute Vol. 37, No. 5, 49-63, ISSN /03 Introduction IT projects have an unenviable reputation for their high failure rate. Successful management of IT projects, in terms of meeting cost, time, and functionality targets, continues to be an elusive achievement (Standish Group, 2001, 2003). Risk management, and in particular, the failure to manage the uncertainties that typically surround an IT project, has been consistently identified as a critical area of IT project management (Charette, 1996a; Nidumolu, 1995; Wallace, Keil, & Rai, 2004). Given the many problems that continue to recur in the IT project arena, the question of how successful project managers handle project risks and problems arising in their projects is clearly of interest. Some recent research has focused on identifying comprehensive lists of potential risk factors for IT projects, and many potential risk factors for IT projects have been identified (see, for example, Barki, Rivard, & Talbot, 1993; Moynihan, 1996; Schmidt, Lyytinen, Keil, & Cule, 2001). There are also suggestions that IT project failures could have been avoided with better risk management methods (Baccarini, Salm, & Love, 2004; Charette, 1996b; Keil, Cule, Lyytinen, & Schmidt, 1998), and a number of researchers have focused on providing guidance on how to manage the risk process for IT projects, in order to ensure the best chance of success (Barki, Rivard, & Talbot, 2001; Boehm, 1991; Charette, 1996b; Fairley, 1994; Heemstra & Kusters, 1996; Keil, Cule, Lyytinen, & Schmidt, 1998). However, there has been little empirical research to support the risk management prescriptions provided, and we still know little about what successful practicing project managers actually do in terms of both managing risks identified, and addressing problems that arise, in their projects (Schmidt, Lyytinen, Keil, & Cule, 2001). The research reported in this paper is part of a larger exploratory field study into risk management practices in IT projects. The first phase of the study reported on key risk factors that vendor project managers actually identified when managing software package implementation projects on client sites (Taylor, 2004). The part of the study reported here examines the strategies used by these IT project managers to address risks in their projects, and the actions they took both to prevent problems in the first place and to deal with problems that did arise in spite of any risk preventive measures they had taken. Understanding more about how risk management and problem resolution is carried out in practice can inform both managers and academics of strengths and shortfalls in this area and help to pinpoint any gaps in theory or practice that, if addressed, could improve the IT project performance record. D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 49

2 IT Project Risk Management The body of knowledge on project management and project risk management is extensive and well established (e.g., Project Management Institute, 2000), and provides a basis for understanding project management techniques that are recommended for the IT industry by researchers such as, for example, Boehm (1991), Powell and Klein (1996) and Barki et al. (2001). Risk Management is one of the project management Knowledge Areas applied over the whole project life cycle, with Risk Identification and Risk Response Planning being part of the Project Planning phase, and Risk Monitoring continuing throughout the project implementation phase. The first stage of Risk Management, Risk Identification, has been a major focus of the IT project risk literature, which reveals an emphasis on developing complete and comprehensive checklists of risk factors to be considered when planning and managing an IT project. There is now a substantial body of work on the typical risk factors faced by software project managers, and also the priorities placed on these risk factors by managers (Alter & Ginzberg, 1978; Barki, Rivard, & Talbot, 1993; Boehm, 1991; Heemstra & Kusters, 1996; Schmidt, Lyytinen, Keil, & Cule, 2001; Wallace, Keil, & Rai, 2004). Although the risk factors strand of research has focused, descriptively, on providing comprehensive identification of the potential sources of risk for IT projects, a more prescriptive risk management strand of articles aims to give guidance on how to manage the risk process for IT projects, in order to ensure the best chance of success. The risk management strategies recommended in the literature revolve round first identifying the specific risks for a particular project, and then, in the response planning stage, formulating and implementing specific actions to address each risk, on a riskby-risk basis (Project Management Institute, 2000). The planned actions may be to eliminate, mitigate, or accept risks depending on their priority, and contingent actions should also be planned to address problems that arise in spite of any eliminating or mitigating actions that may have been taken. Risk management researchers (for example, Barki, Rivard, & Talbot, 2001; Boehm, 1991; Charette, 1996b; Fairley, 1994; Heemstra & Kusters, 1996; Keil, Cule, Lyytinen, & Schmidt, 1998; Powell & Klein, 1996) argue that past IT project failures could have been avoided with better risk management methods, and provide frameworks that are variations and elaborations on the general project risk management processes previously described. However, although this literature provides some prescriptive advice on appropriate strategies to address risks in IT projects, most of the recommended strategies are high level, rather than at the detailed risk-by-risk level recommended in the generic project management literature (Project Management Institute, 2000). For example, Barki et al. (2001) suggested that highrisk projects should have higher levels of planning and user participation and Davis (1982) recommended that the project management strategy should be matched to the degree of uncertainty in the project. Researchers have advocated the outsourcing of IT project work in order to reduce the risk to the client (Choudhury & Sabherwal, 2003; Lacity & Willcocks, 1998; Willcocks, Hindle, Feeny, & Lacity, 2004). Contracting a firm to custom develop a system or to implement a (possibly customized) software package allows the client firm to transfer much of the project risk to the outsource provider (Lassila & Brancheau, 1999; Martin & McClure, 1983; Natovich, 2003; Rao, 2004). In addition, because the vendor firm s core business is the development and implementation of systems on client sites, the client can benefit from the vendor firm s high levels of competence in management of these projects (Levina & Ross, 2003). The fact that the core business of vendor firms is IT project management, suggests that a study of risk management and problem resolution strategies applied by vendor project managers could be particularly fruitful: these project managers are most likely to be expert in their field because their firms future business depends on their ability to deliver their firms products successfully. Indeed, one of the few empirical studies exploring vendor project managers strategies found that the managers proposed broad strategies for a variety of different situations, with no clear links between the type of strategy and the uncertainty level of the situation (Moynihan, 2000, 2002). Moynihan also found that a major underlying principle seemed to be the need for the project managers to protect themselves and their immediate organization against any adverse impact from project problems. With its focus on vendor project managers as a source of expertise in the area of risk management, Moynihan s study is particularly interesting, although the use of hypothetical risk situations may provide more insight into vendor project managers espoused theories about strategies to apply, than into their actual actions on real projects (Argyris & Schön, 1978). Other researchers have also noted the need for more empirical investigation into the countermeasures that may be effective against the many risk factors now known to be possible threats to IT project success (Keil, Cule, Lyytinen, & Schmidt, 1998; Schmidt, Lyytinen, Keil, & Cule, 2001; Wallace, Keil, & Rai, 2004). In particular, drawing on their respondents comments about why certain risks were important, Keil et al. developed a framework to categorize the risks identified in their Delphi study according to the level of perceived importance of each risk and the extent to which the risk was perceived to be controllable. Keil et al. then speculate[d] on possible strategies for managing each type of risk, but their risk categorization framework and suggested risk management strategies remain untested. Given the limited amount and speculative nature of the empirical evidence about what risk management strategies are being used in IT projects, the present study investigates the actions that project managers use in practice to address risks identified for their projects, and the countermeasures that they apply when risks materialize into problems in spite of their risk management efforts. A key objective is to identify strategies used by expert project managers, because such managers are likely to have developed a high level of skill in dealing with the risk and uncertainty that typically 50 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

3 accompanies IT project work. As previously noted, project managers working for vendor firms are particularly likely to have developed high levels of competence in the project management area because the success of their employing firms depends on their ability to ensure high levels of customer satisfaction with the project. Thus, the research focused on vendor project managers working primarily with software package implementation projects, and was guided by the following research question: What strategies do project managers use to address risks in IT projects both to prevent them causing problems in the first place and to deal with the problems if they do arise (i.e., if the risks identified as potential problems turn into actual problems)? The term project manager is used in the context of this research to describe the manager who is assigned overall responsibility for a specific project. In outsourced projects, there will typically be a vendor project manager with responsibility for the project from the vendor perspective, and a client project manager with responsibility for the project from the client perspective. The term risk has typically been used in prior research to refer to any potential problem that threatens the success of the project. However, a particular aim of the present study was to allow the respondents themselves to determine what constituted risk in their projects, in order to avoid the possibility noted by Schmidt et al. (2001) that risks could be overlooked if the researcher predetermines the risks to be discussed. Thus, drawing on the respondents perceptions, risk in the present study refers not only to potential problems threatening the project itself, but also to potential problems arising from involvement in the project that could adversely impact on the respondent s firm. Methodology The exploratory and descriptive nature of the research required a research methodology that met the need for contextual information, as the situation contexts were of paramount importance in gaining an understanding of participants interpretations and actions regarding risk and problem management, and possible differences that might be due to the outsourcing context and vendor perspective. Investigations of this type lend themselves to broadly interpretive methods of research (Walsham, 1995). According to Orlikowski and Baroudi s (1991) seminal article on various research approaches in information systems, interpretive research is based on the epistemological belief that understanding about social reality involves interpreting the meanings assigned to phenomena by those interacting with them. Interpretive research techniques draw on participants own words and experiences and interpretive researchers present their own interpretations of the interpretations made by participants. A purposeful sampling strategy was employed (Miles & Huberman, 1994; Patton, 2002), with the aim of building an information-rich set of data, in order to cover the range and diversity of risk management approaches that might be found in practice. Packaged software implementation firms in Hong Kong were first identified on the basis of their reputation, for the purpose of being able to identify organizations recognized as outsource providers of software package implementation. Within the firms, IT project managers were sought who would be recognized as proficient in their domain, based on their years of experience in the profession, the number of package implementation projects they had been involved with, and peer recognition. A guideline for experience of a minimum of five years and five projects was used. Once an initial contact had been made with a senior executive in a firm, the conversation was used as an opportunity (Miles & Huberman, 1994) to obtain further suitable contacts both within the firm and in other firms. Data collection was quickly followed by preliminary data analysis. As certain themes emerged from the preliminary analysis, it became clear that specific confirming cases (Patton, 2002) were needed in order to probe deeper into key differences between the views of the vendor project managers interviewed and those of the in-house managers who had been interviewed or surveyed in previous research. Thus, the sample was expanded to include three project managers who were working for in-house IT departments in both commercial and government organizations, running both inhouse and outsourced projects, in order to seek cases that could confirm or refute these emerging differences. As shown in Table 1, 25 experienced project managers from 12 different organizations within Hong Kong participated in the research and discussed a total of 60 implementation projects. Although most of the projects (80%) fell into a broad category of package implementation project, only two were described as straightforward package implementations with little or no customization. More typically, the projects included extensive customization, front-end web development work, and/or major infrastructure upgrades. The remaining projects involved mainly custom development and some consulting work. The respondents included 20 males and five females; 18 were Chinese and the remaining seven were American, Australian, British, and South African. Four of the respondents finished their formal education at the high school level, 10 had bachelor s degrees, and 10 had postgraduate Type Organizations Respondents Projects Local boutiques/warehouses Mid-range vendor firms (local HK and some mainland China projects) Large multinational vendor firms (HK branch, Asia-Pacific projects) In-house department, large commercial firm In-house department, government organization Total Table 1: Organization and project details D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 51

4 qualifications. All were well-versed in typical project management methodologies, such as A Guide to the Project Management Body of Knowledge (PMBOK Guide Third Edition) (Project Management Institute, 2000). All the respondents had trained or worked abroad for part of their careers, and all had experience working with team members from a wide variety of cultures. In order to learn how managers actually manage risks and problems in their projects, it was important to choose a method that would bring forth respondents actual actions, rather than simply eliciting their espoused theories about risk management (Argyris & Schön, 1978). Thus, the study used a critical decision interview approach (DuBois, 2002; Klein, Calderwood, & MacGregor, 1989), which is a variant of the critical incident technique (Flanagan, 1954), to explore and describe the risk and problem management approaches used by experienced vendor project managers during software package implementation projects. Critical incident and critical decision methods have been used extensively in research for eliciting expert knowledge in situations where the experts have difficulty accessing their knowledge (Klein, Calderwood, & MacGregor, 1989; Koh, Ang, & Straub, 2004; Sternberg et al., 2000; Taylor, 2005; Wagner, 1987; Wagner, Sujan, Sujan, Rashotte, & Sternberg, 1999). One of the key strengths of the critical decision method is its concentration on identifying the detailed context of a given critical situation and specific environmental and situational cues, thus, facilitating the surfacing of possibly subtle differences in perspective that could be involved in vendor project managers risk management and problem resolution strategies. Semi-structured interviews were conducted with respondents at their offices, with interviews typically lasting about one hour. Respondents were asked to choose a recently completed project that they thought was interesting from a risk perspective. After they had given a brief description of the overall story of the project, respondents were probed for specific incidents that were challenging from a risk management perspective. A structured series of probe questions was used to elicit information about the situational cues surrounding these situations and the actions taken. With the consent of the participants, the interviews were recorded, and additional notes were taken during the interviews, in order to provide descriptive elaboration of the elicited information. The transcripts of the interviews were provided to the respondents for their comments and feedback, and a confidential summary report was provided to participants at the end of the study. Feedback received from respondents was uniformly positive, providing support for the validity of the findings. Coding and Analysis All interviews were analyzed with a qualitative content analysis procedure, using NVivo 1 software version 2.0 to support the process. Typically, respondents discussed two or three projects, and also made general comments about their project management experience. The general analysis 1 NVivo, a qualitative research software product, is a registered trademark of QSR International, Pty Ltd. process followed three key stages (Miles & Huberman, 1994; Wolcott, 1994): description (i.e., summarizing what happens); coding and analysis (i.e., systematically identifying key factors and relationships); and interpretation (i.e., drawing conclusions from the research data). The descriptive stage of the analysis involved summarizing the transcripts of each respondent on a project-by-project basis and separating out any general comments. Within each project, passages were identified that described initial risk management situations and that related to actual problem situations that the respondent had to address during the course of the project. During the analysis phase, the focus was on the detailed coding of individual transcripts. In the first pass of coding, a thematic analysis approach was used to develop a risk factor framework to categorize the risks that were of concern to these respondents. The framework, shown in Table 2, classified the risks into four broad themes relationships, project management, business environment, and solution ambiguity categorized by Keil et al. (1998). In addition, the framework also reflected the respondents emphasis on the importance of identifying the source of the risk, namely whether the risk arose primarily from the vendor side, from the client side (including client-vendor relationships), or from a third-party involvement. The next pass of coding addressed strategies used by the respondents. Drawing on the generic risk management literature that prescribes a risk-by-risk approach to risk response planning (Project Management Institute, 2000), passages describing risk assessment and actual problem situations were studied closely with the aim of identifying the specific strategies being applied to the specific risks or problems being discussed. It was difficult to find evidence of a risk-byrisk approach in the initial risk assessment situations. Rather, respondents generally described broad strategies that were applied to cover a wide range of risks and problems, and typically, these strategies were intertwined with respondents risk perceptions and frequently indicated an implicit assessment of particular risks. In addition to the specific situations described by respondents, their general comments were scrutinized and it was determined that, particularly for the initial risk response planning stage of their projects, respondents had typically developed heuristic risk management strategies that they applied to all of their projects. Thus, in order to determine the risk to which a particular strategy was being applied to, the situation description passages and general comments were correlated with the risk framework coding and the wider context of the projects being discussed and the cues the managers were describing in relation to the situation were also considered. The interpretation stage of the analysis process set the groundwork for the discussion of results reported later. An underlying aim of the data analysis process was to understand the domain characteristics as a framework for interpreting practitioner performance (Potter, Roth, Woods, & Elm, 2000). Domain characteristics, such as the level of complexity of the projects, the various demands of key players during the course of the projects, variability among proj- 52 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

5 Theme Source of Risk Vendor Third Party Client Project Management - Staffing resources - Change management - Schedule and budget - Documentation - Staffing resources - Deliverables control - Staffing resources - Sign-off control - Readiness - Project management Relationships - Team morale - Internal negotiations - Top management support - Cooperation - Expectation - Trust - Top management support - Users - IT department - Bad news Solution Ambiguity - Customization - Newness - Complexity - Development choice - Requirements understanding - Integration and compatibility - Deliverables - Data conversion - Technical environment - Requirements understanding - Functionality Environment - Reputation - Overseas head office - Competition - Legal and credit risk - Contract terms and conditions - Non-local third party - Multiple third parties - Multiple sites / countries - Organization culture - Multiple departments - Business changes Table 2: Summarized risk factors ects, and complicating factors and limitations particular to each project, provided a framework for attempting to interpret the expertise, knowledge, and strategies of the project manager practitioners. Initially, the plan was to group together strategies identified from the situations and from the respondents general comments into categories relating to the types of risk or problem situation that they seemed intended, either explicitly or implicitly, to address. As can be seen from Table 3, the strategies fell into four categories, and although some risks and problems fell into more than one strategy group, for the most part each strategy group was directed at quite different sets of risks and problems within the risk framework shown previously in Table 2. Finally, an evaluation was completed of the detailed work that had been carried out during the description and analysis stages in order to consider questions such as What complexities in the domain triggered the choice of particular strategies for this set of risks or problem situations? and What key aspects of the domain limited the choice of actions for this set of risks or problem situations? Consideration of questions such as these enabled the development of original interpretations of the factors that prompted project managers to choose one strategic approach over another, in order to address the particular risk or problem situation they were facing. Results and Discussion With the exception of the three in-house managers, the respondents described rigorous risk assessments carried out by the pre-sales team prior to the project commencement. The in-house managers reported a mixed approach to preproject risk assessment, relying on their vendors risk management processes for outsourced projects and regarding risk management as part of project management, in general, on their in-house projects. The primary focus in this study, however, was on risk management and problem resolution strategies applied by the vendors project managers after the project commenced. In total, 90 risk management and problem resolution situations were identified and analyzed in 47 different projects. In 41 of the situations, project managers described actions they took when they started work on the project. The remaining 49 situations related to actions taken to address problems that arose during the progress of the project. There was little evidence that respondents either revisited the pre-sales risk checklist, or made their own independent risk assessment when they actually started the project work. Instead, managers relied on the contingencies already built into the schedule by the pre-sales team and applied broad project management strategies to all of their projects, regardless of specific risks that had been identified. Thus, in the integrated presentation of results and discussion that follow, the general framework is shown in Table 3, categorized by the four strategy groups used by respondents to manage their projects and linked to the broad risk themes shown in Table 2. Using this framework, the specific strategies applied by respondents in each of the four groups are discussed, and finally, the framework and the strategies applied by respondents are compared with recommendations from the literature on risk management. D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 53

6 Risk Theme Strategy Type Control Negotiation Research Monitoring Project Management Relationships Solution Ambiguity Business Environment - Vendor staffing resources - Change management - Schedule and budget - Third-party deliverables control - Client sign-off control - Multiple sites / countries - Non-local and multiple third parties - Change management - Client project management - Client staffing resources - Third-party staffing resources - Expectation - Trust - Bad news - Cooperation - Internal support - Team morale - All technology risks - Unclear requirements and/or functionality - Client top management support - Development choice - Client readiness - Multiple sites / countries - Client organization culture - Vendor reputation - Competition - Contract terms and conditions - Client business changes Table 3: Strategies and associated risks General Categorization of Risk Management and Problem Resolution Strategies When focusing on the start-up stages of their projects, respondents generally described broad strategies that they applied to cover a wide range of risks. When dealing with problems arising during the implementation of the project, managers typically drew on the same set of possible strategies to address the problems they faced. As can be seen from Table 3, the strategies fell into four categories, for the most part directed at quite different sets of potential and actual problems. Strategies in the first category, control, were typically applied in those situations that the project managers saw to be directly within their locus of control, and were mainly described in situations and contexts relating to the risks grouped in the project management risk theme, namely vendor staffing resources, change management, schedule and budget, documentation control, client sign-off control, and third-party deliverables control. Control of these risks was seen as being at the heart of a project manager s skill set, and as discussed further later, strong management of the schedule and budget was seen as the foundation for project success. The environment risks related to multiple sites/countries, multiple departments, and non-local and multiple third parties were also partly addressed by control strategies, because close management of logistical issues was seen to be paramount in addressing these risks. The second category, negotiation, describes strategies used in situations seen by the respondents as only partly within their locus of control. Success in these situations was dependent not only on the project manager s skills but also on the willingness of other parties to work together toward a successful outcome. These situations were typically the soft situations involving inter-personal interactions and relationship issues. Such situations could not be addressed by a simple application of a control strategy as they required cooperation from other parties, and such cooperation had to be negotiated and carefully managed. Change management appeared in both the control and the negotiation strategy categories because although respondents clearly considered it important to exercise very close control over change requests, they also recognized the inevitability of requirements change and the need to negotiate a satisfactory solution for requested changes with the client. Thus, the risks related to change management issues were addressed both with strong control strategies and also with careful negotiation strategies as discussed next. The third set of strategies, research, like the control strategies, was applied in situations that the respondents regarded as fully within their control, namely situations that required further information in order to determine what action to take. In particular, these strategies were applied when any of the technology risks had been identified, and also when there was a lack of vendor understanding of the client requirements. These situations were the ones that the managers felt most confident about, because if potential problems were identified in these areas they could be resolved simply by getting more information. If the new information revealed that the problem was unavoidable, then at least 54 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

7 they had a definitive answer and could either look for a different solution, or escalate the issue for resolution at the executive level. Finally, for a large number of risks, respondents used no mitigating strategies at all, and instead adopted a monitoring strategy. Most of these risks fell into the environmental risk theme. Environmental risks were potential problems that respondents were aware of that formed the context of the project and that could influence the approach used for a particular project situation, but they were not risks that managers actively attempted to address or change. These risks formed part of the environment of the project and provided situational cues that colored the respondents approach to managing the project. The monitoring strategy was also used to address one risk each from the relationship and technology/solution risk themes: client top management support and development choice, respectively. Respondents appeared to view these two risks as non-negotiable parts of the project environment that they had little control or influence over, and, just like contract terms and conditions, this was something to be accepted and monitored. Table 4 shows the number of specific situations describing the use of the different strategies. Note that the specific situations did not include descriptions of the monitoring strategy. Evidence for this strategy came from the analysis of respondents general comments, as well as a close reading of the cues that were taken into account when applying control, negotiation, or research strategies. Control strategies tended to dominate in the initial stages of projects, whereas negotiation strategies were more prominent when problems arose during the course of projects. The detailed strategies in each group, with indicative interview extracts, are shown in Table 5. Initial Assessment Problem-Arising Totals Situations Situations Control 26 (29%) 12 (13%) 38 (42%) Negotiation 8 (9%) 35 (39%) 43 (48%) Research 7 (8%) 2 (2%) 9 (10%) Totals 41 (46%) 49 (54%) 90 Table 4: Strategy groups used in initial assessment and problem-arising situations Control Strategies The control strategies used by respondents were typically proactive actions initiated at the beginning of projects and applied throughout the projects in order to maintain control and meet targets. The key control strategy, described in 26 of the 41 initial risk situations, and in 12 of the 49 problem-arising situations, was an exhaustive analysis and control of the work breakdown structure (WBS), which frequently triggered the application of strategies in the other three groups by revealing potential gaps in the project plan. Respondents paid particular attention to the development of the project plan and WBS as the substance of their risk management strategy, and regarded it as essential for guarding against schedule and budget risks, which were the most frequently mentioned threats to project success. Work Breakdown Structure (WBS) The typical control approach used at the start of projects was an intensive analysis of the WBS, with the aim of developing a very detailed task breakdown. This focus had the implicit side effect of helping the manager to develop a detailed understanding of the client requirements and the proposed solution and technology for the project. The required vendor staff skill-set was established from this analysis and technical and logistical requirements, and any gaps in vendor understanding of requirements, were revealed. Hence, the project manager established an in-depth understanding and control of the project and its potential risk areas without actually assessing risks explicitly. The risk that most managers were concerned about at this start-up stage of their projects was the potential failure to meet an over-ambitious schedule. Where this risk was identified, managers would quickly move to mitigate the risk by alerting their senior executive and attempting to negotiate internally for more resources. Here, similar to Moynihan s (2000, 2002) findings discussed earlier, managers motives appeared to be related both to protecting themselves in the event that the timeline targets were not achieved, as well as to concern about the project itself going over time or budget. Respondents from vendor firms were aware that if they made a strong enough case they could sometimes negotiate for a particular project to be run on a lesser profit margin than the standard one applied in their firm, thus allowing them to draw in more staff resources to meet difficult schedules. However, more often, respondents recognized that commercial realities presented limitations on the size of budget and schedule and acknowledged the expectation that they would use their project management skills to meet the required targets. Progress Control Once the WBS was well-established, the second control strategy involved very close monitoring and control of each task s progress to identify any slippage. Respondents regarded the first two to four weeks as critical for establishing early and effective control of the project and identified the management and control of the WBS as the single-most crucial activity. Problems that arose in the projects even after being anticipated as risks when the original WBS was developed were most often the following five: schedule and budget, vendor staffing, newness of technology, multiple sites/countries, and vendor understanding of requirements. These problems were seen to be almost always caused by underlying inadequacies in the original schedule and budget. Even if the immediate problem was a shortfall in vendor staffing, or unexpected problems with technology, or logistical difficulties with multiple sites, or more complicated requirements than originally assumed, the respondents saw the fundamental issue as one of insufficient time or budget. With more time (person-hours), and hence more budget, risks such as insufficient staffing or technical difficulties could be addressed before they materialized into problems. D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 55

8 Change Control Trust & Relationship Building Managing Client Expectations Balancing Cost, Schedule, and Scope Escalation Team Building Table 5: Detailed strategies and indicative interview excerpts This view of the interrelatedness of project risks and problems helps to explain the respondents reliance on detailed project planning and monitoring as the key risk reduction strategy. If most of the key potential problems have their source in inadequate schedule and budget, then they will also immediately show an impact in this area. Thus, by focusing on performance against a detailed WBS, managers both maximize their ability to control their projects and also ensure the earliest possible warning of any problems arising. Change Control Although close management of the WBS was generally seen as the most fundamental control strategy for the management of a project and its risks, managers also recognized that, in order to achieve the schedule and budget targets underpinning the WBS, clients change requests had to be tightly controlled. This typically involved application of change control clauses in the contract and although use of these clauses appears on the surface to be a simple control strategy, respondents were always aware of the underlying 56 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

9 Table 5: Detailed strategies and indicative interview excerpts (concluded) interaction and negotiation with the clients. Hence, change control strategies are discussed further in the next section on negotiation strategies. However, when faced with particularly difficult schedules and budgets, respondents typically took a control approach to the use of the change control clauses in the contract in order to ensure that they met their targets. Indeed, two managers reported that, with astute use of these clauses, they were able to negotiate additional time or budget to alleviate some of the deadline pressure in their projects, and, in fact, this approach enabled them to turn very tight projects into more profitable ones for their vendor firms. Documentation There was general agreement that, no matter how the issue of change control was approached, documentation was essential. Even when the contract change control terms were waived, respondents emphasized that it was still important to record client change requests that had been accommodated in the event that disputes arose at a later stage. This strategy was viewed as a very basic part of project management discipline, because failure to document agreed-upon changes could lead to major difficulties later in the project. Indeed, some managers personally controlled documentation such as meeting minutes in order to position themselves to advantage if disputes arose later. Finally, if a schedule/budget problem arose during a project, the managers typically first attempted to correct the problem with one of the following two reactive control strategies. Increasing Available Person-Hours Within the Schedule Timeline Once a schedule/budget problem arose during a project, the first defense was always a control strategy of increasing the overtime hours worked by the project team. Because team members were, for the most part, salaried employees, extending their hours of work effectively increased the person-hours available to meet the schedule without increasing costs at all. Of course, regular use of this approach had a negative impact on staff morale, and a discussion of the managers strategies for maintaining morale is presented in the next section on negotiation strategies. Rescheduling and Problem Isolation A second control strategy for dealing with schedule/budget problems involved isolating the issue causing the problems, and rescheduling the remaining work so that progress could continue to be made. This approach allowed work to resolve the problem to carry on in parallel with other tasks so that the problem did not impact on the overall schedule. Negotiation Strategies Negotiation strategies were discussed by respondents in 8 of the 41 initial assessment situations and in 35 of the 49 problem-arising situations (see Table 5). The three key external negotiation strategies, change control, trust- and relationship-building, and managing client expectations, were applied at the start of projects to establish a good working relationship with the client and to control risks from unrealistic client expectations. These negotiation strategies were also used throughout the project to maintain the relationship and to attempt to correct a situation when issues arose. Internally, managers used similar skills to negotiate the support they needed in order achieve their targets, and applied strong team-building skills to build and maintain commitment within their teams. Although respondents indicated in their general comments that change control, relationship-building, and expectation management were general risk-management strategies initiated at the start of projects, in the specific situation descriptions most managers mentioned negotiation strategies in the context of addressing problems that arose during D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 57

10 their projects. These problems fell into two categories: (1) those that had been anticipated as risks but still occurred in spite of any mitigating actions and (2) those that were unanticipated. General communication and negotiation skills underpinned the respondents approaches in all these situations, particularly in addressing problems that arose as a result of unexpected difficulties with external relationships with clients and third parties, and also where the respondents encountered problems gaining required levels of support internally. In addition to the three key negotiation strategies mentioned in the previous paragraph, three other negotiation strategies for problem resolution were identified: 1. The strategy of balancing cost, schedule, and scope with the client was used as a second line of defense when control strategies failed to address schedule/budget inadequacies or underlying risks that were anticipated and still materialized into problems; 2. When unexpected problems related to staff morale, various team-building strategies were applied; and 3. An escalation strategy was the fallback position when the manager exhausted all other strategies in his or her repertoire. These six strategies are discussed in more detail later. Change Control There was general agreement among respondents of the importance of establishing limits or boundaries to customer expectations from an early stage in the project, and their approach typically involved application of change control clauses in contracts. However, managers differed in their views on the best way to approach change control. Some managers believed that it was important to start out with an insistence on applying strict change control procedures from the very beginning of the project. These managers felt that an initial softer stance could increase the risk of a loss of control that was difficult to recover from. On the surface, this approach appears to be an example of the application of a control strategy rather than a negotiation strategy. However, even the managers advocating the toughest initial stance were aware that they were building and managing a relationship with their client, and that this involved maintaining a delicate balance. Other managers argued for an initially more cooperative approach, especially where requested changes could be accommodated without much extra effort. This way they established an atmosphere of trust and support with their clients that they found invaluable if the project encountered difficulties at later stages. When the respondents were questioned more closely regarding the type of approach to use for change control, they responded that they typically tried to assess their clients at the pre-sales stage in order to help determine the best approach for change control. Thus, while the change control strategy was always applied to manage risks relating to client expectations about requirements changes, the manner in which it was applied was influenced by cues derived from applying a monitoring strategy during the pre-sales stage to observe risks arising from the client s organizational culture and general approach to the project. Trust and Relationship Building Managers described various general trust- and relationshipbuilding strategies underlying their entire project management approach, which seemed to be intended to serve as a general mitigation for all the soft relationship risks that were typically seen as difficult to anticipate and difficult to address if they arose during the progress of the project. Respondents described how they tried to view the project from the customer s perspective in order to understand how best to establish trust between the parties. Managing Client Expectations One key aspect of the change control and relationshipbuilding strategies was the managers underlying objective of managing client expectations. Client expectations featured as a key issue in both anticipated and unanticipated problems, and ranked second only to schedule and budget management in respondents concerns. As such, it deserves separate consideration here, even though, for the most part, managers did not identify specific strategies targeted to manage client expectations. Rather, part of the purpose of the change control and relationship-building strategies mentioned previously was to set the client expectations at a level that could be met. Managers reported facing an arduous task, because typically, the pre-sales team had painted a favorable picture of the project during the pre-sales negotiations. The implementation manager was then faced with the task of guiding the client through the realities of what could actually be achieved and a key part of this process involved trying to recalibrate the client expectations from the start. Satisfaction with vendor or consultant performance is a key dimension of overall success for customers when engaging an external firm (Gable, 1996), and this customer satisfaction is a function of both the initial expectations and the degree to which those expectations are met (Szymanski & Henard, 2001). If client expectations were unrealistic at the start, then managers realized that they must act quickly to bring them down to levels that matched what could be achieved, particularly as the detail of requirements included and not included in the original contract became clearer. However, managers found this to be a difficult task. One tactic mentioned several times was simply to be honest with the customers about any areas that were not achievable. Although such honesty might be initially unpalatable for the customer, it was seen to be the first step toward realigning the customer expectations again so that even a problematic project could be turned around and result in a satisfied customer. Balancing Cost, Schedule, and Scope As noted in the previous section, the first approach to dealing with anticipated problems that arose in spite of mitigation was most often to apply the control strategy of increasing person-hours with extra overtime. If increasing 58 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

11 the available person-hours was not sufficient to address the problem, managers strategies typically involved a negotiation to balance the conflicting demands of minimizing cost and time and maximizing the deliverables. Typically, the managers would first attempt to negotiate with the client for a reprioritization of time, budget, or quality in terms of what functionality could be delivered. Managers would also try to use the change control clauses of the contract to gain extra time to meet commitments. Often this involved negotiating a postponement of less critical requirements to a second phase of the project, so that all resources could be channeled into getting the essential core of the system running by the schedule date. Sometimes, in this negotiation situation, managers found that tight and immovable deadlines set by their clients could actually be used to their advantage, because the fixed nature of the schedule made it easier to negotiate a reduction in the requirements, or extra budget, when faced with difficult problems. On other occasions, however, managers saw themselves in a weak negotiating position with their clients, either because the clients were very tough negotiators or because the managers recognized some shortcoming in their own team s performance, perhaps because of an over-ambitious schedule set by the pre-sales team or an underestimation of the resources required to deal with the complexity of the project. In these situations, managers from the larger firms would turn to internal negotiation with their own senior executive to attempt to obtain extra resources, thus increasing their own budgeted costs and reducing their firm s profit margin on the project, while still meeting the schedule, budget, and deliverables commitments made to the customer. These managers were aware that they could fall back on the fact that their firms employed a large pool of permanent staff whose salaries were already accounted for in the firms overall budgets. If things got very difficult, they could negotiate the assignment of extra staff from this pool, even though it meant a cost overrun in their own projects, because other more profitable projects could take up the slack. Managers from the smaller firms faced more difficult negotiations with the client because they did not have such buffers to fall back on, and failure to meet budget targets had a very direct impact on their firms profitability. However, when faced with difficult clients who were unwilling to negotiate any balancing in terms of cost, schedule, and scope, these managers typically chose to accede to the client s demands and accept reduced profits rather than continue with protracted argument and disagreement. Indeed, even though contracts included exit clauses, these were rarely invoked, with respondents viewing the potential damage to their reputation as an overriding concern. Escalation Although escalation to the respondents managers was sometimes used concurrently with other negotiation strategies in order to gain extra leverage within the vendor firm, most often it was seen as a last resort to be used when all other attempts to resolve an internal issue had failed. However, managers were generally wary of escalating problems directly to their client s executive, because they saw such a move as destructive and likely to seriously undermine the relationship with client staff that they had put so much effort into building. Team-building Managers were keenly aware of the difference a committed team could make to the success of their projects, and worked hard to negotiate internally for the right mix of staff skills to meet the requirements. Most of the staff were salaried employees with no overtime pay available, and managers had to rely on their own motivational skills to encourage their staff to make the necessary effort to achieve results. When tight schedules had been set, managers tried to set the team s expectations from the start that the project would be hard and challenging and would require long hours of work from them. Once they had set expectations for long-working hours, managers used a leadership-by-example approach to maintain morale if their staff had to work overtime, the managers also worked similar hours. They also sought to build commitment and enthusiasm by fostering a team spirit and involving their staff in decisions about the project. Sometimes, managers felt the need to protect key staff from outside pressures, particularly from the client. They were especially protective of their key technical staff who were often seen as being poorly equipped with interpersonal skills to deal with the customer liaison side of the project. A strong concern about the welfare of the project team permeated most of the interviews. The care and attention that managers paid to maintaining a strong team spirit was striking, especially given that team morale, or lack of it, had not been identified in the literature previously as an important risk factor. But clearly the managers in this study considered a strongly motivated team to be one of the keys to success, and they worked hard to ward off any threat to their team s well-being. Research Strategies Respondents were especially alert for technical risk at the start of their projects and technical risks were discussed in 7 of the 41 initial assessment situations, and 2 of the 49 problem-arising situations. Technical risks, for example, risk associated with new or unfamiliar technology, seemed to generate interest and enthusiasm in the project staff, and managers typically tried to have part of their team make an early start, well ahead of any task deadline, on researching and self-educating about the new technology in order to identify any potential pitfalls. Indeed, this new technology risk was not one that raised many concerns, partly because project staff typically found such problems interesting and enjoyable to work on. If an unexpected technical problem arose during the project, particularly any problem that might involve a package customization, managers moved on to a negotiation strategy, with the aim of avoiding customization if at all possible, because of D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 59

12 the implications for package integrity across other customers. Thus, they would first try to convince their client that a workaround solution would be satisfactory, or that the client should adapt to whatever functionality was provided as standard in the package. Only as a last resort would they propose customization because this would then typically involve them in internal negotiation with their own developers to get the customization approved. Monitoring Strategies Although monitoring strategies were not often explicitly described by respondents, there was an ever-present undercurrent of situation awareness throughout their descriptions of their work. Some managers noted that their more implicit approach to risk management was embedded in their overall management, and involved a constant surveillance of the situation for potential problems. At the start of their projects, respondents described an often implicit assessment and evaluation of the surroundings and context of their projects that helped to alert them on particular aspects of the control strategy and the WBS that would require their attention. So for example, if the environment of the project involved overseas third parties; although this did not trigger off any new strategy, it did alert the project manager to be particularly vigilant in monitoring and controlling the deliverables expected from that third party, especially in watching out for time-zone differences that could cause additional and potentially unplanned delays in meeting deadlines. Similarly, although managers were aware that difficult contract terms and conditions could be a serious risk, they accepted whatever terms and conditions had been agreed upon at the presales stage, and then focused on management skills to monitor any risks resulting from these conditions. Although managers recognized there was often no specific action they could take about an environment risk, this did not mean that they ignored the risk. Rather, they monitored the situation but took no preventative action and indeed often made no decision about how to respond until and unless the situation actually arose. Active monitoring was particularly important to help the managers determine the level of support or resistance and general client readiness for the new system. These kinds of observations, along with assessment of their clients general attitude and organizational culture were essential cues in determining how to approach their negotiation strategies. However, equally important was self-awareness and monitoring of their own firm s position, and managers alluded to a constant alertness and reflection about what was happening both within the project with their clients, with third parties, with their own team and in the larger project context of the business environment including such things as competition, business changes, and government regulatory changes. Comparison of Risk Strategies With Prescribed Processes In contrast to the risk-by-risk response planning recommendations in the generic risk management literature (for example, Project Management Institute, 2000), managers in the present study did not attempt to plan specific responses to each of the many possible risks that might be identified for a project. Instead, they relied on (a) their own general project management skills to control the project s progress and (b) constant situation evaluation and assessment to give them sufficient warning to react when risks materialized into problems. Indeed, these managers seemed to be generally reluctant to attempt to make elaborate plans to address, particularly, the intangible and uncertain risks that were difficult to articulate and specify, and also difficult to quantify. As Pfleeger (2000) noted, such quantification is particularly difficult in situations of high uncertainty, and can often amount to little more than guesswork. For example, although managers were clearly aware that a project involving an overseas third party was a higher risk, whether that risk evolved into an actual problem depended on many uncertainties and unknowns, such as the skills of the third-party staff, their willingness and cooperativeness, other work on the third party s agenda, unknown technical problems that would surface only at the stage of testing and integration of products, and logistical issues of import and delivery. All these uncertainties could occur simultaneously and produce a major impact on the project s progress, or none of them could happen at all. Thus, similar to Moynihan s findings (2000, 2002), respondents described general control, negotiation, and monitoring strategies that would enable them to respond quickly to any problems if they arose, but that did not involve any extra work or planning at the outset that would be wasted effort if everything ran smoothly. The findings from the present study also provide support for the contention made in previous studies (Cule, Schmidt, Lyytinen, & Keil, 2000; Keil, Cule, Lyytinen, & Schmidt, 1998) that the risk-by-risk approach quickly becomes unmanageable when there are a large number of risks to consider. The results discussed here also provide an opportunity to examine the mixed advice in the literature on the use of risk management strategies in IT projects. As noted earlier, some researchers advocate a contingency approach (Barki, Rivard, & Talbot, 2001; Davis, 1982), linking the choice of risk management strategy to risk level and uncertainty in the project, while Keil et al. (1998) proposed a framework linking strategies to two key characteristics of the risks identified for the project: their perceived importance and the extent to which they could be controlled. However, Moynihan (2000, 2002) found no clear links between respondents proposed strategies and the type of situation or the level of uncertainty of the project. A close examination of the comments of respondents in the present study suggests that, rather than first identifying risks, and then choosing strategies to address the risks, they placed more emphasis on the importance of implementing certain strategies, no matter what risks might be present, in order to ensure the best possible likelihood of success. By focusing on the dual strategies of establishing strong control of the project and building a solid foundation of trust with the client at the start, these managers placed themselves in the best possible position to cope with whatever problems arose. Simply by applying these key strategies, they implicitly took action to protect the project from the most important threats: the risk of not meeting targets and problems arising from relationship difficulties with the client. Examined from this light, the mixed findings in the literature are perhaps easier to understand. The contingency approaches rely on first 60 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

13 identifying the risk and the level of risk in the project, and then selecting appropriate strategies. However, if risk-by-risk identification and quantification are largely unmanageable tasks in practice, then the strategy of being prepared for any problems that could arise, by applying a few key broad strategies, is a very pragmatic approach. Finally, one of the reasons for choosing vendor project managers for this study was the conjecture that this group of managers would likely represent the most experienced and successful group, because they are employed by firms whose core business is IT project implementation. The success of these businesses relies on the skills of their project managers, in particular, to deliver projects both to the satisfaction of the customer and within the vendor firm s budget. The most commonly cited studies about poor IT project performance the Standish Group Chaos Reports are drawn from surveys of in-house IT executive managers, reporting on primarily custom-development projects (Standish Group, 1994, 2001, 2003). The top 10 list of factors reported by the Standish Group which cause in-house projects to be challenged shows unrealistic time frames as the ninth factor, and does not include unrealistic budget at all (Standish Group, 1994). Yet, in this present study, a consistent emphasis on achieving schedule and budget targets permeated through the respondents risk management descriptions. Indeed, the risk of failing to meet schedule and budget targets was the risk most frequently mentioned by this group of respondents. This emphasis on schedule and budget risk is also in marked contrast to the risk rankings reported by Keil et al. (1998) in their recent Delphi study of project managers involved primarily with in-house custom development projects: schedule and budget risk did not appear at all in their top 11 most highly ranked risk factors. In order to explore this substantial difference in more depth, the transcripts from the in-house project managers were reviewed again. These managers described long and frustrating experience of having to acquiesce to continuous user requirements changes on in-house projects, and hence budget and schedule overruns, with little or no support from senior management to limit or control these requests. These managers welcomed the discipline that came with outsourced projects, and were keen to support the external contractors in enforcing a structured change control approach upon the users, noting that this experience would have a positive spin-off in subsequent projects. Although it is not possible to draw conclusions from only three in-house respondents, the contrast between these in-house managers resigned acceptance of schedule and budget changes on in-house projects, and the intensive focus of the vendor project managers on controlling their projects in order to achieve schedule and budget targets was striking. This suggests the possibility that some of the reported failures of IT projects to meet schedule and budget targets may well be due to a user perception on in-house projects that the budget is a soft target, and is flexible and negotiable. If the senior managers of the firm give tacit endorsement to this view by allowing interdepartmental politicking to override the original budgetary targets, then it will indeed be very difficult for the project manager to ensure that the project completes successfully on time and within budget. Thus, for in-house project managers the risk of lack of top management support is the most important of all risks (Schmidt, Lyytinen, Keil, & Cule, 2001). In contrast, once a contract has been signed with an external party, the agreed schedule and budget are more likely to be viewed by both parties as fixed hard targets that must be achieved. In this latter situation, vendor project managers are likely to perceive that they have a mandate from the client senior executive to exercise the control necessary to meet the agreed targets, and so consider that the risk from lack of top management support is low. However, the personal performance and long-term career prospects of vendor project managers will be measured by their own employers in terms of their success in delivering both a profitable project and a satisfied customer. Thus, it is not so surprising that vendor project managers place such a high emphasis on ensuring control of their projects in order to minimize the threat from schedule and budget risk. Summary and Conclusions This investigation into strategies applied during the project implementation has uncovered a substantial variation from the prescriptions in the literature, which recommends a risk-by-risk approach to plan actions to address each of the risks identified for a project. Respondents in the present study described risk management and problem-resolution strategies that can be summarized within the four broad categories of control, negotiation, research, and monitoring, which they applied generally to all their projects, regardless of specific risks that had been identified. Under this view of risk management for IT projects, the large number of potential risks is consolidated into four major categories according to which strategy is best applied to them. Thus, most of the risks placed in the project management theme fall into the category addressed by the control strategy. Most of the risks in the relationships theme are addressed by negotiation strategies. Technology and solution related risks are amenable to a research strategy, and all the remaining risks fall under the monitoring strategy. The gap between prescription and practice revealed by this study can be viewed from two perspectives: (a) the extent to which IT project managers do not adhere to formal risk management prescriptions, and (b) the inability of the formal prescriptions to provide practical guidance in the situations faced by project managers. From the first perspective, the research presented here clearly demonstrates that this group of experienced project managers did not follow formal prescriptions. However, the in-depth analysis of managers descriptions of the situations they faced suggests that, from the second perspective, formal prescriptions fall short of addressing project managers practical needs, with the result that managers developed their own pragmatic model of applying general strategies to guard against all potential problems. Although this model has much to recommend to practitioners as a workable and well-tested approach to managing risks in IT projects, it is important to recognize that the application of this practice, rather than the structured risk management prescriptions, could be the explanation for on-going problems in IT projects. Thus, further research is needed as to why managers find the formal prescriptions difficult to implement in practice. D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 61

14 The risk factors mentioned by the respondents were similar to those already identified in the literature, which suggests that one difficulty lies, not in identifying the risks, but in quantifying them, and particularly in determining their likelihood of occurrence. The pressure from both clients and top vendor management to meet tight schedule and budget targets may also result in project managers regarding time-consuming risk response planning activities to be a costly luxury. Finally, in the present study, no attempt was made to investigate any link between the risk management strategies used and ultimate success or failure of the project. Nonetheless, the model developed previously of consolidated risks and broad strategies to address those risks, is well grounded in the data analyzed in this study; and provides empirical support for conjectures by researchers such as Keil et al. (1998) and Cule et al. (2000) that a practical solution is needed for the difficulties inherent in the formal prescriptions of risk-by-risk response planning. The use of a few broad strategies enabled the managers to control and respond to a wide variety of potential problems in their projects, without having to attempt to quantify each possible risk in terms of probability and impact. Through their reliance on these broad general strategies, the managers in the present study were able to reduce a lengthy and unwieldy checklist of potential risks to a manageable form and to avoid spending limited time and resources on specifying individual responses to each particular risk. The strategies derived here from a set of project managers experienced in a wide range of projects should provide useful advice to managers about workable approaches to managing the complexities and uncertainties inherent in IT projects. References Alter, S., & Ginzberg, M. (1978). Managing uncertainty in MIS implementation. Sloan Management Review, 20(1), Argyris, C., & Schön, D. A. (1978). Organizational learning II. Reading, MA: Addison-Wesley. Baccarini, D., Salm, G., & Love, P. E. D. (2004). Management of risks in information technology projects. Industrial Management & Data Systems, 104(3/4), Barki, H., Rivard, S., & Talbot, J. (1993). Toward an assessment of software development risk. Journal of Management Information Systems, 10(2), Barki, H., Rivard, S., & Talbot, J. (2001). An integrative contingency model of software project risk management. Journal of Management Information Systems, 17(4), Boehm, B. W. (1991). Software risk management: Principles and practices. IEEE Software, 8(1), Charette, R. N. (1996a). Large-scale project management is risk management. IEEE Software, 13(4), Charette, R. N. (1996b). The mechanics of managing IT risk. Journal of Information Technology, 11(4), Choudhury, V., & Sabherwal, R. (2003). Portfolios of control in outsourced software development projects. Information Systems Research, 14(3), Cule, P., Schmidt, R., Lyytinen, K., & Keil, M. (2000). Strategies for heading off IS project failure. Information Systems Management, 17(2), Davis, G. B. (1982). Strategies for information requirements determination. IBM Systems Journal, 21(1), DuBois, D. A. (2002). Leveraging hidden expertise: Why, when, and how to use cognitive task analysis. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development: State-of-the-art lessons for practice (pp ). San Francisco: Jossey-Bass. Fairley, R. (1994). Risk management for software projects. IEEE Software, 11(3), Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, Gable, G. G. (1996). A multidimensional model of client success when engaging external consultants. Management Science, 42(8), Heemstra, F. J., & Kusters, R. J. (1996). Dealing with risk: A practical approach. Journal of Information Technology, 11, Keil, M., Cule, P., Lyytinen, K., & Schmidt, R. (1998). A framework for identifying software project risks. Communications of the ACM, 41(11), Klein, G. A., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3), Koh, C., Ang, S., & Straub, D. (2004). IT outsourcing success: A psychological contract perspective. Information Systems Research, 15(4), Lacity, M. C., & Willcocks, L. P. (1998). An empirical investigation of information technology sourcing practices: Lessons from experience. MIS Quarterly, 22(3), Lassila, K. S., & Brancheau, J. C. (1999). Adoption and utilization of commercial software packages: Exploring utilization equilibria, transitions, triggers, and tracks. Journal of Management Information Systems, 16(2), Levina, N., & Ross, J. W. (2003). From the vendor s perspective: Exploring the value proposition in information technology outsourcing. MIS Quarterly, 27(3), Martin, J., & McClure, C. (1983). Buying software off the rack: Packages can be the solution to the software shortage. Harvard Business Review, 61(6), Miles, B. M., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). London: Sage. Moynihan, T. (1996). An inventory of personal constructs for information systems project risk researchers. Journal of Information Technology, 11, Moynihan, T. (2000). Coping with requirementsuncertainty : The theories-of-action of experienced IS/software project managers. Journal of Systems and Software, 53, Moynihan, T. (2002). Coping with client-based peopleproblems : The theories-of-action of experienced IS/software project managers. Information and Management, 39, Natovich, J. (2003). Vendor related risks in IT development: A chronology of an outsourced project failure. Technology Analysis & Strategic Management, 15(4), Nidumolu, S. (1995). The effect of co-ordination and 62 D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL

15 uncertainty on software project performance: Residual performance risk as an intervening variable. Information Systems Research, 6(3), Orlikowski, W. J., & Baroudi, J. J. (1991). Studying information technology in organizations: Research approaches and assumptions. Information Systems Research, 2(1), Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications. Pfleeger, S. L. (2000). Risky business: What we have yet to learn about risk management. Journal of Systems and Software, 53, Potter, S. S., Roth, E. M., Woods, D. D., & Elm, W. C. (2000). Bootstrapping multiple converging cognitive task analysis techniques for system design. In J. M. Schraagen, S. F. Chipman, & V. L. Shalin (Eds.), Cognitive Task Analysis. Mahwah, NJ: Lawrence Erlbaum Associates. Powell, P. L., & Klein, J. H. (1996). Risk management for information systems development. Journal of Information Technology, 11, Project Management Institute. (2000). A guide to the project management body of knowledge (PMBOK guide). Newtown Square, PA: Project Management Institute. Rao, M. T. (2004). Key Issues for Global IT Sourcing: Country and individual factors. Information Systems Management, 21(3), Schmidt, R., Lyytinen, K., Keil, M., & Cule, P. (2001). Identifying software project risks: An international Delphi study. Journal of Management Information Systems, 17(4), Standish Group. (1994). The CHAOS report. Retrieved August 25, 2005, from Standish Group. (2001). Extreme CHAOS. Retrieved August 25, 2005, from Standish Group. (2003). Latest Standish Group CHAOS report shows project success rates have improved by 50%. Press release, March 25, Retrieved July 29, 2005, from Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M., et al. (2000). Practical intelligence in everyday life. Cambridge, UK: Cambridge University Press. Szymanski, D. M., & Henard, D. H. (2001). Customer satisfaction: A meta-analysis of the empirical evidence. Journal of the Academy of Marketing Science, 29(1), Taylor, H. (2004, August 6 8). Risk factors in vendordriven IT projects. Paper presented at the Tenth Americas Conference on Information Systems, New York. Taylor, H. (2005). A critical decision interview approach to capturing tacit knowledge: Principles and application. International Journal of Knowledge Management, 1(3), Wagner, R. K. (1987). Tacit knowledge in everyday intelligent behavior. Journal of Personality and Social Psychology, 52(6), Wagner, R. K., Sujan, H., Sujan, M., Rashotte, C. A., & Sternberg, R. J. (1999). Tacit knowledge in sales. In R. J. Sternberg & J. A. Horvath (Eds.), Tacit Knowledge in Professional Practice: Researcher and Practitioner Perspectives. Mahwah, NJ: Lawrence Erlbaum Associates. Wallace, L., Keil, M., & Rai, A. (2004). How software project risk affects project performance: An investigation of the dimensions of risk and an exploratory model. Decision Sciences, 35(2), Walsham, G. (1995). Interpretive case studies in IS research: Nature and method. European Journal of Information Systems, 4, Willcocks, L., Hindle, J., Feeny, D. F., & Lacity, M. C. (2004). IT and business process outsourcing: The knowledge potential. Information Systems Management, 21(3), Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks: CA: Sage Publications. HAZEL TAYLOR is an assistant professor at the Information School, University of Washington, Seattle. She holds a PhD from Queensland University of Technology, Brisbane, Australia. Prior to joining the Information School, Dr. Taylor taught at the University of Waikato in New Zealand, and at the Hong Kong University of Science and Technology, and conducted research in Hong Kong on risk management and tacit knowledge in IT projects. Her teaching and research focuses on IT project management and risk management, and information systems analysis and development, with an emphasis on tacit knowledge and decision-making in these areas. Prior to her academic career, she worked in industry with manufacturing, construction and government organizations, both as a systems manager and an IT project manager. D ECEMBER 2006 PROJECT M ANAGEMENT J OURNAL 63

16