RISK MANAGEMENT AND TACIT KNOWLEDGE IN IT PROJECTS: MAKING THE IMPLICIT EXPLICIT

Transcription

1 RISK MANAGEMENT AND TACIT KNOWLEDGE IN IT PROJECTS: MAKING THE IMPLICIT EXPLICIT by Hazel Ann Taylor BSc, MSc A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy Centre for Information Technology Innovation Queensland University of Technology 2003

2 Certificate of Completion

3 Key Words: Decision-making processes; IS implementation; IS project risk management; software packages; tacit knowledge.

4 RISK MANAGEMENT AND TACIT KNOWLEDGE IN IT PROJECTS: MAKING THE IMPLICIT EXPLICIT Hazel Taylor ABSTRACT This research addressed the need for in-depth investigation of what actually happens in the practice of risk management in software package implementation projects. There is strong official sanction in the IT literature for the use of formal risk management processes for IT projects but there is a confused picture of their application in practice. While many potential risk factors for IT projects have been identified, and formal procedures have been prescribed for the management of these risks, there has been little work investigating how project managers assess these risks in practice and what countermeasures they employ against these risks in their projects. In particular, the study used an interpretive critical decision interview approach to focus on those areas of risk management knowledge that project managers have acquired through experience, i.e. tacit knowledge. A new categorization of risk factors emanating from three sources -- vendor, client, and third party reveals risk factors not previously identified. Some of these new factors arise from the three sources noted, while others arise from the package implementation focus of the projects and from aspects arising from the location of the projects in Hong Kong. Key factors that cause problems even when anticipated and mitigated, and the most often unanticipated problems are also identified. The study further presents an examination of the studied managers risk management practices, and the strategies they use to address both potential and actual problems. This examination revealed close conformance with recommended literature prescriptions at some stages of projects, and significant variation at other stages, with strategies applied being broad and general rather than risk specific. A useful categorization of these strategies into four broad groups relating to different sets of risk factors is presented, reflecting the actual practice of respondents. Tacit knowledge was revealed throughout these investigations in the variances observed between prescribed and actual practice, and particularly from an examination of project managers decision-making practices from two different perspectives - rational and naturalistic. A hybrid decision-making model is proposed to capture the actual processes observed, and to provide prescriptive guidance for risk management practice. The investigation makes a contribution to the field of IT project risk management in three ways. First, the investigation has addressed the need for empirical studies into IT risk management practices and the factors influencing project managers in their choice and application of strategies to manage risk. Second, by examining how experienced IT project managers approach the task of managing risk in software package implementations, the study has extended our understanding of the nature of the knowledge and skills that effective IT project managers develop through experience. Third, the study makes a theoretical contribution to our understanding of IT project risk management by examining the decision-making processes followed by IT project managers from the perspective of two contrasting theories of decision-making the rational method and the Naturalistic Decision Making theory.

5 TABLE OF CONTENTS 1 Introduction Background to the research Research aims Justification for the research Methodology Limitations and key assumptions Definition of key terms Outline of thesis structure Theoretical Foundations Introduction IT software project risk management Project management Project risk management IT project risk management literature Rational decision-making Naturalistic decision making Tacit knowledge Historical beginnings Implicit learning Declarative and procedural knowledge Categories of knowledge Tacit knowledge research in the management field The present study Summary Methodology Introduction Research approach Critical incident technique and critical decision method Research procedures Sample selection (organizations and subjects) Respondent details Project details Instrumentation Data collection - interviews Refinement of interview protocol Reflections on interview process Analysis and coding processes Research Question 1: Key risk factors Research Questions 2 and 3: Risk management practices and strategies Research Questions 4 and 5: Tacit knowledge and gaps between practice and theory Researcher s perspective Ethical considerations Summary...80

6 4 Results and discussion of research question 1: Key risk factors Introduction Risk factors Comparison of risk factors with previous research Risk factors in previous literature not mentioned in this study New risk factors identified in this study Discussion of new risk factors identified in this study Frequency with which risk factors were mentioned Comparison of risk factor rankings with previous research Discussion of comparison of risk factor rankings Anticipated and unanticipated problems, and troubled projects Anticipated problems Unanticipated problems Problems causing projects to become troubled Summary of discussion: Research Question 1 Key risk factors Results and discussion of research questions 2 and 3: risk management practices and strategies Introduction Research Question 2: Risk management practices Pre-project risk management practices Risk management practices at project start-up stage Risk management practices during the course of the project Risk management practices after project completion Comparison of risk management practices with prescribed processes Research Question 3: Risk management strategies Strategies to manage risks emanating from pre-sales practices General categorization of risk management strategies Control strategies Negotiation strategies Research strategies Monitoring strategies Comparison of risk strategies with prescribed processes Summary of discussion: Research Questions 2 and Results and discussion of research questions 4 and 5: tacit knowledge and gaps between practice and theory Introduction Research Question 4: Project managers explicit and tacit knowledge Explicit knowledge applied to project risk management Tacit knowledge applied to project risk management Environmental and situational cues Research Question 5: Gaps between risk management theory and practice, and decision-making processes Decision-making processes in initial assessment situations of routine projects Decision-making processes in initial assessment situations of troubled projects

7 Decision-making processes in problem-arising situations of routine and troubled projects Summary of discussion: Research Questions 4 and Conclusions and implications Introduction Conclusions about each research question Research Question Research Question Research Question Research Question Research Question Implications for theory Implications for practice Limitations Further research Self Assessment of Research Quality Application of sound and rigorous qualitative research methods Recognized principles for conducting interpretive research Summary References Appendix A: Initial Interview Protocol Appendix B: Ethics consent package Appendix C: Practitioner Report

8 LIST OF TABLES AND FIGURES Fig. 2.1: Project risk management processes...23 Fig. 2.2: Recognition-primed decision model (after Klein et al., 1989)...33 Table 2.1: Categories and sub-sets of knowledge...43 Table 3.1: Organization details...60 Table 3.2: Project management perspectives...60 Table 3.3: Types of projects...62 Fig. 3.1: NVivo coding structure...70 Table 4.1: Summarized risk factors...83 Table 4.2: Number of respondents (out of 25) identifying risk factors...84 Table 4.3: Comparison of Risk Factors Identified in Present and Previous Studies...89 Table 4.4: New risk factors identified in the present study Table 4.5: Top 17 Risks (mentioned by at least ten respondents) Table 4.6: Comparison of risk rankings: Schmidt/Keil studies and present study Table 4.7: Problems anticipated, and number still arising Table 4.8: Unanticipated problems arising during the project Table 4.9: Problems responsible for derailing troubled projects Table 5.1: Problem situations Table 5.2: Strategies and risks they addressed Table 5.3: Strategies used in initial assessment and problem arising situations Table 6.1: Examples of tacit knowledge associated with risk management strategies Table 6.2: Strategies and stages in routine and troubled projects Fig. 6.1: Partial rational risk management processes during initial assessment of routine projects Fig. 6.2: Descriptive RPD model of processes at start of routine projects Fig. 6.3: Rational risk management processes during initial assessment of troubled projects Fig. 6.4: RPD model incorporating rational and naturalistic situation assessment during initial assessment of troubled projects Fig. 6.5: RPD model for problem arising situations in all projects Table A.1: Critical decision probe questions (adapted from Klein et al., 1989)

9 STATEMENT OF ORIGINAL AUTHORSHIP The work contained in this thesis has not been previously submitted for a degree or diploma at any other higher education institution. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made. Signed: Date:

10 ACKNOWLEDGEMENTS Thank you to my husband, Paul, for his unfailing support, encouragement and advice.

11 Chapter 1 1INTRODUCTION The research reported in this thesis was motivated by reports in the literature (for example, Boehm, 1991; Drummond, 1996; Keil & Carmel, 1995; McFarlan, 1981) that continuing problems with IT projects can be attributed, at least in part, to failure to apply rational risk management techniques that have for some time now been considered to be best practice in project management. The study set out to explore the practice of risk management in IT projects, with a focus on vendor-driven software package implementation projects. This exploratory research used a critical decision method to surface tacit knowledge held by IT project managers regarding risk management in their projects, and examined the findings from the perspectives of two contrasting theories of decision-making rational and naturalistic. The results reveal new risk factors and pragmatic strategies for managing risk that have not been identified in the literature. In addition, several gaps between practice and theory are highlighted. In these areas the predominant theoretical approach prescribed in the literature does not adequately describe the more complex realities of practice in these uncertain and changing project environments. This introduction outlines the rationale for the thesis and the nature of the problems it addresses. The chapter is organized into six sections: 1.1) background to the research; 1.2) the research questions; 1.3) justification for the research; 1.4) outline of methodology; 1.5) limitations and key assumptions; 1.6) outline of subsequent chapters Background to the research Reports about problems with IT projects have appeared regularly in the popular and academic literature for over 30 years, including several well-publicized major failures (Drummond, 1996; Lyytinen, Mathiassen, & Ropponen, 1998). Risk management practice has been identified as one critical factor of the success of IT development projects (Schmidt, Lyytinen, Keil, & Cule, 2001), and a significant stream of research has focused on identifying risk factors and prescribing risk management frameworks to aid managers in making decisions about risk factors in IT software development projects (see, for example: Alter, 1996; Barki, Rivard, & Talbot, 1993, 2001; Boehm, 1991; Keil, Cule, 11

12 Lyytinen, & Schmidt, 1998). However, reports of troubled IT projects continue, and while there is a presumption that this is at least partially due to the risk management techniques recommended in the literature not being applied in practice (Keil et al., 1998), there is little research on whether this is indeed the case. While many potential risk factors for IT projects have been identified, and formal procedures have been prescribed for the management of these risks, little is known about how project managers assess each of these risks in practice and what countermeasures they employ against these risks in their projects. In particular, we know little about the knowledge and skills being applied by more experienced project managers, nor of the extent to which these knowledge and skills derive from formal education or training. Software packages are especially interesting in the context of IT risk management practice, in that their use is purported to ameliorate or avoid many of the risks associated with custom developments (Lassila & Brancheau, 1999; Martin & McClure, 1983). With the trend away from custom system development towards generic software packages (Russo, 2000), interest in specific risk issues related to software package implementation projects is increasing (Parr, Shanks, & Darke, 1999; Sumner, 2000). And while it appears that these projects have some unique risks (Sumner, 2000), again there is little research addressing the particular practices of managers of these projects (Davis, 1998; Gable, 1998). One common theme in the risk management literature referred to above is that projects with higher levels of uncertainty and complexity are more risky. Research has investigated how experienced professionals make decisions in poorly-defined and uncertain circumstances in a variety of contexts, including fire-fighting, nursing, the armed forces, and hardware and software design (Klein, 1999; Vicente, 1999). This stream of research has led to the development of a descriptive theory - Naturalistic Decision Making (Zsambok, 1997), and a model of decision-making - the Recognition-Primed Decision model (Klein, 1993; Klein, Calderwood, & MacGregor, 1989), which propose that situation assessment, tacit knowledge (i.e. knowledge gained through experience), and judgment are important factors in decision-making in real-life settings, and that the rational processes assumed by academic prescriptive models and frameworks for decision-making (see, for example, Barki et al., 2001; Boehm, 1991; Charette, 1996b; Fairley, 1994; Heemstra & Kusters, 1996; Keil, 1995; Powell & Klein, 1996) are often not applied in practice. Moreover, studies of skilled practitioners in areas such as business management, 12

13 sales, and military leadership have shown that, when dealing with complex and ill-defined situations, these experts supplement their formal education with tacit knowledge and judgments derived from their experience (Sternberg & Horvath, 1999). Rational decision processes are a basic assumption of the risk management frameworks referred to earlier, and are typically learned through a formal education process. However, in practice, experienced project managers of complex and uncertain software projects may rely more on judgment and tacit knowledge than on the prescribed rational frameworks. The thesis reports on an exploratory and descriptive field study of current practice in risk management of software package implementation projects. In particular, the aim was to gain an understanding of the dynamics of the risk management process and the actions taken with respect to possible risks for a given project in its situational context. This study of current practice provides a useful window into inherent work constraints that limit the applicability of the prescriptive models found in the literature. Further, the study: (1) highlights aspects of current practice that may undermine the effectiveness of risk management practices; (2) lays the foundation for a formative model encompassing aspects of both rational and naturalistic decision-making that can provide better descriptions of actual practice; and (3) offers more practical prescriptive guidance in order to support practitioners effectively Research aims The aims of the research were three-fold. The study explored, firstly, the key risks that experienced project managers attended to in software package implementation projects, and how they managed these risks in practice. Secondly, the study identified knowledge (tacit or explicit) that experienced project managers used in order to plan for and address critical risk situations that arose during the course of their projects. Thirdly, risk management practices were compared with generally accepted prescriptions in the literature to identify gaps between theory and practice. These gaps were examined from the perspective of both rational and naturalistic models of decision making in order to investigate how well each of these theories described and explained practices described by respondents. Specifically, the thesis focused on vendor project managers working primarily with software package implementation projects, and investigated the following inter-related research questions: 13

14 1) What key risks do project managers attend to in software package implementation projects, and how do these compare with the risk factors identified in the literature for software packages as well as for custom development? 2) What risk management practices do project managers employ when planning software package implementation projects, and how do these compare with prescriptions in the literature? 3) What strategies do project managers use to address risks in IT projects both to prevent problems in the first place and to deal with the problems if they do arise (i.e. if the risks identified as potential problems turn into actual problems)? 4) What knowledge (tacit or explicit) do experienced project managers use in order to plan for and address critical risk situations that may arise during the course of software package implementation, and in particular, what environmental and situational cues do managers attend to when managing their projects? 5) What gaps are there between actual practice and the rational risk management processes as recommended in the IT risk management literature, and to what extent can decision making theories such as Naturalistic Decision Making explain any such gaps? The underlying theoretical foundations for these research questions include project risk management, tacit knowledge and decision-making. The literature related to these underpinning areas is discussed in Chapter 2, and a context and motivation for the present research is established in that chapter. A brief summary of the rationale for the current study is given in the next section Justification for the research With the growing demand for high quality information systems in organizations, and with the increasing use of commercial packages rather than custom-developed software (Russo, 2000), strategies for managing risk during the implementation of these packages have become increasingly important. The exploration of how experienced project managers approach the task of managing risk in the implementation of software packages on client sites provides insights for understanding when and why recommended risk management processes are not always being applied in practice. Improved understanding of the knowledge, both explicit and tacit, that successful IT project managers draw on in 14

15 determining strategies to address risk in implementation projects can aid researchers and practitioners in developing better implementation project risk management techniques, and can also be used to develop specific training programs for novice project managers. The investigation reported here makes a contribution to the field of IT project management in three ways. Firstly, the investigation addresses the need for more empirical studies into IT risk management practices (Schmidt et al., 2001) and the factors influencing project managers in their choice and application of strategies to manage risk. Secondly, by examining how experienced IT project managers approached the task of managing risk in software package implementations, the study extends our understanding of the nature of the knowledge and skills that effective IT project managers develop through experience. Thirdly, this research contributes to a better understanding of discrepancies between theoretical prescriptions and practical applications of risk management in package implementation projects, by examining the processes and factors identified in the light of both rational and naturalistic decision-making theories. The next section outlines the methodological approach chosen for the research Methodology The exploratory and descriptive nature of the research required research methods that met the need for contextual information, as the contexts of the situations examined were of paramount importance in gaining an understanding of the dynamics of the risk management process and the decisions made with respect to possible risks for a given project. Investigations of this type lend themselves to broadly interpretive methods of research (Walsham, 1995). Thus, a primarily qualitative analysis approach was used. The study used a critical decision interview approach (DuBois, 2002; Klein et al., 1989) to explore and describe the risk management approaches used by experienced IT project managers during software package implementation projects. This semi-structured interview method enabled the elicitation of critical risk situations illustrating key constraints and environmental and situational cues that could be involved in project managers risk assessment and risk management of projects. Interview transcripts were analyzed with a qualitative content analysis procedure, using the software package NVivo version 2.0, to support and manage the detailed coding and analysis process. The risk management decisions made by IT project managers during their projects were examined from two perspectives using frameworks drawn from standard risk management guides 15

16 typically recommended to practitioners (PMI Standards Committee, 1996), and from naturalistic decision making theory and the Recognition-Primed Decision model developed by Klein, Calderwood, and MacGregor (1989) Limitations and key assumptions The purpose of this research was to learn more about risks and risk management in software package implementation projects, and the knowledge, both tacit and explicit, that project managers draw on when managing these project risks. The methodology relied on a variation of the critical incident interview approach to elicit information about project managers approaches to risk management, and as such was reliant on their self-reports and recollections of their plans and actions and the circumstances surrounding these. An assumption of this approach is that people can and will provide information that draws not only on their explicit knowledge about what ought to be done, but also on their tacit knowledge, gained from experience, about what actually works. While it is acknowledged that one aspect of tacit knowledge is that it is difficult to articulate, previous use of the critical incident technique (Sternberg et al., 2000) and the critical decision method (Klein et al., 1989) for eliciting knowledge has demonstrated that these approaches can describe the function served by tacit knowledge in proficient task performance. All of the project managers interviewed for this research were based in Hong Kong, and there is a possibility that the findings were country/culture specific. To some extent this limitation is mitigated by the international nature of the respondents, all of whom had studied or worked abroad during some stage in their careers, and the international nature of the software packages the managers were working with, since, for the most part, these packages are marketed globally. While some of the risks identified by these managers were clearly related to their Hong Kong location, many of the risks correspond to those reported in the literature previously, and the location-specific risks also have implications for managers in other countries, particularly in view of the increasing trend towards multinational projects that cross traditional country/culture boundaries. Finally, the study focused solely on packaged software implementation, and the exploratory nature of the research and use of in-depth interviews placed practical limitations on the sample size. However, the in-depth analysis of information across a range of informants and organizations in the packaged software field has enhanced our knowledge of risk management practices within this range of empirical circumstances, and 16

17 hence can increase our confidence in applying this knowledge to new settings sharing similar circumstances to those investigated in this research (Baskerville & Lee, 1999). In particular, while this research has focused mainly on software package implementation projects, these projects share many features in common with custom development projects and a future research opportunity is to explore the extent to which the findings from the present study are applicable in other IT project settings Definition of key terms There is some variation in the literature in the definition of certain key terms. While detailed discussion of these key terms and differences in definition can be found in chapters 2 and 3 of the thesis, a brief glossary of key definitions that I have applied in this study is given to aid the reader. Critical Decision Method: A variation of the critical incident technique that has been developed specifically for knowledge elicitation applications, where the aim is to reveal aspects of expertise such as the critical cues used in making perceptual and conceptual discriminations, and the underlying basis for judgment decisions (Klein et al., 1989). Critical Incident Technique: A method for obtaining specific, behaviourally focused descriptions of job performance developed by Flanagan (1954). The critical incident technique has been applied to the elicitation of tacit knowledge related to job performance by a group of researchers working to identify tacit knowledge in applied settings (Sternberg et al., 2000; Sternberg & Horvath, 1999; Sternberg & Wagner, 1986). Explicit Knowledge: Knowledge that is formally learned and easily articulated. Explicit knowledge is held in repositories such as libraries, books, formal data media, written rules and procedures and is acquired through formal learning procedures such as schooling, reading, and formal training. Implicit Knowledge: (Also referred to as tacit knowledge). The broad category of knowledge that individuals find difficult to articulate and have learned by experience, by practice ( doing ) or by osmosis. Researchers have used different degrees of granularity, and different terms, in discussing types of implicit knowledge. See Section for a detailed discussion. 17

18 Project Executive: A senior manager, referred to as a programme manager in some firms, who has responsibility for supervising multiple project managers within the vendor firm and overseeing the projects they are working on. Project executives typically take a strategic and long-term management perspective within their firms, act as executive sponsor or steering committee head for the projects under their oversight, are responsible for balancing the competing resource needs of their projects within their firms, and deal with high-level contract negotiations with client firms. Project Manager: The manager who is assigned overall responsibility for a specific project. In vendor-driven projects, typically there will be a vendor project manager with responsibility for the project from the vendor perspective, and a client project manager with responsibility for the project from the client perspective. Vendor project managers were the focus of the present research. Vendor project managers manage the daily activities of the project, co-ordinate tasks, manage the project team, liaise with their counterparts in the client firm, and are responsible for ensuring the project schedule, budget, and requisite quality are achieved. Risk: Potential problems or issues that may arise and adversely impact on the progress or outcome of a project. Note that while typical definitions of risk with respect to project management relate to potential problems that could impact on the project itself, the vendor respondents in this study also consider a risk to be any potential problem or situation arising from involvement in the project that could adversely impact on their own firm See Section 4.3.2, Environment factors for more discussion. Risk Management Practice: An organization s policies and procedures regarding the management of risk, in particular the planning processes prescribed to evaluate and address risk at the start of a project, and prescribed procedures for managing risk during the project. Risk Management Strategy: Specific strategies applied by managers to address particular risks during the course of the project. These strategies may be prescribed as part of the organization s standard risk management practice, or they may be tactics that project managers have developed through their own experience. Tacit Knowledge: see Implicit Knowledge 18

19 1.7. Outline of thesis structure In this introductory chapter I have set the context and foundations for the thesis, and introduced the research aims and questions. I have presented a justification for the research, briefly overviewed the methodology, and discussed key assumptions and limitations. The detailed description of the research is contained in subsequent chapters as set out in the outline of the thesis structure below. The thesis is structured into eight chapters, including this introduction. In Chapter 2 I discuss the theoretical foundations underlying the research questions investigated in this thesis. In particular, I review the literature on project risk management, with a particular emphasis on risk management research in the IT field. I discuss the assumption underlying this project risk management literature of a rational decision making approach, and consider an alternative theory of decision-making, Naturalistic Decision Making, that incorporates a focus on knowledge gained from experience tacit knowledge. I then examine research in the cognitive psychology field on tacit knowledge, and explore how tacit knowledge has been investigated in related fields such as business and management. I show how researchers working from very different starting points decision-making on the one hand, and tacit knowledge on the other hand have converged in developing a practical method for examining the question of knowledge gained from experience. This method forms the basis of the research approach of this thesis. Finally, I provide a basis for the aims of the present research and discuss in detail the research questions noted in Section 1.3 above. In Chapter 3 I discuss the chosen research approach, the critical decision interview method, and show why it is an appropriate method for the investigation of the research questions. I then detail the research procedures followed, and explain the analysis approach. Demographics and simple descriptive results are reported in Chapter 4, while Chapters 5, 6, and 7 contain results and discussion for the five research questions discussed above. I present conclusions and implications from the research in Chapter 8. 19

20 Chapter 2 2THEORETICAL FOUNDATIONS 2.1. Introduction In this chapter I discuss the theoretical foundations underlying the research questions that were outlined in Chapter 1. Section 2.2 contains a review of the literature on project risk management, with a particular emphasis on risk management research in the IT field. Section overviews project management in general, and Section focuses on project risk management. In Section 2.2.3, I review the IT project risk management literature. In Section 2.2.4, I discuss the underlying assumption of a rational decision making approach to risk management that pervades the risk management literature. In Section I discuss an alternative approach to decision making, Naturalistic Decision Making (NDM), based on empirical studies of decision makers in real-life situations. NDM provides a descriptive model of decision-making, the Recognition-Primed Decision model (RPD), which will be explored for its potential to shed light on the research questions of this thesis. NDM introduces the concept of knowledge based on judgment and experience tacit knowledge, and in Section 2.3 I discuss tacit knowledge, and how tacit knowledge has been investigated in related fields such as business and management. Section overviews the historical beginnings of the study of tacit knowledge, and in I discuss the related concept of implicit learning. In Section I review Anderson s (1982) skill acquisition framework of declarative and procedural knowledge, and in Section I draw together various categories of knowledge that have been proposed by researchers in the business and management arena. In Section I focus on empirical tacit knowledge research in the management field conducted by the group of researchers working with Sternberg and Wagner (Sternberg et al., 2000; Sternberg & Horvath, 1999; Sternberg & Wagner, 1986; Wagner, 1987; Wagner, Sujan, Sujan, Rashotte, & Sternberg, 1999). These researchers have used a critical incident approach to examining tacit knowledge that is very similar to the critical decision method developed by NDM researchers (Klein et al., 1989). These two methods form the basis of the chosen research approach discussed in Chapter 3. 20

21 Finally, in Section 2.4, I show how the literature reviewed in the previous sections forms a basis and context for the aims of the present research and discuss in detail the research questions introduced in Chapter 1. Section 2.5 provides a brief summary of the chapter IT software project risk management Reports of problems with IT projects abandoned projects, cost and schedule overruns, failure to deliver required functionality have appeared regularly in the popular and academic literature for over 30 years (Boehm, 1991; Drummond, 1996; Keil, 1995; McFarlan, 1981). During this period there has also been a steady flow of advice for IT project managers in both the academic and practitioner literature on IT project management, development methodologies, and risk management techniques. IT software project management includes project risk management and is a subset of project management in general. In the following sections I first overview project management in general, with illustrations from IT projects (Section 2.2.1), and I then discuss the generally accepted approaches to project risk management (Section 2.2.2). Following this, I review the two major strands, prescriptive and descriptive, of IT project risk management literature, and researchers recommendations for successful management of IT projects (Section 2.2.3). Finally, I discuss the possible implications of the premise of rational decision-making that underlies accepted general and IT project management techniques (Section 2.2.4) and (Section 2.2.5) review an alternative research approach to decision making - naturalistic decision making, showing how this approach is relevant for the concerns of this thesis Project management The body of knowledge on project management and project risk management is extensive and well established (e.g. PMI Standards Committee, 1996), and provides a basis for understanding project management techniques that are recommended for the IT industry by researchers such as, for example, Boehm (1991), Heemstra and Kusters (1996), and Powell and Klein (1996). Generally accepted definitions of projects, project life cycle, project management, and project risk management are summarized next. Projects are distinguished from operations in that projects are temporary and unique activities, with every project having a distinct beginning and end, producing a unique 21

22 product or service. Operations, in contrast, are on-going and repetitive activities, with no distinct end. While projects may have a repetitive aspect in that a new project may seek to produce a similar outcome to previous projects e.g. a new building or a new software application the specific outcome for each new project is unique in that its particular scope, context and requirements will be different from previous similar projects. Projects typically have specific goals, aim to achieve some sort of change, are constrained by limited resources, and require co-ordination in order to meet the goals. All projects go through a project life cycle, comprising several phases including initiation, planning, and implementation, with some further breakdown within each phase. For IT projects, the systems development methodology chosen for a software development project is an instance of a particular approach to the project life cycle. The project life cycle gives a higher-level view for overall control and management of the project, while the choice of systems development methodology determines how the necessary technical tasks are carried out (McLeod & Smith, 1996 pg 73). The initiation phase of the project life cycle typically corresponds to initiation and feasibility stages of traditional systems development methodologies. The planning phase of the project life cycle occurs if management approves going ahead with the project, and would typically include deciding upon the specific methodological approach to take for the systems development (for example, waterfall approach, prototyping, rapid application development). The implementation phase of the project life cycle then utilizes the chosen methodology to complete the project. Project management involves the application of general and specific management knowledge and skills to project activities in order to meet the stated goals of the key stakeholders in the project. Such goals typically include scope, cost, time and quality objectives. IT projects are sometimes claimed to be less amenable to traditional project management techniques because of the high levels of uncertainty in terms of knowledge about, for example, specific requirements, how long a particular application will take to program, and issues relating to the acceptance of the new application by the organization (McLeod & Smith, 1996 pg 7). However, most IT project management researchers argue that more rigorous application of project management techniques, and in particular, risk management techniques, is the best approach to dealing with the uncertainties that are typical of IT projects (see, for example, Barki et al., 2001; Boehm, 1991; Keil et al., 1998). Project risk management is one of the project management knowledge areas applied over the 22

23 whole project life cycle, with risk assessment being part of the planning phase, and risk monitoring continuing throughout the implementation phase Project risk management Project risk management includes four major processes, illustrated in Figure 2.1 and discussed below (PMI Standards Committee, 1996). RISK IDENTIFICATION RISK ASSESSMENT Risk Analysis Risk Prioritization RISK RESPONSE PLANNING Risk Elimination Risk Mitigation Risk Acceptance Contingent Action Planning RISK MONITORING Progress Feedback Progress Analysis Corrective Action Fig. 2.1: Project risk management processes The risk identification process involves identifying both the sources of potential risk to the progress or outcome of the project, and the risk symptoms, sometimes called triggers or cues, that can provide early warning of an actual risk event. Risks are typically identified using some sort of checklist, specific to the application area of the project. Risk interviews of key personnel can also help to identify atypical project risks not included on the checklists. Once the potential risks have been identified they must be assessed in terms of the likelihood of their occurrence, and the potential impact if they do occur. Typical assessment techniques include quantitative analysis of probabilities of occurrence, and cost of impact if the risk event does occur, in order to give an expected monetary value (probability times cost) for each potential risk. Tools such as simulations and decision trees are used to aid this analysis when probability and cost figures are available, but expert judgment is also used if the probabilities or costs are difficult to estimate because of uncertainties about the potential risk. When expert judgment is used, likelihood of risk and impact are usually estimated on a low-medium-high scale. The expert judgment approach is frequently applied in IS projects because of the typically high levels of uncertainty in these projects (McFarlan, 1981; McLeod & Smith, 1996). The analysis of the risks results in a prioritized list of risks, and also an overall assessment of the level of riskiness of the whole project. In IT projects, the level of riskiness of a project is often considered as a contingent factor in determining the management approach to take for the project. For example, Davis (1982), McFarlan (1981) and Barki et al. (2001) all 23

24 recommend a contingency approach to IT project management depending on the degree of uncertainty (i.e. riskiness) about the project. Some potential risks can also represent potential opportunities depending on the outcome of managing the risk (PMI Standards Committee, 1996), and hence a thorough risk assessment phase also includes consideration of potential opportunities and how to take advantage of these. For example, in IT projects, working with a new programming language could be considered a potential risk to the project s success because of the team s inexperience with the language, but it also can open up new future opportunities if the project team can master the skills required with the new language. However, typically in the IT risk literature, the focus is solely on the negative aspects of risk, with little or no consideration of the broader range of possible outcomes (Lyytinen et al., 1998). The list of risks identified, prioritized as discussed above, is the input for the risk response-planning phase, during which plans are developed to address the various risks (and take advantage of potential opportunities). Each risk is considered, to determine what response is required. Preventive steps are taken to eliminate or mitigate the impact of high priority risks, while lower priority risks may simply be regarded as acceptable and no further action is taken. Contingent actions are planned to occur in the event that a particular risk event actually happens in spite of the preventive steps. Finally, in the risk monitoring stage, the project is checked for risk symptoms, feedback on progress is analyzed and compared with the original plan, and corrective measures are taken as necessary in accordance with the pre-planned contingent actions. Given the many problems that continue to recur in the IT project arena, the question of how project risk management is handled in the IT field is clearly of interest and is the aspect of IT project management focused on in this thesis. I next review the recommendations in the IT literature for successful project risk management IT project risk management literature The IT project risk literature falls into two major strands, the descriptive strand, which has focused on the first stage of risk management, identifying risk factors in software development projects; and the prescriptive strand, which gives advice on how to carry out risk management, and in some cases includes contingency frameworks for deciding when to apply certain risk management approaches. 24

25 Descriptive research on IT risk factors. The descriptive risk factors strand of research has focused on developing complete and comprehensive check-lists of risk factors to be considered when planning and managing an IT project. There is now a substantial body of work on the typical risk factors faced by software project managers, and also the priorities placed on these risk factors by managers (Alter & Ginzberg, 1978; Barki et al., 1993; Boehm, 1991; Heemstra & Kusters, 1996; Schmidt et al., 2001; Sumner, 2000). The risk check-lists vary in detail and emphasis, but the risks identified all generally fall within Sumner s (2000) eight categories of i) organizational fit, ii) skill mix, iii) management structure and strategy, iv) software systems design, v) user involvement and training, vi) technology planning, vii) project management, and viii) social commitment. While the risk check-lists developed by researchers are extensive and based on surveys and interviews with experienced and successful IT project managers, these checklists report what the managers say they attend to when considering IT projects, not how they actually use such check-lists. Perhaps the lists tell us more about managers views of what went wrong with their projects, than about what they actually do to try and ensure a project stays on track. We know very little about what managers actually do and how they assess and manage the risks of a given project. Whether project managers actually use risk checklists or not, they would seem a useful aid for assessing potential project risks. Yet, in practice, they are not always so practical. As Powell and Klein (1996) note, there may be a tendency to assume the checklist is complete, and therefore to overlook possible risks specific to a given project, but not included in the checklist. This is especially of concern since the reported checklists vary considerably in the risk factors on which they focus. For example, many studies cite top management support or the existence of a project sponsor as a crucial factor of success (Heemstra & Kusters, 1996; Moynihan, 1997; Parr & Shanks, 2000; Schmidt et al., 2001; Sumner, 2000). These researchers, and Barki et al. s (1993) initial literature survey of risk factors, all included top management support as a risk variable. However, this variable was dropped from the final measure of software development risk offered by Barki et al. as a tool for organizations wishing to undertake risk assessment of projects, because it did not load on any of the five factors they identified in their factor analysis. Similarly, Boehm s (1991) often cited top ten risk items does not include the lack of top management support or a project sponsor as a risk factor. Project managers who base their risk assessment on McFarlan s (1981) approach will find that he provides a primarily task and technology focused view, while more recent checklists place less emphasis on 25

26 technology and more on organizational, environmental and individual characteristics (Schmidt et al., 2001). Further, identifying risks on a checklist is not a substitute for actually taking action on the risks (Powell & Klein, 1996). Schmidt et al. note we still lack a basic understanding of why certain managerial actions serve to mitigate multiple risks, and under what conditions these actions are no longer effective. There is also little known about the extent to which the risk taxonomies and checklists discussed above are applicable in all software development situations and whether different taxonomies are needed for different project contexts (Moynihan, 1997). For example, Heemstra and Kusters (1996), drawing on evidence from five action research projects, note that the generic 36 risk factor checklist they developed was not suitable for one of the projects, which involved a package implementation and required relatively more emphasis on human aspects and less on technical development risks. And after analyzing seven case studies of firms implementing enterprise resource planning (ERP) systems, Sumner (2000) concluded that nine of the twenty detailed risk factors identified were unique to ERP systems and thus had not been adequately covered in previous checklists. These particularly relate to, firstly, the client s commitment to change business processes and data definitions to match the ERP package; secondly, the ERP firm s ability to obtain and retain personnel skilled in both the specific ERP technology and the client business or alternatively to work effectively with outside consultants providing these skills; and thirdly, the adequacy of training given to both the ERP firm s staff and the client firm s staff. These risk factors correspond to Parr and Shanks (2000) critical success factors of using vanilla ERP (i.e. change the client processes, not the ERP package), and having a balanced team (i.e. a mix of internal/external and business/technical expertise). Parr and Shanks do not mention training. Both Sumner and Parr and Shanks also identify full-time commitment from key client personnel and top management support as important factors but Sumner notes these are not unique to ERP projects. Thus the risk taxonomies and suitable strategies for software package implementation are likely to be different from those for in-house software development. We would, however, expect software package implementation risks to be closer to the bespoke project risks that Moynihan (1997) identified, since both package implementations and bespoke projects address an interface between two organizations vendor-client for packages and external developer-client for bespoke development. In 26