A SOUTH AFRICAN PERSPECTIVE ON INFORMATION TECHNOLOGY SOFTWARE PROJECT FAILURE

Transcription

1 A SOUTH AFRICAN PERSPECTIVE ON INFORMATION TECHNOLOGY SOFTWARE PROJECT FAILURE Jurie Smith. PMP. Senior Manager PricewaterhouseCoopers. ABSTRACT We learn from success, but we can learn much more from failure. Further, it is far better, cheaper, and of course, wiser to learn from other people s failures than our own. Making this point in a formal manner, the great German statesman, Otto von Bismarck, observed: Fools you are to say you learn by your experience. I prefer to profit by others mistakes and avoid the price of my own. (Kharbanda, et al, 1996). IT Practitioners all over the world are aware of I.T. software project failure statistics. Groups like Gartner and Standish have done extensive research on the topic, however, very little, if any, research has been done on IT software project failure in South Africa. As part of my MBA, I researched IT Software Project failure in South Africa and he has unearthed some very interesting findings that he would like to share with the audience. His paper will focus on: Defining failure International failure statistics The South African results What about the future? 1. Background Information Technology Software Project failure is rarely studied! The above statement will surprise most Information Technology practitioners because it is generally believed that there are ample studies on the subject. Information Technology practitioners will be quick to highlight the research of The Standish Group, Gartner, possibly KPMG and? In addition when studies are done the full results are unobtainable to researchers because the reports cost thousands of dollars and once converted to South African currency, it is simple unaffordable. As part of my MBA, I am in the process of submitting a dissertation entitled Why Information Technology Software Projects fail in South Africa. As The Standish report is the most comprehensive, I decided to base my research design on the format presented by the Standish Group and supplement the failure reasons with the research of The Gartner Group, KPMG and the additional literature that was available. 2. Why Study Information Technology Software Projects? Is it not more advantageous and more beneficial to investigate only the success of projects? This will enable us to construct checklists, roadmaps and recipes on how to successfully implement Information Technology projects. The answer is simply because we learn from success, but we can learn much more from failure. We should look at both sides of the coin and analyse our successes and also investigate our failure. Only once we have harvested this knowledge and once we have a thorough understanding of successes and failure can we proceed to incorporating this knowledge into best practice. Paper presented at African Rhythm Project Management Conference April 2002, Johannesburg, South Africa Hosted by: Project Management Institute of South Africa (PMISA): ISBN Number:

2 In addition, we should also consider the external environment by continuously taking cognisance of lessons learned. It is far better, cheaper, and of course, wiser to learn from other people s failures than our own. Making this point in a formal manner, the great German statesman, Otto von Bismarck, observed: Fools you are to say you learn by your experience I prefer to profit by others mistakes and avoid the price of my own. Dorsey (2001: 1) claims that Information Systems projects frequently fail and that this is a catastrophe. Depending upon which academic study referenced, the failure rate of large projects is reported as being between 50%-80%. This occurs because of the natural human tendency to hide bad news and consequently the real statistic may be even higher. It is interesting to note that the studies that have been done report a variety of results. One reason for the varied conclusions is that most failed projects are never studied, even by the organisation that experienced the failure. Having wasted so much on a fruitless venture, few organisations will invest more time or money to collect and analyse additional data, whereas any data that had been collected may be massaged or hidden to protect careers or reputations. May (1998: 1) Continuously improving the success rate of our project is of paramount importance for the survival of our organisations. 3. Failure Defined What is failure? According to the Oxford English Dictionary Allen (1990: 421) failure is a lack of success, an unsuccessful attempt or thing, non-performance, breaking down or ceasing to exist, or non-occurrence. The Standish Group (1995: 2) has broken failure down into two descriptions. Project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified and project impaired: The project is cancelled at some point during the development cycle. Gilbreath (1986: 3) argues that the definition of failure is merely the absence of success. Furthermore it has to be recognised that failure is a perception rather that a physical reality, in that we have failed only we, or others, perceive that we have actually failed. No matter how much we feel we have actually succeeded, if the others around us perceive failure, then failure has occurred. People perceive failure when their expectations are not met, of expected or planned accomplishment, irrespective if the expectations are reasonable or not. Failure exists when what should have happened did not happen, thus Failure = Unmet expectations. Glass (1998: 13) referencing the KPMG report Runaway projects causes and Effects defines a runaway or failed project as a project that has failed significantly to achieve its objectives and/or has exceeded its original budget by at least thirty percent (Kharbanda et al, 1996: 37) states that in the old days project managers used the triple constraint to evaluate a project at completion. The three constraints were: time the project had to come in on or under its initially scheduled timeframe money the project had to be completed within its budget limits, and performance the end result had to perform in the manner in which it was intended.

3 This method is very simple to use but unfortunately, it has also become simplistic in the modern business world. He recommends that a fourth parameter of customer satisfaction be added. Client satisfaction refers to the idea that the project is only as successful as it satisfies the needs of the customer. Kerzner (1998: 6) reasons that very few projects are completed within the original scope of the project and as a result scope changes are inevitable. Scope changes must be kept to a minimum and that the changes that are required must be approved by both the project manager and the customer/user. As can be seen from the above definitions, there are a variety of definitions for failure. Because my research was primarily based on The Standish Group, I applied the following definition for my research: A Challenged project is completed and operational but over-budget, or over the time estimate, or delivered with fewer functions than originally specified or the general perception is that the project is a failure. A cancelled project is considered to be a failure if the project is cancelled at some point during the project life cycle. The combination of challenged and cancelled projects therefore constitutes project failure. 4. International Statistics and Reasons for Failure Goldstein (2001:2) presented a paper at the 2001 National Project Management Institute Conference. He said the consulting group KPMG found in its 1995 study that of the projects deemed to have failed by the study respondents: 75% exceeded their schedule by 30% or more, and more than 50% exceeded their budgets by a substantial margin. Glass (1998: 13) also referenced the KPMG report Runaway projects causes and Effects conducted research on 250 United Kingdom based IT companies. The report defined a runaway or failed project as a project that has failed significantly to achieve its objectives and/or has exceeded its original budget by at least thirty percent. KPMG presented the following additional findings: Schedule overruns were eighty nine percent more common than cost overruns which were 62 percent The use of packaged software did not help in reducing the incidents of runaways. The runaway projects that were studied consisted of forty seven percent mixed custom or packaged software, twenty four percent custom software, and twenty two percent packages software Most of the projects failed for various reasons and there may or may not have been a dominant cause, but there were several problems contributing to many of the runaways. Runaway projects showed their true colours early in the project history. More than half started showing failure symptoms during system development, and twenty five percent already showed the above mentioned symptoms during the planning stage Although the respondents thought that there would be runaways in the government and financial sectors, and fewer in service and manufacturing, the findings were that all sectors were equally susceptible.

4 Taylor (2001: 1) had his research on Information Technology project failure published by the British Computer Society. The research involved questioning of 38 members of the British Computer Society, the Association of Project Managers and the Institute of Management. He concluded that out of 1,027 projects covered only 130, or 12.7 per cent, were successful. The 130 successful projects consisted of 2.3 percent development projects, 18.2 percent maintenance projects and 79.5 percent data conversion projects. It is interesting to note that development projects made up half the 1,027 covered by the research, thus out of 513 development projects, only 2.3 percent were successful. These findings compared well with those of the Standish Group, which found a success rate of around 16.2 per cent in a 1995 study. The Standish Group (1995: 1) conducted research into reasons why IT software projects fail. They conducted a survey to which IT executive managers responded. The respondents operated in large, medium, and small companies across major industry segments, e.g., banking, securities, manufacturing, retail, wholesale, heath care, insurance, services, and local, state, and federal organizations. The total sample size was 365 respondents and represented 8,380 applications. The Standish Group further segmented these results by large, medium and small companies. A large company is any company with greater than $500 million dollars in revenue per year, a medium company is defined as having $200 million to $500 million in yearly revenue, and a small company is from $100 million to $200 million. The study classified projects into three resolution types, where Resolution types 2 and 3 therefore constituted project failure. Standish Group (1995: 4): Resolution Type 1, or project success: The project is completed on-time and on-budget, with all features and functions as initially specified. Resolution Type 2, or project challenged: The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified. Resolution Type 3, or project impaired: The project is cancelled at some point during the development cycle. The Standish Group (1995: 1-6) found that in the United States, companies spend more than $250 billion each year on IT application development of approximately 175,000 projects. A great many of these projects will fail. The Standish Group research shows a staggering 31.1% of projects will be cancelled before they ever get completed. Further results indicate 52.7% of projects will cost 189% of their original estimates. Based on this research, The Standish Group estimates that in 1995 American companies and government agencies will spend $81 billion for cancelled software projects. These same organisations will pay an additional $59 billion for software projects that will be completed, but will exceed their original time estimates. On the success side, the average is only 16.2% for software projects that are completed on-time and on-budget. In the larger companies, the news is even worse: only 9% of their projects come in ontime and on-budget. And, even when these projects are completed, many are no more than a mere shadow of their original specification requirements. Projects completed by the largest American companies have only approximately 42% of the originally proposed features and functions. Smaller companies do much better. A total of 78.4% of their software projects will get deployed with at least 74.2% of their original features and functions.

5 This data may seem disheartening, and in fact, 48% of the IT executives in the research sample feel that there are more failures currently than just five years ago. The good news is that over 50% feel there are fewer or the same number of failures today than there were five and ten years ago. Only 9% of projects in large companies were successful. At 16.2% and 28% respectively, medium and small companies were somewhat more successful. A whopping 61.5% of all large company projects were challenged (Resolution Type 2) compared to 46.7% for medium companies and 50.4% for small companies. The most projects, 37.1%, were impaired and subsequently cancelled (Resolution Type 3) in medium companies, compared to 29.5% in large companies and 21.6% in small companies. One of the major causes of both cost and time overruns is restarts. For every 100 projects that start, there are 94 restarts. This does not mean that 94 of 100 will have one restart, some projects can have several restarts. For example, the California Department of Motor Vehicles had many restarts. Equally telling were the results for cost overruns, time overruns, and failure of the applications to provide expected features. For combined Type 2 and Type 3 projects, almost a third experienced cost overruns of 150 to 200%. The average across all companies is 189% of the original cost estimate. The average cost overrun is 178% for large companies, 182% for medium companies, and 214% for small companies. For the same combined challenged and impaired projects, over one-third also experienced time overruns of 200 to 300%. The average overrun is 222% of the original time estimate. For large companies, the average is 230%; for medium companies, the average is 202%; and for small companies, the average is 239%. For challenged projects, more than a quarter were completed with only 25% to 49% of originallyspecified features and functions. On average, only 61% of originally specified features and functions were available on these projects. Large companies have the worst record with only 42% of the features and functions in the end product. For medium companies, the percentage is 65%. And for small companies, the percentage is 74%. The most important aspect of the research is discovering why projects fail. To do this, The Standish Group surveyed IT executive managers for their opinions about why projects succeed. The three major reasons that a project will succeed are user involvement, executive management support, and a clear statement of requirements. There are other success criteria, but with these three elements in place, the chances of success are much greater. Without them, chance of failure increases dramatically. According to Kreitzberg (1999: 30) far too many projects crash and burn. Information Technology Managers surveyed by The Gartner Group considers seventy percent of development projects to be failures. Most Information Technology managers don t blame technology for the problem but surprisingly believe that the top 10 risk factors are all human problems. In a report published by Cable and Wireless ninety percent of information fail to achieve their objective, while forty percent fail altogether. The mistakes can cost as much as two hundred and forty billion dollars a year worldwide. Scotts (2000: 1).

6 Table 1 Standish Group Project Success Factors Sequence Project Success Factors % of Responses 1. User Involvement 15.9% 2. Executive Management Support 13.9% 3. Clear Statement of Requirements 13.0% 4. Proper Planning 9.6% 5. Realistic Expectations 8.2% 6. Smaller Project Milestones 7.7% 7. Competent Staff 7.2% 8. Ownership 5.3% 9. Clear Vision & Objectives 2.9% 10. Hard-Working, Focused Staff 2.4% 11. Other 13.9% Table 2 - Project Challenged Factors Sequence Project Challenged Factors % of Responses 1. Lack of User Input 12.8% 2. Incomplete Requirements & Specifications 12.3% 3. Changing Requirements & Specifications 11.8% 4. Lack of Executive Support 7.5% 5. Technology Incompetence 7.0% 6. Lack of Resources 6.4% 7. Unrealistic Expectations 5.9% 8. Unclear Objectives 5.3% 9. Unrealistic Time Frames 4.3% 10. New Technology 3.7% 11. Other 23.0% Table 3 - Project Impaired Factors Sequence Project Impaired Factors % of Responses 1. Incomplete Requirements 13.1% 2. Lack of User Involvement 12.4% 3. Lack of Resources 10.6% 4. Unrealistic Expectations 9.9% 5. Lack of Executive Support 9.3% 6. Changing Requirements & specifications 8.7% 7. Lack of Planning 8.1% 8. Didn't Need It Any Longer 7.5% 9. Lack of IT Management 6.2% 10. Technology Illiteracy 4.3% 11. Other 9.9% DeMarco (1982: 3) stated that there are some disquieting facts to consider: Fifteen percent of all software projects never deliver anything; that is, they fail utterly to achieve their established goals Overruns of one to two hundred percent are common on software projects.

7 He further noted that a construction job is considered to be debacle if it overruns by six percent. Comparing that measure to software projects, there have been awfully few successful software projects. Hammer and Champy suggest that twenty percent of all new software implementations succeed and nine percent al all major software development applications are worth the cost. The rate of failure therefore is eighty percent. Brendler (2001:1) 5. The South African Survey Results The research project was primarily conducted amongst members of the Project Management Institute of South Africa (PMISA). In addition, companies that are not members of PMISA were also contacted and requested to respond to the questionnaire. Thirty one companies completed the questionnaire. The bulk of the respondents surveyed were in senior management, highest percentage 22.58% are in project management positions and a substantial number have Information Technology as part of their title. Table 4 - Respondent Titles Company Position Frequency Percentage Project / Programme Manager Managing Director Architect Director IT Manager Business Development Manager Business Engineer Chief Information Officer Executive Manager IT Development Manager IT Director IT General Manager IT Regional Manager IT Senior Manager Sales Manager ERP Senior Advisor Information management Senior Analyst Systems Analyst Systems Manager Technical Manager The respondents were also requested to supply the primary company that their companies operated in. The highest concentration of the population occurs in the engineering (16.13%), retail (12.90%), and software development (9.68%) industries, comprising of the total population. The balance of the population (61.28%) representing thirteen of the sixteen industries consists of five industries with an individual representation of 6.45% each and eight industries with an individual representation of 3.23% each.

8 Table 5 - Primary Company Industry Primary Company Sector Frequency Percentage Engineering Retail Software Development Education Financial Services Information technology Manufacturing Pharmaceuticals Electricity Utility Energy Human Resources IT Insurance/Banking Pension Fund Administration Publishing/Advertising Sector Education Training Authority (SETA) Telecommunications Water Treatment Fourteen (14) of the companies had been classed as small companies with an annual sales turnover of less that R 200 million per annum. Six (6) companies were classified as medium size companies with an annual turnover in the range R 200 million to R 500 million. Lastly, eleven (11) companies represented the larger companies, with an annual turnover in excess of R 500 million. For companies with an annual turnover of less than R 200 million the median is R 425,000 and the mean is R 4,155,214. For companies with an annual turnover of between R 200 million and R 500 million the median is R 12,000,000 and the mean is R 18,900,000. For companies with an annual turnover in excess of R 500 million the median is R 16,000,000 and the mean is R 38,827,273. Projects that are initiated per annum and then categorise these projects under the headings of challenged, cancelled and successful. For companies with an annual turnover of less than R 200 million the average project percentages are: challenged 38.57%, cancelled 11.61% and successful 49.82%. For companies with an annual turnover of between R 200 million and R 500 million the average project percentages are: challenged 53.33%, cancelled 10.83% and successful 35.83%. For companies with an annual turnover in excess of R 500 the average project percentages are: challenged 41.09%, cancelled 5.76% and successful 53.15%. The respondents were requested to supply details on approximate cost overrun amount per project for challenged projects. Small companies had a mean cost overrun of R 70,417, medium size companies had a mean cost overrun of R 723,750 and large companies had a mean cost overrun of R 1,645,455. The respondents were requested to supply details on approximate cost percentage overruns per challenged project. Small experienced a mean cost overrun percentage of 21%, medium size experienced a mean cost overrun percentage of 33% and large companies experienced a mean cost overrun percentage of 23%.

9 The respondents were requested to supply details on approximate time overruns per challenged project. Small experienced a mean time overrun of 8 weeks, medium size experienced a mean time overrun of 15 weeks and large companies experienced a mean time of 12 weeks. The respondents were requested to supply details on approximate time overrun percentage per challenged project. Small experienced a mean cost overrun percentage of 15%, medium size experienced a mean cost overrun percentage of 27% and large companies experienced a mean cost overrun percentage of 25%. The respondents were requested to supply details on approximate functionality percentage delivered per challenged projects. Small delivered a mean functionality of 82%, medium delivered a mean functionality of 83% and large companies delivered a mean functionality of 89%. For cancelled projects the respondents indicated that on average 8% of projects in small companies are restarted, 6% of project in medium size companies are restarted and a massive 58% of project in large companies are restarted. For each of the cancelled projects that are restarted, will experience two restarts for small companies, one restart in medium size companies and two restarts in large companies. The respondents were also asked to list the reasons for success in their companies. The table below is a summary of the mean scores per success reason. Sequence Project Success Reasons % of Respondents 1. Clear Vision & Objectives Clear Statement of Requirements Realistic Expectations Proper Planning Competent Staff Hard-Working, Focused Staff Executive Management Support Strong Business Case Smaller Project Milestones Ownership Effective Communication User Involvement Adequate Risk Management Sound Project Management Other: 1.72 The respondents were also asked to list the reasons for projects being challenged in their companies. The table below is a summary of the mean scores per challenged reason.

10 Sequence Project Challenged Reasons % of Respondents 1. Incomplete Requirements & Specifications Changing Requirements & Specifications Unrealistic Expectations Unclear Objectives Lack of User Input Unrealistic Time Frames Lack of Resources Poor Communication New Technology Technology Incompetence Weak Business Case Lack of Executive Support Inadequate Risk Management Other 5.97 The respondents were also asked to list the reasons for projects being cancelled in their companies. The table below is a summary of the mean scores per cancelled reason. Sequence Project Cancelled Reasons % of Respondents 1. Changing Requirements & Specifications Unrealistic Expectations Lack of Executive Support Didn't Need It Any Longer Lack of Planning Weak Business Case Incomplete Requirements Lack of User Involvement Poor Communication Lack of Resources Technology Illiteracy Lack of IT Management Inadequate Risk Management Other What about the Future? The Gartner Group (1998: 1) report titled Want better productivity? Kill projects early and often, forecasts that during the next five years, without significant changes to its project management processes, an Application Development (AD) organisation of 100 developers can expect to spend more than $10 million on cancelled software projects (0.8 probability). Through 2001, organisations using rigorous gating criteria to move projects from the requirements-gathering phase to the development phase will save more than 80 percent in wasted development costs on singleproject cancellations and more than 25 percent in total organisational costs for cancelled projects (0.7 probability). The South African study requested respondents to indicate if more project are challenged than 5 years ago. The results were 39% - No, 58% - Yes and 3% not sure or did not answer. The South African study requested respondents to indicate if more project are cancelled than 5 years ago. The results were 26% - No, 45% - Yes and 29% not sure or did not answer.

11 The good news is that the majority of respondents were of the opinion that we are more successful now than 5 years ago, although a reasonably high percentage of respondents indicated that challenged and cancelled projects are worse off now than 5 years ago. 7. Conclusion The South African results indicate that fewer projects are considered failures in South Africa compared to the Standish results, as follows: Small South African companies have a failure rate of 50.2% compared to 72%, medium size South African companies have a failure rate of 64.2% compared to 83.8% and large South African companies have a failure rate of 16.9% compared to 91%. The overall failure rate for South African companies is 53.7% compared to 82.3%. The South African research supports the opinion expressed by Dorsey (2001: 1) who claims that depending upon which academic study referenced, the failure rate of large projects is reported as being between 50%-80%. This occurs because of the natural human tendency to hide bad news and consequently the real statistic may be even higher. 8. References Allen, R.E. (1990). The concise Oxford Dictionary. England: Clays Ltd. Brendler, W.F. (2001) The people reasons why front office technology projects fail. 23/07/2001 at 10:16. DeMARCO, T. (1982). Controlling software projects. New York: YOURDON Inc. Dorsey, P. (2001). Top 10 reasons why systems projects fail. 20Fail.htm 23/07/2001 at 10:16. Gartner Group. (1998). Want better productivity? Kill Projects early and often. February Gilbreath, R.D. (1986). Winning at project management. What works, what fails and why. New York: John Wiley & Sons, Inc. Glass, R.L. (1998). Runaway projects. New Yersey: Prentice Hall Inc. Goldstein, M. (2001) Knowing Right From Wrong:What Research Tells Us About Ways to Increase the Chances for Project Success. Project Management National Conference Nashville, Tennesee, USA. KERZNER, H. (1998). Project Management A Systems Approach To Planning Scheduling and Controlling. New York: Van Nostrand Reinhold KHARBANDA, O. (1996). What Made Gertie Gallop. New York: Van Nostrand Reinhold. Kreitzberg, C.B. (1999) Divide and Conquer for Better Software. United States: CMP Publications Inc. May, L.J. (1998). Major Causes of Software Failures. 14/04/ :25. Scotts, A. (2000). IT Failures A High Risk. 09/04/2001 at 06:32. Standish Group. (1995). Choas. 13/04/2001 at 19:21 Taylor, A. (2001). IT projects sink or swim. 23/07/2001 at 10:16

12 Jurie Smith, MBA (University of Wales) MBP Diploma, Project Management Diploma, PMP, Certified Information Systems Auditor (CISA). Jurie Smith is a Senior Project Manager and the National Co-ordinator of Project Management Services for PricewaterhouseCoopers in South Africa. Jurie has extensive Programme and Project Management experience and his forte is the facilitation and initiation of large multiple workstream programmes. He has consulted in various industry sectors, including Government, Banking, Distribution, Retail and Oil. Jurie is a past President of the Gauteng branch of PMISA and he has also been involved in the Project Management Standards Generating Body (PMSGB).