The Alberta Report. COAA Major Projects Benchmarking Summary

Transcription

1 The Alberta Report COAA Major Projects Benchmarking Summary February 2009

2 Prepared for: Alberta Finance and Enterprise, Alberta Energy Construction Owners Association of Alberta (COAA) Sponsored by: Alberta Finance and Enterprise, Alberta Energy Construction Owners Association of Alberta (COAA) Under Research Contract: UTA05-782

3 Table of Contents List of Tables... i List of Figures... ii Executive Summary... iv 1 Introduction Background COAA / CII Partnership Research Objectives Scope and Approach COAA Major Projects Benchmarking System and Data Collection Development of Alberta Benchmarking System Hierarchical Structure for Project Comparison Project Key Report COAA Project Performance and Productivity Metrics General Metrics Project Performance Metrics Engineering and Construction Productivity Metrics Practices COAA-Specific Metrics Understanding Benchmarking Reports and Analyses Metrics Explanation of Statistics Data Analysis Description of Alberta Dataset Selected Descriptive Analyses Selected Inferential Analyses Comparison of Alberta and U.S. Project Performance Engineering Productivity Construction Productivity Analysis of Impact Factors Major Findings Project Performance Productivity Impact Factors Project Management Conclusions and Recommendations Appendices Appendix A: Summary of Correlation between Project Performance and Related Factors of Alberta Based Projects Appendix B: Performance Metric Formulas and Definitions Appendix C: Glossary References i

4 List of Tables Table 2-1 Comparison Algorithm of Alberta Project Performance Metrics...10 Table 2-2 Comparison Algorithm of Alberta Engineering and Construction Productivity Metrics...10 Table 2-3 Hierarchical Structure of Alberta Project Types...11 Table 3-1 Additional Study-Specific Performance Metrics...16 Table 4-1 Submitted Projects by Owners and Contractors at Project Completion and Sanction...21 Table 5-1 The Top 5 Factors Affecting Cost, Schedule or Productivity...52 Table A-1 Correlations of Project Characteristics with Project Performance...53 Table A-2 Correlations of Project Characteristics with Project Performance (Cont d)...54 Table A-3 Correlations of Project Characteristics with Project Performance (Cont d)...55 Table A-4 Correlations of Project Characteristics with Project Performance (Cont d)...56 i

5 List of Figures Figure 2-1 Development Process of Alberta Benchmarking System...7 Figure 2-2 Number of Submitted Project Data...9 Figure 2-3 Sample of Project Cost and Schedule Performance Metrics...12 Figure 2-4 Sample of Project Engineering Productivity Metrics...13 Figure 2-5 Sample of Project Construction Productivity Metrics...13 Figure 3-1 Example of Performance Metrics...17 Figure 3-2 Example of Practice Metrics...17 Figure 3-3 Box and Whisker Diagram...18 Figure 4-1 Number of Submitted Projects by Project Nature and Delivery System...21 Figure 4-2 Number of Projects Submitted at Sanction and Completion by Total Project Cost Category ($CDN in 2007)...22 Figure 4-3 Construction Indirect / Direct Work hours (%)...23 Figure 4-4 Construction Indirect Cost / Total Project Cost (%)...23 Figure 4-5 Modularization by Project Nature...24 Figure 4-6 Project Cost Growth by Project Delivery System...25 Figure 4-7 Project Schedule Growth by Project Delivery System...25 Figure 4-8 Effect of % Engineering Completed before Construction Started...26 Figure 4-9 Actual / Estimated Number of Peak Construction Workforce...27 Figure 4-10 Construction Indirect Work-hours/ Direct Work hours (%)...28 Figure 4-11 Construction Indirect Cost Growth by Project Size ($)...28 Figure 4-12 Project Risk Assessment vs. Project Cost Growth...29 Figure 4-13 Constructability vs. Project Schedule Growth...30 Figure 4-14 Planning for Startup vs. Startup Phase Cost Growth...31 Figure 4-15 Workface Planning vs. Construction Schedule Growth...31 Figure 4-16 Project Size ($M CDN, in 2007)...32 Figure 4-17 Contingency Budget (%)...33 Figure 4-18 Project Cost Growth...34 Figure 4-19 Project Schedule Growth...34 Figure 4-20 Development and Scope Change Cost Factor...35 Figure 4-21 Comparison of Project Size ($M CDN, in 2007) for Engineering Productivity Dataset...36 Figure 4-22 Comparison of Concrete Engineering Productivity (WH/ Cubic Meter)...37 Figure 4-23 Comparison of Structural Steel Engineering Productivity (WH/ Metric Ton)...37 ii

6 Figure 4-24 Comparison of Piping Engineering Productivity (WH/ Linear Meter)...38 Figure 4-25 Comparison of Project Size ($M CDN, in 2007) for Construction Productivity...39 Figure 4-26 Comparison of Total Concrete Construction Productivity (WH/ m 3 )...40 Figure 4-27 Comparison of Total Structural Steel Construction Productivity (WH/ Metric Ton)...40 Figure 4-28 Comparison of Instrumentation- Devices Construction Productivity (WH/ Count)...41 Figure 4-29 Comparison of Insulation- Piping Construction Productivity (WH/ Linear Meter)...42 Figure 4-30 Construction Productivity Project Level Index vs. Project Size ($M CDN, in 2007)...42 Figure 4-31 Actual / Estimated Construction Productivity Rate by Work Discipline...43 Figure 4-32 Actual / Estimated Total Installed Unit Cost (TIUC) by Work Discipline...44 Figure 4-33 Factors Impacting Project Cost...46 Figure 4-34 Impact of Factors vs. Cost Growth...46 Figure 4-35 Factors Impacting Project Schedule...47 Figure 4-36 Impact of Factors vs. Schedule Growth...47 Figure 4-37 Factors Impacting Construction Productivity (Field Productivity)...48 Figure 4-38 Impact of Factors vs. Project Construction Productivity (CPM)...49 Figure 5-1 Project Change Cost Factor vs. Project Cost Growth...51 iii

7 Executive Summary The Construction Owners Association of Alberta (COAA) as the principal industry association for heavy industrial construction in Alberta provides leadership to enable our owner members to be successful in their drive for safe, effective and productive project execution. The heavy industrial construction sector represents a significant portion of the GDP in Alberta with announced major projects in excess of $100B. In 2008, the Oil Sands sector in particular was forecasting significant capital expenditures of $10B per year in 2009, rising to $15B by This level of construction activity has strained the industries ability to execute the work effectively and has led to significant concerns about low productivity along with cost and schedule overruns. With these concerns in mind the COAA embarked on a benchmarking initiative in 2003, engaging the Construction Industry Institute (CII) at the University of Texas in Austin to develop a benchmarking system which would assess project performance considering the unique characteristics of Alberta major projects. The CII has extensive experience benchmarking projects in the United States and globally, permitting comparisons between Alberta projects and similar projects in the CII database. This report summarizes the results of the first series of project assessments completed in October A total of 78 Alberta projects were initiated in the benchmarking system since December 2005 and of these 37 completed the data input required for benchmarking analysis at the cut off for this report in August Twenty-seven of the 37 projects in this study were related to the Oil Sands sector with only natural gas processing (4) and pipeline (2) sectors submitting more than one project. About half of the projects were grass roots with additions (38%) and modernizations (11%) making up the balance. Execution strategies varied with almost half of the projects using parallel primes; designbuild was the second most frequent strategy at 32% of the projects. Project sizes varied from <$5M (1) although 75% were over $100M and 8 of these exceeded $1B; the average project size was $368M. Project performance metrics included cost, schedule, safety, change and re-work. Productivity metrics assessed both engineering and construction productivity, overall and in specific disciplines. The effectiveness of 14 project Best Practices were assessed for their impact on performance metrics. Eighteen COAA-specific metrics were included, such as comparison for direct and indirect costs, use of modularization, overtime, peak workforce and overtime. Finally a comparison was made between Alberta projects and comparable projects in the CII database for the United States. The reader should note that this is still a relatively small dataset, so comparisons and conclusions should be drawn with caution. iv

8 The average cost growth for Alberta projects was 19% and average schedule growth was 17%. Cost growth was generally lower as the % engineering completed increased and more effective Project Risk Assessment also reduced project cost growth. Constructability assessments lead to reduced schedule growth but had no impact on cost growth. Indirect costs averaged about 21% of total project costs and indirect cost growth increased as the project size increased. As for other project best practices, Planning for Start-up reduced the cost growth in start-up but there was no detectable correlation between Workface Planning and construction schedule growth, although there were only 7 data points in this assessment. A number of comparisons were made between Alberta and comparable U.S. projects. Although the comparison is for similar industrial projects, no adjustment has been made to account for the differences in project size, the prevailing economy while the projects were built and other potentially significant project drivers. The median project size in the Alberta dataset is $186M vs. $40M for the U.S. dataset. Project cost growth was much higher in Alberta (19%) vs. the U.S. (3%) and Alberta project cost growth had much wider range (-27% to 69%). Development and scope changes were similar between Alberta and the U.S.. Engineering productivity is measured as the ratio of direct engineering hours per installed quantity in the field (e.g., for structural steel, hours per ton of steel; lower is better). In a similar way, construction productivity is measured as the ratio of field direct work hours per installed quantity (e.g., for structural steel, hours per ton of steel; again, lower is better). All comparisons noted below between the Alberta and U.S. data sets are based on weighted averages (i.e., larger projects count more in the average productivity than smaller projects). Engineering productivity for concrete was better in Alberta vs. the U.S. (3.5 vs. 6.3); structural steel engineering productivity was worse in Alberta vs. the U.S. (12.6 vs. 5.9) while piping engineering productivity is comparable (1.28 vs. 1.23). Construction productivity for concrete is worse in Alberta vs. the U.S. (13.1 vs. 9.8) and instrumentation devices productivity is much worse in Alberta (21.4 vs. 8.3), although the nonweighted average between the two was comparable, so further research into this comparison is warranted. Construction productivity for structural steel was comparable between the two datasets (about 38) while insulation productivity was better (1.4 vs. 2.2) in Alberta. To sum up the results to date, productivity is better in Alberta for some disciplines but worse (or much worse) for others, so the productivity picture is mixed. Average wage rates in Alberta are v

9 higher than the U.S., particularly when compared to the U.S. Gulf Coast where many of the heavy industrial projects occur, so improved productivity in Alberta will enhance competitive advantage. Furthermore, indirect costs are higher in Alberta. All this helps explain the significantly higher cost growth in Alberta vs. the U.S. data. The COAA and its members have developed and are implementing a number of initiatives such as Work Face Planning and Re-Work Reduction that will help improve project productivity. The reduced pace of project development in Alberta in the near term should also contribute to improved project execution. Companies that submitted data are given customized reports which show the comparison of their projects against the Alberta and U.S. datasets. This will help determine strengths and weaknesses and should lead to better project execution in future, which in the end is the goal of all benchmarking work. The COAA is considering extending the benchmarking initiative for another 2 years to allow submission of additional projects which will strengthen the data analysis and improve our insight into causes of and potential solutions to poor project performance. The COAA would like to thank the Benchmarking Committee and in particular the current and past committee co-chairs, Steve Revay of Revay and Associates, Larry Sondrol of Suncor and Donald Mousseau of Husky Energy Inc. for their outstanding efforts. The COAA is also indebted to CII for their expertise and efforts and to the Government of Alberta who helped fund this study. vi

10 1 Introduction The oil sands industry plays a crucial role in Canada s global economic position and the delivery of energy to the world. In fact, Canada s oil reserves are second in the world behind Saudi Arabia (OSDG, 2008). Of these reserves, 97 percent are oil sands. Commercial production of the oil sands began over 40 years ago and current output is expected to triple by 2020 (ibid.). The advances that have been made in surface mining and in-situ production technologies have been driving the rapid pace of development of the oil sands reserves. Over the past 11 years, a total of $102 Billion (CDN) was spent on construction and operation capital necessary to develop these resources. Some have projected that through 2012, an additional $205 Billion (CDN) could be invested given favorable economic conditions. The realities of the oil sands resource and the Canadian energy industry place tremendous demands on companies engaged in the efficient and effective execution of capital projects. This report chronicles the efforts made by owners, contractors, and other stakeholders in their delivery of capital projects in the heavy industry sector in Alberta. Using estimated and completed capital projects as its basis, the report examines project performance from cost, schedule, change, rework, safety, and productivity standpoints. It recognizes the uniqueness of heavy industrial projects in Alberta, projects often characterized by their remote locations and challenges posed by severe weather. The story of their development is a compelling one. The Construction Industry Institute (CII) was selected by COAA to explore the performance and productivity concerning the execution of capital projects in Alberta. This selection was premised on the extensive experience of CII in researching and benchmarking industrial facilities in the United States and around the world. Extending CII s reach into Alberta permitted tremendous understanding of the performance of these projects, especially when compared with similar projects in the United States. The results obtained through this study are both surprising and expected. Using quantitative methods, the report dispels common myths regarding project execution in Alberta while establishing a solid footing for the future study of additional projects. 1.1 Background Benchmarking has long been used to improve the process of manufacturing. It is the continuous and systematic process of measuring one s own performance against the results of recognized leaders for the purpose of finding best practices that lead to superior performance when implemented. In the capital projects industry, benchmarking is primarily used at the project level to help participants 1

11 identify gaps in their work processes which lead to compromised performance. For a given company, benchmarking provides sets of external comparisons to its peer group that can be used to establish improvement goals and objectively understand what best in class performance means. The execution of capital projects in Alberta is truly unique. It is one of few geographic areas that has such a great prevalence of capital projects. At last estimate, over 240,000 people were engaged in the development of the oil sands resources in Alberta (OSDG, 2008). In fact, construction comprised 9.0% of Alberta s gross domestic product (GDP) in 2007 (AFE, 2008). Spending on the Athabasca Oil Sands resource in particular rose to $37.7 Billion (CDN) in 2007 (ibid.). However, this dramatic amount of growth has also brought its challenges. Increasing pressures on capital projects have been created due to significant worldwide cost escalations and labour shortages. This has led to the creation of many perceptions regarding the potential loss of productivity or excessive indirect costs, for example. The purpose of this study was to quantitatively assess the performance of capital projects in Alberta. The combined resources of COAA, CII, and Alberta Finance and Enterprise were directed to objectively measure the performance of actual projects planned and executed in Alberta within the past seven or eight years. While it was not possible to obtain measures of every aspect of project performance, this study does provide data necessary to gain new insights to the results of Alberta s heavy industry sector projects. It directly addresses many common perceptions regarding engineering and construction productivity and it provides a baseline of project data that can be used to help improve the work processes used by companies developing projects in Alberta. 1.2 COAA / CII Partnership As the principal industry association for capital projects in Alberta, the Construction Owners Association of Alberta (COAA) strives to provide leadership to enable owner members to be successful in their drive for safe, effective and productive project execution. Principal members of COAA include the users of construction services in capital expansion plans. Indeed, COAA represents a broad cross-section of owners' interests which are associated with many sectors of the Alberta construction community. COAA also includes Associate Members which provide construction services and other activities. COAA s mission is to assist its members in achieving excellence in the execution of capital projects by: Creating and promoting Best Practices in the construction industry Serving as a voice for owners to stakeholders that can make a difference 2

12 Providing a forum for dialogue and debate among owners, contractors, labour providers and government Bringing new ideas to the construction industry and to government leaders Headquartered at the University of Texas at Austin, CII is a consortium of leading owners, engineering and construction contractors, and suppliers that have come together to improve the cost effectiveness of capital projects. As the major public benchmarking resource in the capital projects industry, CII has over 15 years experience in benchmarking capital project delivery and best practices. CII was formed in 1983 by 28 organizations based on recommendations from an intensive five year study of the engineering and construction industry, known as the Construction Industry Cost Effectiveness (CICE) project. Today, there are 117 members around the world engaged in capital projects. Over the past 25 years, CII has partnered industry practitioners with academia to study the capital projects industry to create a vast array of knowledge. In fact, CII research products have been widely disseminated throughout the industry through publications, conferences and workshops and have led to the creation of a number of best practices. CII started its Benchmarking and Metrics (BM&M) program in 1993 with an initial purpose to validate the benefit of best practices and to support CII research. Today, CII s BM&M program employs 10 staff members to advance project performance through benchmarking research. Over the years, an online benchmarking system known as Project Central has been developed to allow benchmarking participants known as Benchmarking Associates (BA s) to enter project data and get real-time feedback 24 hours per day. BA training is provided three times a year to ensure understanding of CII metrics and compliance with standard data definitions. As of 2008, over 800 BAs have been trained and a total of 1,738 projects representing over $81 Billion (USD) have been collected from leading construction owners and contractors around the world. Building on the collective expertise of COAA and CII, a research contract was established in 2005 between the two organizations for the purpose of benchmarking capital projects in Alberta. It was funded by COAA with assistance from Alberta Finance and Enterprise, a component of the provincial government of Alberta. Besides this research report, the contract established a comprehensive benchmarking system comprised of a customized questionnaire, a dedicated database, and a suite of individualized reports for each company submitting project data. The relationship between COAA and CII has been very productive and has yielded many discoveries regarding Alberta s heavy industry sector capital projects, many of which are presented here. 3

13 1.3 Research Objectives The purpose of the research was to develop a benchmarking system to assess the performance of Alberta major projects considering factors unique to their execution, to permit analysis of this performance over time, and to include measures of engineering and field productivity. In particular, specific research objectives included: 1) Identification of Alberta metric requirements 2) Development of a customized benchmarking questionnaire based upon the CII questionnaire, but tailored to the characteristics and environment of Alberta projects 3) Establishment of a set of benchmarks for Alberta projects using the customized questionnaire 4) Documentation of Alberta project performance against the Alberta benchmarks 5) Identification and documentation of factors and practices impacting project performance As the research evolved from 2005 to 2008, COAA s benchmarking committee worked directly with CII benchmarking and metrics staff members to continually refine the research program, its questionnaire and its information technology (IT) tools. For example, besides including additional data definitions for Alberta projects, specific COAA best practices such as Workface Planning were added to the research s customized questionnaire. Taken together, these efforts have produced a premier benchmarking research program for Alberta projects. 1.4 Scope and Approach This research program used the principal components of CII s benchmarking research program as its foundation. CII s existing large project questionnaire for heavy industry sector projects was used as a basis for the COAA questionnaire. A series of development meetings was held in 2005, 2006, and 2007 between COAA s benchmarking committee and CII benchmarking and metrics staff to create and prioritize new metrics specific to Alberta capital projects. This led to the programming of a customized web-based data collection instrument and key report. Throughout the study period, and into 2008, CII conducted seven training sessions for COAA participants in this study. These individuals, known as COAA Benchmarking Associates (BA s), were given access to the online system and key reports. Using the knowledge gained in training, these BA s collected project-specific data and entered them into the online system. Subsequently, they worked with CII staff to validate their data to ensure conformance to accepted definitions. Finally, 4

14 the BA s used the information contained within the key reports to communicate knowledge gained about their projects to their individual firms in order to improve key work processes. The final aspect of this research program was the creation of this report, entitled the Alberta Report. This report is intended to examine all the projects collected through this research to identify common factors or new findings concerning the execution of capital projects in Alberta. It is also the means of communication regarding the entirety of this research. Consequently, this report describes not only the interesting findings of this research, but also the system used to collect, analyze, and disseminate this information. The contents of this report have been distilled to provide commentary only on the most critical aspects and results of this research effort. Certainly, other queries regarding the collected project data were investigated, but only the most statistically significant are presented here. This research was intended to provide the first step in the dedicated study of heavy industry sector capital projects in Alberta. Future steps are planned. Principally, this report provides quantitative assessments of Alberta oil sands projects. It can be used to: 1) Aid understanding of generalized, current perspectives of project performance in Alberta 2) Aid understanding of the benefits obtained through best practice use in the management of capital projects 3) Aid understanding of the drivers for improved capital project performance, especially in the areas of planning, estimating, and productivity 4) Plan for improvements to work processes to execute capital projects more effectively Importantly, this report should not be used to estimate any current or future projects. Results should not be extrapolated to projects beyond those studied as, by definition, every project is both temporary and unique. Results contained herein pertain only to those projects submitted for analysis; projects which were executed by particular individuals in particular periods of time. Continued benchmarking is recommended to maximize the benefits received. Acknowledgement Funding for this research was provided by both the Government of Alberta and COAA. In addition, this study would not have been possible without the endless support from the COAA Alberta Major Projects Benchmarking Committee. The individuals who collected and submitted their project data through this study are greatly appreciated, though their names are not listed due to confidentiality 5

15 policies which are in effect. Members of the COAA Alberta Major Projects Benchmarking Committee are listed next. The COAA Alberta Major Projects Benchmarking Committee Steve Revay*, Revay and Associates Limited Larry Sondrol*, Suncor Energy Inc. Donald Mousseau**, Husky Energy Inc. Patricia Armitage, Alberta Finance and Enterprise Aamer Ahmed, Shell Canada Limited Billy Bai, Ledcor Bob Montgomery, Colt Engineering Dale Elmer, Flint Energy Dave Williams Bantrel Douglas Shako, Flint Energy Ed Catolico, WorleyParsons Ltd. Mahendra Bhatia Suncor Energy Inc. Mel Otteson, Imperial Oil Resources Ltd. Greg Sillak, BA Energy Greg Taylor, Nexen Inc. Hans Raj, Colt Engineering Jared Wharton, EPCOR Johnnas Jagonos, Flint Energy Jennifer Koivuneva, Jacobs Korey Jackson, Stantec Lea Chambers, Golder Associates Ltd. Lubo Iliev, Petro-Canada Renee Roberge, Flint Energy Rheal Guenette, Shell Canada Limited Richard Haack, Shell Canada Limited Stephan Chudleigh, Flint Energy Tim Silbernagel, Bantrel Umesh Krishnappa, Suncor Energy Inc. Vladimir Deriabine, Petro-Canada Warren Rogers, Flint Energy Include past members and denote them as such *current chairs **past chairs 6

16 2 COAA Major Projects Benchmarking System and Data Collection Benchmarking has been recognized as a core component of continuous improvement programs in the capital projects industry. Implementing specific benchmarking approaches on Alberta-based projects will provide the participating companies with a systematic process to measure project performance, enable external comparisons with peers projects, and establish project objectives. Moreover, a comprehensive benchmarking system can identify areas for work process improvement. This was the basis for the development of the Alberta Benchmarking System. 2.1 Development of Alberta Benchmarking System This research comprises the first round of benchmarking heavy industry sector projects in Alberta. Accordingly, a significant amount of time and effort was spent by COAA s benchmarking committee and CII benchmarking program staff to develop the Alberta benchmarking system. The development process of this system can be seen in Figure 2-1. c Figure 2-1 Development Process of Alberta Benchmarking System Principally, the development of a benchmarking system includes the following aspects: development of metrics and a survey instrument, development of a data collection and reporting system, and validation of submitted data. Each is discussed next. 7

17 a) Development of Metrics This study applies most of the project performance, best practice, and engineering and construction productivity metrics developed by CII s Benchmarking and Metrics (BM&M) program. Definition of these metrics can be seen in Appendix B. This study also incorporates many additional metrics focused on specific areas of interest concerning projects executed in Alberta. These additional metrics were developed through meetings with COAA s benchmarking committee, industry experts, and CII s BM&M staff. Development activities for all additional metrics are described in section 3.4. b) Development of Survey Instrument Once the metrics were defined, CII s existing Large Project Questionnaire was modified to include additional metrics for projects executed in Alberta. Primarily, this was accomplished through input obtained from the COAA benchmarking committee. In addition, the questionnaire was refined using the feedback and input from 180 industry representatives who attended COAA benchmarking training over three years. To ensure the reliability and consistency of questionnaire responses, all questions were reviewed and validated by an expert in survey design at the University of Texas at Austin. It should be mentioned that the questionnaire was developed with both owner and contractor data in mind. The final questionnaire was subsequently programmed by CII staff from 2005 to 2007 and can be downloaded through COAA s website. Notably, the questionnaire requires each project to report data concerning general project information, budget, schedule, change orders, rework, safety, practice use, productivity, and factors known to impact project performance. c) Data Collection System CII has developed a robust web-based data collection system over the last eight years. This development activity has resulted in a mature, online system that is recognized as a cost-effective tool that companies can use to benchmark a large number of projects. This system also supports the collaboration of data entry among multiple project participants and allows benchmarking at two milestones: at project sanction (i.e., Approval for Expenditure (AFE)) and after project completion (see definitions of terms in Appendix B). Benchmarking at AFE uses project estimates, while benchmarking after completion relies upon both estimates and actual data. In addition, the Alberta benchmarking system supports both imperial and metric systems of measurement. Here, the system is capable of converting concrete quantities between cubic yards and cubic meters, and wire and cable quantities between linear feet and linear meters, for example. This feature supports projects using a hybrid quantity unit system which is advantageous to large projects managed by multiple companies working in different unit environments. 8

18 d) Data Collection and Validation Figure 2-2 contains a history of COAA project data collection beginning in November Data collection was planned to complete in October 2008 for the first round. As can be seen in the Figure, a total of 78 projects were created by 19 COAA member companies through the end of These 19 firms include ten owner companies and nine contractors. However, only 37 projects containing complete project data were submitted before the deadline in October For this reason, only these projects were validated for inclusion in this study. 100 Number of Project Data in COAA DB by Month (last updated Oct. 24th, 08) Number of Project Data Nov Dec-05 Jan-06 Training # 1 Nov. 05 Feb-06 Mar-06 Apr-06 May Jun-06 Jul-06 Aug-06 Sep-06 Oct-06 Nov-06 Dec-06 Training #2 Oct Jan-07 Feb-07 Number of projects created in database Total number of submitted projects Number of projects submitted by months Mar-07 Apr-07 May-07 Jun-07 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Training #3 Nov Jan-08 Feb-08 Mar-08 Apr-08 May Training #4 May Jun-08 Jul-08 Aug-08 Sep Oct-08 Nov-08 Dec-08 Round 1 Data Cut Off : Aug. 1st, 08 Figure 2-2 Number of Submitted Project Data To ensure the quality and integrity of data included in the Alberta database, a comprehensive data validation process was established by the research team. This process consists of two phases. First, COAA benchmarking associates (BA) validate their project data through internal comparisons and submit these data only once they have been verified. Secondly, the COAA Account Manager at CII examined the submitted data using comparisons with additional Alberta projects, primarily to identify outliers, thereby generating a series of questions to the responsible BA. 9

19 2.2 Hierarchical Structure for Project Comparison To provide meaningful benchmarking results, comparisons are made amongst projects that are as similar as possible using five different project characteristics. These characteristics are used in a hierarchical structure and programmed as a comparison algorithm, the logic of which can be seen in Tables 2-1 and 2-2. The COAA and CII development teams collaboratively created these algorithms in order to mine the database based upon: 1) project cost category, 2) project nature, 3) project type level 2, 4) project type level 1, and 5) respondent types. In order to achieve reasonable benchmarking by project size ($CDN), time adjustments of project costs are required. The year during the middle of the project was used to normalize project cost dollar values to July Table 2-1 Comparison Algorithm of Alberta Project Performance Metrics Loop # # 1 no slices found, go to 2 #2 no slices found, go to 3 #3 no slices found, go to 4 4: Stop! Data Slice found with n=10!! Respondent Type Owner Owner Owner Level 1 Level 2 Nature Upstream Upstream Upstream Oil Sands SAGD Oil Sands SAGD Oil Sands SAGD Grassroots Cost Category $ MM Grassroots ALL ALL ALL Owner Upstream ALL ALL ALL #5 Owner ALL ALL ALL ALL #6 ALL ALL ALL ALL ALL Table 2-2 Comparison Algorithm of Alberta Engineering and Construction Productivity Metrics Loop # # 1 no slices found, go to 2 #2 no slices found, go to 3 #3 no slices found, go to 4 4: Stop! Data Slice found with n=10!! Respondent Type Owner Owner Owner Level 1 Level 2 Nature Upstream Upstream Upstream Oil Sands SAGD Oil Sands SAGD Oil Sands SAGD Grassroots Cost Category $ MM Grassroots ALL ALL ALL Owner Upstream ALL ALL ALL #5 Owner ALL ALL ALL ALL #6- Second Round with All Response Type ALL Upstream #7 ALL Upstream #8 ALL Upstream Oil Sands SAGD Oil Sands SAGD Oil Sands SAGD Grassroots $ MM Grassroots ALL ALL ALL #9 ALL Upstream ALL ALL ALL #10 ALL ALL ALL ALL ALL 10

20 The hierarchical structure of Alberta project type (level 1 and level 2) can be seen in Table 2-3. Alberta projects were divided to four types (level 1) which includes upstream and downstream oil and gas, natural gas, and pipeline projects. This was done for data comparison and analysis purposes. Level 1 projects are also further broken down to a second level (level 2). For example, upstream oil and gas is divided into oil sands Steam Assisted Gravity Drainage (SAGD) and oil sands mining. After metric values are calculated for each project, metrics are compared with the closest specific data slice according to the previously-developed algorithms (e.g., $100M- $250M project size, grassroots, oil sands SAGD, upstream, heavy industry, and contractor). This can be seen in loop 1 of Table 2-1. Additionally, if the comparable dataset has less than 10 projects or data from less than 3 companies, the comparison was moved to the next loop (and so on) until enough data are available. Table 2-3 Hierarchical Structure of Alberta Project Types Level 1 Upstream Level 2 Oil Sands SAGD Level 3 Cogeneration Central Plant Processing Facilities (Oil Exploration/ Pad and Gathering Production) Oil Sands Mining Oil Sands Mining/ Extraction Central Plant Processing Facilities Naptha Hydrotreater Unit Oil Sands Upgrading Downstream Hydrogen Plant Oil Refining Utilities and Offsite Natural Gas Natural Gas Processing Process Pipeline Pipeline Pipeline SAGD Pipeline (Gas Distribution) 2.3 Project Key Report For each participating company in this research, a standardized report was created that contained all metric values that could be calculated based on questionnaire responses for a given project. This standardized report is known as the project key report which was generated to provide comparisons of selected project performance with other similar projects in the Alberta benchmarking database following the procedures discussed previously. The key report presents metric scores, database means, performance quartiles, and sample size of the comparable dataset. The key report was customized for Alberta based projects based on a series of discussions with the COAA Benchmarking committee. A sample of the report can be seen in Figure 2-3, while a complete sample report can be downloaded at the COAA website. Generally, metrics scores are presented in 11

21 quartiles with the first quartile (1Q) being preferred. However, some metrics scores are not presented using quartiles, but rather, are presented as a continuum of observed project performance. In the case of Figure 2-3, schedule metrics are qualified as the percent of projects spending more time. Figure 2-3 Sample of Project Cost and Schedule Performance Metrics The key report also contains comparisons for both engineering and construction productivity. A sample of engineering productivity metrics can be seen in Figure 2-4. In this figure, a calculation of unit rate is provided that divides the total design work hours by its corresponding issued for construction (IFC) quantity. Again, comparisons for each metric are provided with the database mean and all comparable projects are organized into quartiles (e.g., with n=12, three projects would reside in each quartile). While the quartiles appear to be a uniform width from a unit rate perspective, this rarely happens in practice. Rather, project metrics may be clustered or spread out for each observed quartile. Figure 2-5 provides a sample of the key report for construction productivity. In this figure, calculations and comparisons for each metric are provided in much the same way as previously discussed. In general, additional detail is provided for construction productivity when compared with engineering productivity for the same disciplines. In addition, the construction productivity section 12

22 also provides for the ability to compare estimates of construction productivity generated at sanction with actual data from completed projects. Figure 2-4 Sample of Project Engineering Productivity Metrics Structural Steel Metric Wk-Hrs Installed Quantity (MT) Unit Rate (Wk-Hrs/MT) Database Mean n Structural Steel 15, * Pipe Racks & Utility Bridge 5, * Miscellaneous Steel 11, * : Total Structural Steel Productivity Rate 32,577 1, Est. Wk-Hrs Est. Quantity (MT) Est. Unit Rate (Wk-Hrs/ MT) : Total Estimated Structural Steel Productivity Rate 29,000 1, n 14 : Total Installed Unit Cost Actual ($/MT) Estimated ($/MT) Actual DB Mean ($/MT) 3,200 3,000 3, n Figure 2-5 Sample of Project Construction Productivity Metrics 13

23 3 COAA Project Performance and Productivity Metrics This section introduces the metrics used in this research. It provides an explanation of interpretation of the project key report. In addition, this section also provides an overview of statistical terms used in conducting the analyses for this report. As discussed previously, the COAA benchmarking system adopted most of the proven CII project performance and practice metrics, plus some specific metrics created for the unique factors found on projects in Alberta. 3.1 General Metrics The category of general metrics pertains to those metrics used by CII s benchmarking and metrics (BM&M) program for many years. The use of these metrics was necessary to ensure the compatibility of comparisons between project data collected in Alberta and by CII in the U.S. and in other countries around the world. There are three sub-categories of general metrics. These are discussed next Project Performance Metrics The CII BM&M program measures five aspects of project performance, notably: 1) cost, 2) schedule, 3) safety, 4) change, and 5) field rework. Project cost and schedule performance metrics evaluate the amount of variation from planned cost and schedule estimates at sanction. These performance metrics are further decomposed to address five primary phases of capital project execution. Known as phase cost and schedule factors, these metrics portray the proportion of total project time and money expended during each phase of the project. Safety, change, and rework are measured in terms of overall project performance at project completion. The definitions of these metrics are described in detail in Appendix B. The only aspect of project performance metrics that differs between the CII and COAA system concerns safety metrics. For this research, the safety metrics commonly used in Canada were included Engineering and Construction Productivity Metrics The productivity metrics used in this research are based on the engineering and construction productivity measurements used by CII s BM&M program. Metrics are defined as ratios of work hours (WH) to quantities. For most, these metrics are easy to understand and are consistent with most estimating and cost accounting systems. For these metrics, a lower productivity rate is generally preferred. 14

24 Engineering productivity metrics are defined as actual engineering work hours per Issued for Construction (IFC) quantity, which is the number of actual direct work hours required to design a particular unit of work. This calculation can be seen in Equation 1. Engineering productivity metrics were captured for significant work activities for the following design disciplines: 1) concrete, 2) structural steel, 3) equipment, 4) piping, 5) electrical, and 6) instrumentation. A definition of direct labour for all engineering productivity metrics can be seen in the questionnaire and in Appendix B. Engineering Productivity Input Actual Design Work Hours = = [Equation 1] Output IFC Quantity Construction productivity metrics are defined as actual direct work hours required to install a unit quantity. This calculation can be seen in Equation 2. In this research, construction productivity rates were captured for significant work activities for the following disciplines: 1) concrete, 2) structural steel, 3) equipment, 4) piping, 5) electrical, 6) instrumentation, 7) insulation and 8) scaffolding. Additionally, in this research, both estimated and actual quantity and work hours are captured for construction activities. Construction Productivity Input Actual Installed Direct Work Hours = = [Equation 2] Output Installed Quantity Practices This study also assessed the use of 14 project best practices during the execution of a capital project including: Front End Planning, Project Risk Assessment, Team Building, Alignment during Front End Planning, Constructability, Design for Maintainability, Material Management, Project Change Management, Zero Accident Techniques, Quality Management, Automation/Integration (AI) Technology, Planning for Startup, Prefabrication/ Preassembly/ Modularization and Workface Planning. Excluding Workface Planning, all these best practices were adopted from the original CII benchmarking questionnaire. A complete list of these 14 Best Practices and definitions for each are provided in Appendix B. 3.2 COAA-Specific Metrics This research included additional COAA-specific metrics to quantify Alberta project performance and productivity. These metrics are listed in Table 3-1. Many of these additional metrics relate to indirect 15

25 and direct construction costs, mechanical and equipment costs, scaffolding work hours, the use of offsite modules, as well as various workforce metrics. The additional metrics were developed to evaluate suspected major causes of cost overruns and schedule delays common to large projects (Flyvberg 2003; COAA 2005). In addition, for projects in Alberta, metrics regarding estimated construction productivity, estimated total installed unit cost (TIUC), and metrics related to actual versus estimated productivity and TIUC were captured in the construction productivity section. A list of all metrics developed specifically for projects in Alberta is provided in Appendix B. Table 3-1 Additional Study-Specific Performance Metrics Metrics Related to Project Cost Metrics Related to Workforce Metrics Related to Construction Productivity Direct Construction Cost = Direct Construction Costs Total Construction Cost Indirect Construction Cost = Indirect Construction Costs Total Construction Cost Indirect/ Direct = Indirect Construction Costs Direct Construction Costs Major Equipment = Major Equipment Cost Total Project Cost Mechanical & Process Equipment = Mech. and Process Equipment Costs Total Project Cost Direct-Indirect Workhours = Total Construction Indirect Work-Hours Total Construction Direct Work-hours %Offsite Construction WH = Offsite Construction WH of Modules x 100 Total Construction Hours %Overtime Work-hours = Overtime Craft Work-hours x 100 Total Construction Field Work-hours Peak Construction Workforce = Actual Peak Workforce Planned Peak Workforce Mode of Travel to Worksite % Workers Living in Camps and Living Out Allowance Scaffolding WH Factor = Scaffolding WH Total Direct WH Scaffolding Cost Factor = Total Scaffolding Cost Total Direct Cost Modules Installation : Pipe Rack, Process Equipment, and Building Modules Total Installed Unit Cost ($/ Unit Quantity) Productivity Estimate Accuracy = Estimated Productivity Rate Actual Productivity Rate Cost Estimate Accuracy = Estimated Total Installed Unit Cost Actual Total Installed Unit Cost Practices Workface Planning 16

26 3.3 Understanding Benchmarking Reports and Analyses The project key report provides the feedback to a company regarding how their selected project(s) performed. It compares the project against the most comparable set of projects available for each individual metric. Importantly, each participating company can use their key report(s) to identify performance gaps in order to set objectives on future projects and to initiate improvements to key work processes Metrics COAA Project performance metrics consist of cost, schedule, safety, change, field rework, engineering and construction productivity, and estimating performance (actual / estimated productivity rate and total installed unit cost). A lower score generally indicates better performance. For each individual metric, a typical comparison is provided in Figure 3-1. This figure shows that the sample project overran the budget by 3.6%, while the comparable dataset has 35 projects with an average cost growth of -4.0% (i.e., actual cost was 4% less than initially predicted). Overall, this project ranks in the third quartile on cost growth when compared with its peer projects. First quartile metrics are considered best in class. Figure 3-1 Example of Performance Metrics Practice metrics were scored using a ten point scale, with a higher number (i.e., 10) indicating better implementation of the selected practice. As can be seen in Figure 3-2, this sample project received a score of for the use of Front End Planning (FEP). In comparison with 36 similar projects, this project ranks in the second quartile, which indicates that the project implemented FEP relatively well. Figure 3-2 Example of Practice Metrics 17

27 3.3.2 Explanation of Statistics In addition to descriptive analyses previously presented and available in project key reports, this research also employed various statistical techniques to analyze projects residing in both COAA and CII databases. Primarily, box and whisker plots, pie charts and tabular descriptions were used to portray descriptive statistics for both databases. Where inferential statistics were used, methods of correlation including regression with trend lines and statistical tests of significance were incorporated in this research. Where used, box and whisker plots also incorporate a variety of test statistics including the standard T-test or Analysis of Variance (ANOVA) techniques, depending on the number of comparison groups and distribution of sample variances (Agresti and Finlay 1999). Figure 3-3 provides an example of a Box and Whisker plot and associated terminology. Mean refers to the arithmetic average of a set of values, which is the sum of the variable value divided by the number of samples. Median is the number separating the higher half of a sample from the lower half. Median is equivalent to the second quartile (Q2). Sample Box and W hisker Diagram OutlierSymbol Third Quartile (Q 3) Last Obs ervation below (Q IQR) Median Mean First Quartile (Q1) Las t Observation above (Q1-1.5IQR) Figure 3-3 Box and Whisker Diagram First Quartile (Q1) is also called as the 25 th percentile or lower quartile which refers to the threshold below which 25% of the sample have observed value(s). 18

28 Third Quartile (Q3) indicates the 75 th percentile and delineates the highest 25% of data. Interquartile Range (IQR) refers to the range between the first quartile and the third quartile. Correlation (r) measures the strength of the linear relationship between variables (metrics) ranging from -1 to 1. However, a strong correlation does not prove that a causal relationship exists between observed variables. The magnitude close to -1 and to +1 merely indicates that a strong negative or positive relationship is observed between the two variables (i.e., when the relationship between them follows a straight line on a scatter plot). Notably, a correlation close to 0 indicates virtually no linear relationship. In this study, r<0.3 is defined as low correlation, between 0.3 to 0.5 is considered to have a moderate amount of correlation, while r>0.5 is considered to have a high degree of correlation. Trend Line is based on the subjective evaluation of the best fit for the data and should not be used for the purpose of extrapolation. In the case where no evident trends exist, trend lines were omitted. The Coefficient of Determination (R 2 ) is the most frequently quoted measure representing the goodness of linear fit of the least square regression line. R 2 can be interpreted as the percentage of variation of the response variable explained by the regression line with the independent variable as the only explanatory variable. The better fit the line possesses, the closer R 2 should be to 1. Significant Value (p) is defined as the probability of making a decision to reject the null hypothesis when the null hypothesis is actually true. Usually, social science research accepts any probability value below 0.05 (or alpha level = 0.05) as being statistically meaningful. Consequently, any probability value below 0.05 is regarded as indicative of genuine effect (Field, 2005). 19

29 4 Data Analysis This chapter presents selected results of significant analyses discovered by the research team. Instead of examining each project, this chapter describes how the databases of CII and COAA projects were used to evaluate different hypotheses regarding the performance of projects in Alberta. The first three sub-sections provide a perspective of the database through the use of descriptive statistics. Subsequent sub-sections provide selected inferential analyses surrounding factors known and suspected to affect project performance. Notably, the fourth sub-section compares the project performance of Alberta and U.S. projects. The fifth and sixth sub-sections present analysis results for the observed engineering and construction productivity (respectively) in Alberta in comparison with U.S. projects. Finally, the results of factors impacting project cost, schedule and construction productivity are presented in the last sub-section. 4.1 Description of Alberta Dataset The first round of data collection was completed over a period of 36 months. During this time, 78 Alberta-based projects were established in the Alberta benchmarking system, though not all were finalized and submitted. By the end of October 2008, a total of 37 projects were submitted, validated and analyzed in this research. 28 projects were submitted by owners and 9 projects were submitted by contractors at either project sanction or completion. Table 4-1 and Figure 4-1 contain further descriptions of the dataset by type, nature, and delivery system. As can be seen in these exhibits, the majority of submitted projects are oil sands SAGD and upgrading facilities. Most submitted projects are grassroots facilities using parallel prime and design-bid-build (DBB) delivery systems. Importantly, all submitted projects used cost reimbursable contracts for their construction phases. These terms, along with others used in this study, are presented in a glossary in Appendix C. It should be noted that due to the limited number of projects in the first round of this research, the lowest level of analysis that can be presented contains 10 or more projects in accordance with guidelines established by CII and COAA. These provisions assure statistical significance and confidentiality and conform to published policies of the COAA Benchmarking Committee. 20

30 Table 4-1 Submitted Projects by Owners and Contractors at Project Completion and Sanction Submitted at Project Types # Projects Completion Sanction Owner Contractor Owner Contractor Oil Sands Upgrading Oil Sands SAGD Natural Gas Processing Oil Sands Mining/Extraction Pipeline Cogeneration Oil Refining Electrical (Generating) Gas Distribution Total Modernization 4 (11%) Traditional D-B-B 3 (8%) CM at Risk 5 (14%) Grass Roots 19 (51%) Addition 14 (38%) Parallel Primes 17 (46%) Design-Build 12 (32%) # of Projects (%of Total) # of Projects (%of Total) Figure 4-1 Number of Submitted Projects by Project Nature and Delivery System In this research, the total project cost is defined as the total installed cost for owners, whereas contractors reported total cost of their work scope. The distribution of all submitted projects at either sanction or completion and total project cost ($CDN) in 2007 can be seen in Figure 4-2. While all projects were normalized to July 2007, all submitted projects were also completed after Time adjustments were accomplished by using the historical index values contained in RS Means in order to produce valid comparison bases. The total number of projects shown in Figure 4-2 is 35, reflecting the fact that 2 projects did not provide project cost information. As a common practice in Alberta, most mega projects were split into several smaller projects (sub projects) and managed as a portfolio. Among the 37 submitted projects, half of these are considered as sub projects. Consequently, all are treated as individual projects for purposes of data analysis and comparison in the following sections. 21

31 Number of Submitted Projects at Sanction& Completion Number of Projects <$5MM 3 $5MM - $15MM 4 $15MM - $50MM ($M CDN, in 2007) 1 $50MM - $100MM 2 3 $100MM - $250MM $250MM - $500MM $500MM - $1B Sanction Completion >$1B Total Figure 4-2 Number of Projects Submitted at Sanction and Completion by Total Project Cost Category ($CDN in 2007) 4.2 Selected Descriptive Analyses The analyses in this sub-section were conducted to provide an appreciation of the baseline metric values for the projects submitted by both owners and contractors. For these analyses, the distribution of metrics mostly related to project characteristics and performance and is presented with mean, median, range and quartile statistics using box and whisker diagrams. Construction Indirect Cost and Work-Hours Figure 4-3 contains the distribution of construction indirect work hours for 20 projects as a proportion of direct work hours (i.e., the n value below the graph indicates the number of projects reporting this particular data). On average, the amount of indirect construction work hours for Alberta-based projects is about 34% of direct construction work hours. Moreover, the average indirect construction cost is 20.71% of total project cost. This can be seen in Figure

32 Construction Indirect/ Direct Work-Hours (%) (N=20) 34.09% Figure 4-3 Construction Indirect / Direct Work hours (%) Construction Indirect Cost/ Total Project Cost (%) % (N=20) Figure 4-4 Construction Indirect Cost / Total Project Cost (%) % Modularization 1 Figure 4-5 contains a comparison of the percentage of project cost spent using modularization by project nature. Here, the dataset includes 17 grass roots projects and 13 addition projects with modularization data (there was insufficient data to report for modernization projects). On average, grassroots projects spent 19.4% of their total project cost on modularization, compared to 12.3% spent on addition projects. This difference, however, is not statistically significant (t = 1.517, p>0.05). 1 % Modularization is a percentage value that describes the level of modularization (offsite construction), and defined as a ratio of the cost of all modules divided by total installed cost. 23

33 Modularization/Total Proejct Cost(%) Grass Roots (N=17) Addition (N=13) Figure 4-5 Modularization by Project Nature Project Delivery Systems Figures 4-6 and 4-7 compare the effectiveness of parallel primes and all other project delivery methods combined by cost and schedule growth, respectively. The research found that 46% of the projects in the Alberta benchmarking database used a parallel primes project delivery method. Other project delivery methods included traditional design/bid/build (D/B/B), design/build (D/B), multiple design-build and construction management (CM). These methods were combined due to insufficient numbers of projects in each of these categories. Nonetheless, these results show a slight advantage to the use of parallel primes over other delivery methods with respect to schedule, but not with respect to project cost. In addition, parallel primes projects had slightly lower average project schedule growth (0.15) in comparison with all other project delivery methods (0.19) and, simultaneously, higher project cost growth (0.23 vs. 0.13). These differences are not statistically significant (t = , p>0.05) for project schedule growth, nor are they significant for project cost growth (t = 0.756, p>0.05). These results are presented here merely for the reader s enhanced understanding of the project data used for this report. 24

34 Project Cost Growth Parallel Primes (N=14) Other (N=10) Figure 4-6 Project Cost Growth by Project Delivery System 1.00 Project Schedule Growth Parallel Primes (N=14) Other (N=10) Figure 4-7 Project Schedule Growth by Project Delivery System 4.3 Selected Inferential Analyses Inferential analyses were used to explore the relationships amongst project characteristics and the implementation of best practices on project performance. This sub-section provides samples of analyses used to determine if any trends or relationships exist between variables for the purpose of identifying the potential root causes that may help explain the performance of Alberta-based projects. Importantly, it should be noted that the trends or relationships presented in this sub-section should 25

35 not be used to predict, or forecast the performance of any current or future project. Regression and trend lines shown here are considered explanatory only for the projects being analyzed. Percent Engineering Completed before Construction Started The relationship between percent engineering completed before construction started and construction phase cost growth can be seen in Figure 4-8. Note that a complete list of project metrics is included in Appendix B. Given the expert opinions of members of the COAA Benchmarking Committee, the relationship displayed in Figure 4-8 uses a cubic polynomial pattern due to the fact that as more design is completed before construction begins, the project tends to have less construction phase cost growth. This trend holds true until, at a certain point, the cost growth curve flattens and subsequently increases. Thus, an optimum value is found at approximately 60% engineering complete. This result is consistent with other studies completed by CII and other industry forums. The results are also statistically significant, meaning that a strong relationship exists between the percentage of engineering completed prior to construction start and construction phase cost growth (R 2 = 0.63, p = 0.016). Likewise, the results also demonstrate a statistically significant correlation with r = , p = Due to limited number of data, predictability is not inferred, nor concluded, in this research Construction Phase Cost Growth % Design completed before construction started 100 Figure 4-8 Effect of % Engineering Completed before Construction Started 26

36 Peak Construction Workforce Analyses were conducted to examine how a change in construction workforce peak numbers affects project performance. This can be seen in Figure 4-9. The results indicate that the projects which estimated their actual peak workforce numbers with greater precision experienced higher levels of project cost and schedule performance. In fact, the relationship between the ratio of actual to estimated peak workforce and project cost growth is statistically strong (r = 0.787, p =0.001). The regression model also indicates a statistically strong high R-square (R 2 = 0.62, p< 0.001). A medium to strong relationship was also identified between project schedule growth and the ratio of actual to estimated peak construction workforce (r = 0.486, p = 0.011) Project Cost Growth Actual/ Estimated Peak Construction Workforce Figure 4-9 Actual / Estimated Number of Peak Construction Workforce Construction Indirect Work-Hours/ Direct Work hours (%) The relationship between construction indirect work hours and project schedule factor can be seen in Figure This figure indicates that projects with high ratios of construction indirect work hours tend to have better project schedule performance. Here, the lower schedule factor may mean that better schedule performance may be obtained by isolating the schedule impact of change orders. The analysis shows that the relationship between these metrics can be characterized as medium to strong and statistically significant (r= , p=0.05). The linear regression model is also statistically significant, yet presents a low R-square value (R 2 = 0.20, p= 0.05). 27

37 Project Schedule Factor Construction Indirect WH/Direct WH (%) Figure 4-10 Construction Indirect Work-hours/ Direct Work hours (%) Project Size ($) Figure 4-11 contains the analysis of how project size ($ CDN, in 2007) affects construction indirect cost growth. Construction indirect cost growth is calculated as actual indirect cost divided by estimated indirect cost. The results indicate that larger projects tend to have higher levels of construction indirect cost growth, based on their own estimates. These results are statistically strong (r= 0.599, p=0.011) and indicate that a relationship does exist between project size and construction indirect cost growth. The regression model also possesses a medium R-square (R 2 = 0.36, p= 0.011) value Construction Indirect Cost Growth Adjusted Total Project Cost ($M CDN, in 2007) 1200 Figure 4-11 Construction Indirect Cost Growth by Project Size ($) 28

38 Project Risk Assessment (PRA) The effects of Project Risk Assessment (PRA) on project performance are well known and were investigated for this study. As defined by CII, PRA is the process needed to identify, assess and manage risk. In PRA, the project team evaluates risk exposure for potential project impacts in order to provide focus for mitigation strategies. The analysis of COAA data indicate that high levels of implementation success of PRA are accompanied by better project cost performance as can be seen in Figure For this practice, the statistical relationship is medium-strong and statistically significant (r = , p = 0.048). However, for PRA as implemented on Alberta projects, the regression model exhibits low R-square values (R 2 = 0.19, p= 0.048) Project Cost Growth Project Risk Assessment Figure 4-12 Project Risk Assessment vs. Project Cost Growth Constructability Constructability is defined as the effective and timely integration of construction knowledge and experience into the conceptual planning, design, construction, and field operations of a project to achieve the overall project objectives. As can be seen in Figure 4-13, increased use of constructability on Alberta project leads to better project schedule performance (i.e., lower project schedule growth). This relationship is statistically significant (r = , p = 0.008), (R 2 = 0.28, p = 0.008). Interestingly, the effect of constructability on cost performance is not significant for the COAA dataset. These results are consistent with previous analyses developed using CII s benchmarking and metrics database. 29

39 1.25 Project Schedule Growth Constructability Figure 4-13 Constructability vs. Project Schedule Growth Analysis of Other Best Practices Analyses of other best practices were also conducted. Two are included here: planning for startup and workface planning. Other evaluations of additional surveyed best practices are not presented here due to limited space and the fact that they are summarized in a table which can be found in Appendix A. Planning for startup is defined as the effective facilitation of the activities that occur between mechanical completion (i.e., plant construction completion) and the commencement of commercial operations. As can be seen in Figure 4-14, the analysis conducted by this research indicates that improved use of planning for startup methods tends to improve the cost performance of the startup phase. However, this conclusion is made with caution due to the small sample size involved. Finally, the COAA best practice of workface planning was also assessed. Workface planning is defined by COAA as the process of organizing and delivering all elements necessary, before work is started, to enable craft persons to perform quality work in a safe, effective and efficient manner. The relationship between workface planning and construction phase schedule performance can be seen in Figure However, no regression line is plotted due to the limited number of projects reported in this first round. Nonetheless, the figure is shown here as it is believed that workface planning does help improve jobsite productivity by assuring that the required resources, tools, equipment and material are made available to craft workers in a timely fashion. More data will be needed for a more 30

40 comprehensive evaluation of this specific best practice. However, even then, as discussed by Kellogg et al (1981), the optimization of jobsite performance may be limited in comparison with the planning and engineering. 3 Startup Phase Cost Growth Planning for Startup 9 10 Figure 4-14 Planning for Startup vs. Startup Phase Cost Growth 0.8 Construction Schedule Growth Workface Planning Index Figure 4-15 Workface Planning vs. Construction Schedule Growth 31

41 4.4 Comparison of Alberta and U.S. Project Performance A primary focus for this research effort was to obtain comparisons of project performance in Alberta with projects executed in the United States using the CII Benchmarking and Metrics (BM&M) database. To accomplish this objective, U.S. projects are limited to projects with an adjusted total installed cost greater than $5 million (CDN), normalized to In this study, we consider U.S. EMR projects to include oil exploration / production, oil upgrading / refining, natural gas, pipeline, chemical manufacturing, power generation, and mining. In the following analyses, the symbol is used to indicate arithmetic mean and the symbol is used to indicate the median of a particular group. Alberta-based projects are unique projects. There are significant differences when comparing them to U.S.-based projects. Principally, most of the COAA projects are larger than their U.S. counterparts in terms of cost. They are also located in remote locations and are subjected to extreme (northern climate) weather conditions. Often, work camps are built and transportation for large numbers of workers becomes necessary. However, the analyses presented in this sub-section are not intended to quantify these differences, but rather, examine differences in project size, contingency, and cost, schedule, and change performance. Comparison of Project Size ($) for two Datasets Figure 4-16 provides the distribution of Alberta-based and U.S.-based projects included in this study in terms of project cost. Notably, the average size and range of the 353 included U.S. projects are Adjusted Project Cost ($M, in 2007) Alberta (N=23) U.S. (N=353) Figure 4-16 Project Size ($M CDN, in 2007) 32

42 notably smaller when compared with Alberta-based projects. Based on past research and benchmarking experience, this difference is a significant factor in quantifying performance and should be considered in understanding the analyses presented here. Comparison of Contingency Budget (%) As can be seen in Figure 4-17, the amount of contingency for Alberta and U.S.-based projects are quite comparable. Project data shows a slightly higher average contingency rate (8.04%) for Alberta-based projects when compared to U.S.-based projects (7.77%). However, the difference is not statistically different (t= 0.33, p= 0.742). Total Contingency Budget/ Total Project Cost (%) % Alberta 7.77% U.S. (N=17) (N=52) Figure 4-17 Contingency Budget (%) Comparison of Project Cost and Schedule Performance Figures 4-18 and 4-19 compare the project cost growth and project schedule growth of projects executed in Alberta and in the U.S. Results show significantly higher average cost growth and schedule growth for the Alberta projects. These projects also demonstrate that a much wider range of performance exists as well. On average, Alberta-based projects experienced 19% project cost growth and 17% project schedule growth, while U.S. projects experienced 3% and 6% cost and schedule growth, respectively. The test of mean difference between these two groups also indicates that the average cost and schedule growth of Alberta project is statistically significant (t = 3.89, p = for cost and t = 3.838, p = for schedule). Additionally, these figures show that the 33

43 Alberta-based projects possess a much wider range of performance (i.e., -27% to 69% for cost growth and -15% to 35% for schedule growth) when compared to projects executed in the U.S Project Cost Growth Alberta (N=24) U.S. (N=352) Figure 4-18 Project Cost Growth Project Schedule Growth Alberta (N=24) U.S. (N=338) Figure 4-19 Project Schedule Growth Comparison of Change Cost Factor Figure 4-20 contains an analysis of the comparison of development change cost factor and scope change cost factor for the selected Alberta and U.S. projects. In this analysis, the Alberta-based projects have a slightly higher average development change cost factor (0.06) when compared to U.S.-based projects (0.04). In contrast, the average scope change cost factor for projects in Alberta 34

44 is slightly lower than that of U.S. projects. However, these differences are not considered to be statistically significant (p > 0.05). Finally, an analysis of total field rework cost (defined as a ratio of total direct cost of field rework to actual construction phase cost) indicated that Alberta projects are also in line with U.S.-based projects Change Cost Factor Alberta U.S. Alberta U.S. Development_Change Scope_Change (N=11) (N=14) (N=13) (N=14) Figure 4-20 Development and Scope Change Cost Factor 4.5 Engineering Productivity Twenty-three of the 37 Alberta-based projects submitted for this research provided measures of engineering productivity. Of these 23 projects, their average project cost was $367 Million (CDN), normalized to July As can be seen in Figure 4-21, the CII BM&M database was able to return 57 EMR projects that also contained engineering productivity data. These projects reported an average cost of $90 Million (CDN), also normalized to July As previously described, this differential in average project cost may impact the direct measures of engineering productivity reported here. Also, in order to ensure appropriate comparisons, the unit of measure of both U.S. and Alberta projects used to calculate engineering productivity is the metric system (e.g., linear meter, metric ton) and data have been converted to meet this standard. In general, engineering productivity metrics use direct engineering work hours in metrics comparing them with specific issued for construction (IFC) quantities for specific disciplines. These are discussed next. 35

45 Adjusted Project Cost ($M, in 2007) Alberta U.S (N=23) (N=57) Figure 4-21 Comparison of Project Size ($M CDN, in 2007) for Engineering Productivity Dataset Comparison between Alberta versus U.S. Projects by Engineering Disciplines Selected disciplines of engineering productivity are presented in this section. Comparisons between Alberta-based and U.S.-based projects are presented by using both arithmetic mean value (indicated by the symbol ), and weighted mean value (represented by the symbol ). Here, the weighted mean is calculated as an aggregated productivity rate and is weighted by project size. Essentially, this mean creates one large, imaginary project where total work hours and total quantities are assimilated. Previous analysis by CII indicates that this approach is valid and that large projects typically experience better productivity rates (due to larger quantities) when compared with smaller projects possessing smaller quantities and lower levels of repetitive work and economies of scale. Concrete Engineering Productivity 2 As can be seen in Figure 4-22, the results of engineering productivity metrics for Alberta-based projects and U.S.-based projects are mixed. Alberta-based projects have comparable concrete engineering productivity in line with U.S.-based projects when considering mean values. In fact, after considering project size, the weighted average concrete engineering productivity rate of Alberta projects is actually better than that of U.S. projects. 2 Total Concrete include slabs, foundations, and concrete structures. 36

46 Total Concrete- Eng. Prod. Rate (WH/CM) Alberta (N=17) U.S. (N=31) Figure 4-22 Comparison of Concrete Engineering Productivity (WH/ Cubic Meter) Total Structural Steel Engineering Productivity 3 Figure 4-23 indicates that U.S.-based EMR projects perform the engineering of structural steel with higher levels of productivity when compared with Alberta-based projects (10.96 WH / ton versus WH / ton). The weighted average productivity rate of U.S. projects is also better (5.86 WH / ton versus WH / ton). This difference of average structural steel engineering productivity rates is statistically different (t = 2.501, p = 0.02). No cause of this difference is indicated. Steel- Eng. Prod. Rate (WH/Metric Ton) Alberta (N=19) U.S. (N=56) Figure 4-23 Comparison of Structural Steel Engineering Productivity (WH/ Metric Ton) 3 Total of structural steel include structural steel, pipe racks & utility bridges, and miscellaneous steel. 37

47 Piping Engineering Productivity 4 As can be seen in Figure 4-24, Alberta-based projects demonstrate higher levels of piping engineering productivity when compared with U.S.-based projects. This holds true for comparisions of small bore pipe, large bore pipe, and all pipe sizes combined. These results are consistent using both measures of average and weighted productivity as can be seen in the figure. It should be noted that the difference portrayed here in the average piping engineering productivity rates is statistically significant only for large bore pipe (t = , p = 0.012), yet this difference is negligible when examining only the projects which reported only total piping data (i.e., no reporting of bore size). Again, no cause of the differential reported here is indicated. 8 Piping- Eng. Prod. Rate (WH/LM) Alberta U.S. Alberta U.S. Alberta U.S. SmallBore(<2.5") LargeBore(>3") Total_Piping (N=10) (N=23) (N=12) (N=26) (N=22) (N=61) Figure 4-24 Comparison of Piping Engineering Productivity (WH/ Linear Meter) 4.6 Construction Productivity Thirty-three Alberta-based projects provided data concerning construction productivity for this research study. Many of these reported both estimated and actual work hours and quantities. Compared to engineering productivity on Alberta-based projects, construction productivity is considered to be more susceptible to variance due to environmental factors such as weather. As can be seen in Figure 4-25, the average project cost of the 33 projects submitted in this study is $460 Million (CDN) after time adjustment to July By contrast, the 29 projects used from the CII Benchmarking Database for comparison have an average project cost of $122 Million (CDN) after 4 Total piping includes small bore (diameter 2-1/2 and Smaller) and Large Bore diameter 3 and Larger. 38

48 time adjustment to July While the project sizes differ, the comparisons which follow are considered to be valid because of how this study defined construction productivity as the ratio of field direct work hours (WH) per applicable installed quantity Adjusted Project Cost ($M, in 2007) Alberta (N=33) U.S. (N=29) Figure 4-25 Comparison of Project Size ($M CDN, in 2007) for Construction Productivity Comparison Between Alberta and U.S. Projects by Work Disciplines Construction productivity for selected disciplines are presented in this sub-section. As was done with measures of engineering productivity, the arithmetic mean is represented by the symbol ( ), and the weighted mean (i.e., a hypothetical single large project) is represented by the symbol ( ) for purposes of comparisons of construction productivity rates between Alberta and U.S.-based projects. Concrete Construction Productivity 5 Figure 4-26 provides an assessment of concrete construction productivity. As can be seen in the figure, U.S.-based projects place concrete more efficiently than do Alberta projects (U.S. average total concrete productivity rate is WH/m 3, compared to WH/m 3 for Alberta projects). The results are also considered to be consistent even given the differences in the size of the projects used for this analysis. Notably, the weighted average productivity rates of U.S.-based projects is 9.72 WH/m 3, compared to WH/m 3 for Alberta-based projects, although this difference is not considered to be statistically significant. 5 Total Concrete includes slabs, foundations, and concrete structures. 39

49 Total Concrete Productivity Rate (WH/ CM) Alberta (N=12) U.S. (N=32) Figure 4-26 Comparison of Total Concrete Construction Productivity (WH/ m 3 ) Total Structural Steel Construction Productivity As can be seen in Figure 4-27, U.S.-based projects are more productive in erecting structural steel than Alberta projects are (53.95 WH/MT versus WH/MT). This difference is not statistically significant. However, the weighted average productivity rate of Alberta-based projects is slightly better than that of U.S.-based projects by 1.06% when considering project size. Total Steel Prod. Rate (WH/Metric Ton) Alberta (N=21) U.S. (N=32) Figure 4-27 Comparison of Total Structural Steel Construction Productivity (WH/ Metric Ton) 40

50 Instrumentation Devices Construction Productivity Figure 4-28 provides an assessment of the productivity of installation of instrumentation devices. Project data reported for this study revealed that the arithmetic mean of instrumentation devices productivity rate of Alberta-based projects is comparable to that of U.S.-based projects (13.37 WH/Count versus WH/Count, respectively). However, the weighted average productivity rate of U.S.-based projects is significantly better (i.e., 158% better) than that of their Alberta-based counterparts. This is primarily due to the discrepancies that exist for instrumentation devices productivity rates between small projects and large projects. Instrumentation- Devices Con. Prod. Rate (WH/ Count) Alberta (N=9) U.S. (N=22) Figure 4-28 Comparison of Instrumentation- Devices Construction Productivity (WH/ Count) Insulation Piping Construction Productivity As can be seen in Figure 4-29, the average (i.e., arithmetic mean) piping insulation productivity rate of Alberta-based projects is comparable to that of U.S.-based projects (i.e., 1.90 WH/LM versus 1.93 WH/LM, respectively). However, when the weighted mean calculation is used, the Alberta-based projects outperformed their U.S.-based counterparts by 35.6%. One hypothesis for this observed difference is that Alberta-based projects likely have much more piping insulation, on average, and that the metrics used for this study are possibly indicating the benefits of repetition for this particular construction activity. 41

51 Insulation- Piping Con. Prod. Rate (WH/LM) Alberta (N=16) U.S. (N=15) Figure 4-29 Comparison of Insulation- Piping Construction Productivity (WH/ Linear Meter) Construction Productivity Project Level Index (CPM Index) Recently, CII has produced a method for examining construction productivity at the project level by weighting and combining the productivity rates for various disciplines for each observed project. This method, known currently as the Construction Productivity Method (CPM) Index, provides a macroview of project performance. As a relative productivity performance measure, the CPM Index ranges from -3 to 3, with -3 indicating the poorest observed productivity performance. Here, one unit difference in the CPM index is equivalent to a 100% observed difference in productivity. Construction Prod. Project Level Index Adjusted Total Project Cost ($M CDN, in 2007) Figure 4-30 Construction Productivity Project Level Index vs. Project Size ($M CDN, in 2007) 42

52 The project-level CPM index is critical to the examination of factors affecting construction productivity because, most of the time, these factors affect all disciplines and not individual disciplines. As can be seen in Figure 4-30, the project data for Alberta-based projects indicates that large projects had better overall construction productivity than small projects did. The analysis provides for a statistically significant, medium strength correlation (r = , p = 0.008), although the linear regression line is not statistically significant (R 2 = 0.13, p = 0.115) and should not be used for estimating or forecasting purposes. However, the idea that larger projects have better construction productivity is made with caution since the CPM index includes only measures direct construction productivity. Indeed, larger projects tend to have larger installed quantities and higher amounts of repetitive work and these factors may impact overall construction productivity figures. Notably, these results are consistent with previous analyses conducted by CII. This research explored other aspects of construction productivity as well. One example involves the analysis of the effect work schedule (days on and off) has on construction productivity. This study concluded that a work schedule of 10 days on and 4 days off was more productive than a work schedule of 5 days on and 2 days off. In fact, an 11% difference was observed using the CPM Index, although strong statistical significance was not present due to low numbers of project observations. Actual / Estimated Construction Productivity Rate by Work Discipline Figure 4-31 provides an assessment of the accuracy of field productivity estimates for three crafts. This research discovered that Alberta-based projects significantly underestimate construction productivity. Observed data demonstrate that piping, structural steel, and concrete actual rates exceeded their estimated rates by 4%, 22% and 45% (respectively), on average. 3.0 Actual/ Estimated Productivity Rate Total_Concrete Total_Steel Total_Piping (N=8) (N=17) (N=10) Figure 4-31 Actual / Estimated Construction Productivity Rate by Work Discipline 43

53 Actual / Estimated Total Installed Unit Cost (TIUC) 6 by Work Discipline To give some context to Figure 4-31, Figure 4-32 provides an assessment of the accuracy of unit cost estimates by craft. Due to the fact that labour accounts for about 30% of total installed unit cost (TIUC), a moderating effect exists on field productivity when other factors are considered. In fact, underestimates of 2%, 11%, and 10% were observed for piping, structural steel, and concrete, respectively. Consequently, construction labour productivity rates must be estimated with caution. Actual/ Est. Total Installed Unit Cost Total_Concrete Total_Steel Total_Piping (N=9) (N=15) (N=8) Figure 4-32 Actual / Estimated Total Installed Unit Cost (TIUC) by Work Discipline 4.7 Analysis of Impact Factors Working with CII, the COAA Benchmarking Committee developed a list of 18 potential factors known to impact project cost, schedule, and engineering and construction productivity. This sub-section contains analyses of these impact factors and their relationship to cost, schedule, and overall construction productivity performance. Importantly, these analyses rely upon the subjective knowledge of industry professionals working on the Alberta-based projects contributing data for this study. Beyond the 12 impact factors routinely used by CII, the COAA Benchmarking Committee added an additional 6 factors, specifically for Alberta-based projects: 6 Total installed unit cost (TIUC) is defined as the burdened cost of direct labour, bulk material, final asset equipment, and civil and sitework equipment by pro rata share including overhead and profit from both direct hire and subcontract. Burden cost of direct labour includes insurance, welfare and other fund and charges associated to labour by regulations. 44

54 1) Quality of Field-Level Supervision 2) Amount of Scheduled Overtime 3) Amount of Unplanned Overtime 4) Engineering Labour Skill 5) Percentage of Engineering Completed Prior to Project Sanction 6) Percentage of Engineering Completed Prior to Construction Start For this study, the industry professionals contributing data for each project were requested to assess whether each of these factors adversely or positively affected project performance (beyond which was planned for) using a scale ranging from highly negative, to highly positive. The results in this sub-section show the combined, relative impact of factors affecting project cost, schedule and construction productivity. In Figures 4-33, 4-35, and 4-37, each factor is ranked using the average degree of impact as reported on all projects as indicated by the symbol ( ). In these figures, the whisker line spreading out from the average indicates the dispersion of impact rating by using one standard deviation (S.D.) in both directions (i.e., -1 to +1). Notably, the factors with less than 10 responses were not reported. Consequently, some figures contain more comparisons of impact factors than others. As can be seen in Figure 4-33, the factor having the most impact on project cost (when compared to that which was planned) was the amount of unplanned overtime. This was followed closely by the percent of engineering completed prior to construction, business market conditions, craft labour skill, and coordination of plant shut down. These are the top four impact factors on project cost and demonstrate an average impact of , , , and , respectively. Figure 4-34 provides a perspective of the impact of 16 factors on project cost growth. The cumulative impact is significant and a very strong correlation exists (i.e., R 2 = 0.56). As a result, this research found that the 16 factors seen in Figure 4-33 had a large, negative impact on the cost performance of the Alberta-based projects. In all cases, these impacts (on average) were worse than the projects had planned for. 45

55 Figure 4-33 Factors Impacting Project Cost Hi Positive Impact Hi Negative Impact Sum Degree Impact of Top 16 Factors vs. Cost Growth Sum Degree of Impact Poor Cost Performance Figure 4-34 Impact of Factors vs. Cost Growth Project Cost Growth As can be seen in Figure 4-35, the factor having the most impact on project schedule (when compared to that which was planned) was the percentage of engineering completed prior to construction start. This was followed closely by business market conditions, craft labour skill, quality of field-level supervision and weather conditions. The top five impact factors on project schedule demonstrate an average impact of , , , , and , respectively. 46

56 Figure 4-36 provides a perspective of the impact of the top 18 factors on project schedule growth. In this case, no regression line has been plotted as no significant correlation exists. Nonetheless, the figure provides some evidence that the surveyed project factors do provide an impact to the schedule performance of the project. In all but one factor (on average) the impact was worse than the projects had planned for. Figure 4-35 Factors Impacting Project Schedule Sum Degree Impact of Top 18 Factors vs. Project Schedule Growth Hi Positive Impact Hi Negative Impact Sum Degree of Impact Poor Cost Performance Project Schedule Growth Figure 4-36 Impact of Factors vs. Schedule Growth 47

57 As can be seen in Figure 4-37, the greatest impact factor on construction productivity (when compared to plan) was the percentage of engineering completed prior to construction start. This was followed by the amount of unplanned overtime, business market conditions, the quality of field-level supervision, and craft labour skill. The top five impact factors on construction productivity demonstrate an average of , , , and , respectively. Figure 4-38 provides a perspective of the impact of the top 17 factors on construction productivity. Notably, a medium-strength correlation exists (i.e., R 2 = 0.27) between the CPM Index of construction productivity and the cumulative impact factors observed for each project. Indeed, this research found that the 17 factors seen in Figure 4-37 had an overall, negative impact on the construction productivity performance of the Alberta-based projects. In all cases, these impacts (on average) were worse than that which the projects had planned for. Figure 4-37 Factors Impacting Construction Productivity (Field Productivity) 48

58 Sum Degree Impact of Top 17 Factors vs. Project Construction Productivity (CPM) Hi Positive Impact Hi Negative Impact Sum Degree of Impact CPM Unproductive Direct Construction Productivity Figure 4-38 Impact of Factors vs. Project Construction Productivity (CPM) 5 Major Findings Through the participation of COAA members in this research, a number of major findings were made concerning the performance of major capital projects in Alberta s heavy industry sector. However, this is only seen as a first step on the journey of continuous improvement for many COAA members. Because the COAA benchmarking system enables each member to identify gaps in their project execution performance relative to that of their peers, the system may empower them to modify work processes or implement best practices. This type of introspection, development, and deployment is critical. Without it, industry-wide improvement is not possible. This report contains selected perspectives of observed data across multiple COAA member companies and their unique projects. Taken collectively, there are a number of significant findings amongst the projects analyzed for this research. These are discussed next. 5.1 Project Performance Alberta-based projects demonstrated significant issues with cost and schedule performance as evidenced by cost growth and schedule growth metrics used in this study. Compared to similar U.S.- based projects, these metrics were 533% and 183% higher, respectively. These overruns are 49

59 beyond unpredictable they are alarming. Due to the fact that these two metrics compare actual and planned costs and durations, the estimation of anticipated costs and schedules is seemingly an issue and potentially at the heart of the capital project performance which was observed in Alberta. Several factors may be at work: 1) Alberta-based projects have comparatively high proportions of indirect (to direct) cost. This study found, on average, that indirects account for 20.71% of total project cost. These costs may be due to factors such as large projects in remote jobsites and executed in harsh climates. However, the estimation and management of indirects deserves close attention. 2) This research revealed that the actual peak construction workforce was highly correlated with project cost growth. Yet, the research also discovered that, in many cases, construction productivity was not highly differential to that experienced on U.S.-based projects. Again, the proportion of indirect labor is an issue. While this study did note that higher amounts of construction indirect labor hours results in better schedule performance, a renewed effort to accurately estimate peak workforce, indirect cost, construction productivity, and unit cost is suggested. Additional benchmarking data can be useful for this purpose. 3) A number of Alberta-based projects submitted for this study began construction with less than 30% engineering complete. CII has conducted many previous studies that show that the most appropriate time to start construction is when more than 60% of engineering is complete. This dichotomy reveals itself in metrics related to construction phase growth and construction productivity, amongst others. Mobilizing to the field too soon is often accompanied by negative cost growth and construction productivity rates. 4) Alberta-based project management teams frequently fail to recognize that project cost growth is driven by, and managed through, scope and project development changes. As can be seen in Figure 5-1, U.S.-based projects report virtually all variance in actual project costs as change. This is visually represented as the near-overlay of the U.S. trend line with a line of unity between cost growth and change cost factor. Such is not the case with the Alberta project data set, which consistently experience cost growths which far exceed their associated change cost factors. 50

60 Location Alberta U.S. Project Cost Growth Project Change Cost Factor Figure 5-1 Project Change Cost Factor vs. Project Cost Growth 5.2 Productivity Despite the wide range of observed capital project performance in Alberta, this research found that engineering productivity as well as construction rework and productivity were comparable to similar U.S.-based projects in some disciplines. However, construction productivity in some disciplines was much worse than U.S. projects. These points are especially applicable when considered and measured at the discipline level for both engineering and construction labour productivity. However, it should be noted that this research measured productivity as a ratio of direct work hours to issued for construction (IFC) quantities for engineering and to installed quantities for construction. 5.3 Impact Factors An array of factors exist that impact the performance of capital projects. Using the subjective evaluation of experienced project professionals, this research was able to categorize the primary factors impacting the cost, schedule, and construction productivity of Alberta-based projects. Surprisingly, there was a fair amount of consistency with the objective findings of this research. The top five ranked factors are listed in Table 5-1 according to their impact on cost, schedule, and construction productivity performance. In many cases, the top five factors were contrary to popular opinion (as surveyed) as they dealt with both managerial and site-related issues. 51

61 Table 5-1 The Top 5 Factors Affecting Cost, Schedule or Productivity Rank Cost Schedule Productivity 1 Amount of Unplanned Overtime 2 % Engineering completion prior to Construction Start % Engineering completion prior to Construction Start Business Market Conditions % Engineering completion prior to Construction Start Amount of Unplanned Overtime 3 Business Market Conditions Craft Labour Skill Business Market Conditions 4 Craft Labour Skill Quality of Field Level Supervision 5 Coordination with Plant Shutdown Weather Conditions Quality of Field Level Supervision Craft Labour Skill 5.4 Project Management Project management best practices make a huge difference in the performance of capital projects. Better implementation of Project Risk Assessment is shown to significantly reduce project cost growth. Constructability reviews can reduce project schedule growth as much as 50%. Not surprisingly, planning for startup improves startup phase cost performance. While insufficient data were available to examine the effects of workface planning on project performance, it is believed that its influence may be limited in an environment where effective project management is lacking. The dedicated implementation of proven project management practices such as front-end planning, constructability reviews, and project risk assessment during a project s earliest phases will maximize the project s potential to achieve high levels of performance. 52

62 6 Conclusions and Recommendations In recent years, numerous global forces have been at work dramatically altering the worldwide marketplace for energy. These forces have also led to a significant increase in the amount of investment and project activity in Alberta surrounding its oil sands resources. Owner companies holding leases in the oil sands accelerated their development of capital projects needed for increased production during the study period. This acceleration of the pace of development may be explanatory to the findings and results of this study. To be sure, the increased pace and amount of capital projects in Alberta resulted in many effects observed in this research. These effects were further compounded by extremes experienced in Alberta related to such things as labour availability, harsh weather conditions, and remote project locations, amongst others. Yet, the benefit of this research is that it was able to objectively quantify the performance of observed projects and the impact of certain factors submitted for this study. The research began with an overarching focus on engineering and construction productivity. The implied belief was that productivity in Alberta was a prime component in the observed performance of its capital projects. The definitions of productivity adopted by COAA s benchmarking committee were those used for many years by CII. Notably, these primarily measured direct productivity, that is, the ratio of work hours to installed quantities. By definition, they did not include indirect labour or costs. Because the COAA productivity definitions were common to CII, comparisons to U.S. productivity data were possible. For the projects included in this study, actual direct productivity for both engineering and construction was very similar to that observed in the United States. However, the observed project performance between Alberta and U.S. projects was differential in favor of the latter. Further investigation revealed that Alberta projects typically overestimate their direct productivity. For example, on average, actual construction productivity was 45% and 22% worse than estimated for concrete and steel craft workers, respectively. This overestimation is compounded by much higher amounts of indirect labour and cost required for Alberta projects when compared to U.S. projects. On average, 25% of hours worked were for indirect labour and almost half of the projects submitted experienced construction indirect costs more than 25% higher than estimated. Taken together, the underestimation of total labour required yielded significant resource peaks much higher than estimated which showed strong correlations with upward project cost growth. The analysis of this trend indicates that projects experiencing labour (direct and indirect) peaks of 50% more than estimated experienced project cost growth approaching 40%. 53

63 It is worth mentioning that all the projects reported for this research that were executed in Alberta used cost reimbursable contracts for their construction phase. This may have an impact on some of the project performance metrics reported here. Importantly, the Alberta-based projects presented here consistently demonstrate that large projects often experience much higher amounts of construction indirect cost growth when compared with their smaller counterparts. Given the anticipated number of planned projects exceeding $1 Billion (CDN) of total installed cost, the proper estimation and control of both direct and indirect costs is paramount. Future projects should not experience a range of project cost growth from -25% to 69% if the learnings from this report are applied effectively. Predictability in estimating and project management is needed. Here, management best practices such as project risk assessment, planning for startup, and constructability reviews are already showing significant abilities to impact project performance. Proper application of best practices would alleviate situations where 19% average cost growth was addressed using only 8% contingency (on average), for example. Indeed, improvement is needed in management-related aspects of planning, estimating, and controlling work. This is not just a contractor issue; owners routinely gave full funding authorization to projects with as little as 10% engineering complete. A thorough evaluation of management policies and procedures is recommended. The path forward is bright. While a focus on improved engineering and construction productivity is always warranted, quicker improvement is possible through increased focus on application of better project management practices. For owners, adherence to effective planning through asset development processes (ADP s) tailored to Alberta projects may be helpful. For contractors, revised emphasis on effective project execution plans (PEP s) may be needed. Intensive implementation of industry best practices and stern adherence to basic project management practices is recommended. Fortunately, the lessons of the past few years have created an improved awareness and added experience to the abilities of Alberta-based companies and personnel to manage the unique projects found in Alberta. Regardless, the only way to truly and objectively know whether or not project execution is improving is through continued measurement. Continued use of benchmarking products in current and additional aspects will generate improved intelligence concerning Alberta-based projects. There is ample reason to suspect that tomorrow s projects will be much better than those executed today. 54

64 Appendices 55

65 Appendix A: Summary of Correlation between Project Performance and Related Factors of Alberta Based Projects Table A- 1 Correlations of Project Characteristics with Project Performance Project Characteristics Performance Metric 1 Total Project Cost ($M CDN) Total Project Duration (weeks) Construction Duration (weeks) % Contingency Budget % Eng. completed before con. started N r N r N r N r N r COST Project Cost Growth Project Budget Factor Construction Phase Cost Growth * Construction Indirect Cost Growth * Startup Cost Growth SCHEDULE Project Schedule Growth Project Schedule Factor Construction Phase Schedule Growth Startup Schedule Growth CHANGES Total Change Cost Factor Development Change Cost Factor Scope Change Cost Factor REWORK Field Rework Cost Factor SAFETY Lost time Frequency (LTF) Lost Time Severity (LTS) PRODUCTIVITY Engineering Productivity (EPM Index) Construction Productivity (CPM Index) * Metric and phase definitions are provided in Appendix A. indicate small sample size (N<8). r = Pearson correlation; Shading indicates statistically significant correlation between factors and performance metrics at alpha level of * indicates variables and relationship which their boxplots or scatter plots with statistical results provided in chapter 4. 56

66 Table A- 2 Correlations of Project Characteristics with Project Performance (cont d) Performance Metrics % Modularization * % Offsite Work-hours Project Characteristics % Construction Indirect/Direct WH* % Construction Ind./ Dir. Cost* Actual/ Est. Peak Workforce % Scaffolding/ Direct WH N r N r N r N r N r N r COST Project Cost Growth * Project Budget Factor Construction Phase Cost Growth Construction Indirect Cost Growth Startup Cost Growth SCHEDULE Project Schedule Growth Project Schedule Factor * Construction Phase Schedule Growth Startup Schedule Growth CHANGES Total Change Cost Factor Development Change Cost Factor Scope Change Cost Factor REWORK Field Rework Cost Factor SAFETY Lost time Frequency (LTF) Lost Time Severity (LTS) PRODUCTIVITY Engineering Productivity (EPM Index) Construction Productivity (CPM Index)

67 Table A- 3 Correlations of Practices with Project Performance Practices Performance Metrics FEP PDRI PRA Team Building Alignment Design for Maintainability Constructability Material Mgmt. N r N r N r N r N r N r N r N r COST Project Cost Growth * Project Budget Factor Construction Phase Cost Growth Construction Indirect Cost Growth Startup Cost Growth SCHEDULE Project Schedule Growth * Project Schedule Factor Construction Phase Schedule Growth Startup Schedule Growth CHANGES Total Change Cost Factor Development Change Cost Factor Scope Change Cost Factor REWORK Field Rework Cost Factor SAFETY Lost time Frequency (LTF) Lost Time Severity (LTS) PRODUCTIVITY Engineering Productivity (EPM Index) Construction Productivity (CPM Index)

68 Table A- 4 Correlations of Practices with Project Performance (cont d) Performance Metrics Change Mgmt. Zero Accident Tech. Practices Quality Mgmt. A/I Technology Planning for Startup PPMOF Workface Planning N r N r N r N r N r N r N r COST Project Cost Growth Project Budget Factor Construction Phase Cost Growth Construction Indirect Cost Growth Startup Cost Growth * SCHEDULE Project Schedule Growth Project Schedule Factor Construction Phase Schedule Growth * Startup Schedule Growth CHANGES Total Change Cost Factor Development Change Cost Factor* Scope Change Cost Factor* REWORK Field Rework Cost Factor SAFETY Lost time Frequency (LTF) Lost Time Severity (LTS) PRODUCTIVITY Engineering Productivity (EPM Index) Construction Productivity (CPM Index)

69 Appendix B: Performance Metric Formulas and Definitions Performance Metric Category: COST Formula: Metric: Project Cost Growth Actual Total Project Cost - Initial Predicted Project Cost Initial Predicted Project Cost Metric: Delta Cost Growth Formula: Cost Growth Metric: Project Budget Factor Formula: Actual Total Project Cost Initial Predicted Project Cost +Approved Changes Metric: Delta Budget Factor Formula: 1- Budget Factor Metric: Phase Cost Factor Metric: Phase Cost Growth Definition of Terms Actual Total Project Cost: Owners o o All actual project cost from front end planning through startup Exclude land costs but include in-house salaries, overhead, travel, etc. Contractors Total cost of the final scope of work. Initial Predicted Project Cost: Owners Budget at the time of Project Sanction. Contractors Cost estimate used as the basis of contract award. Formula: Formula: Actual Phase Cost Actual Total Project Cost Actual Phase Cost Initial Predicted Phase Cost Initial Predicted Phase Cost Actual Phase Cost: All costs associated with the project phase in question. See the Project Phase Table in Appendix C for phase definitions. Initial Predicted Phase Cost: Owners Budget at the time of Project Sanction. Contractors Budget at the time of contract award. See the Project Phase Table in Appendix C for phase definitions. Approved Changes: Estimated cost of owner-authorized changes. 60

70 Performance Metric Category: SCHEDULE Metric: Project Schedule Growth Formula: Actual Total Proj. Duration - Initial Predicted Proj. Duration Initial Predicted Proj. Duration Metric: Delta Schedule Growth Formula: Schedule Growth Metric: Project Schedule Factor Formula: Actual Total Project Duration Initial Predicted Project Duration + Approved Changes Metric: Delta Schedule Factor Formula: 1- Schedule Factor Metric: Phase Duration Factor Formula: Actual Phase Duration Actual Overall Project Duration Metric: Total Project Duration Actual Total Project Duration (weeks) Metric: Phase Schedule Growth Formula: Actual Phase Duration Initial Predicted Phase Duration Initial Predicted Phase Duration Definition of Terms Actual Total Project Duration: (Detailed Engineering through Start-up) Owners Duration from beginning of detailed engineering to turnover to user. Contractors - Total duration for the final scope of work from mobilization to completion. Actual Overall Project Duration: (Front End Planning through Start-up) Unlike Actual Total Duration, Actual Overall Duration also includes time consumed for the Front End Planning Phase. Actual Phase Duration: Actual total duration of the project phase in question. See the Project Phase Table in Appendix C for phase definitions. Initial Predicted Project Duration: Owners Predicted duration at the time of Project Sanction. Contractors - The contractor's duration estimate at the time of contract award. Approved Changes Estimated duration of owner-authorized changes. 61

71 Performance Metric Category: SAFETY Metric: Lost Time Frequency (LTF) Formula: Total Number of Lost Time cases x 200,000 Total Site Work-Hours Metric: Medical Aid Frequency (MAF) Metric: First Aid Frequency (FAF) Metric: Total Recordable Injury Frequency (TRIF) Formula: Formula: Formula: Total Number of Medical Aid Cases x 200,000 Total Site Work-Hours Total Number of First Aid Cases x 200,000 Total Site Work-Hours Total Number of Recordable Cases x 200,000 Total Site Work-Hours Metric: Total Injury Frequency (TIF) Formula: Total number of all injury or illness cases x 200,000 Total Site Work-Hours Metric: Restricted Work Frequency (RWF) Formula: Total Number of Restricted Work Cases x 200,000 Total Site Work-Hours Metric: Lost Time Severity Rate (LTSR) Formula: Total Number of Lost Time Workdays x 200,000 Total Site Work-Hours Metric: Total Severity Rate (TSR) Formula: Total Number of Recordable Lost Time Cases and all Restricted Work Cases x 200,000 Total Site Work-Hours 62

72 Performance Metric Category: SAFETY (cont d.) Definition of Terms Lost Time Days: Equals the number of scheduled work days away from work as a result of an occupational injury or illness, disabling injury or illness which prevents a worker from reporting to work on next regularly scheduled. Medical Aid Case: Any occupational injury or illness requiring medical treatment administered by a physician, not including first aid treatment First Aid Case: Any one time treatment which does not require medical care or further medical aid e.g. minor scratches, cuts, burns, splinters. Recordable Case: A work event or exposure that is the discernable cause of an injury or illness or of a significant aggravation to a pre-existing condition. A recordable case requires medical aid, restricted work in relation to either medical aid or lost time, or fatality. Total number of all injury or illness cases: Equals the number of lost time (LT) cases, medical aid (MA) cases, first aid (FA) cases and the number of restricted work cases for lost time (RWLT), medical aid (RWMA) and first aid (RWFA). Total Number of Restricted Work Cases: Equals the number of restricted work lost time cases, restricted work medical aid cases and restricted work first aid cases. Lost Time Case: Lost Time cases are the result of an occupational injury or illness including any disabling injury which prevents a worker from reporting to work on his/her next regularly scheduled. Restricted Work Case: Includes restricted work lost time cases, restricted work medical aid cases and restricted work first aid cases. Restricted Work Days: Equals the number of scheduled work days that the worker was unable to work their regular duties as a result of an injury or illness as defined in restricted work. Total Number of Recordable Lost Time Cases and all Restricted Work Cases: Includes the number of lost workdays plus the number of restricted work days for all lost time, medical aid and first aids. 63

73 Performance Metric Category: CHANGES Metric: Scope Change Cost Factor Metric: Project Development Change Cost Factor Formula: Formula: Total Cost of Scope Changes Actual Total Project Cost Total Cost of Project Development Changes Actual Total Project Cost Definition of Terms Total Cost of Scope Changes: Total cost impact of scope and project development changes. Total Cost of Project Development Changes: Total cost impact of project development changes. Actual Total Project Cost: Owners o All actual project cost from front end planning through startup o Exclude land costs but include in-house salaries, overhead, travel, etc. Contractors Total cost of the final scope of work. Performance Metric Category: REWORK Metric: Total Field Rework Factor Formula: Total Direct Cost of Field Rework Actual Construction Phase Cost Definition of Terms Total Direct Cost of Field Rework: Total direct cost of field rework regardless of initiating cause. Actual Construction Phase Cost: All costs associated with the construction phase. See the Project Phase Table in Appendix C for construction phase definition. 64

74 Construction Productivity and Total Installed Unit Cost (TIUC) Metrics Categories and Breakouts Concrete - Total Concrete o Slabs (CM) On-Grade (CM) Elevated Slabs/On Deck (CM) Area Paving (CM) o Foundations (CM) < 4 CM 4 15 CM CM 38 CM o Concrete Structures (CM) Structural Steel - Total Structural Steel (MT) o Structural Steel (MT) o Pipe Racks & Utility Bridges (MT) o Miscellaneous Steel (MT) Instrumentation - Loops (Count) - Devices (Count) Piping - Small Bore (2-1/2 & Smaller) (LM) o Carbon Steel (LM) o Stainless Steel (LM) o Chrome (LM) o Other Alloys (LM) o Non Metallic (LM) - Inside Battery Limits (ISBL) (LM) Large Bore (3 & Larger) (LM) o Carbon Steel (LM) o Stainless Steel (LM) o Chrome (LM) o Other Alloys (LM) o Non Metallic (LM) - Outside Battery Limits (OSBL) (LM) Large Bore (3 & Larger) (LM) o Carbon Steel (LM) o Stainless Steel (LM) o Chrome (LM) o Other Alloys (LM) o Non Metallic (LM) - Heat Tracing Tubing (LM) Electrical - Total Electrical Equipment (Each) o Panels and Small Devices (Each) o Electrical Equipment below 1kV (Each) o Electrical Equipment over 1kV (Each) - Conduit (LM) o Exposed or Above Ground Conduit (LM) o Underground, Duct Bank or Embedded Conduit (LM) - Cable Tray (LM) - Wire and Cable (LM) o Control Cable (LM) o Power and Control Cable below 1kV (LM) o Power Cable above 1kV (LM) - Transmission Line (LM) o High Voltage above 25kV (LM) - Other Electrical Metrics o Lighting (Each) o Grounding (LM) o Electrical Heat Tracing (LM) Equipment - Pressure Vessels (Field Fab.& Erected) (Each), (MT) - Atmospheric Tanks (Shop Fabricated) (Each), (MT) - Atmospheric Tanks (Field Fabricated) (Each), (MT) - Heat Transfer Equipment (Each), (MT) - Boiler & Fired Heaters (Each), (MT) - Rotating Equipment (Each), (HP) - Material Handling Equipment (Each), (MT) - Power Generation Equipment (Each), (kw) - Other Process Equipment (Each), (MT) - Modules & Pre-assembled Skids (Each), (MT) Insulation - Equipment o Insulation Equipment (SM) - Piping o Insulation Piping (ELM) Module Installation - Pipe Racks (MT) - Process Equipment Modules (MT) - Building (SM) Scaffolding - Scaffolding Work-Hours/ Total Direct Hours Construction Work-Hours - Construction Indirect/ Direct Work-Hours Construction Productivity Unit Rate = Productivity Estimating Performance = Direct Work hours Installed Quantity Actual Productivity Rate Estimated Productivity Rate Cost Estimating Performance = Actual TIUC Estimated TIUC 65

75 Engineering Productivity Metrics Categories and Breakouts Concrete - Total Concrete (CM) o Total Slabs (CM) Ground and Supported Slab (CM) Area Paving (CM) o Total Foundations (except Piling) (CM) Foundation (<4CM) (CM) Foundation ( 4CM) (CM) o Concrete Structures (CM) o Total Piling (Each) Structural Steel - Total Steel (MT) o Combined Structural Steel / Pipe Racks & Utility Bridges (MT) Structural Steel (MT) Pipe Racks & Utility Bridges (MT) o Miscellaneous Steel (MT) Electrical - Total Electrical Equipment (Each) o Electrical Equipment 600V & Below (Each) o Electrical Equipment Over 600V (Each) - Conduit o Conduit (LM) o Conduit (Number of Runs) - Cable Tray (LM) - Wire & Cable o Wire & Cable (LM) o Wire & Cable (Number of Terminations) - Other Electric Metric o Lighting (Each Fixtures) o Electrical Heat Tracing (LM) Piping - Total Piping (LM) o Small Bore (2-1/2 and Smaller) (LM) o Large Bore (3 and Larger) (LM) o Engineered Hangers and Supports (Each) - Heat Tracing Tubing (LM) Instrumentation - Loops (Count) - Tagged Devices (Each) - I/O (Count) Equipment (Individual Design and Total Quantity) - Total Equipment (Each) o Pressure Vessels (Each) o Atmospheric Tanks (Each) o Heat Transfer Equipment (Each) o Boiler & Fired Heaters (Each) o Rotating Equipment (Each) o Material Handling Equipment (Each) o Power Generation Equipment (Each) o Other Process Equipment (Each) o Vendor-Designed Modules & Pre- Assembled Skids (Each) Engineering Productivity = * Per Design Component ** IFC (Issued for Construction) Direct Design-Hours* IFC Quantity** 66

76 Project Phase Definition Table Project Phase Start/Stop Typical Activities & Products Typical Cost Elements Front End Planning Typical Participants: Owner Personnel Planning Consultants Constructability Consultant Alliance / Partner Start: Single project adopted and Formal project team established Stop: Project Sanction Options Analysis Life-cycle Cost Analysis Project Execution Plan Appropriation Submittal Pkg P&IDs and Site Layout Project Scoping Procurement Plan Arch. Rendering Owner Planning Team Personnel Expenses Consultant Fees & Expenses Environmental Permitting Costs Project Manager / Construction Manager Fees Licensor Costs Detail Engineering Typical Participants: Owner Personnel Design Contractor Constructability Expert Alliance / Partner Start: Contract award to engineering firm Stop: Release of all approved drawings and specs for Construction (or last package for fast-track) Drawing & spec. preparation Bill of material preparation Procurement Status Sequence of operations Technical Review Definitive Cost Estimate Owner Project Management Personnel Designer Fees Project Manager / Construction Manager Fees Procurement Typical Participants: Owner personnel Design Contractor Alliance / Partner Start: Procurement plan for engineered equipment Stop: All major equipment has been delivered to site Vendor Qualification Vendor Inquiries Bid Analysis Purchasing Expediting Engineered Equipment Transportation Vendor QA/QC Owner project management personnel Project Manager / Construction Manager fees Procurement & Expediting personnel Engineered Equipment Transportation Shop QA / QC Note: The demolition / abatement phase should be reported when the demolition / abatement work is a separate schedule activity (potentially paralleling the design and procurement phases) in preparation for new construction. Do not report the demolition / abatement phase if the work is integral with modernization or addition activities. 67

77 Project Phase Table (Cont.) Project Phase Start/Stop Typical Activities & Products Typical Cost Elements Construction Typical Participants: Owner personnel Design Contractor (Inspection) Construction Contractor and its subcontractors Start: Commencement of foundations or driving Piles Stop: Mechanical Completion Set up trailers Procurement of bulks Issue Subcontracts Construction plan for Methods/Sequencing Build Facility & Install Engineered Equipment Complete Punchlist Demobilize construction equipment Warehousing Owner project management personnel Project Manager / Construction Manager fees Building permits Inspection QA/QC Construction labour, equipment & supplies Bulk materials (including freight) Construction equipment (including freight) Contractor management personnel Warranties Start-up / Commissioning Note: Does not usually apply to infrastructure or building type projects Typical Participants: Owner personnel Design Contractor Construction Contractor Training Consultant Equipment Vendors Start: Mechanical Completion Stop: Custody transfer to user/operator (steady state operation) Testing Systems Training Operators Documenting Results Introduce Feedstocks and obtain first Product Hand-off to user/operator Operating System Functional Facility Warranty Work Owner project management personnel Project Manager / Construction Manager fees Consultant fees & expenses Operator training expenses Wasted feedstocks Vendor fees 68

78 Appendix C: Glossary General Terms Addition (Add-on) A new addition that ties in to an existing facility, often intended to expand capacity. Grass Roots, Green Field A new facility from the foundations and up. A project requiring demolition of an existing facility before new construction begins is also classified as grass roots. Modernization, Renovation, Upgrade A facility for which a substantial amount of the equipment, structure, or other components is replaced or modified, and which may expand capacity and/or improve the process or facility. Percent Offsite Construction Labour Hours The level of offsite labour hours for building modules. This value should be determined as a ratio of the offsite labour hours of all modules divided by total construction hours. Rework - is defined as activities in the field that have to be done more than once in the field or activities which remove work previously installed as part of project. Total Construction Hours The summation of all direct and indirect hours associated with the construction phase. Project Delivery System Design-Bid-Build Serial sequence of design and construction phases; Owner contracts separately with designer and constructor. Design-Build (or EPC) Overlapped sequence of design and construction phase; procurement normally begins during design; owner contracts with Design-Build (or EPC) contractor. CM at Risk Overlapped sequence of design and construction phases; procurement normally begins during design; owner contracts separately with designer and CM at Risk (constructor). CM holds the contracts. Multiple Design-Build Overlapped sequence of design and construction phases; procurement normally begins during design; owner contracts with two Design-Build (or EPC) contractors, one for process and one for facilities. Parallel Primes Overlapped sequence of design and construction phases; Procurement normally begins during design. Owner contracts separately with designer and multiple prime constructors. Cost Definition Construction Costs include the costs of construction activities from commencement of foundation or driving piles to mechanical completion. The costs include construction project management, construction labour, and also equipment& supplies costs that are used to support construction operations and removed after commissioning. See Instruction for Construction Direct and Indirect Costs for detail of typical cost element. Contingency Contingency is defined as an estimated amount included in the project budget that may be required to cover costs that result from project uncertainties. These uncertainties may result from incomplete design, unforeseen and unpredictable conditions, escalation, or lack of project scope definition. The amount of contingency usually depends on the status of design, procurement and construction, and the complexity and uncertainties of the component parts of the project. 69

79 Direct Costs Direct costs are those which are readily or directly attributable to, or become an identifiable part of, the final project (e.g., piping labour and material) [AACE]. Direct Cost of Field Rework The sum of those costs associated with actual performance of tasks involved in rework. Examples include: Labour, Materials, Equipment, Supervisory personnel, Associated overhead cost. Modularization Modularization refers to the use of offsite construction. For the purposes of the benchmarking data, modularization includes all work that represents substantial offsite construction and assembly of components and areas of the finished project. Examples that would fall within this categorization include: Skid assemblies of equipment and instrumentation that naturally ship to the site in one piece, and require minimal on-site reassembly. Super-skids of assemblies of components that typically represent substantial portions of the plant, intended to be installed in a building. Prefabricated modules comprising both industrial plant components and architecturally finished enclosures. Modularization does not include offsite fabrication of components. Examples of work that would be excluded from the definition of modularization include: Fabrication of the component pieces of a structural framework Fabrication of piping spool-pieces Indirect Costs Indirect costs are all costs that cannot be attributed readily to a part of the final product (e.g., cost of managing the project) [AACE]. Schedule Definition Project Sanction is defined as the milestone event at which the project scope, budget, and schedule are authorized. Project Sanction is the start of the execution phase of the project. Commissioning and Startup The transitional phase between construction and commercial operations; major steps include turnover, checkout, commissioning, and initial operations. Commissioning is the integrated testing of equipment and facilities that are grouped together in systems prior to the introduction of feedstocks. Detail Engineering Detail engineering is the project phase initiated with a contract to the firm providing detail engineering for the project. The typical activities included in this phase are: preparation of drawings, specifications, bill of materials, development of a definitive cost estimate, technical reviews, and engineering procurement functions. The detail engineering phase terminates with release of all approved drawings and specifications for construction. Mechanical Completion - The point in time when a plant is capable of being operated although some trim, insulation, and painting may still be needed. This occurs after completion of pre commissioning. Changes Definition Change - A change is any event that results in a modification of the project work, schedule or cost. Owners and designers frequently initiate changes during design development to reflect changes in project scope or preferences for equipment and materials other than those originally specified. Contractors often initiate changes when interferences are encountered, when designs are found to be not constructable, or other design errors are found. Change Order - A contractual modification executed to document the agreement and approval of a change (See definition of Change above). 70

80 Project Development Changes Project Development Changes include those changes required to execute the original scope of work or obtain original process basis. Examples include: 1) Unforeseen site conditions that require a change in design / construction methods 2) Changes required due to errors and omissions 3) Acceleration 4) Change in owner preferences 5) Additional equipment or processes required to obtain original planned throughput 6) Operability or maintainability changes. (See Change above) Scope Changes Scope Changes include changes in the base scope of work or process basis. Examples include: 1) Feedstock change, 2) Changed site location, 3) Changed throughput, 4) Addition of unrelated scope Practice Definition Front End Planning is the essential process of developing sufficient strategic information with which owners can address risk and make decisions to commit resources in order to maximize the potential for a successful project. Front End Planning is also known as pre-project planning, front end loading, feasibility analysis, conceptual planning/ schematic design, and early project planning. Project Risk Assessment Project risk assessment is the process to identify, assess and manage risk. The project team evaluates risk exposure for potential project impact to provide focus for mitigation strategies. Team Building is a project- focused process that builds and develops shared goals, interdependence, trust and commitment, and accountability among team members and that seeks to improve team members problem- solving skills. Alignment during Front End Planning is the condition where appropriate project participants are working with acceptable tolerances to develop and meet a uniform defined and understood set of project objectives. Constructability is the effective and timely integration of construction knowledge into the conceptual planning, design, construction, and filed operations of a project to achieve the overall project objectives in the best possible time and accuracy at the most cost- effective levels. Design for Maintainability Design for maintainability is the optimum use of facility maintenance knowledge and experience in the design/engineering of a facility to pertain the ease, accuracy, safety and economy in the performance of maintenance action; a design parameter related to the ability to maintain. Material Management the planning, controlling, and integrating of the materials takeoff, purchasing, economic, expediting, transportation, warehousing, and issue functions in order to achieve a smooth, timely, efficient flow of materials to the project in the required quantity, the required time, and at an acceptable price and quality, and the planning and controlling of these functions (CII Publication SP-4) Project Change Management is the process of incorporating a balanced change culture of recognition, planning, and evaluation of project changes in an organization to effectively manage project changes. Practices related to the management and control of both scope changes and project changes. Zero Accident Techniques include the site- specific safety programs and implementation, auditing, and incentive efforts to create a project environment and a level of that embraces the mind set that all accidents are preventable and that zero accidents is an obtainable goal. 71

81 Quality Management Quality management incorporates all activities conducted to improve the efficiency, contract compliance and cost effectiveness of design, engineering, procurement, QA/QC, construction and startup elements of construction projects. Automation/Integration (AI) Technology The Automation and Integration Technology practice addresses the degree of automation/level of use and integration of automated systems for predefined tasks/work functions common to most projects. Planning for Startup is the effectiveness of planning on startup activities that facilitate the implementation of the transitional phase between plant construction completion and commercial operations, including all of the activities bridging these two phases. Critical steps within the startup phase include systems turnover, checkout of systems, commissioning of systems, introduction of feed stocks, and performance testing. Prefabrication/ Preassembly/ Modularization Prefabrication/Preassembly/Modularization (PPMOF) is defined as several manufacturing and installation techniques, which move many fabrication and installation activities from the plant site into a safer and more efficient environment. For each technique, more specific definitions are provided below. Prefabrication: a manufacturing process, generally taking place at a specialized facility, in which various materials are joined to form a component part of a final installation. Prefabricated components often involve the work of a single craft. Preassembly: a process by which various materials, prefabricated components, and/or equipment are joined together at a remote location for subsequent installation as a sub-unit: generally focused on a system. Module: a major section of a plant resulting from a series of remote assembly operations and may include portions of many systems: usually the largest transportable unit or component of a facility. Offsite Fabrication: the practice of preassembly or fabrication of components both off the site and onsite at a location other than at the final installation location. This practice consists of two part, constructability at AFE phase and constructability at mechanical completion. Please fill out one part of this practice according to your current project phase. Workface Planning The process of organizing and delivering all elements necessary, before work is started, to enable craft persons to perform quality work in a safe, effective and efficient manner Engineering Productivity Engineering Direct Work hours - should include all detailed design hours used to produce deliverables including site investigations, meetings, planning, constructability, RFIs, etc., and rework. Specifically exclude work hours for operating manuals and demolition drawings. - Engineering work hours reported should only be for the categories requested and may not equal the total engineering work hours for the project. (See Instructions for Computation of Work hours and Rework-Hours reference table) - Exclude the following categories: architectural design, plumbing, process design, civil/site prep, HVAC, insulation and paint, sprinkler/deluge systems, etc. Within a category, direct work hours that cannot be specifically assigned into the provided classifications, and have not been excluded, should be prorated based on known work hours or quantities as appropriate. IFC Drawing Issued for Construction drawings. Construction Productivity Actual Quantities and Work hours - are all quantities and work hours of actual installation and include rework hours for these quantities and work-hours. 72

82 Estimated Productivity are the estimated productivity of direct labour work hours required for installation according to the estimated quantity. For owners: For contractors: Estimated Quantity, Work hours and Total Installed Unit Cost at the time of Project Sanction (or as soon as available following sanction) Estimated Quantity, Work hours and Total Installed Unit Cost used as the basis of Contract Award. Estimated Quantities and Work hours are the estimated quantity to be installed, the estimated work hours required for the installation and include all change orders. Estimated Total Installed Unit Cost including labour and material cost at the time of project sanction (or as soon as available following sanction). Estimated Total Installed Unit Costs (TIUC) is the burdened direct cost of labour, material and equipment by pro rata share which are directly attribute to, or become a part of the final product including overhead and profit at the time of project sanction (or as soon as available following sanction). Actual Total Installed Unit Costs (TIUC) the burdened direct cost of labour, material and equipment by pro rata share which are directly attribute to, or become a part of the final product including overhead and profit from both direct hire and subcontract. The direct labour costs are considered as the costs of the labours listed as Direct in the Instructions for Computation of Actual Work-Hours, Rework-Hours, and Installed Costs table in Construction Productivity Section. 73

83 References AACE (2004). Estimating Lost Labour Productivity in Construction Claims. TCM Framework: 6.4- Forensic Performance Assessment, AACE International Recommended Practice No. 25R-03. Agresti, A. and Finlay, B. (1999). Statistical methods for the social sciences, 3rd ed., Prentice Hall, Inc., Upper Saddle River, NJ, 07458, Prentice Hall, Inc., Upper Saddle River, NJ, Alberta Finance and Enterprise (AFE) (2008). Highlights of the Highlights of the Alberta Economy, Aminah R. (2005). Results of a Survey of Performance Deviations on Major Industrial Construction Projects in Alberta ( ). Report to Construction Owner Association of Alberta, Spring, COAA (2008). Construction Owners Association of Alberta, Field, A. (2005). Discovering Statistics Using SPSS. Second Edition, SAGE Publications. Flyvbjerg, B., Bruzelius, N., & Rothengatter, W. (2003). Megaprojects and risk: An anatomy of ambition. Cambridge, UK: Cambridge University Press. Kellogg, J., Taylor, D., and Howell, G. (1981). Hierarchy Model of Construction Productivity. Journal of the Construction Division, 107(1), March 1981, pp ASCE, OSDG (2008). Oil Sands Important to Canada s Present and Future. Oil Sands Developers Group. [email protected]. 74

84 Companies Participating in COAA Benchmarking Training Sessions: Air Products & Chemicals Inc. Alberta Economic Development Alberta Infrastructure and Transportation Ascension Systems Inc. BA Energy Bantrel BIRD Construction Company Canadian Natural Resources Canonbie Contracting Ltd. Cobra Group of Companies Colt Corporation ConocoPhillips CPI Construction DriverCheck Enbridge Pipelines Inc. EPCOR Esso Petroleum Canada Flint Energy Fluor Corporation Husky Energy Inc. IBEW Local 424 Imperial Oil Resources Ltd. Intergraph Corporation Jacobs Kellogg Brown & Root Laird Electric Inc. Ledcor Murdoch International Inc. Nexen Inc. OPTI Canada Inc., PCL Petro-Canada Revay and Associates Limited RSC Equipment Rental SafeTech Consulting Group Ltd. Shell Canada Limited Stantec Steeplejack Industrial Group Inc. Suncor Energy Inc. Syncrude Canada Ltd. Tartan Canada Corporation ThyssenKrupp Safway, Inc. TransCanada Pipelines, Ltd. Westwood Companies WorleyParsons Limited CII Staff: Stephen Mulva, Ph.D., Associate Director Benchmarking and Metrics Stephen R. Thomas, Ph.D., Associate Director, Research, Academic, and Breakthrough Jiukun Dai, Ph.D., Research Engineer Arpamart Chanmeka, Graduate Research Assistant Deborah DeGezelle, Senior Systems Analyst Hong Zhao, Senior Systems Analyst 75

85 The Construction Industry Institute The University of Texas at Austin 3925 West Braker Lane Austin, TX (512) FAX (512) Construction Industry Institute Bureau of Engineering Research The University of Texas at Austin