Benchmarking Initiatives in the Construction Industry: Lessons Learned and Improvement Opportunities

Transcription

1 Benchmarking Initiatives in the Construction Industry: Lessons Learned and Improvement Opportunities Dayana B. Costa 1 ; Carlos T. Formoso 2 ; Michail Kagioglou 3 ; Luis F. Alarcón 4 ; and Carlos H. Caldas, M.ASCE 5 Abstract: Benchmarking is a systematic process of measuring and comparing an organization s performance against that of other similar organizations in key business activities. The lessons learned from other companies can be used to establish improvement targets and to promote changes in the organization. The benchmarking process can create a fertile ground for ideas, but only in a receptive environment; companies that share good practices and compare their performance against others benefit most. Recently, industry groups in several different countries have initiated benchmarking programs focused mainly on construction performance measures. This paper describes the scope of these initiatives and discusses the lessons learned and improvement opportunities that were identified in their design and implementation. This investigation is focused on four initiatives, carried out in Brazil, Chile, the United Kingdom, and the United States. This paper concludes by recommending some further directions on this research topic. DOI: / ASCE X :4 158 CE Database subject headings: Bench marking; Construction industry; Evaluation; Organizations. Introduction Despite their importance, performance measurement data have not been widely identified and collected in construction companies. As a result, information on the performance of the construction industry as a whole tends to be scarce. This is because, to a great extent, managers have a reluctant attitude and lack training Formoso and Lantelme In fact, several companies measure and control a wide range of project variables, but only a few have performance measurement processes, which should provide key support for decision-making processes Lynch and Cross 1995; Kaplan and Norton Moreover, some companies have too many measures, most of them linked to support functions rather than to the key processes that companies most need to control Costa and Formoso This tends to make it difficult for the company staff to determine priorities and to define the key indicators that should be used for 1 Ph.D. Candidate, Federal Univ. of Rio Grande do Sul UFRGS, Building Innovation Research Unit NORIE. dayana@ cpgec.ufrgs.br 2 Professor, Federal Univ. of Rio Grande do Sul UFRGS, Building Innovation Research Unit NORIE. formoso@ufrgs.br 3 Manager of SCRI/Reader, Univ. of Salford, Salford Centre for Research and Innovation SCRI in the Build and Human Environment. m.kagioglou@salford.ac.uk 4 Professor, Univ. Católica de Chile, Escuela de Ingeniería. lalarcon@ing.puc.cl 5 Assistant Professor, Univ. of Texas at Austin, Dept. of Civil, Architectural, and Environmental Engineering. caldas@ mail.utexas.edu Note. Discussion open until March 1, Separate discussions must be submitted for individual papers. To extend the closing date by one month, a written request must be filed with the ASCE Managing Editor. The manuscript for this paper was submitted for review and possible publication on July 27, 2005; approved on December 16, This paper is part of the Journal of Management in Engineering, Vol. 22, No. 4, October 1, ASCE, ISSN X/2006/ /$ comparison with other companies Schiemann and Lingle However, the effective implementation of performance measurement systems is not simply a matter of selecting the right measures. It also implies a much deeper change in the decisionmaking processes and the learning approaches adopted within an organization Lantelme et al An important role of performance measurement is to enable a company to do benchmarking. Benchmarking is a systematic process of measuring and comparing an organization s performance against that of other similar organizations in key business activities. Then, lessons learned from other organizations are used to establish improvement targets and to promote changes in the organization KPI 2000; Barber Benchmarking must be an integral part of the planning and ongoing process of improvement to ensure a focus on the external environment as well as to strengthen the use of factual information in developing plans Camp According to Garvin 1993, the greatest benefits of the benchmarking process are that it allows more efficient work and that it involves managers proactively in the process rather than depending exclusively on results. It can be used to improve performance by helping managers understand the methods and practices required to achieve higher performance levels Camp Therefore, the general purpose of benchmarking its function as an assessment process is to encourage continuous learning for both managers and organizations Barber The benchmarking process can also generate innovation, but only in a receptive environment Garvin This open environment can be created in benchmarking clubs, forums for individuals to learn from the best practices within a local support network Constructing Excellence In general, these initiatives involve a group of similar companies that compare results and share practices with the goal of continuous improvement. In recent years, benchmarking programs have been established in countries such as Australia Karim et al. 1997, Brazil Costa et al. 2004, Chile CDT 2002, Denmark Byggeriets Evaluerings 158 / JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006

2 Center 2002, the United Kingdom Constructing Excellence 2004, the United States CII 2000, Hong Kong, Singapore, and the Netherlands Bakens et al These programs have mainly been related to the creation and implementation of performance measurement systems. Such initiatives typically aim to 1 offer guidance for performance measurement; 2 provide benchmarks that can be used by individual companies to establish business goals and objectives; and 3 identify and disseminate the best practices in the industry through reports and benchmarking club networks. The objective of this paper is to describe the scope of these initiatives and to discuss the lessons learned and improvement opportunities that were identified in the design and implementation of these systems. This investigation is focused on four initiatives carried out in Brazil, Chile, the United Kingdom, and the United States. The Role of Performance Measurement in the Benchmarking Process The evolution of the benchmarking concept is similar and inextricably linked to the evolving pattern of performance measurement Anderson and McAdam In general, performance measurement is limited to traditional broad financial measures, such as return on investment, profit, and liquidity ratios, indexes that have been strongly criticized for being out of step with recent changes in the industry Kaplan and Norton Among other problems, these measures tend to be focused backward, make it difficult to trace operational costs, and fail to monitor the benefits of new investments Berliner and Brimson Moreover, existing performance measurement systems are often used for imposing centralized control Senge 1999 and, for that reason, are often rejected by employees. Several models aiming to redress these problems have been proposed to support the development of performance measurement systems; these models focus mostly on establishing a balanced set of measures that consider financial measures as well as client satisfaction, production efficiency, organizational learning, the environment, and the supply chain. For example, Lynch and Cross 1995 devised a pyramid of measures that integrates performance measurement throughout the hierarchy of the organization. Neely et al proposed a framework for specifying performance measures, while Maskell and Baggaley 2004 suggested a set of guidelines for devising and implementing performance measurement in lean production systems. Finally, Kaplan and Norton 1992 designed a performance measurement framework, called the Balanced Scorecard, which has received a favorable academic industry response. All these models and approaches are usually multidimensional in the sense that they focus on a broad set of both financial and nonfinancial measures and are concerned with different managerial levels. They provide mechanisms to facilitate the alignment of performance indicators with a company s strategic objectives, linking them to key managerial processes. Previous studies indicate that the use of performance measures has had a significant effect on the development and effectiveness of benchmarking; when performance is tracked, outdated uncompetitive management practices are identified, and changes are investigated Anderson and McAdam 2004; KPI 2000; CII 2000; Alarcón et al Therefore, companies must be flexible and able to adapt to unanticipated organizational changes; this in the long run will improve organizational competitiveness Anderson and McAdam Conversely, benchmarking adds value to performance measurement because it allows companies to compare their data; it also allows for better decision making based on these comparisons Beatham et al Authors concur to argue that most organizations involved in benchmarking initiatives tend to focus more on the comparison of performance metrics over time than on understanding and learning from other companies good practices Welch and Mann 2001; Hilton et al. 2000; Anderson and McAdam According to Anderson and McAdam 2004, the use of benchmarking should be extended beyond the comparison of lagging performance measures. Although lagging indicators are important to assess the achievement of a company s strategic objectives, leading performance measures are also necessary because they are proactive and preventive in nature. Leading measures help to anticipate the impact on future desired results Manoochehri Therefore, the benchmarking process should involve a set of indicators that includes both leading and lagging indicators. Creating Knowledge through the Benchmarking Process Spendolini 1994 found that, although the major focus of benchmarking is usually planning and organizing, one of its main objectives is to introduce new ideas to an organization. One of the implications of the benchmarking process is that organizations need to overcome their internal boundaries in order to assess opportunities and threats in the external environment. In this context, benchmarking can be used to identify innovation by helping management understand the external environment and by promoting organizational learning Spendolini Garvin 1993 also stresses learning from the experience and best practices of others as essential to promoting a learning organization. Barber 2004 argues that the benchmarking of managerial practices helps to accelerate and manage organizational changes by creating a culture of continuous improvement. To him, benchmarking properly fosters incremental changes along a gradient of continuous improvement and learning Zairi and Leonard Despite the arguments for the positive relationship between the benchmarking process and organizational learning, the literature shows that it is important to understand how practices and measures can be translated into practical knowledge; such useful information makes it possible to identify and adopt superior performance standards Hinton et al According to Lillrank 1995, the ability to understand practices and their application in a local context depends on how such transfers of knowledge happen. He suggests that practices can be more effectively carried across different contexts and cultures if they are first translated into abstract ideas, as illustrated in Fig. 1. According to Lillrank 1995, a successful abstraction must go beyond the simple description of the directly observable processes because in such cases know-how remains context bound. Effective transportation of concepts usually demands a deeper and, by implication, more abstract understanding of the meaning and dynamics of processes than simply viewing them in their original context Lillrank However, the level of abstraction is determined by the complexity of the system or idea itself. In this context, complexity refers to the number and types of social interfaces, as well as to the degree to which knowledge is tacitly embedded in management practices Lillrank Therefore, the transmission of knowledge within a benchmarking process is JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006 / 159

3 Table 1. Sources of Evidence for Each Benchmarking Initiative Initiatives KPI Sources of evidences Information was obtained from the project s website construction/kpi/ and published papers Beatham et al. 2004; Kagioglou et al Fig. 1. Knowledge transfer model adapted from Lillrank 1995 more likely to succeed when the essence of a particular practice has been grasped abstractly and translated into practical applications for other settings. Drew s research 1997 also finds that the benchmarking process is affected by a strong tacit element in the practices being studied. For example, a particularly difficult situation is encountered when operators are themselves not consciously aware of what they do or how it contributes to success. Consequently, large amounts of tacit knowledge in a business process add significantly to the difficulties of benchmarking that process. It is equally important that the benchmarking team share what they have learned with the aim of creating an atmosphere in which knowledge transfer is actively encouraged Hinton et al Learning cycles are set in motion once people absorb the transferred knowledge and apply it to their local conditions. NBS-Chile CII BM&M SISIND-NET Project One of the authors of the present report participated in the conception of the program. Information was also obtained from the project s website from published papers Alarcón et al. 2001, Ramírez et al. 2004, and also by means of semistructured interviews with representatives from seven construction companies involved with the coordination of the program. Information was obtained from the project s website and also from two interviews, one conducted as correspondence and one as conversation with the associate director of the program. Two authors of the present report have participated in the conception and implementation of this program. Information was also obtained from documents, meetings, site visits, interviews, the project s website norie/benchmarking, and the project database. Research Method Four benchmarking initiatives were analyzed: 1 Key Performance Indicators KPI from the United Kingdom; 2 the National Benchmarking System for the Chilean Construction Industry NBS-Chile ; 3 the Construction Industry Institute Benchmarking and Metrics CII BM&M from the United States; and 4 the Performance Measurement for Benchmarking in the Brazilian Construction Industry SISIND-NET Project. The information about the KPI initiative was obtained from its website, published papers, and reports. For the CII BM&M initiative, the information was obtained from its website, reports, and interviews with a project researcher and the associate director. The information about the NBS-Chile initiative was obtained from its websites, from published papers and reports, and from its existing database. Also, semistructured interviews were conducted with managers of seven construction companies involved in the program and the current coordinator of the initiative. Finally, two of the authors of the SISIND-NET Project have been directly involved in this project. Table 1 shows the main sources of evidence used for each initiative. Three main issues were chosen for analyzing each initiative Table 2 : 1 type of benchmarking; 2 scope of the performance measurement system; and 3 implementation of the initiatives. The data were analyzed in two stages. In the first one, each initiative was analyzed independently within-case study analysis to determine the results of each and to organize the existing data. In the second stage, the initiatives were compared cross-case study analysis to identify similar or divergent patterns between them. Description of the Benchmarking Initiatives in the Construction Industry Key Performance Indicators KPIs in the United Kingdom The KPI program was launched by the U.K. Best Practice Programme in This program is supported by the British Table 2. Main Issues Involved with the Analysis of the Benchmarking Initiatives Issues Within-case study analysis Cross case study analysis Type of benchmarking Principal purpose Sector, firm or project Principal purpose Scope of the PMS Development Types of measures of the system Common measures Lack of measures Implementation Data collection Positive issues of the PMS process Difficulties Data analysis process Availability of data Company involvement Difficulties 160 / JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006

4 government through national and regional offices. Recently, the Constructing Excellence body, the amalgamation of Rethinking Construction and the Construction Best Practice Programme CBPP, was created. Constructing Excellence aims to achieve a step change in construction productivity by tackling the market failures in the sector and by selling the business case for continuous improvement Constructing Excellence Four integrated programs were created aiming to deliver Constructing Excellence s strategic objectives: 1 innovation, aimed at identifying and promoting tomorrow s best practices; 2 productivity, aimed at improving the competitiveness of the U.K. construction industry; 3 best practice knowledge, aimed at creating continuous improvement through the sharing of best practices; and 4 engagement, aimed at working with people, businesses, and organizations to change the culture of the construction industry. These programs are complementary, and in each one is a set of integrated activities Constructing Excellence KPI is currently part of the Constructing Excellence productivity program. The first set of KPI data was produced in November 2000, and the current set of KPIs is based upon projects completed in 2002 KPI The design of the first set of KPIs was the result of an initiative involving extensive reviews of construction projects by a panel of experts, followed by the publication of a report. The set of KPIs is annually updated by the Construction Best Practice Programme Constructing Excellence To implement the KPIs, companies receive a support handbook, guidance for measurement, and access to online software. The companies are responsible for collecting data, introducing them into the database, and updating them. The software supports the analysis of a project s performance in relation to the benchmarks. The companies involved can also access reports and wall charts, containing graphs of performance ranking curve and radar chart for 10 key issues for the construction sector, such as client satisfaction, cost, and time. The wall charts show the benchmark scores and allow an organization s score to be benchmarked against a large sample across the industry. A few hundred companies have been voluntarily participating in the Best Practice Knowledge Program, testing it on demonstration projects. The companies present their projects for review by a panel of experts. The companies involved can participate in a benchmarking club and are given access to information from all main benchmarking initiatives, clubs, and organizations that provide services to the construction industry Constructing Excellence The clubs are local services, and each new club is formed by the people in the local construction industry who are inspired by the promise of benchmarking Constructing Excellence Companies join the KPI program for two main reasons: marketing advantages and improved performance opportunities. According to Beatham et al. 2004, the most significant problem with the KPIs in their current format is that they do not offer the opportunity for organizational change. They are designed to be used as post-result lagging KPIs. The authors recommend the use of leading measures to provide early warnings, identify potential problems, and highlight any need for further investigation. Kagioglou et al made the following observations regarding the KPI initiative in the United Kingdom: The KPIs are specific to projects and offer very little indication of the overall performance of the organizations from a business point of view, apart from the perspective of customer satisfaction; It is important not only to use the right measures to measure the right things, but also to show the relationships between the different measures from a holistic viewpoint, since this is a way of identifying potential mechanisms for improvement; The main difficulties of the entire KPI process are the availability and validity of data. National Benchmarking System for the Chilean Construction Industry NBS-Chile The National Benchmarking System was developed in 2000 by the Corporation for Technical Development CDT of the Chilean Chamber of Construction and the Program for Excellence in Production Management of the Pontificia Universidad Católica de Chile GEPUC, being supported by the Building Research Establishment BRE from the United Kingdom and sponsored by CORFO Foundation of Development and Innovation of the Chilean Government. The program consists of two initiatives: 1 performance measurement for benchmarking, which aims to devise and implement performance measurement in the construction industry; and 2 benchmarking clubs, which are groups of companies that aim to share managerial practices and information and to compare performance through meetings and visits to construction sites Grillo and Garcia A selection of performance indicators was based on previous studies that included extensive literature reviews and empirical research Alarcón and Serpell 1996; Grillo Initially, over 30 performance indicators were discussed in several meetings involving company representatives. This set was later reduced, based on the experience and needs of the companies. The final set of indicators is concerned with five subsectors of the construction industry: 1 high-rise building; 2 low-rise building; 3 civil works; 4 heavy industrial construction; and 5 light industrial construction. For the implementation of these indicators, the companies involved received a support guidebook and had access to computer software that enabled performance comparisons CDT The National Benchmarking System mainly used quantitative tools, such as mean, ranking curves, radar graphs, and tables displaying companies results, for data analysis CDT By March 2005 the NBS-Chile had a database of 247 projects provided by 39 Chilean companies. An evaluation system for managerial practices has been developed recently as an additional part of the benchmarking initiative Ramirez et al This system seeks to incorporate qualitative data as a complement to quantitative performance indicators, using a set of analysis techniques such as Pearson s correlation, factor analysis, and multivariate linear regressions Ramirez et al This evaluation system aims to compare management practices, identify relationships between performance data, and determine industry trends. It can be applied independently of the presence of hard performance data, thereby increasing the feasibility of applying the system periodically as part of a continuous improvement program. The benchmarking club initiative was carried out independently, using the following approach: 1 researchers of the program interviewed construction managers aiming to identify the best industry practices; 2 researchers and members of the benchmarking clubs visited construction sites to observe the best practices; and 3 technical presentations were promoted twice a year for members of the benchmarking club. Based on the interviews with representatives of seven partici- JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006 / 161

5 Table 3. Summary of Findings Initiatives Positive factors Negative factors Performance measurement Comparison among competitors Exclusive use of lagging measures system for benchmarking Guidance concerning the implementation Excess of measures and use of measures Overload of information to be collected Fast transmission of information Collection of data imposed by top manager Use of measures in real time Comparison between projects that are very different High cost and low benefit Lack of connection between measures and practices Benchmarking clubs Visits to projects sites Very long meetings Exchange of experiences among engineers Lack of continuity between the meetings and site visits Presentation and discussion of managerial techniques Lack of focus on site visits and building systems Provide more information than receive Fast transmission of information Involvement of top managers pating Chilean construction companies, researchers identified the main difficulties, the improvement opportunities, and the main benefits from the point of view of the companies. Interviewees identified the main difficulty for the implementation of the performance measures for benchmarking in the companies, as the lack of an internal systematic collection of measures. Most companies did not have a well structured performance measurement system. Most existing measures were concerned with cost accounting and were not clearly linked to the company s goals. Moreover, there was no external demand for performance measurement systems in Chile, as is the case in other countries; such demand exists in Brazil Costa et al because there is a demand for quality management systems from public clients there, and the demand exists in Denmark because of an industrywide qualification system Byggeriets Evaluerings Center Furthermore, a number of improvement opportunities were identified by the interviewees: 1 most measures are concerned with outcomes and do not provide real-time information that enables managers to take actions during a project. Therefore, there is a need to introduce leading performance indicators; 2 the dissemination of results takes a relatively long time and creates the necessity of frequent contacts between the companies and the program; and 3 most companies have not developed the necessary internal capabilities for effectively implementing a performance measurement system. Some kind of technical support for implementing data-collection processes seems to be desirable for most companies. The main benefits observed by the participating companies were the opportunities to share and discuss experiences, including difficulties and good practices, with other companies with similar problems. Moreover, they found that learning through real-world experiences happens faster than through using only measures. The research team perceived that most companies found it difficult to continually implement the chosen set of measures. This difficulty was largely caused by the lack of corporate commitment to benchmark at the company level. Researchers saw the need for a regular and committed team of representatives from each company involved in the benchmarking initiative. Table 3 presents a summary of factors that have positively and negatively affected the success of the Chilean initiative based on the interviews carried out with representatives of the construction companies. Construction Industry Institute Benchmarking and Metrics CII BM&M The CII BM&M was created in 1993 CII 2000 with four main objectives: to provide the construction industry with a common set of metric definitions, to provide performance norms to the industry, to quantify the use and value of best practices, and to help focusing CII research and implementation efforts. A committee of industry representatives working with the CII staff has developed the BM&M policy and is in charge of overseeing the execution of the program. This committee has defined critical performance measures that can be used in practice and developed a strategic approach to CII s collection, analysis, and dissemination of industry data. The CII BM&M first collected data in 1996, and established a core set of indicators in 2000 CII The CII BM&M collects project data as an ongoing process through its web-based system. Each participating company nominates a benchmarking associate, a liaison who has a leading role in in-house training and who validates project data before submission to the CII BM&M database. Each participant fills out the web-based questionnaire based upon his or her best knowledge, updating data online over the life of a project CII Participants receive real-time evaluation of their projects performance using the web-based Progress Key Report. In this system, projects can be immediately compared to other projects also in the database, and reports show metrics scores, performance quartiles, and graphic comparisons of individual project performance overtime CII In October 2004, the CII BM&M database held over 1,240 projects from CII owners and contractor member companies, ECI European Construction Institute companies, and Benchmarking & Metrics Participant Program BMPP organizations. This involvement represents $60 billion in total construction costs. One of the noticeable trends in the database is a recent increase in international projects. Currently, approximately 25% of the projects are from Europe, the Middle East, Asia, and South America. The projects are from the heavy industrial, building, light industrial, and infrastructure industry sectors, with the majority representing heavy industrial CII Besides performance measurement, CII also promotes a forum called BM User Platforms, a support network similar to bench- 162 / JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006

6 marking clubs. In this forum, participants and independent experts discuss the best practices in the industry and how to apply them for better performance. In the interview conducted for the present report, the CII associate director for benchmarking identified data collection and analysis processes as key elements of this initiative. CII conducts mandatory training for companies that benchmark with them. The program emphasizes standard definitions and then uses a multitiered validation process; data is validated at the time of their entry with computer algorithms. Trained company benchmarking associates for each company must review and approve submitted projects for validation, and their CII-trained account managers examine all data for validation before it is submitted. The main difficulty in data collection is getting companies to complete surveys once they have started them. Beyond that, the validation noted above is difficult and time consuming. CII data indicate that companies that benchmark more often tend to have better cost, schedule, and safety performance. These companies, however, tend to already employ continual improvement programs. Participating companies report that the three main benefits of the program are 1 access to a cost-effective system for benchmarking many projects; 2 agreement on common definitions for metrics of performance and practice use; and 3 creation of norms for many performance indicators that were previously unknown or not available. One of the main challenges faced by program participants is a lack of resources for implementation. Most companies lack the personnel to enter data and evaluate the reports that are returned. Complicating this obstacle, CII staff is also small and part time and is sometimes delayed in returning reports to companies. Additional challenges include getting corporate commitment to benchmark at the company level and implementing companywide improvement processes based on the findings from the benchmarking program. CII has also found that companies have been using benchmarking with varied results and that success depends mostly on the commitment at the organizations uppermanagement levels to improvement through the use of benchmarking. Performance Measurement System for Benchmarking in the Brazilian Construction Industry SISIND-NET Project The SISIND-NET Project was initiated in April 2004 by the Building Innovation Research Unit NORIE of the Federal University of Rio Grande do Sul UFRGS and the Association of Construction Companies from the State of Rio Grande do Sul SINDUSCON/RS, with the support of the National Council for Scientific and Technological Development CNPq. It involves the design and implementation of a performance measurement system for benchmarking in construction companies. The first stage of the SISIND-NET Project was the definition of the set of measures for benchmarking. The starting point for this standardization task was the knowledge gained from three previous international experiences CII 2000; KPI 2000; CDT 2002 and from past studies carried out in Brazil Oliveira et al. 1995; Lantelme et al. 2001; Costa The lessons learned from other programs have been very important in the conception and implementation of this initiative, especially in terms of encouraging the participating companies pursuit of performance improvement. Several meetings involving both representatives of the companies and members of the research team were held to define the set of measures. In each meeting an aspect of the measurement process, including objectives, formulae, and data collection and analysis procedures, was discussed. The final version of the set of measures was defined by the end of August So far, 18 construction companies from the State of Rio Grande do Sul have been involved in this initiative, most of them from the residential, commercial, and industrial building markets. A training course was provided for company staff members involved in the implementation of performance measurement. In October 2004 the implementation process was started, and companies began to send data to the database. Currently, the companies are sharing results and practices in the benchmarking club through monthly meetings, some of them carried out on the construction sites of member companies. In these meetings, several themes such as health and safety, layout and logistics of construction sites, cost management, and the best practices in the implementation of the performance measurement system for benchmarking are discussed. In these meetings, linking the practices discussed to the benchmarking measures is paramount. In these meetings facilitators abide by three rules: 1 members must participate regularly in the meetings and site visits; 2 member companies must share information with other member companies; and 3 participants must engage actively in discussion to ensure proper understanding of the practices being presented. These rules have helped the information exchange between member companies and have fostered the learning process. Another important component of this project is the individual support offered for the implementation of performance measurement in each company, especially those that face implementation difficulties. This support is aimed at enabling the incorporation of the measures in the organizational routine and has increased the engagement of the companies in data collection and analysis. A project website has been developed in which data can be directly input by the member companies. In the private area of the website, these companies also have access to some project reports and graphs derived from the existing database. Moreover, a validation procedure has been implemented to assess the consistency and coherency of the data sent by the companies. This procedure includes two types of controls: 1 the first one is carried out in the online system, since data must be approved by the research staff before they are incorporated into the database; and 2 the second type is carried out through audits of the companies. Both types of validation guarantee that the information that is provided to the companies and to the construction industry is reliable. One positive result of this initiative is the participatory process that has emerged from the benchmarking PMS definition meetings. Through the meetings, the companies experience the decision-making process for the definition of measures, including the negotiation of data-collection criteria. As a consequence, the representatives of the companies involved have come to understand the relevant set of measures well. The preliminary assessment of this participatory process indicates that the meetings and visits, the availability of a private site for the benchmarking club members, and the exchange of s can be regarded as triggers for starting the learning process concerning the sharing of practices and the comparison of measures among participants. However, for the consolidation of the learning process, companies need to develop mechanisms to facilitate the incorporation of the improvement actions identified through benchmarking. In other words, the prescribed actions need to be modified, enriched, and translated so that they can be applied to JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006 / 163

7 the real-world context of a company. This enables the creation of new knowledge and effective organizational learning. Lessons Learned and Opportunities for Improvement Based on the analysis of benchmarking initiatives in the United Kingdom, the United States, Chile, and Brazil, and also on the literature review, some key issues for the design and implementation of benchmarking systems were identified. First, there are several barriers to the implementation of performance measurement systems in the construction industry, due the peculiarities of this sector. These barriers arise because 1 construction is a project-oriented industry and each project is unique in terms of design and site conditions; 2 establishing a project performance measurement system and incorporating the measures into the company routine require a fairly intense effort; 3 the responsibilities for data collection, processing, and analysis, in general, are not well defined at the beginning of the project; 4 each project usually has a different managerial team and the use of measures will depend on the capabilities and motivation of each manager Costa and Formoso To combat these barriers, a benchmarking initiative demands a joint effort from several organizations, such as governmental entities, construction clients, individual companies, research institutions, and industry organizations. Regarding the type of benchmarking used, the four initiatives have nationally or regionally carried out performance comparisons within the construction sector. In all of them, the aim is to maintain continual data collection so that up-to-date statistics can be easily derived. The participating companies submit project data to a database manager. Furthermore, these initiatives have also sought to promote change through the identification and description of best practices. In Brazil, Chile, and the United Kingdom, benchmarking clubs tend to be focused around a particular client group, a type of firm, or a geographic area. Using a different structure, the U.K. s Demonstration Projects initiative and the United States BM user platforms both aim to stimulate industry improvement by making explicit links between the adoption of certain management principles or technologies and superior performance. It appears that benchmarking clubs as well as demonstration projects may have an important impact on the construction industry because they encourage the development of a culture of innovation, leading to improvement action instead of only data comparison. In general, the set of measures for benchmarking should be simple and well designed to effectively support improvement initiatives and should include a mixture of leading and lagging indicators. Table 4 summarizes the main set of performance measures adopted in each of the four initiatives. Although a wide range of measures is involved, some of the more common in the programs are cost and cost predictability total project cost and cost deviation, time and time predictability total project duration or time deviation, and safety accident statistics or measure of site management quality. Understanding the differences between the approaches adopted in different countries makes it possible to identify the generic measures that can form the basis for international benchmarking. However, it is important to determine why different approaches have been used in different regions before assessing what seems essentially to be a common goal. As discussed above, the KPI and NBS-Chile program involve mostly lagging measures that, by definition, are based on project outcomes. Such measures are important for assessing the success of company strategies but do not help to identify improvement opportunities during the period for which the measure has been taken. By contrast, the CII BM&M and SISIND-NET Project include several leading performance measures that can be used for support improvement programs and real-time control. No measures related to suppliers performance, employee satisfaction, site management, and quality management were found in the CII, NBS-Chile, and KPI initiatives. Because these were considered to be of major importance for the Brazilian companies, they were included in the SISIND-NET Project. The analysis of these benchmarking initiatives confirmed that the procedures for data collection should be objective and simple; streamlining the creation of the database makes it easier to compare project performance evaluation in real time. All four initiatives offer an interactive online tool for the collection and evaluation of the benchmarking measures. Such a tool allows users to access an assortment of documents and provides immediate feedback for the benchmarking club members. Beatham et al suggest that the online tool must be used over the life of a project to create continuous opportunities to analyze the results and to promote improvements. Only the CII BM&M and SISIND-NET Project provide this kind of access. Also crucial to the implementation of the online tool is data confidentiality, security, and information quality control, all of which guarantee the reliability and validity of the information provided by companies. Table 5 summarizes the positive issues and difficulties regarding the implementation of the four initiatives. One aspect that has not been sufficiently explored by any of the initiatives is the alignment of the benchmarking measures to company strategy. Therefore, benchmarking club members should be encouraged to design their own performance measurement system according to their companies strategies and capabilities, while making sure that the set of measures is relevant and feasible for them. According to Hudson et al. 2001, a strategic performance measurement system for small-sized companies must be very resource effective and produce noticeable short-term results. In addition, it must be dynamic and flexible enough to accommodate strategic changes, since those companies tend to experience sudden contingencies. For Hudson et al., in practical terms, this means that the process should be iterative, so that the strategic relevance of performance measurement is consistently maintained. There are other important issues to attend to in this kind of initiative, mainly concerning its implementation. These issues include: 1 Promoting training for the companies involved, including instruction in data collection, processing, and analysis, and communication of results to the companies; 2 Providing closer support for the companies, mainly to guide them to a better use of the information provided by the measures and practices exchanged. Such support encouraged the application of these practices and the creation of knowledge in the companies; 3 Motivating internal benchmarking and, afterward, external benchmarking; 4 Motivating the companies to share equivalent information and also stimulating discussion to generate understanding of the practices being presented; and 5 Creating a learning environment within the companies through the benchmarking clubs, motivating them to transfer the knowledge from that forum into the local condition. 164 / JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006

8 Table 4. Scope of Measures Comparison among Benchmarking Initiatives Scope of measures KPI United Kingdom CDT Chile CII Benchmarking and Metrics USA SISIND-NET Brazil Lagging measures Client satisfaction a Cost deviation by project a Project cost growth Cost deviation a Defects a Deviation of construction due date a Project budget factor Time deviation a Predictability cost a Change in amount contracted Project schedule growth Degree of client Predictability time a Rate of subcontracting Project schedule factor satisfaction user Profitability Cost client Total project duration Degree of client Safety Efficiency of direct labor Change cost factor satisfaction owner Productivity a Accident rate a Recordable incident rate Average time for Risk rate a RIR selling unit Risk rate a Lost workday case Contracting index incident rate LWCIR Ratio between the number of accidents and total man-hour input Nonconformity index in the unit delivery Leading measures Effectiveness of planning Total field rework factor PPC percentage of plan Urgent orders phase cost completed Productivity performance a Factor phase cost Construction site best growth owner data practice only Supplier performance Phase duration factor subcontractors, Construction phase material duration suppliers, and designers Number of nonconformity in audits Degree of employee satisfaction Rate of training courses Rate of employees trained a Measures that could be used if measured during the process. Table 5. Comparison of Implementation among Benchmarking Initiatives Implementation KPI United Kingdom NBS Chile CII Benchmarking and Metrics USA SISIND-NET Project Brazil Positive issues Online software Online software for users Online software for users Online software for users for users Outlined system qualitative Annual training Validation process like an audit Benchmarking club analysis Annual questionnaire for Benchmarking club Demonstration Benchmarking club companies evaluation Visits to construction sites project Visits to construction sites Difficulties Exclusive use of Exclusive use of lagging Commitment of the companies Analysis of the results by lagging measures measures Implementation of companies Availability of data Support for the improvement process based Use of the information for and validity of data measurement on the findings of the decision making Lack of Commitment of the companies benchmarking program Continuous measurement opportunities for Continuous measurement real control time JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006 / 165

9 Finally, the current format of the four initiatives discussed in this paper has focused on metrics for benchmarking, an objective aimed at improving the productivity of the industry. However, the benchmarking process can lead to benefits beyond productivity, if it is used for encouraging innovation. For example, benchmarking club initiatives have a huge potential for providing learning opportunities to participating companies; by identifying and sharing their own and other companies best practices, participants are immersed in an atmosphere of improvement. Still and yet, it seems that many construction companies involved in these initiatives have a limited view of benchmarking. They maintain a singular interest in comparing their performance to other companies, especially those from the same market segment, and continue in their habit of copying managerial practices rather than understanding the principles and concepts involved in those practices. Such companies should instead see benchmarking as a source of new ideas, or as a route to support performance improvement based on existing best practices. Recommendations for Future Research Directions Based on the analysis of the four performance measurement systems developed in different countries, four improvement opportunities were identified and new research directions were suggested. Future research should address the main problems and limitations of the existing systems, as follows: (1) Establishing a classification of performance measures. Clear distinctions should be made between measures that are proper to the construction process, organizational performance, client-oriented product specification, facility performance, etc. The product-development process is only a starting framework, one which needs to be extended to the whole lifecycle. Such comprehensive approach is increasingly popular in the industry because it allows for retrospective assessment of completed projects, in particular if this evaluation is extended to the organizational level. Useful lessons can be learned from such comparisons of final product with their original conceptions. (2) Developing frameworks that allow migration from performance measurement processes to performance management systems. This is the arena in which relationships between performance measures can be examined and used to improve overall performance of projects and organizations. New and improved frameworks are needed to consider the cause and effect relationships of actions taken at the operational and strategic levels. Performance modeling Alarcón et al is one such framework, but many of its existing indicators should be reviewed to make them more adaptable to individual projects. Only when these mechanisms are fully understood will it be possible to fully develop benchmarking systems. Dynamic simulation of such models can be used to perform what-if analyses. (3) Developing collaborative learning processes. Further investigation is needed on collaborative initiatives in which companies can mutually share their benchmarking experience and learn from each other. Recent experiences in benchmarking clubs around the world have demonstrated the benefits of this win-win approach to accelerating the improvement process. Such initiatives should be fully described and critically analyzed, with a focus on the transmission of knowledge from the benchmarking club to the companies local conditions. (4) Devising new measures. There is a need for new performance measures that can better explain the performance of production systems in the new operations management paradigm. For instance, measures for uncertainty and variability that exist in the project processes are probably necessary to understand the dynamics of the production system. Moreover, qualitative measures of the organizational and cultural characteristics of companies and projects are also needed to enable more comprehensive explanations of project performance. (5) Developing a theoretical framework for performance management. The establishment of a theoretical framework for performance management can be used for assessing existing performance measurement systems at two different levels: first, extensive reviews of literature in general management and social science disciplines and, second, through good practice implementation. For instance, data-mining analyses can be performed on large amounts of data to identify unrecognized critical success factors, such as the relationships between safety and lean production systems. Conclusions This article has discussed initiatives to develop performance measurement systems for benchmarking in four different countries the United Kingdom, Chile, the United States, and Brazil. This study has pointed out some of the benefits, problems, limitations, and opportunities for improvement of these initiatives. The lessons learned should be used for upgrading the existing initiatives and devising new ones. A joint effort involving several organizations is necessary for the successful design and implementation of benchmarking programs. Such an effort should not be limited to data collection but should also provide data analysis and training, as well as enable the exchange of best practices among the companies aiming to promote innovation. Moreover, the set of measures should be assessed and revised periodically, according to the needs of the companies involved. The commonalities among these initiatives indicate that they potentially could be used for international benchmarking. International benchmarking refers not only to the identification of common measures for data comparison among companies from different countries but also to learning opportunities that exist by sharing managerial practices among companies. Such enables sharing of new ideas in construction management. There are several mechanisms to promote the transfer of knowledge between companies from different countries: technical visits to foreign construction companies and projects; workshops and seminars involving companies from different regions of the world; and the development of web portals that make information on managerial practices, new technologies, and performance measures widely available. The existing benchmarking initiatives in different countries can potentially improve the effectiveness of those mechanisms, by making the participating companies more prepared to transfer knowledge and data that can be used for comparing performances. References Alarcón, L. F., et al Learning from collaborative benchmarking in the construction industry. 9th Int. Conf. of the Int. Group for Lean Construction, National Univ. of Singapore, Singapore, Alarcón, L. F., and Serpell, A Performance measuring, benchmarking and modeling of project performance. 5th Int. Conf. of the Int. Group for Lean Construction, Univ. of Birmingham, U.K. Anderson, K., and McAdam, R A critique of benchmarking and 166 / JOURNAL OF MANAGEMENT IN ENGINEERING ASCE / OCTOBER 2006