Benchmarking and operational performance: some empirical results

Transcription

1 1046 and operational : some empirical results Christopher A. Voss, Pär Åhlström and Kate Blackmon London Business School, London, UK International Journal of Operations & Production Management, Vol. 17 No. 10, 1997, pp MCB University Press, Introduction In order to understand what practices are necessary to reach world class standards, many organizations have begun to use benchmarking as a way of acquiring knowledge. can be defined as the search for industry best practices that lead to superior (Camp, 1989). Pioneered by Xerox, benchmarking has been widely adopted by companies as an improvement initiative (Port and Smith, 1992). At the core of successful benchmarking lies a regular and documented worldwide scan for organizations that are skilled at what they do, regardless of industry (Garvin, 1991). These organizations are then visited with the aim of gaining a better understanding of their processes and ways of working. These findings are used to stretch the imagination and develop new goals, as well as finding new ways of operating. Thus, benchmarking is both a means by which new practices are discovered and understood, as well as a goal setting process (Camp, 1989). As a process for measuring your against best-in-class companies, then using the analysis to meet and surpass the best-in-class companies (Pryor and Katz, 1993, p. 7), benchmarking is one way of identifying and understanding the practices needed to reach new goals. By identifying how superior companies organize their processes, a company can seek to adopt and adapt these practices. can also be used as a goal-setting process, an aid in setting objectives to achieve improvements (Venetucci, 1992). If benchmarking is carried out using best-in-class companies, these goals are likely to be stretch goals, which are important for improvements and learning (Roth et al., 1994). can be an effective tool for planning and implementing change processes that lead to organizational improvement when the knowledge gained is converted into a detailed action plan to improve competitive advantage (Pryor and Katz, 1993). The inclusion of benchmarking in the Baldrige Quality Award reflects its popularity (Hackman and Wageman, 1995). This is also reflected in the management literature (see for example Sweeney, 1994; Vaziri, 1992; Venetucci, 1992), which mainly focuses on:

2 how to go about benchmarking; what to benchmark (Partovi, 1994); and learning from the point of view of the host in a benchmarking undertaking (Langowitz and Rao, 1995). Existing research on the links between operational and benchmarking has relied mainly on anecdotal evidence for the benefits of benchmarking and has taken the link between benchmarking and as self-evident. This article sets out to address this gap in knowledge by testing whether there is a relationship between benchmarking and. The main questions investigated in doing so are: To what extent have firms adopted benchmarking? Does benchmarking lead to higher? Apart from treating the link between benchmarking and as given, research on the mechanisms through which benchmarking (supposedly) leads to increased is lacking. Therefore, a second aim of this article is to identify possible mechanisms through which benchmarking may lead to increased. We propose that benchmarking promotes directly through identification of practices and goals. Furthermore, benchmarking increases a firm s understanding of its position relative to competitors, which is beneficial for. Finally, we argue that benchmarking is a part of a learning organization, which has links to increased. This article begins by examining empirical evidence about the extent to which a large sample of European manufacturers have adopted benchmarking, using data collected from structured interviews with over 600 managers in the UK, Germany, The Netherlands, and Finland. It next looks at whether benchmarking has led to improved operational in this sample. The mechanisms by which benchmarking is linked to higher are then examined, as well as factors associated with higher levels of use of benchmarking. and operational 1047 Research methodology The data used in this article were taken from Made in Europe, a study of operational practice and in European manufacturing companies conducted between 1993 and This study focused on the relationship between operations practice and (Voss and Hanson, 1993). The model underlying the study is that companies competitive capabilities are based on six critical areas of manufacturing organization and culture, logistics, manufacturing systems, lean production, concurrent engineering, and quality. Data were collected on each company s adoption of best practice and in each of these crucial areas (see Figure 1).

3 1048 The central hypothesis of the study was that the adoption of best practice has a direct link to the attainment of high operational, which in turn leads to superior business and competitiveness: Practice refers to the established processes which an organization has put in place to improve the way it runs its business, ranging from organizational aspects such as teamwork and employee involvement to the use of techniques such as kanban. Operational refers to the measurable aspects of the outcomes of an organization s processes, such as reliability, production cycle time, and inventory turns. Operational in turn affects business measures such as market share and customer satisfaction. Figure 1. The world class manufacturing model (adapted from Voss and Hanson (1993)) Total quality Concurrent engineering Lean production Manufacturing systems Logistics Organization and culture Business A structured interview technique was used to gather the data in a sample of 660 European plant sites. A plant was defined as a self-contained unit, with its own personnel, facilities, products and management structure. The sample is described in more detail in Voss et al. (1995). Copies of the interview guide were pr ed to the research co-ordinator, who formed a multifunctional team spanning several organizational layers to prepare responses. The teams were typically co-ordinated by the plant manager or the head of a functional area such as manufacturing or quality. The aim of this stage of the process was to achieve a consensus between the various participants in the team on the organization s practice and. The plant was then visited by a trained facilitator, who met with management and the team on site, to review with them the assessment of the plant s practices and. The site visit also included a plant tour. The process was focused on achieving a convergence between the team s scores and the facilitators impressions. Feedback to each plant included a report comparing its to other plants in its industry and to the sample as a whole. Practices and questions were asked in each of the six areas outlined in Figure 1, plus business. For each item, respondents

4 were asked to position their manufacturing site on a 1 to 5 scale, with three descriptive anchor points explicitly defined for each item: 1 = basic level of practice or ; 3 = average levels; and 5 = good practice or. Use of benchmarking by companies Respondents were asked to indicate the extent to which the manufacturing site used benchmarking, using a 1 to 5 scale (see Figure 2). Table I indicates the percentage of respondents at each point on the benchmarking scale. Most sites (88 per cent) were participating in some benchmarking; however, few (7 per cent) were using it at best practice level, and the majority of sites (39 per cent) had attained only average levels. and operational None Benchmarks within own organization or group Regular, documented benchmarks against competition and world class standards beyond company s industry Figure 2. Use of benchmarking by companies use Total Number Percentage Table I. practice levels Increased levels of benchmarking were associated with higher levels of both practice and (See Table II). Overall practice was computed as the total of all manufacturing practices measured, and overall as the total of all measures. Both practice and averages increased markedly at higher levels of benchmarking. To facilitate an initial comparison between high and low performers, we defined a high performing group the leaders as the top 10 per cent and the low performing group the laggers as the bottom 10 per cent of sites in terms of aggregate practice and. The leaders had nearly twice the score of the laggers in benchmarking usage (see Table III). use Total Overall practice Overall Table II. Overall practices and at different levels of benchmarking

5 1050 Linking benchmarking and The high usage of benchmarking by successful sites leads to the proposition that benchmarking is associated with superior. To test whether there is a link between the use of benchmarking and, we constructed two indexes of : operational and business. Details of scale construction and statistical methodology are presented in the Appendix. use Table III. Leaders vs laggers Leaders (top 10 per cent of the sample) 3.80 Laggers (bottom 10 per cent of the sample) 2.06 Average 2.94 Operational is an index constructed by summing 17 questionnaire items measuring in quality, productivity, and cycle time. Indicators of good included: process variance less than product design tolerance; low product failure rates in service; low warranty claims costs; low defect rates; and low scrap, rework, and recycling costs; per cent of production cycle time devoted to performing valueadded work on materials; processing orders in hours, rather than days, with minimum expediting; the ability to perform equipment changeovers in minutes, rather than days; high inventory turns; the ability to deliver products faster than competitors without holding finished goods inventory; consistently meeting 95 per cent or better customer delivery commitments; new products delivered to market faster than competitors, with high initial yields. Business is an index constructed from the six items that measured the site s ability to deliver value to customers, including: high levels of customer satisfaction; an increasing market share; positive cash flow; return on assets better than competitors ; low product costs; high productivity growth. In order to test whether benchmarking has a significant effect on operational and business, ordinary least squares (OLS) regression was used to test the following hypotheses:

6 H1: Operational is positively related to benchmarking. H2: Business is positively related to benchmarking. The results of the tests are shown in Table IV. As can be seen from Table IV, the use of benchmarking is significantly related to both operational and business. This result supports the assertions on the benefits of benchmarking made in the literature. However, the causality of this relationship is not clear from these tests. Whether benchmarking leads to better, or whether better performing companies use benchmarking more, is an unanswered question. and operational 1051 Operational Business Dependent variable Coefficient p (t-test) Coefficient p (t-test) Constant use Goodness of fit R 2 = 0.112, p = R 2 = 0.052, p = Table IV. and two measures of regression results The mechanism of benchmarking In the above section we have put forward evidence that benchmarking is associated with at the site level. The next step is to consider the mechanisms that may lie behind its effectiveness. We will examine two areas, and use the study data to test propositions concerning each. and understanding It can be argued that one mechanism through which benchmarking leads to improved is through helping a company gain an understanding of its strengths and weaknesses in both its processes and its. Understanding will lead in turn to focused and effective improvement activities. We therefore set out to test the relationship between benchmarking and understanding. To develop a measure of understanding, we compared respondents views of their competitiveness with their competitive capability as measured in the study. Study participants were asked: To what extent do you consider your site able to compete with the best of your competitors elsewhere in the world? The results of this question are displayed in Table V. From the Table we see that two-thirds of respondents (68 per cent) thought that they were completely Completely Mostly Partially Not at all Total Frequency Percentage of responses Table V. Ability to compete with best competitors anywhere in the world

7 1052 or mostly competitive with their best competitors, while only one-third (33 per cent) thought that they were only partially or not at all competitive. Clearly, not all competitors within an industry can measure up to best practice standards. Managers self-opinions, in the absence of evidence to the contrary, seem to be biased towards over-estimating their sites competitiveness. We can postulate that companies that do not use benchmarking are likely to be over-optimistic about their position relative to competitors. The value of benchmarking is that the data help companies make a realistic assessment of their capabilities. This in turn provides a greater incentive and focus for improvement. To investigate the existence of an indirect link between benchmarking and operational, we constructed the Optimism Index, which indicates the extent to which the manufacturing site is realistic in its assessment of its own position, strengths, and weaknesses vis-à-vis competitors. The site s selfassessment was measured using the self-opinion item described above. The optimism index compares Ability to compete successfully, which reflects the manager s opinion of the company s position, strengths, and weaknesses relative to competitors, with the actual measure of operational. Three positions were defined: If managers opinion of how well the company performs is greater than actual, they are optimistic. If managers opinion of how well the company performs is equal to actual, they are realistic. If managers opinion of how well the company performs is lower than actual, they are pessimistic. We would expect that companies that benchmark are likely to be less optimistic in their assessment of their own capabilities than companies that do not make competitive comparisons. Over-optimism from lack of information, on the other hand, leads to complacency, and can be taken as an indicator of lack of understanding. This leads to the hypothesis: H3: Over-optimism is negatively related to benchmarking. Furthermore, realistic companies will focus their improvement agendas on real needs, while companies that do not use benchmarking are likely to be overoptimistic about their position relative to competitors, and may choose agendas that are not related to real internal and external needs. Thus, the higher the degree of understanding of its competitive positioning, the more likely a company is to use the increased knowledge gained through benchmarking appropriately. This leads to the hypothesis: H4: Over-optimism is negatively related to operational. The two hypotheses were statistically tested using ordinary least squares (OLS) regression analysis. The results are shown in Tables VI and VII below.

8 As the results in Tables VI and VII indicate, there is a moderately strong relationship between benchmarking and understanding, as well as between understanding and operational. Both relationships were significant at the p = 0.05 level and were in the expected direction. However, particularly the percentage variance in understanding that was explained by benchmarking was low. This indicates that there are many other factors that affect the respondents understanding of their own competitive positioning, which is to be expected. The important finding in view of the purpose of this study is that the effect of benchmarking was found to be significant. and operational 1053 Understanding Dependent variable Coefficient p (t-test) Constant use Goodness of fit R 2 = 0.006, p = Table VI. vs understanding Operational Dependent variable Coefficient p (t-test) Constant use Goodness of fit R 2 = 0.052, p = Table VII. Understanding vs operational and organizational learning As well as understanding the company s competitive positioning vis-à-vis its competitors, benchmarking may also be linked to broader concepts of organizational learning, specifically the learning organization (Senge, 1990). The learning organization, the subject of considerable recent attention in the management literature, argues that learning is necessary for a company to survive in the increasingly turbulent competitive environment. Along these lines, the concept of the knowledge factory has been proposed as an accelerated learning organization driven by dynamic processes that create superior knowledge and translate that knowledge into competitive capabilities and core competencies (Roth et al., 1994). The knowledge factory has been used to describe a learning organization in which knowledge, as opposed to capital, is the limiting factor of production (Roth et al., 1994), and the strategy for acquiring knowledge how the organization will adapt to and exploit knowledge from its environment is key (Roth and Marucheck, 1994). This definition contains two important elements. Strategy to acquire knowledge concerns how the organization will adapt to and exploit knowledge

9 1054 from its environment. can be seen as a strategy for knowledge acquisition, a relationship that has been previously suggested by Garvin (1993), who cites learning from others through benchmarking as a building block in his vision of the learning organization. Predisposition for organizational learning indicates the capabilities that can be identified and leveraged within the organization to bolster learning. It is described as organizational awareness that a portfolio of learning opportunities exists (Roth and Marucheck, 1994). Without at least some prior related knowledge, firms are not likely to benefit from benchmarking. This concurs with the finding reported by Port (1992): benchmarking will not improve unless a company already has a comprehensive quality programme. Based on this argument, we predict a positive relationship between what we choose to term an organization s predisposition for learning and benchmarking. This relationship is based on our proposition that benchmarking is a vital part of the learning company s repertoire for improvements. To test this prediction, we developed an index termed learning orientation for measuring an organization s predisposition for learning by summing scores from five questionnaire items: vision, shared vision, mission and goals; quality vision, management style; and employee involvement. Central to the learning organization is a corporate-wide initiative to move the employees in a unified direction (Roth and Marucheck, 1994). Evidence of a best practice learning orientation included: leadership by management; extensive cross-functional communication, with management vision communicated to individuals and departments; written mission statement and published improvement plans; relatively flat organizational structure; key manufacturing goals of leadership in quality, service and product cycle time, rather than cost reduction; a high level of employee involvement; participation, empowerment and teamwork; an emphasis on total quality management, including zero defects, quality control in process, design for manufacturability, and quality as everyone s job. The items in the index together reflect this initiative to guide the actions and learning of employees. Management style and employee involvement reflect the usage of what Senge (1990) terms team learning, one of the five disciplines of the learning organization, where employees at all levels are involved in the learning process and allowed to learn. The organization must open its

10 boundaries to ensure a fresh flow of ideas and the chance to consider competing perspectives (Garvin, 1993). The relationship between learning orientation and benchmarking was tested using ordinary least squares (OLS) regression. The following hypothesis was tested and the results are shown in Table VIII: H5: Learning orientation is positively related to benchmarking. The results show a strong association between benchmarking and the organization s learning orientation. The percentage variance explained is satisfactory and the correlation is significant at the p = level. This supports the notion that companies that are oriented towards learning are likely to use benchmarking more. can, in this light, be seen as an excellent way of learning from the external environment. and operational 1055 use Dependent variable Coefficient p (t-test) Constant Learning orientation Goodness of fit R 2 = 0.133, p = Table VIII. vs learning orientation Discussion This study clearly points to a link between benchmarking and, which has been proposed in the literature but not previously empirically tested with a large sample. The results of the regression analysis show that the use of benchmarking is linked strongly to both improved operational and business. This result supports the assertions on the benefits of benchmarking made in the managerial literature. Besides this more direct relationship between benchmarking and, an indirect link between benchmarking and has been demonstrated. has been shown to increase understanding about where a unit is positioned relative to its competitor, which leads to a number of benefits: Companies who over-estimate their competitiveness (viewing themselves through rose-coloured spectacles) may become complacent, and thus delay adopting improvement programmes and be overtaken by competitors. Companies who are realistic, however, will increase their chances of identifying the areas in which improvement will have the most competitive leverage, and thus be able to compete on a more level playing-field. It can be argued that companies who underestimate their competitiveness may become stuck in a vicious cycle of failing to improve and falling further behind;

11 1056 or panicking and adopting any or every programme in an attempt to catch up and ending up in another vicious cycle. Finally, learning orientation has been linked to benchmarking. This supports both the knowledge factory concept, which has been proposed to be important in the manufacturing context (Roth and Marucheck, 1994; Roth et al., 1994), and the overall concept of the learning organization (Senge, 1990). Based on these arguments and on the results of the data analysis, we now propose a relationship between learning, benchmarking, understanding and, summarized in the research model (Figure 3): is a vital part of the learning company s repertoire for improvements and learning organizations will be more likely to use benchmarking. promotes higher directly through helping a company to identify practices and set challenging goals. increases a company s understanding of its strengths and weaknesses relative to competitors. This understanding in turn benefits, since improvement agendas will be focused on real needs. Learning orientation Understanding Manufacturing Figure 3. The research model Business Conclusions This study contributes to operations management by integrating theory and empirical data to investigate whether benchmarking, as an organizational learning tool, leads to improved. This is important because few studies have to date tested the links between best practices and. Three mechanisms by which benchmarking leads to improved operational and business were proposed and tested. describes a set of empirical practices that have been diffused through the practitioner community as part of a broader set of best practices, including total quality management (TQM), just-in-time (JIT) and many others. In common with other such practices, researchers have initially focused on

12 describing the content of these practices and the extent to which they have been adopted, without fully comprehending how they fit into the broader theoretical context of manufacturing strategy, or considering the way that they contribute to. In many cases, the links between best practices and improved are assumed to be self-evident. Without demonstrable positive links to improved, however, improvement initiatives such as benchmarking may be classed by managers as just more fads or buzzwords, the proliferation of which have led to initiative burnout in many cases. Disseminating information on best practices is one of the key tasks of management research, but it is not sufficient to offer prescriptions based on small samples or vague generalities. Operations management in particular must set, as one of its research goals, the testing of whether such initiatives are indeed linked to improved, and what mechanisms come into play. and operational 1057 References Camp, R.C. (1989), : The Search for Industry Best Practices that Lead to Superior Performance, ASQC Quality Press, Milwaukee, WI. Garvin, D.A. (1991), How the Baldrige Award really works, Harvard Business Review, November-December, pp Garvin, D.A. (1993), Building a learning organization, Harvard Business Review, July-August, pp Hackman, J.R. and Wageman, R. (1995), Total quality management: empirical, conceptual, and practical issues, Administrative Science Quarterly, Vol. 40 No. 2, pp Langowitz, N.S. and Rao, A. (1995), Effective benchmarking: learning from the host s viewpoint, for Quality Management & Technology, Vol. 2 No. 2, pp Partovi, F.Y. (1994), Determining what to benchmark: an analytic hierarchy process approach, International Journal of Operations & Production Management, Vol. 14 No. 6, pp Port, O. (1992), Quality, Business Week, 30 November, pp Port, O. and Smith, G. (1992), Beg, borrow and benchmark, Business Week, 30 November, pp Pryor, L.S. and Katz, S.J. (1993), How benchmarking goes wrong (and how to do it right), Planning Review, Vol. 21 No. 1, pp Roth, A.V. and Marucheck, A. (1994), The knowledge factory: paradigm for accelerated learning, University of North Carolina, Working Paper, Kenan-Flagler Business School, Chapel Hill, NC. Roth, A.V., Marucheck, A.S., Kemp, A. and Trimble, D. (1994), The knowledge factory for accelerated learning practices, Planning Review, Vol. 22 No. 3, pp Senge, P.M. (1990), The Fifth Discipline: The Art and Practice of the Learning Organization, Century Business, London. Sweeney, M.T. (1994), for strategic manufacturing management, International Journal of Operations & Production Management, Vol. 14 No. 9, pp Vaziri, H.K. (1992), Using competitive benchmarking to set goals, Quality Progress, October, pp Venetucci, R. (1992), : a reality check for strategy and objectives, Production and Inventory Management Journal, Vol. 33 No. 4, pp

13 1058 Voss, C. and Hanson, P. (1993), Made in Britain: The True State of Britain s Manufacturing Industry, IBM United Kingdom Limited/London Business School, London. Voss, C., Blackmon, K., Hanson, P. and Oak, B. (1995), The competitiveness of European manufacturing a four country study, Business Strategy Review, Vol. 6 No. 1, pp Appendix: Details of statistical methodology This appendix contains information and details on the measures that were used in the study. The means, standard deviations, and Pearson product-moment correlations for each of the four variables that were utilized are shown in Table AI. Construct reliability of each of the multi-item indices was examined by calculating Cronbach s alpha for learning orientation, manufacturing, and business ), as shown in Table AII. The minimum cut-off point of at least 0.70 was met by each of these scales. Construct validity was examined using factor analysis with varimax rotation. The regression equations were calculated using the multiple regression routine of the computer program SPSS using standard ordinary least squares (OLS) procedures. Variable Key n Mean SD X1 X2 X3 Y1 Y2 Table AI. Descriptive statistics X Understanding X Learning orientation X Operational Y Business Y Variable n Items Cronbach s alpha Table AII. Construct reliability and validity Understanding Learning orientation Operational Business