1 DEVELOPING BENCHMARKING METHODOLOGIES FOR RAILWAY INFRASTRUCTURE MANAGEMENT COMPANIES Richard Anderson Robin Hirsch Mark Trompet William Adeney Railway Technology Strategy Centre, Centre for Transport Studies, Imperial College London United Kingdom 1. INTRODUCTION Since the 1990s, the European Commission (EC) decided that to allow fair competition on the European railway network, the separation of essential functions in the sector was required. This has led to the separation of functions such as licensing, capacity creation and allocation, charging, the monitoring of public service obligations and the provision of train services. EC Directives such as 91/440 have far-reaching implications for the management of the railway infrastructure. Significantly, separate, dedicated Railway Infrastructure Management companies 1 (Infracos) have been created in most EU countries, providing the essential separation between the management of the rail infrastructure and use of the infrastructure. The creation of separate Infracos necessitates a new approach to international railway benchmarking. European Railways have only recently been vertically separated and in many countries, the process is not yet complete. To date, therefore, most national railway benchmarking has either encompassed the traditional vertically integrated railway, or it has focussed on internal benchmarking. The recent exception to this has been the INFRACOST study performed by the UIC, of which the early stages focussed on the activities of both building and maintaining the infrastructure. External benchmarking requires valid comparisons of performance between different companies, so attaining a reasonable level of comparability is essential. Railway Infracos are hugely complex and comparing performance between them can be hindered by the large extent of external factors (such as geography and train service density) that influence performance. Moreover, different Infracos throughout the EU define and measure their own performance (such as reliability due to good infrastructure) in many different ways. In particular, Infracos often wish to focus on measures of performance that are different to the total (integrated) railway. In July 2001, the European Commission, as part of the 5th framework growth programme, commissioned a 2-year project, known as IMPROVERAIL 2. Its objective was to support the establishment of railway infrastructure management in accordance with Directive 91/440, and other newly issued
2 railway infrastructure directives, by developing the necessary tools for modelling railway infrastructure and access management. IMPROVERAIL dedicated a Work Package for the development of benchmarking methodologies specific to railway infrastructure companies. The project addressed the issue of comparability by developing a consistent set of Key Performance Indicators for railway Infracos along with a clear set of harmonised definitions. However, the consortium also recognised that defining a framework and process to enable successful international benchmarking was even more important. Therefore this has been an essential element of the IMPROVERAIL research The results of this work, developed in collaboration with the IMPROVERAIL consortium, are described in this paper. This paper will first put the proposed methodology into context in Section 2 by explaining the value of benchmarking for infrastructure companies. The methodology will then be described in summary form in Section 3. Section 4 will cover the keys to successful Infraco benchmarking and will describe the implementation phase and its challenges, leading up to an overall conclusion in Section VALUE OF BENCHMARKING FOR RAILWAY INFRASTRUCTURE COMPANIES In order to understand the need for the development of a benchmarking methodology for railway Infracos, it is useful to explain why benchmarking will be of value, and to review previous experiences with railway benchmarking. 2.1 Why should Infracos undertake benchmarking? The primary reason why any company should undertake benchmarking is the search for and emulation of best practices to achieve performance improvement. It is this philosophy that has driven the methodologies developed by this research. There may be reasons for benchmarking other than to improve performance. These include: Explaining the situation to stakeholders (e.g. regulator, financiers, government). Justifying an appropriate level of financing from the government. Justifying an appropriate level of track charges from the regulator. Providing a better understanding and forecasting of costs and revenues, leading to better project predictability. Setting target cost levels. Monitoring contractual performance (e.g. train operating companies, maintenance suppliers, etc.). Many of theses objectives may also imply performance improvement. However, if benchmarking does not lead to improvement in the long term, it is a waste of resources. Moreover, if improvement is not the main objective,
3 and if top management is not committed to opening up to outside ideas and implementing best practice, little benefit will be achieved. 2.2 Previous experience of railway benchmarking The IMPROVERAIL project necessarily built upon prior experiences in international, national and internal benchmarking projects. As vertical separation is in its infancy in most European countries, few benchmarking projects have so far been dedicated to the Infraco in isolation, and have mainly concentrated on the total, integrated railway. Those projects that were undertaken by Infracos have mainly focussed on unit cost / efficiency benchmarking to a large degree. However IMPROVERAIL established that there was a need for the development of an Infraco-specific benchmarking framework that was dedicated to the search for best practices: that is, a methodology which was focussed on identifying practical, transferable and implementable results. Amongst others, the results and methodologies of the CoMET and Nova metro railway benchmarking programmes and the UIC INFRACOST (The Cost of Railway Infrastructure) studies were reviewed. The CoMET and Nova metro benchmarking programmes each include ten metros from across the world CoMET (Community of Metros) including most of the largest world metros, and Nova, the second or new group, mainly for medium sized ones. The process is owned and led by the metros and is focussed towards identifying best practices which can be emulated. Benchmarking is undertaken on an annual cycle and uses both performance and process benchmarking to examine all areas of the business. The UIC INFRACOST study commenced in 1997 and initial phases were focussed on developing benchmark comparisons of harmonised life cycle costs for railway infrastructure assets. The project eventually involved participants from nineteen countries in Europe, North America and the Far East. Comparing the CoMET/Nova, INFRACOST and IMPROVERAIL approaches, the main differences have been in the objectives of the participants, the level of detail of each study, and the choice of topics covered. The first three stages of INFRACOST were mainly concerned with lowering the life cycle cost of infrastructure, while CoMET/Nova have been more concerned with optimising processes and investment to deliver higher quality services (though also at lower cost where possible). However, the basic methodology of CoMET/Nova, INFRACOST and IMPROVERAIL is very similar: in each case, the project starts out from an analysis of relative performance, which identifies high and low performers. It then passes to an analysis of the reasons for variances in performance, which inevitably leads to an investigation of the differences in process methods or uses of different technologies.
4 2.3 Benefits of benchmarking The most common reason for undertaking benchmarking has been to bring cost levels down and efficiency levels up to those of the best performing companies. Railways and metros have achieved some substantial results from benchmarking over the past few years. However, it is the adoption of best practice which generates the clearest immediate benefits. More or less all of an Infraco s functional areas can benefit from benchmarking to improve performance. Some examples of ways in which integrated railways or Infracos have already benefited from benchmarking projects in the past, and functions which have benefited, include the following: Adopting world best practice in maintenance methods led to significant reliability improvements for London Underground (up to +137%) and MTRC (Hong Kong) (+61% overall). London Underground increased line capacity by 4%-19% and New York City Transport by 5%-17 %, by copying proven practices from East Asia, in both cases employing almost no capital investment. Station Management rationalisation by MTRC (Hong Kong) resulted in an 8% reduction in station staff. This came from comparing their functional profile and degree of supervision with other metros. MTRC asset replacement programme was expected to achieve savings of US$150 million by experimenting with life extension methods used in other railways, notably in London. Some successful cost comparisons have been made. A comparison of national railways telecommunications activities identified potential savings of 34% for one railway. 3. PROPOSED BENCHMARKING METHODOLOGY The objectives of the research project were as follows: Development of benchmarking methodologies for Infracos. Development and definition of performance indicators for Infracos. Stimulation of the exchange of information between stakeholders. Promotion of standardisation and contributing to the better management of information within the railway industry. As such, the research was dedicated to the development of benchmarking methodologies, frameworks and indicators for future benchmarking. Its primary aim was not to undertake a definitive comparison of performance between Infracos, although practical exercises were undertaken in order to validate the approach, and to identify the opportunities and pitfalls for Infraco benchmarking in the future. It should be noted that the Benchmarking methodology developed is prescribed in terms of a practical process and framework for benchmarking, rather than any technical benchmarking tools or statistical methodologies such as Total Factor Productivity (TFP) or Data Envelopment Analysis (DEA).
5 3.1 The need for Infraco-specific approach to benchmarking Integrated-railway international benchmarking projects have tended not to focus on infrastructure, but in areas, such as rolling stock maintenance and customer service, where performance and processes are well recorded and where there are easily identifiable best practices. Conversely, integrated information systems relating to railway infrastructure (such as asset registers) are often poor or non-existent, yet they are necessary for successful benchmarking. Vertical separation has meant that benchmarking of national railways in the traditional manner has to be adapted to reflect the Infraco s specific priorities. The Infraco has to consider performance in the context of what is within its managerial control. Therefore, how success is measured and the focus of priorities for improvement will be different to those of the integrated railway. Railway infrastructure companies are large, complex, and their scopes of operation differ between European countries. This adds an additional level of complexity to the application benchmarking to this industry. Moreover, between European railways, the existing mechanisms for the measurement performance (such as the measurement and definition of train delay) vary considerably. Even when performance measures are comparable, the influence of geography and the local economy can hide differences in performance which are truly due to those superior practices. A successful benchmarking study should be committed to the search for and emulation of these superior practices. 100% 95% 3 min 4 min 10 min 3 min % of trains on time 90% 85% 80% 3 min / 5 min 2 min 75% 70% Infraco 4 Infraco 7 Infraco 9 Infraco 11 Infraco 12 Infraco Figure 1: Percentage of trains on time for six European Railways and their delay thresholds
6 For example, Figure 1 (above) compares the percentage of passenger trains on time for a selection of railway infrastructure companies. The threshold for which a train delay is recorded varies between 2 minutes and 10 minutes, even for similar train types (such as intercity services). For most railways, the thresholds for this important performance measure are set for the ultimate client: the travelling public. However, the Infraco may be more concerned with a performance measure that represents how well it is managing the network and serving its direct client, the Train Operating Company. Here a universal delay threshold (for example 3 minutes) for all trains might be more appropriate. Moreover, the Infraco should also be interested in a Key Performance Indicator (KPI) which measures only those delays to trains due to its own failures, excluding those due to the the TOCs. A key output of the research was the establishment of KPIs (along with harmonised definitions and measurement methods) that are appropriate for the Infraco (as opposed to the integrated railway). 3.2 Improverail Benchmarking concept The benchmarking process in Figure 2 below is a suitable benchmarking model for adoption by Infracos. This figure introduces certain concepts described in this paper. The process starts by defining Success Dimensions which enables the Infraco to view success not in terms of functional output, but in terms of an integrated aim such as delivering service quality or a safe railway. Factors that are critical to delivering success are then defined, and KPIs are developed to describe these. A holistic approach is essential to define KPIs which are useful to the Infracos and which can help to identify cause and effect and best practice. Measuring Success Vision & Strategy Success Dimensions Critical Success Factors Entities to compare, e.g. Products, Processes KPIs to describe them Creating Success Take action, monitor and recalibrate Set goals and action plans Identify best practice Eliminate impact of exogenous conditions Analyse reasons for differences Figure 2: The benchmarking cycle adopted by IMPROVERAIL KPIs could be applied to the whole of the Infraco as a single unit, but the research proposed a process for splitting the Infraco into comparable units, here defined as entities. The benchmarking process should continue to understand the reasons for differences in performance, identifying and eliminating external factors affecting performance upon which the Infraco s
7 management has no control. Best practices and processes under the control of management can then be identified and a successful benchmarking project should then continue to emulate those best practices, set goals and action plans which are then implemented. The KPIs should then be used once again to monitor the effect of change. 3.3 The need to define benchmarking entities Standard benchmarking methodologies are relevant to Infracos, but such companies are different in certain important ways from the majority of commercial manufacturing, trading and service companies for whom benchmarking was developed. They are more complex, with less standardisation of business units. Much more adjustment is needed to make a standard comparable entity. This is therefore a separate step in methodology developed for IMPROVERAIL. To make valid comparisons, wherever data is available, for operational benchmarking it therefore makes sense to attempt to benchmark comparable parts of railways, rather than the whole company. The entities to be benchmarked can be defined as one or more of the following, usually in combination: A product or service such as High speed, Inter-city, Regional, Local or Freight lines. A process or set of processes such as operation, maintenance, administration, monitoring. An asset or set of assets such as ticket machines, signals, stations or track. A relationship or set of relationships e.g. with authorities, regulators, customers or suppliers. The research has shown that separation of the infrastructure in this way into comparable similar sections of track or corridors, with similar service characteristics is often essential to reach comparability and meaningful results. 3.4 Set of success dimensions The Infraco s vision and strategy will be critical to what the company considers to be success or failure. The Balanced Scorecard, used by most leading commercial companies, offers a good framework for selecting Success Dimensions to measure using KPIs. A diagram showing its use to develop such success dimensions for Infracos is shown below:
8 Success Dimensions Balanced Scorecard EFFICIENCY ASSET UTILISATION INTERNAL FINANCIAL INNOVATION & GROWTH SERVICE QUALITY & RELIABILITY FINANCIAL EFFECTIVENESS SAFETY CUSTOMER LEARNING & GROWTH ACCESSIBILITY Figure 3: Development of Success Dimensions using the Balanced Scorecard model (developed from Kaplan & Norton,1993) These Success Dimensions were used in order to categorise those factors which are critical to the success of the Infraco and thereby define the KPIs that should be used to measure success. 3.5 Role of KPIs in benchmarking The development of a set of Key Performance Indicators (KPIs) for Infraco benchmarking was a key output of the IMPROVERAIL research. The purposes of KPIs developed in IMPROVERAIL include the following: To identify priority areas for improvement within each Infraco. To identify the best performing Infracos in the different success dimensions and critical success factors: these will be the ones to emulate. To help determine how much improvement is possible (when normalised). To monitor progress over time: if performance has improved, what practices have led to the improvement? To monitor an Infraco s own progress over time, in order to determine if it is progressing towards its vision and objectives. To estimate the effect of differences in processes and practices on actual outcomes. To set targets for improvement.
9 These are the main uses of KPIs in general. During the IMPROVERAIL exercise they were also used for four additional purposes, which are equally valid for Infracos starting out in benchmarking: 1. To enable Infracos to identify areas to benchmark in detail. 2. To validate the process and validity of collecting top-level KPIs. 3. To examine how performance for specific corridors is different from that of the railway as a whole in other words, the scope for internal benchmarking to spread the use of best practice within each Infraco. 4. To identify what data was available, and evaluate Infracos information and internal performance measurement systems. 3.6 KPI structure The KPIs developed in IMPROVERAIL can be used at many levels: the total railway, the Infraco, segmented entity or sub-entity. Such segmentation and disaggregation of total company performance is not only essential for reasons of comparability, as already discussed, but also makes it easier to assess individual performance relatively free from influences beyond the control of railway management. In addition to the KPIs developed for each dimension of success as described in the last section, a number of background indicators were developed to provide an understanding of the railway s structure and environment. These indicators do not directly lead to the success of the Infraco, but are included in a further category. The KPIs were divided into four hierarchal categories: Core KPIs: For use by top level management (a small sub-set of KPIs which have the most bearing on the success of the Infraco). Second Order KPIs: KPIs which support, or provide a deeper understanding of the performance described by the core KPIs, or whose impact on the performance of the Infraco is of a second order magnitude. Contributory PIs: Performance indicators which break down the components of a Core or Second Order KPI. Pre-cursor PIs: Detailed (often process-level) performance indicators which describe incidences of events or conditions which may have a significant influence on the core or second order KPIs. 3.7 Ideal set of Key Performance Indicators The success dimensions and the KPI structure were used to develop a wish list or ideal set of Key Performance Indicators 3. This set represents the complete group of KPIs that, in a perfect scenario (i.e. where data is available and measured in a consistent manner), should be produced to identify best practices in all areas. This list is accompanied by a set of proposed clear definitions and measurement methods for the Infracos to take forward.
10 Figure 4 shows some example KPIs against their relevant success dimensions. Accidental equivalent Fatalities Per Train km Train km per track km ASSET UTILISATION SAFETY EFFICIENCY Planned track possession km hours per track possession km hour SERVICE QUALITY & RELIABILITY % trains delayed due to infrastructure Harmonised lifecycle Costs per gross hauled Tonne km (using Infracost measures) FINANCIAL EFFECTIVENESS INNOVATION & GROWTH ACCESSIBILITY Passenger train km per route km Average relative age of assets Figure 4: Examples Key Performance Indicators for each of the seven success factors Figure 5, below, shows how KPIs can be used to monitor performance over time. Here, a measure of asset utilisation, (train kilometre per track kilometre) is shown for 13 European Infracos. An immediate question to examine with more detailed process benchmarking might be how IM 12 has increased the use of its assets, year-on-year, from an already high level, and the extent to which this might have affected service quality and reliability and/or profitability. 25 Train km (000) per track km IM1 IM2 IM3 IM4 IM5 IM6 IM7 IM8 IM9 IM10 IM11 IM12 IM Figure 5: KPI: Train km (000) per track km for 13 Infracos 4 (here denoted as IM) over five years
11 Figure 6 is shown below to demonstrate the problems in achieving comparability in safety indicators throughout Europe. The KPI shows the number of train accidents per million train kilometres. However, the existing data available from the Infracos is not comparable due to the different ways in which different countries define a train accident. Moreover, IM5, for example, has a very high number and density of level crossings, which are responsible for the vast majority of all train accidents. These problems lead to enormous variations in apparent performance levels and demonstrate the need for developing consistent definitions and measurements. However, the problem also highlights the need to develop a structured, benchmarking approach to identifying those precursor events that lead to high or low levels of performance. 35 Train accidents per m train km Inf 1 Inf 2 Inf 3 Inf 5 Inf 6 Inf 7 Inf 8 Inf 9 Inf 10 Inf 11 Inf 12 Inf Figure 6: KPI: Train accidents per million train km for 12 Infracos 5 over five years 3.8 Achieving comparability in Infraco benchmarking We have already discussed some of the problems of comparability above, in section 3.1. IMPROVERAIL and previous work have shown that any benchmarking study can take many years to achieve a reasonable level of comparability, but that full comparability is not necessary before significant benefits can be achieved. KPIs must be developed with full involvement and active participation from the Infracos. This was not fully achievable within the scope of IMPROVERAIL, so the set of KPIs developed are currently only at their second level of iteration. The choice of KPIs and their definitions must be finalised by the stakeholders themselves, the Infrastructure managers. The KPIs were developed from the accumulation of research, existing measures used by Infracos and experience with a series of previous
12 benchmarking projects, together with practical exercises and pilot studies in benchmarking undertaken throughout the project. In addition to the definitional and measurement issues mentioned above, the practical benchmarking exercises in IMPROVERAIL illuminated the inadequacy of many railways information systems, including asset registers. They also emphasized how KPIs must measure performance under the control of the IM, rather than the integrated railway, but that a vision of the success of the total railway must be retained. To interpret and get the best value out of KPI data, the following methodology for the harmonisation of KPIs and using them to identify and implement best practice has been developed: Top-level KPIs (macro benchmarking) Forget it - look at other issues Different Y definitions? N N N Exogenous factors? Is there substantial difference in performance? Y Are KPIs directly comparable? Differences due to. Y Types of service / equipment / nature of assets? Y Processes / practices? Monitor over time, identify trends in performance Identify High Performers Do definitions explain most of the difference in performance? Y Re-define definitions and re-measure N Identify & normalise for differences (e.g. regression analysis) Normal Track Segment by entity / product (e.g. by route / links / services) Micro benchmarking (e.g. further statistical analysis and precursor indicators) Benchmark processes and identify best practices Immediate identification of some best practices Fast Emulate Track Figure 7: Harmonising KPIs and using them to identify & implement Best Practice Figure 7 shows that on occasions it is possible to identify high performers directly from a review of top-level KPIs and to see very quickly what those participants are doing differently. Thus, the benchmarking approach recommended by IMPROVERAIL does not rely on the achievement of 100% comparability in order to identify and emulate best practices. Nonetheless, it is sometimes imperative to segment each Infraco into different entities as described above in section 3.3. Statistical methods can be employed to achieve comparability, but they rely on significant levels of
13 resources and data that are frequently not available. Thus, IMPROVERAIL offers a framework for IM benchmarking that is focussed towards finding and emulating best practices and achieves comparability in a manner that does not necessarily rely on detailed analysis demanding extensive data and resources. 3.9 Case study approach The IMPROVERAIL project has strongly recommended that Infracos adopt both performance and process benchmarking. First, performance benchmarking uses Key Performance Indicators to provide direction and objectivity for identifying which areas show the greatest variability and share of the total value chain. In this way it can indicate which areas have the greatest potential for profit and service quality improvement, and are thus the primary business processes to be analysed in process benchmarking. Then it can help to determine best practice leaders, which process benchmarking can confirm or refute. Three process benchmarking pilot case studies were undertaken during the course of the research. Of these, one was concerned with safety improvement, another with improving service reliability and the third with optimising procurement processes. Like the CoMET/Nova studies, these have all been relatively short, low-input studies to identify those railways with best practice management approaches and indicate what those are. The aim has not been to provide detailed cost targets and cost driver analysis, which only a major high-input study such as INFRACOST can provide. The service reliability case study exemplified the above approach. First, it identified the best performers in this case in terms of punctuality. A few railways were significantly better than most of the others. So it then looked at how those railways managed punctuality and asset monitoring. It did not analyse in detail the exact degree to which their performance was better, but went straight to the differences in their methods and processes. One of the first lessons was that they used a much more sophisticated Pareto analysis of causes of delay, prioritised both by frequency of incident and by length of delay. This is was made possible by having the data available for delays by cause of incident, as shown on the next page in Figure 8. This shows, for example, that although trains are frequently delayed by stations and remote control centres, the delays from this source are relatively low, so additional investment for this would be a waste of money compared with ensuring that the locomotive is there, which takes twice as long. Such information enables a railway to prioritise investment more effectively and address the highest priority problems first. Most of the European Infracos did not have this information, but the best performers all did.
14 Annual No of Incidents 10,000 9,000 8,000 7,000 6,000 5,000 4,000 3,000 2,000 1,000 0 Number of incidents Delay from previous railway Train formation Traction vehicle failure Car or wagon failure Safety installations Passengers Loco drivers Clearing of the track Loco is absent Stations / remote control centres Average delay minutes per incident Change of traction Other - External Others regarding production Customs / Police Handicapped person Other - Maintenance Track Weather factors Other - Ops Management Release of line and track Figure 8: Frequency and average delay by type of incident Average delay (mins) In general, the case studies found that good information systems and easy information availability were closely correlated with good punctuality performance. This is hardly a surprising discovery, but it emphasizes the need to copy the good practices of the better performers, even to the types of information they collect. A second finding was that high performing railways worldwide tended to use shorter thresholds and more rigorous definitions to report delays. Here, the lesson is that those railways with high ambitions use more rigorous forms of reporting. This applies not only to reliability but to all functional areas: worldwide comparisons and benchmarking lead to better target setting, while those who do not benchmark tend to be more complacent and to be satisfied with performance levels that, looked at on an objective basis, are unacceptable. A third finding was that the best performance was not in Europe, but in East Asia, where a climate of continuous improvement has been in place for some decades. It was supported by a much greater use of remote and automatic monitoring. Here again, this does not just apply to reliability, but also to safety and customer service, including passenger information systems. A fourth finding was that there was a lot of commonality between railways in the categories of delay causes they used for classification, but that there were important differences and in some cases, an excess of incidents in the other
15 category. The conclusion was drawn that an agreed set of categories across all European railways, together with agreed thresholds (set at the shortest levels currently used) would greatly enhance the ability of railways to improve their performance through sharing best practice. Overall, this case study brought out the fact that the benchmarking method could work well, if there was enough data to assess that some railways were particularly successful. Such railways are almost always ready to share their leading practice with others. 4. STEPS FORWARD TO IMPLEMENTING STRUCTURE MANAGEMENT COMPANIES BENCHMARKING OF INFRA- A benchmarking project is never finished until action plans have been set in place and implemented to start using best practices and processes. This should condition the entire project from the choice of success factors and KPIs through to the organisation of resources, as better practices are identified during the project. As the benchmarking process should be changeand implementation-oriented, commitment of the actors involved is essential through the whole benchmarking process. The actors should organise their part in the project in such a way that commitment and quality is guaranteed throughout the process. This section will therefore discuss keys to successful railway Infraco benchmarking and finalises by stating which prerequisites Infracos should comply with. 4.1 Keys to successful railway Infraco benchmarking The following conditions are essential before benchmarking will deliver the results of which it is capable (not specifically in order of importance): 1. The initiators of the project must be the benchmarking participants themselves, i.e. the stakeholders must drive the benchmarking process. 2. A strong confidentiality agreement is essential to protect commercially sensitive information and to ensure trust and cooperation in the supply of data and information There should be 5 to 10 actively participating members of the benchmarking project. Up to 10 is helpful, but more than that becomes unwieldy Successful benchmarking relies on a culture (and related systems) that is supportive of change and openness 8. It is essential that those undertaking the analysis have full access to data and people across the organisation. 5. Top management support 9 is therefore necessary. 6. A senior manager should be responsible for the progress of the project, and put in enough time to make sure that the direction of the project is in line with the priorities of the company.
16 7. In most cases, it is essential to set up a project manager to ensure that all actions for improvement are planned and implemented at the time that is scheduled. 8. It is essential to ensure that the process owners are involved in the benchmarking programme from the outset, and come to understand the advantages that they might be able to draw from it for their own areas of responsibility An external professional facilitator for the project is normally essential. 10. The benchmarking programme should be integrated with other corporate programmes that are in process. 11. There should be enough meetings of the steering committee of the participants to create a community of interest 11 and a desire to form informal networks to exchange information on new ideas and best practice beyond what is being covered in the project itself. 12. The approach to analysis should be action and value-oriented 12. Work should be focussed on issues which form a large proportion of the value chain or which are critical to customers top priorities. 13. While concise questionnaires are important in collecting data, they should be followed up with face-to-face interviews with the relevant managers where appropriate, and with site visits to see other railways, particularly those that are suspected to be good performers 13. More can often be achieved by visual observation than by detailed analysis. 14. It is of the utmost importance that the collected data is comparable. Common definitions of data items and common data sources and measurement techniques are therefore important. 15. Understanding and treatment of the context and the environment of the Infracos is required to be able to better interpret the collected data. 16. For case studies, it helps to prepare a set of hypotheses at the outset, which are then tested in the course of the study. 4.2 Infrastructure company benchmarking prerequisites as stated earlier the IMPROVERAIL project s three pilot benchmarking exercises showed that the following steps should be undertaken in order to take forward Infraco benchmarking: Establish the stakeholders (the Infracos) as leaders and directors of the project. Benchmarking does not work if imposed from outside. Infraco information systems should be improved (most are currently inadequate). Infracos should collectively agree and prioritise KPIs. Infracos need to agree on harmonised definitions for KPI data items and they should develop consistent measurement techniques. A large enough number of Infracos should be persuaded to participate. Participant Infracos should be outward looking with a desire to improve performance. Set up a clearing-house for Infracos who wish to benchmark specific topics.
17 When these steps and all other keys to successful benchmarking are taken into consideration, the proposed Infraco benchmarking methodology can be of great value to the sector, leading to gradually improving performance. 5. CONCLUSIONS Benchmarking has proven to be a valuable methodology to improve performance in the railway sector. However, vertical separation implies that benchmarking of national railways in the traditional manner has to be adapted to reflect the Infraco s specific priorities. The Infraco has to consider performance in the context of what is within its managerial control. Therefore, how success is measured and the focus of priorities for improvement will be different to those of the integrated railway. A key difference relates to the more operational focus of the Infraco and the more commercial focus of the TOC, which leads to different processes that are chosen to be benchmarked. The key characteristics of the Infraco benchmarking methodology developed are as follows: Balanced: covers all dimensions of the Infraco i.e. not just cost, or reliability. Progressive: Committed to continuous improvement and search for best practices. Multi-faceted: Recommends that Infracos adopt both performance and process benchmarking. Practical: A system to achieve transferable and implementable results and obtain benefits even without achieving 100% comparability. Credible: including recommendations for achieving harmonisation and comparability. Customised: Key Performance Indicator (KPI) system tailored specifically for Infracos for them to apply in the future. Complementary: Recommends that for unit cost / efficiency benchmarking, the INFRACOST benchmarking model (UIC) is maintained. Stakeholder-led: When applied, benchmarking process must be led by the stakeholders (the Infracos). The benchmarking methodology can be of great value for Infracos to improve their processes, but the methodology itself does not guarantee results. Along many keys to successful benchmarking, the Infracos should be fully committed to the programme, they should exchange harmonised data on a full confidence basis and they should be willing to implement change, otherwise Infraco benchmarking will be a waste of time and resources.
18 BIBLIOGRAPHY Adeney, W.E. (2003), CoMET and Nova deliver tangible benefits, Railway Gazette; Metro Report 2003, vol 159, p15-18 BSL (2000), report to the UIC: the cost of railway infrastructure investment and maintenance / renewal (INFRACOST), June 2000, 58p. Hirsch, R.C.d A and Adeney, W.E. (2000), Making a real difference to performance: Global Benchmarking in railways, Judge Institute proceedings, Cambridge IMPROVERAIL consortium (2003), Improved tools for railway capacity and access management, Deliverable 2: Benchmarking methodologies and Harmonisation of concepts in the Railway sector, European Commission 5 th framework (growth) programme, 129p. IMPROVERAIL consortium (2003), Improved tools for railway capacity and access management, Deliverable 3: Benchmarking exercise in Railway Infrastructure Management, European Commission 5 th framework (growth) programme, 250p. Kaplan, R. S. and Norton, D. P (1993): The Balanced Scorecard Measures that Drive Performance, Harvard Business Review, No Vaglio, M. (2003), Creating the conditions for successful benchmarking Railbench experience, Brussels, , 13 p. Internal documents from: BEST, Benchmarking European Sustainable Transport CoMET, Community of Metros Nova studies, Community of medium sized metro systems PRORATA, Profitability of rail transport and adaptability of railways PROMAIN, Progress in Maintenance and Management of Infrastructure RAILBENCH, Best practice benchmarking consortium of national railways
19 NOTES 1 This type of organisation can also be found in literature identified as: Infrastructure Manager, Infrastructure Provider and Infrastructure Company (Infraco). Infraco is the term used in this paper. 2 IMPROVEd tools for RAILway capacity and access Management. 3 The list is added as Appendix A in IMPROVERAIL Deliverable D3. 4 The Infracos are anonymous because of confidentiality agreements. 5 The Infracos are anonymous because of confidentiality agreements. 6 Noted in CoMET / Nova and an alternative, anonymous approach used in INFRACOST. 7 As noted in CoMET/Nova. 8 As noted in BEST. 9 As noted in the CoMET/Nova, BEST, INFRACOST and PROMAIN reports. 10 As Promain, CoMET/Nova and INFRACOST showed. 11 Noted in PRORATA, PROMAIN, CoMET/Nova and BEST. 12 Noted in CoMET/Nova. 13 As noted in PROMAIN, INFRACOST, and in CoMET / Nova studies and confirmed in the IMPROVERAIL pilot studies.
International Bus System Benchmarking: Performance Measurement Development, Challenges, and Lessons Learned Paper submitted for Transportation Research Board 86 th Annual Meeting (2007) Initial date submitted:
SUPPORTING THE RAIL INDUSTRY UNIQUE SOLUTIONS FOR UNIQUE SITUATIONS WWW.SGS.COM/RAIL OUR UNIQUE SOLUTION TO YOUR UNIQUE SITUATION SGS is recognised as the global benchmark for quality and integrity. We
Interserve and Sheffield Hallam University market research 2012 Page 2 www.commercial.interserve.com How are companies currently changing their facilities management delivery model...? we have a strategy
Asset Management Policy March 2014 In February 2011, we published our current Asset Management Policy. This is the first update incorporating further developments in our thinking on capacity planning and
Key Performance Indicators in Railways- Who measures what? Contact: Victor Vaugoin (email@example.com) Key Performance Indicators- how to define and measure performance for Infrastructure Managers
IRG-Rail (13) 2 Independent Regulators Group Rail IRG Rail Annual Market Monitoring Report February 2013 Index 1 Introduction...3 2 Aim of the report...3 3 Methodology...4 4 Findings...5 a) Market structure...5
: Major challenges facing rail operators, maintainers & owners and the role of ict 1 Executive summary The challenge is clear: today, more than ever, rail operators are expected to operate more efficiently,
Sustainable Development Strategy Our vision and strategy: A railway fit for the future 2013 2024 Document Ref: SBPT204 Version 0.71 Executive summary Network Rail exists to generate outstanding value for
Measure 9: Updating the interoperability directives on high-speed and conventional railway networks First page: Policy package: 2A: Rail liberalisation and harmonisation Measure 9: Updating the interoperability
Rail and Underground Panel Date: 11 April 2014 Item 8: Rail and Underground International Benchmarking Report This paper will be considered in public 1 Purpose and Decision Required 1.1 This paper presents
Level 5 Diploma in Managing the Supply Chain (QCF) Qualification Specification Created: May 2012 Version: 1.0 Accreditation Number: 600/5605/8 Qualification Start Date: 1 st June 2012 Qualification Last
Stakeholders' consultation on a proposal for an EU coordinated approach to R&I in the rail sector under Horizon 2020 in support to the completion of the Single European Railway Area This consultation asks
TEC Capital Asset Management Standard January 2011 TEC Capital Asset Management Standard Tertiary Education Commission January 2011 0 Table of contents Introduction 2 Capital Asset Management 3 Defining
Managing Performance and Cost Information Benefiting from Existing Experience and Tools Frank Zschoche civity Management Consultants Asset Management Seminar Daejeon, October 7th, 2011 Agenda Why benchmarking?
Purchasing and Managing Software Licences REPORT BY THE COMPTROLLER AND AUDITOR GENERAL HC 579 Session 2002-2003: 1 May 2003 LONDON: The Stationery Office 8.25 Ordered by the House of Commons to be printed
London Underground Environment Strategy 2008 2013 mayor of london Transport for London Contents Introduction 3 London Underground Environment strategy 4 Managing environmental impacts 5 Embedding environment
Asset Management UIC involvement Teodor Gradinariu Senior Technical Advisor UIC The environment The preparations for the European Single Market 1993 led to liberalisation of transportation markets Restructuring
STARS Europe Accreditation Scheme Accreditation Guidance for STARS SCHOOLS Primary Schools This document brings together all the information needed for STARS schools implementing the STARS campaign Mobiel
BENJAMIN J CONDRY Nationality: British Citizen Year of Birth: 1975 Key Skills and Experience Currently Senior Research Associate and International Suburban Rail Benchmarking Project Manager at the Railway
Practitioner Certificate Software Asset Management Syllabus Version 2.0 June 2010 Practitioner Certificate in Software Asset Management The ISEB Practitioner Certificate in Software Asset Management (SAM)
trapeze Rail System English Software for Rail Modelling and Planning Trapeze Rail System Simulation and Planning www.trapezegroup.com Enabling future railway plans Cost reductions through integrated planning
IT Professional Standards Information Security Discipline Sub-discipline 605 Information Security Testing and Information Assurance Methodologies December 2012 Draft Version 0.6 DOCUMENT REVIEW Document
1. Introduction The Department of Provincial and Local Government (DPLG) defined performance management as a strategic approach to management, which equips leaders, managers, employees and stakeholders
ATOC Approved Code of Practice Incident Response Duties of Primary Support Operators Submitted by --------------------------------------------------------- Gary Cooper Director, Ops., Engineering & Major
THE RAILWAYS ACT [ No. 4 OF 2002] REGULATIONS Made under section 28 THE RAILWAYS (ACCIDENT INVESTIGATION AND REPORTING) REGULATIONS THE RAILWAYS ACT [ No. 4 OF 2002] REGULATIONS Made under section 28 THE
Executive Summary 1. The consultation seeks views on the vision, targets and measures for improving road safety in Great Britain for the period beyond 2010. 2. We have made good progress in reducing road
Guidance for building and civil engineering projects Procurement requirements for reducing waste and using resources efficiently Model procurement wording for clients and contractors to cut waste on construction
European Distribution System Operators for Smart Grids Position paper on Electric Vehicles Charging Infrastructure European Distribution System Operators for Smart Grids Position paper on ELECTRIC VEHICLES
9909 Mira Mesa Blvd. San Diego, CA 92131 Phone: 800-768-3729 Phone: 858-225-3300 Fax: 858-225-3390 www.smartdraw.com Visual Strategic Planning Thinking and Communicating Visually in the Strategic Planning
www.pwc.nl Procurement Transformation: Towards Sourcing & Procurement Excellence PwC firms provide Industry-focused Assurance, Tax and Advisory services to enhance value for their clients. More than 161.000
SC21 Manufacturing Excellence Process Overview Prepared by:- The SC21 Performance, Development and Quality (PDQ) Special Interest Group (SIG) Acknowledgement The scoring methodology used in the Management
Procurement and Contract Management Strategy 2013-2017 Title Owner Version 1 Distribution Procurement and Contract Management Strategy 2013 2017 Procurement and Performance Manager Corporate Issue date
KEY PERFORMANCE INDICATORS TO APPLY TO WESTNET RAIL ARTC SUBMISSION The Acting WA Independent Rail Access Regulator ( Regulator ) has requested submissions from interested parties with regard to a consultancy
15.4 DELIVERY PROGRAM PROGRESS REPORT JANUARY TO JUNE 2015 & UPDATE ON FUTURE PERFORMANCE MEASUREMENT & REPORTING ATTACHMENT: RESPONSIBLE OFFICER: AUTHOR: DELIVERY PROGRAM ITEM: DELIVERY PROGRAM PROGRESS
Prepared by James Horton - james.horton@ - +61 8 9324 8400 August 2012 Overview The intent of this paper is to describe how rail automation can be integrated with logistics scheduling. It provides general
Kinapse Consulting, 2011 Advise Build Operate www.kinapse.com As Medical Affairs evolves and takes a more prominent role in the development and commercialisation of medicines, it needs a more robust approach
European GNSS Applications in Horizon 2020 Official International Space Information Day 2015, Brussels 10 November 2015 Marta Krywanis-Brzostowska European GNSS Agency Integrated approach: understanding
Vodafone Global Supplier Management Vodafone Global Enterprise One global communications supplier for streamlined processes, reduced costs and fully centralised management. Vodafone Power to you 78% of
1 - Introduction Since the late 1980 s the European ANS system has gradually become unable to cope with growing traffic demand. The Single European Sky initiative of the European Commission is meant to
Request for Information (RFI) Concerning Procurement of Electrified Rolling Stock ("Rolling Stock") November 2013 1 Request for Information ( RFI ) Concerning the Procurement of Electrified Rolling Stock
Driving Transformational Change Innovative Solutions for Optimising Performance in Quarrying & Mining Fleets Babcock International Group: In Brief The UK s leading engineering support services organisation
Passenger Rail Service Satisfaction 2014-15 Quarter 2 Statistical Release 18 December 2014 Responsible Statistician: Dr Fazilat Dar 020 7282 3705 Further information Media Enquiries: 020 7282 2094 Contents
PORTFOLIO, PROGRAMME & PROJECT MANAGEMENT MATURITY MODEL (P3M3) 1st February 2006 Version 1.0 1 P3M3 Version 1.0 The OGC logo is a Registered Trade Mark of the Office of Government Commerce This is a Value
EUROPEAN COMMISSION Brussels, 17.12.2013 SWD(2013) 524 final COMMISSION STAFF WORKING DOCUMENT A call to action on urban logistics Accompanying the document COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN
APM Procurement Guide : Draft7_RevA_Chapter 2.1_Project Procurement Strategy_Jan12 1 2.1 STAGE 1 PROJECT PROCUREMENT STRATEGY In this stage, the project definition is developed so that decisions can be
Report EU Databases and Interfaces Study Management Management to Final Report Version 2.. Document control Superseded documents Version history Version Date Comments 1 14.01.2012 to Final Report 2 10.02.2012
Documentation on Noise-Differentiated Track Access Charges Information on Status, Background and implementation Documentation on Noise Differentiated Track Access Charges: Executive Summary > Railway noise
Prefeasibility Study for the High Speed Line HU-RO Border Bucharest - Constanta Description and Objectives Timisoara - 13 th of September 2012 1 The European Vision for Railway Transport The European Commission's
Calculation of charges for the Use of Railway Infrastructure THE RAILCALC PROJECT OVERVIEW & RESULTS Paulo Teixeira - RAILCALC WORKSHOP Brussels, February 15 th, 2008 Team European Commission: DG Energy
Network Rail October 2007 Strategic Business Plan Supporting Document 2 Executive Summary The capacity strategy in the SBP is drawn from a range of sources. The HLOSs specify a small number of key schemes
Fundamental rights & anti-discrimination THE COSTS AND BENEFITS OF DIVERSITY European Commission Emplo 2 THE COSTS AND BENEFITS OF DIVERSITY A Study on Methods and Indicators to Measure the Cost-Effectiveness
156 Activities Activity 9.1 (PP. 357 358) [Project planning exercise] You are required to construct a project plan for the following information system development project. Your objective is to schedule
1. Background On different occasions, Eurogas is asked to present its views on the future of the European gas industry. The forecasts are mainly needed for conferences and bilateral discussions with European
Rail Infrastructure Costs in Hungary 1 Katalin TANCZOS 2, Ferenc MESZAROS 3, Adam TOROK 4 Budapest University of Techonology and Economics Department of Transport Economics H-1111 Budapest, Bertalan L.
The Cranfield Project an opportunity for your business Applying learning to real business issues At Cranfield School of Management we believe that Logistics and Supply Chain excellence is both a desirable
ANNEX 1. BUDGET 1.1. Budget heading 06020101 - Removing bottlenecks and bridging missing links; 06020102 - Ensuring sustainable and efficient transport in the long run; 06020103 - Optimising the integration
Maximising supply chain throughput with existing infrastructure Improve customer service without capital outlay 1 the CHALLENGE SUPPLY CHAIN AND LOGISTICS Increasing global complexity and uncertainty is
www.pwc.co.uk November 2011 Managing contractors involved in high impact activities A study of practices adopted by major organisations across six different sectors Contents 1. Introduction 2 2. Executive
Better Practice Contract Management Framework Better practice element Comment: What is required and why Governance Agencies contract delegations are clear and consistent with general financial delegations
Bulletins of the Number Manufacturing Performance Survey June 2005 33 Innovation: More than Research and Development Growth opportunities on different innovation paths Steffen Kinkel, Gunter Lay and Jürgen
PR18 working paper Working paper 3: Initial views on the regulatory framework for Network Rail s system operator function Date of publication: 7 June 2016 Overview This paper is part of a series of working
Procurement Strategy 2013-2017 Delivering Social Value for our Community Making Bath & North East Somerset an even better place to live, work and visit 1 Picture courtesy of Bath & News Media Group Our
National Passenger Survey Autumn 2012 Main Report Who are Passenger Focus? Passenger Focus is the independent consumer watchdog for Britain s rail passengers and England s bus, coach and tram passengers.
1 Photo courtesy of Bechtel Photo Library HS2 Jobs Analysis Leo Eyles June 2013 Chapter 1 Introduction Overview was asked by the Industry Leaders Group (ILG) established by Greengauge 21 (GG21) to undertake
Public Record Office Victoria PROS 10/10 Strategic Management Guideline 5 Records Management Strategy Version Number: 1.0 Issue Date: 19/07/2010 Expiry Date: 19/07/2015 State of Victoria 2010 Version 1.0
Do Better Bus Area Monitoring and Evaluation Framework March 2014 The Department for Transport has actively considered the needs of blind and partially sighted people in accessing this document. The text
Rail Asset Management Rail Rail Asset Management Calibre Global understands the key drivers facing rail owners and operators are both financial and operational performance. The end objective for any company
Achieving Excellence in Construction A manager s checklist Construction projects Construction projects A manager s checklist This document outlines the key questions that a Senior Responsible Owner (SRO)
Energy, Utilities and Chemicals the way we see it Call Center Optimization Utility retail competition is about customer satisfaction, and not just retail prices Customers critical awareness; emancipation
Network Rail Consultation on Traction Electricity Consumption Rates for Train Operating Companies August 2008 2 1 Introduction... 3 Context & background... 3 2 Calculation of The New Electricity Consumption
Our Group strategic framework The Executive Committee focuses on evolving and delivering the Group strategy. For 2012, the Group strategic framework has been refreshed. Our purpose (why we exist) Network
Step by Step Project Planning Contents Introduction The Planning Process 1 Create a Project Plan...1 Create a Resource Plan...1 Create a Financial Plan...1 Create a Quality Plan...2 Create a Risk Plan...2
Finance Transformed Changing the focus Finance Business Partnering Contents Out of the shadows of the back office 1 Striking the right balance in finance 2 Finance finds multiple barriers to becoming a
English Trapeze Rail System www.trapezegroup.com Trapeze Rail System Enabling future railway, tram and metro transport The worldwide growth in demand for travel and increasing competition between all modes
WEST YORKSHIRE BUS STRATEGY Phase 1 Consultation on Vision and Principles Introduction and Purpose There is a statutory requirement for the West Yorkshire Combined Authority (WYCA) to produce a Local Transport
Definition: Financial management is the system by which the resources of an organisation s business are planned, directed, monitored and controlled to enable the organisation s goals to be achieved. Guidance
Vodafone Global Enterprise Global Mobility Outsourcing Whitepaper Vodafone Global Enterprise The bottom line: By 2012, the number of mobile workers is expected to reach one billion worldwide (Source IDC,
White Paper from Global Process Innovation by Jim Boots Fourteen Metrics for a BPM Program This white paper presents 14 metrics which may be useful for monitoring progress on a BPM program or initiative.
DSDM Case Study An Agile Approach to Software Systems Development for the Highways Agency Government agencies are constantly striving to develop software systems that support business objectives, deliver
Strategy for regulation of health and safety risks - chapter 7: Rolling Stock Asset Management ORR strategy for Rolling Stock Asset Management How ORR will address this topic: The industry, regulator and