Rail Finance Quality Report

Transcription

1 Rail Finance Quality Report Release Date: 11 September 2015 We have now published a separate quality report for the Rail fares index accompanying the Rail fares index statistical release. Rail Finance Quality Report 1

2 Contents Introduction 3 Historical background 5 Relevance 11 Accuracy and reliability 14 Timeliness and punctuality 16 Accessibility and clarity 17 Coherence and comparability 19 Trade-offs between output quality components 21 Performance, cost and respondent burden 22 Summary of methods used to compile the output 23 Quality performance indicators table 26 Rail Finance Quality Report 2

3 Introduction This is a report on the quality of ORR Rail Finance statistics. This helps users to understand the quality of our statistics, and also ensures ORR is compliant with principle 4 of the Code of Practice for Official Statistics 1. The quality report covers the following areas: Historic background a background to each statistic and detail of changes throughout the time series Relevance of the data - the users of the statistics, and user-engagement we have done Accuracy and reliability the accuracy of each statistic Timeliness and punctuality our timelines for the production, quality assurance and publication of each statistic Accessibility and clarity the format of our statistics and where they can be found Coherence and comparability comparisons to similar statistics published elsewhere Trade-offs between output quality components Performance, cost and respondent burden Methodology additional detail on the methodology behind each statistic 1 Principle 4: Sound methods and assured quality. Statistical methods should be consistent with scientific principles and internationally recognised best practices, and be fully documented. Quality should be monitored and assured taking account of internationally agreed practices. The Code of Practice can be accessed here Rail Finance Quality Report 3

4 Rail finance statistics are supplied by a number of different sources and are measured using a range of metrics: Government support to the rail industry the government revenue support grants to domestic passenger and freight services in Great Britain, other government expenditure in the rail network and loans issued to Network Rail; Government subsidy per passenger train kilometre - the government subsidy paid to or premiums received from each train operating company, subsidy per passenger kilometre by train operating company, and government support to Passenger Transport Executives (PTE) 2 ; Private investment in the railway industry private investment statistics showing the expenditure and disposal of fixed assets from private companies in the rail industry; and Rail fares indices - the rail fares price indices provide a measure of the change in prices charged by train operating companies to rail passengers. ORR publishes quality reports on other ORR statistics which can be found on the statistical release page. 2 PTE s are strategic transport bodies serving the six largest city regions outside London. They cover the West Midlands (Centro), Greater Manchester (Transport for Greater Manchester), Merseyside (Merseytravel), West Yorkshire (Metro), South Yorkshire (Travel South Yorkshire) and Tyne & Wear (Nexus) Rail Finance Quality Report 4

5 Historical background Government support to the rail industry Prior to , government support to the rail industry comprised grants to British Rail and the Passenger Transport Executives (PTEs), and borrowing by British Rail from the National Loans Fund. The privatisation of the rail industry in April 1994 led to changes in the basis of government funding. Grant levels were set to allow the newly formed rail companies to earn commercial returns. Support for passenger services was channeled through the Office of Passenger Rail Franchising (OPRAF) and the PTEs, which were funded by the Revenue Support Grant and an additional Metropolitan Grant. Any cash surpluses that were earned were returned to the Exchequer and used to reduce the net level of support to the industry while the rail companies were still in the public sector. In 2001 OPRAF support was replaced with Strategic Rail Authority (SRA) support, which in 2005 was replaced with grants made by the Department for Transport (DfT), the Welsh Government and Transport Scotland. Since its creation in 2001 Government grants have also been paid to Network Rail. Government support to the rail industry now chiefly consists of support grants paid to Network Rail and Train Operating Companies (TOCs), PTE Special Grants and spending on major projects. This Government support to the rail industry statistics are comprised of: Central government grants - these are net payments to/from TOCs and performance receipts; Grants made to Passenger Transport Executives (PTEs) - net payments to PTEs for national rail services; Direct rail support is the grant payment to Network Rail and, during the construction of HS1, to London and Continental Railways (LCR) 3 ; Freight grants grants paid to encourage the movement of freight by rail; Other elements of Government support expenditure on major projects (e.g. Crossrail), support to arms-length bodies, including British Transport police and Transport Focus, and other ad hoc rail projects; and Loans issued by DfT to Network Rail (as of 1st September 2014) following Network rail re-classification to the public sector. 3 LCR is a UK Government-owned company that specialises in the management, development and disposal of property assets within a railway context and, in particular, property assets associated with major infrastructure projects Rail Finance Quality Report 5

6 Until 1993/94, central government grants consisted of Public Service Obligation (PSO) Grant and Level Crossing Grant to British Rail. From 1994/95 onwards PSO grants were replaced by OPRAF support and grants to British Rail and, from the point of franchise, to the private sector TOCs. Level Crossing Grant was paid to Railtrack in 1994/95 and 1995/96 and discontinued at the start of 1996/97 with the transfer of Railtrack into private ownership. Central government grants to TOCs include payments contracted in their franchise agreements and performance receipts. The latter vary depending on the financial performance of the TOC, and can result in payments from a TOC to the government as well as from government to the TOC. These figures also include contract payments from the Department for Transport to Merseytravel PTE and Transport for London which are made under funding arrangements for the devolved Merseyrail and London Overground operations. Negative values in the table show where the Government was in receipt of payments. PTE grants are paid to the five Passenger Transport Executives (PTEs) and Transport for Greater Manchester. These figures include grants made to PTEs specifically for national rail, so will not include any spending on rail made by PTEs from general grants. All grants to PTEs are currently made by the Department for Transport. Following privatisation, between and PTE rail funding also included loan repayments under Deeds of Assumption (DoA). These were payments to the PTEs relating to their past capital investment in the railway. They were made by British Rail and DoA Ltd until 2001, when the Strategic Rail Authority took over responsibility for the payments until they were paid off. Direct rail support comprises direct payments to Network Rail. These originated in 2001 and were initially paid to Railtrack, and then to Network Rail following its creation in Prior to this, Railtrack was funded by network access charges paid by train operators. These figures are net of any rebates to Government and interest fee payment by Network Rail to DfT. These figures also include payments that were made to London and Continental Railways during the construction of the High Speed 1 route. Rail freight grants are also paid by Government to encourage the movement of freight from road to rail and are currently secured through the Mode Shift Revenue Support (MSRS) Scheme. Further details on the recipients and levels of freight grants awarded by DfT can be found at Freight Grants The Other elements of Government support category includes other Government expenditure on national rail. It includes expenditure on major projects, rail pensions, support to arms-length bodies including the British Transport Police and Transport Focus, a grant to British Rail to finance its residual activities, and other ad hoc rail projects. Prior to privatisation it chiefly comprised the changes in the net indebtedness (borrowing minus lending) of the rail industry. The proceeds from the sales of rolling stock operating companies (ROSCOs) and British Rail non-passenger business in and are also included in this category. Rail Finance Quality Report 6

7 Loans issued by DfT to Network Rail (as of 1 September 2014) were arranged as part of Network Rail s re-classification to the public sector. Funding is provided by DfT in the form of a loan to Network Rail to continue the programme of investment in the rail network. The year by year profile of Government support to the rail industry was affected by the decision in the Access Charges Review 2003 to re-profile Network Rail's income. This led to Government support being lower than it otherwise would have been in and , with the shortfall being made up in later years. Further details are available on the ORR website 4. Government subsidy per passenger kilometre Government subsidy per passenger kilometre shows the net payments for: Currently active train operating companies; Inactive train operating companies; Contracted grant payments; and Payments to Passenger Transport Executives These statistics show subsidies paid directly to TOCs and premia received from them. They do not take into account the grants to Network Rail that form the large majority of government support to the rail industry, and which provide an indirect subsidy to the TOCs. Changes to franchise boundaries can affect year-on-year comparisons of these figures. Negative values in the table show where the government was in receipt of payments. Subsidy figures for individual TOCs are prepared on an accruals basis and represent all payments made or received under the relevant franchise agreement between the Department for Transport, the Welsh Government, Transport Scotland and individual passenger train operators. 4 National Rail review: data/assets/pdf_file/0010/2116/343.pdf Rail Finance Quality Report 7

8 Private investment in the rail industry Private investment statistics are collected by the Office for National Statistics (ONS) every quarter on behalf of ORR. ORR select up to 40 rail industry companies, such as train and freight operating companies, to take part in the survey every quarter. The survey asks for investment in the following categories: Track and signalling including new routes and new electrification; Rolling stock including eligible refurbishment work; Stations including retail outlet buildings; and All other expenditure associated with the rail business, such as non-rail vehicles and business related costs such as IT and web related costs. Negative numbers can occur due to a company's disposal of rail assets. Network Rail investment data is excluded from these statistics but is included in the government support to the rail industry statistics through the direct support provided by government to Network Rail. Information is provided by a number of private companies and is supplied to ONS confidentially and because of this agreement, ORR cannot provide any further disaggregation. Rail fares indices The rail fares indices provide a measure of the change in prices charged by train operating companies (TOCs) to rail passengers for the following categories: Average change in price of rail fares by ticket type; and Average change in price of rail fares by regulated and unregulated tickets 5 ; The average change in price of rail fares by ticket type shows statistics back to January 2004 and is disaggregated by sector (London & South East, Long Distance and Regional) and ticket type (Advance, Anytime, Off Peak, Seasons, Super Off Peak and others). The average change in price of rail fares by regulated and unregulated tickets shows statistics back to January 1995 and is disaggregated by sector, class (1 st Class and Standard Class), and regulated/unregulated 5 fare. 5 Regulated fares include season tickets for most commuter journeys and Off-Peak fares on most journeys between major cities. Unregulated fares are those which operators are free to determine according to market forces and willingness to pay. Rail Finance Quality Report 8

9 Both statistics are calculated from transaction data held in the Latest Earnings Nationally Networked Over Night (LENNON) system, the rail industry s ticketing and revenue database. The indices include franchised and open access operators operating on the mainline rail network with the exception of Heathrow Express and Heathrow Connect as they are not included in LENNON. Eurostar fares are also not included. The index shows the change in prices from one January to the next as, generally, new fare prices are introduced at the beginning of January. The average change in fares is presented alongside the January all items Retail Price Index (RPI). The all items RPI measures the changes in the cost of a representative basket of goods and services bought by consumers within the UK. Presenting these measures alongside one another illustrates how the average change in rail fares compares with the average change in the cost of goods and services. More information on the RPI and other measures of inflation is available on the Office for National Statistics website 6. The RPI is also used to calculate the change in fares in real terms 7. The expenditure weights (%) are the proportions of market share in terms of revenue for the calendar year preceding the publication of the statistics (e.g. the January 2015 index weightings are based on expenditure during 2014). They are then used to calculate the weighted average price change. In 2014, ORR undertook a review of the rail fares index, which was carried out by the Office for National Statistics (ONS) Methodology Advisory Service. They made a number of recommendations which we have implemented from this year. A summary of the changes we made are below. The full report can be found on the ORR website: Sampling From 2014, ORR is calculating the fares index based on a sample of fares rather than a population of matched prices (i.e. exact same fare present in both the current and previous years). The previous methodology generated an annual dataset of between 1.6m and 3.5m records. The production of the index from this annual dataset was resource intensive and time consuming. ONS recommended that we switch to probability sampling. This has been implemented in 2014, using a sample size in excess of 97,000, approximately 3% of the population dataset Corrected for the effect of inflation Rail Finance Quality Report 9

10 Advance fares There is a greater product turnover in advance fares than for other fare types which means that calculating a consistent matched price index can be difficult. ONS recommended that rather than calculate the change in advance fares through the matched price method, ORR should calculate an overall average price covering all advance ticket sales for each origin destination pair. This has been implemented in the index from 2014 onwards. Price change thresholds Between 2004 and 2013, price changes between -40% and +60% were included within the final index calculations. Due to the size of the dataset, it was not possible to check the validity of all price changes and the flows with the largest weightings were checked and validated. ONS recommended that switching to a sample would enable ORR to quality assure a greater number of fares so all fares outside the price change limits of -20% and +20% have been checked for their validity from 2014 onwards. In addition, those that remain outside of these limits are excluded from the final index calculation as these are deemed to be erroneous. Excluded revenue Prior to 2014, only the revenue from those flows that had a matched price in the current and previous year was included as weighting in the index calculation. From 2014, expenditure on those flows that have been excluded at the various stages of compiling the index (e.g. no price record in Jan 2013 and/or Jan 2014, price change exceeding the allowable thresholds) has been reintroduced to compile the aggregated sector/national/ticket category indices. This means that revenue included as weighting has increased from between 85% and 90% prior to 2014, to in excess of 95%. Rail Finance Quality Report 10

11 Relevance The degree to which the statistical product meets the user in both coverage and content. Measures of rail financing are important in understanding how the rail network and delivery of services is financed in Great Britain with information on government support to the rail industry providing an insight into the burden of the rail industry on public finances. Comparisons of the levels of financing over the time series with railway outputs and other modes of transport provides an insight in to the efficiency of service delivery over time. Statistics on the changes in rail fares provides passengers and policy makers with important information on the cost of travel by rail both within the industry and in comparison to the wider cost of living. Users of ORR statistics: Official Statistics on the ORR data portal are diverse and there are many different reasons people will use the statistics we release. From this, we have identified five groups that use our statistics for various purposes. These are: 1. Policy makers including briefing and local government use the statistics to inform ministerial briefing, answer ad hoc queries and include in their publications. 2. Funders of the rail industry use these statistics to make informed decisions about where public money should be targeted in order to improve punctuality on the rail network. They also use these statistics in briefings and publications. 3. Findings from joint research between ORR and Transport Focus showed that passengers value more rail statistics being brought into the public domain. This would help increase the transparency of the rail industry and enable greater scrutiny. This research also showed that some passengers use the information when making life choices, such as choosing the location of their home. 4. Activists need access to a wide variety of statistics to provide evidence to challenge the status quo. 5. Rail enthusiasts potentially have no limit to the statistics they wish to explore. Rail Finance Quality Report 11

12 Registered users of ORR data portal: Rail finance statistics published on our data portal are used by a range of individuals for planning, analysis, decision making and data validation. Analysis of the registered users of the data portal identified the key users of rail finance statistics to include: Government organisations such as DfT for fares policy and regulation; ORR for supporting evidence for the periodic review and consumer programme; Businesses including train operating companies to understand market conditions; Media organisations for press releases and news articles; Academic institutions such as Universities for dissertations and research purposes; and Personal users. Meeting the needs of users: In December 2014, ORR conducted an online survey to gather feedback on ORR statistical releases and quality reports. A summary of the key results is available on our website which stated: 71% were able to find what they need easily. 61% felt the statistics releases met their needs very well or well, and 77% rated the content of the statistical releases excellent or good. From May 2012 ORR asked users to complete a survey asking what they used each rail statistics table for. ORR communicated the survey through the data portal and notes within each rail statistics report. Rail Finance Quality Report 12

13 The results of this survey identified that individuals use rail finance statistics for a range of reasons: Government support statistics are used for analysis and regional comparisons, understanding levels of investment and government decision making. Private investment statistics are used for understanding levels of investment and government decision making Fares information is used for informing the periodic review, understanding market conditions, government decision making on fares policy and regulation and comparison with other datasets. The last data portal focus group for registered users took place on 5 March 2013 and reviewed the improvements and changes we are planning to make to the data portal. The main objectives of the re-design were to focus on transparency, user experience, increased patronage and gaining UKSA accreditation. The findings from the focus group are published on the user engagement page of our website ORR has gathered feedback on the statistical releases and quality reports during the months of December 2014 and January 2015 through an online survey. Part of the feedback received has been incorporated in the rail finance statistical release and the key findings from the survey are published on our website: If you would like to provide feedback about the data portal please dataportal@orr.gsi.gov.uk Rail Finance Quality Report 13

14 Accuracy and reliability The proximity between an estimate and the unknown true value. Government support to the rail industry and Government subsidy per passenger kilometre The government subsidy per passenger kilometre statistics represent subsidy paid directly to TOCs. Changes to franchise boundaries can affect year-on-year comparisons of these figures. Payments (network grant) made directly to Network Rail, that would have otherwise been paid through the TOCs are not included. In million was paid to the Department by the Olympic Delivery Authority to compensate for increased subsidies paid to the TOCs during the Olympic period, which arose as a result of Olympic ticketing products being exempt from support calculations. This payment cannot be accurately attributed to individual TOCs but is included in the overall TOC total as a balancing item. These figures are audited by three data providers DfT, Transport Scotland and the Welsh Assembly. Therefore, the figures can be subject to change. Private investment Private investment statistics are sourced from the Office for National Statistics (ONS) Railway Investment Survey. These statistics are often revised because companies update their financial data as more information becomes available. Annual and quarterly Q3 revision: In Quarter 3 incorrect data was supplied by ONS for one company. ORR is currently investigating this and will ask ONS to have extra checking procedures when providing the figures for the Investment survey. Further details can be found at: Revisions Log Rail Finance Quality Report 14

15 Rail fares indices The rail fares index is calculated from transaction data held in the LENNON database, the rail industry s ticketing and revenue database. The index has been constructed so that, as far as possible, it covers the cost of travel only. This is done by excluding fares that include extras so as not to distort the index, such as the following: Additional services such as multi modal tickets for urban areas, bus and car packing tickets, entrance fees to attractions etc Short-term temporary fares/promotions; Passengers switching ticket types following the introduction/deletion of certain tickets are not taken into account immediately; and Flows for which we were unable to find price information for either of the two reference years, for example a ticket type that is introduced after the first reference date. London Travelcard purchases are included in the index because such tickets are so important in the earnings of train operators and purchases by rail passengers. In addition, TOCs influence price changes associated with these tickets. From 2014, the rail fares index is calculated based on a sample of approximately 100,000 records. Based on the 2012 dataset, ONS has produced some estimates of the error associated with using probability sampling rather than the population dataset. These can be found in annex B of the ONS report available here: Rail Finance Quality Report 15

16 Timeliness and punctuality Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates. The statistics contained within the finance statistical release are published annually on the ORR data portal approximately 140 days after the end of the financial year, though the rail fares indices have been published as a separate release in 2015 to improve the timeliness of the statistics. The publication schedule outlines the publication dates for National Statistics quarterly and annual statistical releases and other statistics up to 12 months in advance These publication dates are determined by availability of the information and are the earliest possible dates which we can publish the information. Approximately 20 days are required to collect, process, quality assure, sign off the statistics and to prepare the statistical release itself. In previous years, the pre-announced release dates for these outputs have always been met. In the unlikely event of a change to a pre-announced release date, attention would be drawn to this on the data portal together with a full explanation of the reason for the change. Feedback from the data portal online survey, conducted in March 2015, indicated that 80% of respondents were satisfied with the timeliness of the statistics available on the data portal. Rail Finance Quality Report 16

17 Accessibility and clarity Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice. All rail statistics data tables can be accessed free of charge on the ORR Data Portal. Charts and commentary about the statistics and trend are provided in the thematic statistical releases. The rail finance reports currently published on the ORR Data Portal are: Government support to the rail industry - Table 1.6 Government subsidy per passenger kilometre by train operating company - Table 1.7 Private investment in the rail industry (excludes Network Rail investment) - Table 1.9 Index showing average change in price of rail fares by ticket type Table 1.8 A full list of publication dates for the next twelve months can be found in the release schedule on the ORR website. For further information about these statistics please contact the rail statistics team at rstats@orr.gsi.gov.uk Confidentiality, transparency and security The procedures and policy used to ensure sound confidentiality, security and transparent practices. ORR is fully compliant with the Statistics and Registration Service Act 2008 and principle 4 of the Code of Practice for Official Statistics. ORR receives rail finance statistics from DfT, Transport Scotland, the Welsh Government, LENNON and ONS. The statistics are supplied electronically and stored in a data warehouse (ORRbit). Only selected members of staff in ORR have access to the data warehouse and access is password protected. The data provision and storage processes have been independently assessed by external consultants Amour Group to ensure they are secure. Internal and external IT vendors also conduct periodic assessments of our systems. The rail fare indices which are calculated from transaction data held in the LENNON database are not disaggregated by TOC as it is commercially sensitive information. Rail Finance Quality Report 17

18 ORR has systems and processes in place to safeguard personal identities/details and commercially restricted information. The statistics are internally validated by two teams before it is granted a publication status and viewable on the data portal. Connections to remotely hosted databases are within a secure network and penetration testing has confirmed that the data portal is secure against external attacks. ORR wants the rail industry to be more open and transparent. We have established a transparency programme to drive our ambitious vision for the industry, and govern a number of projects to support our aims and objectives. More information on the ORR transparency programme can be found at ORR s transparency programme. Rail Finance Quality Report 18

19 Coherence and comparability Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain. Government support to the rail industry and Government subsidy per passenger kilometre Due to the figures presented in this report being based on DfT, Transport Scotland and Welsh Government audited accounts they do not match the figures quoted within ORR s GB rail industry financial information publication 8, where the figures shown for individual TOC subsidies/premiums are based on the TOC's management accounts. Differences between the figures are likely to be due to payments being accounted for differently and recorded in different years in the two sets of accounts. Government support to the rail industry and government subsidy per passenger kilometre by train operating company are obtained from DfT, Transport Scotland and the Welsh Government. DfT publishes figures showing government subsidy per passenger mile for the TOCs that it franchises on their website 9. These are based on the same information used in these statistics, although the figures also include an estimate of the indirect subsidy received by each of the train operators from the DfT grant to Network Rail. DfT published financial information for franchises they let but this does not include four devolved operators. Also please note that the data produced by DfT is based on passenger miles. Private investment ORR publishes investment in the railway industry by private companies. This does not cover any government subsidy. Rail fares indices Since 2013, ORR has included revenue per passenger journey by sector alongside the rail fares index. This information is calculated from the passenger revenue and passenger journeys tables, available in the passenger rail usage section of the ORR data portal. The rail fares indices measure price change from one January to the next, so in order to make the information comparable, the equivalent index for revenue per journey has been calculated using Q4 data, which covers January to March. It is also important to understand the differences between the rail fares indices and revenue per passenger journey when making comparisons between the two measures. Given the exclusions above, in general we would expect revenue per passenger journey to grow at a Rail Finance Quality Report 19

20 slower rate than a matched price index although this is not always the case. For example, there are instances when revenue from particular products are recorded in the most recent quarter, but not recorded in the equivalent quarter last year, or vice versa. Therefore any comparisons between the two should be treated with caution. The revenue per journey metric reflects changes in customer behaviour and reflects when passengers switch to cheaper tickets. The fares index does not capture this phenomenon, other than in the expenditure weighting for those products in the following year. Furthermore, revenue per journey reflects changes in usage as well as the cost of fares and can be affected by different growth rates on different parts of the network as some sectors have a higher rate per journey than others. Comparability to European statistics The difference in the structure of internal rail markets in European countries means that finance statistics are difficult to compare across member states. The statistical office of the European Union, Eurostat, collects no financial statistics on the rail market. Limited financial information is collected by the Independent Regulators Group (IRG-Rail) for their Market Monitoring Report, including information on passenger and freight revenues although data are not supplied by all European countries. More information on IRG-Rail is available on their website Rail Finance Quality Report 20

21 Trade-offs between output quality components Trade-offs are the extent to which different aspects of quality are balanced against each other. We work to publish the rail finance statistics as soon as possible and currently work to a tight timeline based on when the year ends and when the information is received from the appropriate data sources. We recently reviewed the timescales associated with the publication process and identified that bringing forward the annual rail finance publication date could impact the quality and accuracy of the statistics published. Rail Finance Quality Report 21

22 Performance, cost and respondent burden The effectiveness, efficiency and economy of the statistical output. Following a project in 2008 to establish the best way to store rail data and improve the efficiency and effectiveness of ORR outputs, a number of changes were made to the data publication process. As a result of these changes, amendments have been made to the processes for the publication of official statistics. The introduction of a data warehouse has allowed ORR to automate a number of processes used to check the quality and accuracy of the information we receive, reducing the staff resource required to produce these publications. We have automated various processes involved in the production of these statistics, reducing the overall time and burden involved in publishing these statistics. Furthermore, we have memorandums of understanding (MoUs) with DfT, Transport Scotland and the Welsh Government detailing the scope and timeliness of each dataset supplied. This ensures consistent and timely information is received each period. The MoUs are reviewed on an annual basis. We source the ONS statistics to calculate the real term change in average ticket price from the ONS website. Therefore, there are no MoUs in place with ONS for this information. Approximately 20 days are required each quarter to extract, load and quality assure the statistics and prepare and publish the reports on the data portal. A maximum of 7 staff are involved in this process. These estimates exclude the time taken to undertake changes to the underlying statistics, structured query language (SQL) used to create the reports or the redesign of the reports. Such changes are relatively infrequent but can occur. Rail Finance Quality Report 22

23 Summary of methods used to compile the output Rail finance statistics are supplied by DfT, Transport Scotland, the Welsh Government, LENNON and ONS and stored in a secure data warehouse maintained by ORR. The statistics supplied are subject to an extensive quality assurance process, including a suite of validation checks to ensure the information meets the required specification and is in line with previous trends. Any arising issues are flagged with the data suppliers who must confirm the anomalies or correct the information and re-submit. Explanations from the data suppliers regarding data anomalies are included within our commentary to explain the statistics and trends. These statistics are then prepared for publication. The process includes quality assuring the tables and charts produced and providing supporting commentary regarding the key trends, methodology and quality measures. These reports are subject to peer review. The final stage of the quality assurance process is a sign off by the statistics Head of Profession confirming the statistics meets the quality standards and are fit for onward use. Government support to the rail industry and Government subsidy per passenger train kilometre Statistics are provided annually by DfT, Transport Scotland and the Welsh Government and these are combined to provide a total level of government support to the rail industry for Great Britain for the most recent financial year (April to March). In the statistical release the total government support figures are presented in prices using the GDP deflators 10. These three organisations also provide information on the subsidies paid to the train operators they franchise during that financial year. These statistics are then combined with annual passenger kilometres statistics for each train operator, sourced from the LENNON ticketing & revenue database and the train operators themselves, to provide a figure for subsidy per passenger kilometre. Private investment in the rail industry Statistics are provided by ONS each quarter. The statistics are based on calendar quarter and for the purposes of our publications, we covert this to financial quarters. The methodology for producing this dataset also includes re-assigning the private investment survey categories to align with the categories featured in the private investment report. We 10 Rail Finance Quality Report 23

24 also ensure that all acquisition figures are positive numbers and disposal figures are negative values (multiply the disposal figures by -1). In previous statistical releases, the total investment for all the years were presented in prices using the July 2013 Retail Price Index (RPI). As from this year, the total investment figures for all years are presented in prices the GDP deflators 11. Rail fares indices To calculate the index weights, an extract is taken from the rail industry s ticketing & revenue database (LENNON) for the calendar year, detailing the revenue taken for each available fare on every flow (origin-destination pair). Each year, this dataset contains in excess of 10 million records. To calculate the index itself, the price of each of these fares is taken from a combination of LENNON and the National Fares Manual (NFM) so that we have a price for January in the current year (Jan x ) and a price from January in the previous year (Jan x-1 ). Not all flows/fares have prices for Jan x and Jan x-1 because: 1) The flow and/or fare were introduced in the calendar year so there is no price for Jan x-1 2) The flow and/or fare were discontinued in the calendar year so there is no price for Jan x Currently, if either 1) or 2) apply, then that particular record is omitted from the index calculation (i.e. only those flows that have a price in both Jan x and Jan x-1 are included) Each record within the final dataset is assigned to a sector based on the train operating company (TOC) running the service, and assigned to a ticket type based on the product code and description. The sectors are as follows: London and South East: c2c, Chiltern, First Capital Connect (Govia Thameslink Railway from 14 th September 2014), London Overground, Greater Anglia, Southeastern, Southern and South West Trains. Long distance: Cross Country, East Coast, East Midlands Trains, First Great Western, First Hull Trains, Grand Central and Virgin Trains. Regional: Arriva Trains Wales, First Scotrail, First Transpennine Express, London Midland, Merseyrail, Northern Rail Finance Quality Report 24

25 These differ from sectors in the passenger rail usage statistics as these disaggregate some operators across multiple sectors (e.g. First Great Western operate services in all three sectors in the passenger rail usage statistics) The indices are produced by calculating the price change between Jan x and Jan x-1 weighted by earnings. Following a 2014 review into the rail fares index by the ONS Methodology Advisory Service, from the indices are calculated from a sample set of flows. The selection of flows is based on probability sampling, which ensures that those flows which have the highest expenditure weights (and consequently the largest impact on the index) are included in the final index calculation. The sampling interval is chosen to ensure a sample size of approximately 100,000 records. In 2015, the sampling interval was just over 20,000 so all flows that had revenue in excess of 20,000 were auto-selected and included in the index. The other flows were ordered by revenue and systematically selected at every 20,000 interval. Rail Finance Quality Report 25

26 Quality performance indicators table Quality measure Indicator Description Value Accuracy Number of mistakes made, by type since the time series began The number of serious mistakes in calculation or presentation of aggregates that are not found until after publication. Government subsidy, government support, rail fares index and private investment: 1. For further details please see: Revisions Log Accuracy Average size of revisions The average of the difference between a later and an earlier estimate expressed as the average revision, the average absolute revision, and/or the corresponding relative quantity(ies). Government subsidy, government support rail fares index and private investment: 0 Timeliness and punctuality Time lag between the end of the reference period and date of final results Average number of days from the last day of the reference period to the day of publication of final results Approximately 140 days (130 days for rail fares index) Timeliness and punctuality Punctuality of publication The number of days separating a previously announced date of publication and the actual date. 0 Rail Finance Quality Report 26

27 Accessibility and clarity Number of subscriptions/purchases of each of the key paper reports Average number of users who access the data portal on publication day. Please note other reports are also published on each date. 151 Coherence and comparability Lengths of comparable time series Number of reference periods in time series from last break. Government subsidy: From Government support: From Rail fares index: From 2004 (1995 for regulated/unregulated fares) Private investment: From Assessment of user needs and perceptions Length of time since most recent user satisfaction survey Length of time since most recent user satisfaction survey Survey on Statistical Releases and Quality Reports: January 2015 Performance cost and respondent burden Number of days per year taken to produce the reports and number of staff required for this Average number of days and maximum number of staff involved in loading and quality assuring the statistics, preparing the reports and publishing them on the data portal. 20 days with a maximum of 7 staff Rail Finance Quality Report 27

28 Crown copyright 2015 This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned. This publication is available at orr.gov.uk Any enquiries regarding this publication should be sent to us at orr.gov.uk Rail Finance Quality Report 28