ORR Rail Delay Compensation Mystery Shopping Train operator results. June P7045 (August 2012)

Transcription

1 ORR Rail Delay Compensation Mystery Shopping Train operator results June 2016 P7045 (August 2012)

2 Individual train operator results Introduction and background The Office of Rail and Road (ORR) commissioned research agency PRS/ESA to conduct mystery shopping to inform our response to the super-complaint we received from Which? regarding rail delay compensation 1. The mystery shopping was primarily concerned with investigating the quality of compensation information provided to passengers by train operating company (TOC) staff. The main findings from this mystery shopping research were published 2 alongside our supercomplaint response report 3 on 18 March The research objectives, methodology and the mystery shopper scenario are described in full in Sections 1-3 of the main research report. While the overall sample of 392 mystery shops delivered robust analysis at an industry level, at individual TOC-level the sample sizes are significantly smaller and so less statistically reliable as a result. There were practical reasons why we could not achieve a larger TOC-level sample at the time. The number of train operators we were required to mystery shop (22 across the whole of GB) made it costly and time-consuming to conduct fieldwork. The effect of this was that it imposed constraints on the number of mystery shops we could reasonably undertake in the time we had, given the need to gather all our evidence and respond to the super-complaint within 90 calendar days. Therefore whilst applying bigger TOC-level sampling quotas was desirable, it was impractical given the circumstances. Given our resultant TOC-level analysis was less robust than our industry-level analysis, it was necessary to quality assure these results by discussing them with the TOCs concerned. This allowed us to ascertain if the positive or negative aspects of their performance we observed 1 Which? turns super-complaint powers on rail industry, 21 December 2015: 2 Rail Delay Compensation Mystery Shopping, 18 March 2016: data/assets/pdf_file/0009/21105/rail-delay-compensation-mystery-shopping-findingsreport.pdf 3 ORR super-complaint response report, 18 March 2016: data/assets/pdf_file/0009/21141/which-super-complaint-response-report.pdf 1

3 were indeed reflective of their wider operations, or potentially more localised events 4. This helped us validate the accuracy of our findings against the backdrop of a small TOC-level sample, and gave the train operators immediate feedback to begin investigating the prevalence of the issues we uncovered. TOC engagement We met with the TOCs 5 assessed as part of our mystery shop during April and May This has been an important aspect of our final analysis because it has allowed us to more fully understand why some TOCs have performed markedly better than others in some areas. We have subsequently learnt, for example, that most of the information failures we observed were caused by common factors. These information failures broadly fell into four categories: Company policy. Where we found systemic information failures in some areas, often this was because staff were giving responses consistent with company policy, but that the company s policy itself was found to be inconsistent with our view of the legal or regulatory requirements. The companies this concerned therefore agreed to review their policies in light of this. Complexity. Where we found a pattern of seemingly inconsistent responses from staff to specific questions, it was particularly associated with some TOCs that operate less straightforward compensation schemes (e.g. non-delay Repay). This might go some way to explaining why some staff at those TOCs find it more difficult to understand the compensation arrangements they operate, and thus relay accurate information to passengers. Better training may therefore be an important way of mitigating this. Training. In other instances (e.g. where there were lots of don t know responses) we simply found that some TOCs processes for training their staff on their compensation arrangements were inadequate. A common issue here seemed to be the lack of refresher training to check and reinforce prior learning. To underline this point, those TOCs who performed comparatively better were usually found to be those with more 4 For example, where a TOC scored 0% overall based on a sample of 12 mystery shops this does not mean t hat none of their staff are able to provide accurate compensation information to passengers. Our mystery shop provides a limited snapshot of their staff s performance as observed by our mystery shoppers during fieldwork. Engaging with the TOCs allowed us to establish the extent to which we can generalise about the specific practises we observed. 5 Cross Country was the only exception to this having rescheduled their meeting with us for Thursday 2 nd June

4 robust, comprehensive procedures in place for staff refresher training that incorporated regular knowledge and comprehension checks. Localised issues. There were also instances where specific failures or weaknesses were somewhat localised e.g. issues with staff at particular stations. In these cases the TOCs have been able to immediately identify those stations and to begin putting in place measures to address the problems. Engaging with train operators has therefore been valuable in enabling us to more easily identify the causal factors which drive good and bad performance. This has given us a platform to begin identifying and promoting the changes that will empower passengers by ensuring they receive useful, accurate information when they make delay compensation enquiries. Next steps At our meetings with the train operators whose performance we assessed as part of our mystery shopping research, we have been pleased by their willingness to accept the findings as constructive feedback and encouraged by their readiness to act upon them. We have now agreed a work plan and timetable with each train operator to make improvements over the coming months in the areas for action we have highlighted to them. We will be undertaking a follow-up mystery shop before the end of the regulatory period to assess the progress each operator has made. 3

5 1 Accurate information regarding the minimum delay to be eligible for compensation Accurate information provided regarding the minimum delay required to be eligible for compensation % GTR Great Northern # % c2c % Virgin Trains East Coast # % London Midland % Virgin Trains West Coast # % Southeastern % Abellio Greater Anglia % Arriva Trains Wales % GTR Southern % East Midland Trains % GTR Thameslink % Chiltern Railways # % ScotRail % Northern Rail % TfL Rail # % Great Western Railway % South West Trains % First TransPennine Express # % London Overground % Merseyrail % All non-station managing TOCs* # % % 4

6 2 Accurate information regarding the proportion of ticket cost payable Accurate information provided regarding the proportion of ticket cost the passenger might be eligible for % Southeastern % Chiltern Railways # % Virgin Trains West Coast # % Abellio Greater Anglia % GTR Great Northern # % c2c % Virgin Trains East Coast # % East Midland Trains % Great Western Railway % London Midland % Arriva Trains Wales % GTR Thameslink % First TransPennine Express # % GTR Southern % ScotRail % South West Trains % London Overground % Northern Rail % TfL Rail # % Merseyrail % All non-station managing TOCs* # % % 5

7 3 Correctly advised that compensation could be paid in cash, or that vouchers could be exchanged for cash Correctly advised that the passenger could receive compensation in cash, or could exchange vouchers for cash % Southeastern % Chiltern Railways # % Virgin Trains West Coast # % Virgin Trains East Coast # % Abellio Greater Anglia % GTR Great Northern # % c2c % Great Western Railway % East Midland Trains % London Midland % Arriva Trains Wales % GTR Southern % First TransPennine Express # % GTR Thameslink % ScotRail % South West Trains % London Overground % Northern Rail % TfL Rail # % Merseyrail % All non-station managing TOCs* # % % 6

8 4 Advised how to make a claim Provided information on how to make a claim % Southeastern % Virgin Trains West Coast # % c2c % Abellio Greater Anglia % Chiltern Railways # % Virgin Trains East Coast # % GTR Thameslink % London Midland % Great Western Railway % Merseyrail % London Overground % TfL Rail # % Northern Rail % ScotRail % GTR Great Northern # % East Midland Trains % First TransPennine Express # % Arriva Trains Wales % South West Trains % GTR Southern % All non-station managing TOCs* # % % 7

9 5 Full & accurate explanation provided of the arrangements and conditions for claiming compensation Full & accurate explanation provided of the arrangements and conditions for claiming compensation (all 4 of the above were correctly advised) % Southeastern % Virgin Trains West Coast # % c2c % Abellio Greater Anglia % Chiltern Railways # % Virgin Trains East Coast # % GTR Great Northern # % GTR Thameslink % GTR Southern % East Midland Trains % ScotRail % London Midland % Arriva Trains Wales % First TransPennine Express # % Great Western Railway % South West Trains % Northern Rail % Merseyrail % London Overground % TfL Rail # % All non-station managing TOCs* # % % 8