Mystery Shopping Training Course Information Pack Roger Saunders E: roger@choosingchange.org.uk T: 01373 453 339 W: www.choosingchange.org.uk
What is mystery shopping? Mystery shopping is a method some service-providers use to check the quality of services they provide. Individuals are trained and briefed to approach the organisation with a query or problem, and to keep a detailed record of how they are treated as customers. Although it s called mystery shopping, it is increasingly used in organisations other than retail. Any service involving close inter-action between the provider and customer can benefit from mystery shopping. All good service-providers want to know how well they are treating their customers. Mystery shopping is a way of finding out about the actual experience of the customer, and is objective and measurable. It is particularly effective at investigating how staff inter-act with customers. Of course it is always possible to ask real-life customers about their experience, for example by asking them to complete satisfaction slips or questionnaires. However, the limitation of real-life customer feedback is that many customers can t be bothered to complete a form after their query has been dealt with as customers, we often don t notice or we forget the details of what takes place. Mystery shoppers, on the other hand, approach the service-provider with a definite request or issue, and are primed to observe and note what takes place in an objective and rigorous manner. For example, a tenant acting as a mystery shopper might be asked to phone their landlord to request a particular piece of information. They would monitor how long it took for the call to be answered, how they were greeted, and how professionally their request was handled. The mystery shopper would then note accurately and objectively what took place by completing a standard form. The same process would be conducted by several mystery shoppers over a period of time, and the results could then be collated to form an overall picture of how phonecalls are being handled. 2
What s needed to make mystery shopping work? Although mystery shopping can provide service-providers with valuable insights into their customer service, there are several conditions that are essential to its success: Mystery shoppers must be properly selected, trained and briefed to carry out the role The forms used to record what takes place must be properly designed, so that useful information is captured and is comparable Staff must not be aware, during the inter-action, that they are dealing with a mystery shopper Enough mystery shops need to be carried out using similar scenarios and over a limited period of time, for them to reveal patterns or trends If fictional scenarios are used, they must be able to be resolved in a single inter-action; otherwise the mystery shoppers will waste staff time in dealing with non-existent problems Each scenario can only be used on a limited number of occasions, otherwise it will become obvious to staff that they are mystery shops The information collected by a mystery shopping programme must be properly collated and analysed, and the findings acted upon Mystery shopping should be approached by organisations as a way of continuously improving customer service, and of identifying best practice, rather than as a punitive exercise intended to catch out recalcitrant staff. Individual staff should not be singled out as a result of a mystery shopping exercise. 3
Resident involvement in social landlords services Most social landlords (housing associations and local authorities) recognise that tenants and leaseholders are customers, who have a right to decent services in return for the rents and service charges they pay. Hence many social landlords have developed a range of ways to inform and consult their customers. 1 However, social housing is different than many services in that as a resident, our home is a profoundly personal commodity, essential to our quality of life as customers, we have little or no choice; it s often extremely difficult to go to another provider if we re dissatisfied with an aspect of the service. For this reason social landlords seek ways of involving residents in decisions about services. In this way, they aim to enable residents to participate actively in the way their housing is run, rather than being passive consumers. In recent years, this approach has been strongly encouraged by central government, through its social housing regulator 2. It has introduced what it calls co-regulation. This means that social landlords are now expected to work in partnership with residents to make sure housing services are being delivered appropriately and effectively. As a result many social landlords have established tenant scrutiny panels (or similar bodies) to investigate in depth how services are being delivered, and to recommend and monitor improvements. Tenant scrutiny panels can be particularly effective if they can access independent information and insight into services through, for example, tenant mystery shoppers or tenant inspectors. 1 See Checking the quality of housing services below 2 The government s regulator for all social housing is now the Homes and Communities Agency (HCA) 4
Checking the quality of housing services Besides mystery shopping, there are many other different ways a social landlord can check the quality of housing services they are providing. The following is a summary of useful methods. Each method is useful in particular ways and each has its own limitations. Satisfaction slips / cards Residents are asked to complete and return a standard slip or card immediately after receiving a service (such as a repair), rating their level of satisfaction. Questionnaires Residents are asked to complete a questionnaire, giving feedback about their experience of one or more services. Questionnaires can be administered by post, via newsletters, in housing offices, in person (e.g. door to door), by phone, by email, or from websites. Questionnaires might be intended for every resident, or might be targeted at a sample of residents. They might be sent to residents soon after they have received a particular service, while the experience is still fresh in their minds. Complaints and compliments Landlords might gain some understanding of residents experiences of a service through the complaints and/or compliments received about it. Website The landlord s website (or the website of a body such as a Tenants and Residents Federation) can encourage users to submit comments about their experiences of a service, and to participate in a discussion forum. In-depth interviews While self-completed questionnaires tend to elicit fairly superficial feedback, in-depth one-to-one interviews with a small number of service users are a way of finding out about the quality the felt experience of a service. 5
Focus groups A cross-section of residents who have used a service can be asked to come to a meeting to share and discuss their experiences of it, and to explore ways of improving it for the customer. Customer panels / User groups Whereas focus groups are usually one-off, some landlords ask residents to join ongoing panels to monitor and assess particular services, such as repairs, rent collection or tenancy management. Neighbourhood or Estate Walkabouts By getting out and walking around an area with officers, residents are able to monitor and highlight particular problems and needs in communal areas and streets. Walkabouts might be held on a regular basis or one-off. Standard check-lists can be a helpful accompaniment. Resident inspectors Some landlords train residents to carry out physical inspections for example, of properties intended for re-letting, environmental improvements or repairs. Resident auditors All social landlords collect performance data, measuring services against targets and standards. Some landlords have helped residents form audit panels, offering interpretations and evaluation of the data from a customer perspective. Shadowing In order to get a real feel for what particular staff do and how a particular service operates, residents might shadow them for a period (e.g. a day). Shadowing can offer residents excellent insight into a service, but is unlikely to give a true picture of its usual quality. 6
What do we mean by housing services? Most social landlords provide a wide range of services to their residents and prospective residents, including the following: Allocations and lettings Maintaining a waiting-list Operating a choice-based lettings system Managing empty homes and preparing them for re-letting Managing transfers and mutual exchanges Tenancy management Issuing tenancy and leaseholder agreements Dealing with neighbour disputes and anti-social behaviour Enforcing tenancy agreements when necessary Repairs, maintenance and improvements Carrying out day-to-day repairs, including emergencies Organising some repairs and improvements in a planned way Servicing and checking (e.g. gas) Aids and adaptations Major improvements Rents and service charges Setting and collecting rents and service charges, and providing advice Preventing and managing arrears Neighbourhood and community Managing, maintaining and cleaning the communal areas associated with the housing, both inside and outside blocks Working with other service providers in the neighbourhoods Sheltered and supported housing Housing designed specifically for older people and/or for people needing extra support services Customer support Housing-related advice and support A first point of contact, helping customers access the right services Dealing with complaints and learning from them. 7
What can mystery shoppers test? Mystery shopping can be used to test aspects of any of the housing services (allocations, repairs, tenancy management etc). Elements of the service that are often tested through mystery shopping include: speed of response to phone-calls politeness and courtesy of staff accuracy and usefulness of information given (by phone or in person) queue waiting-times responses to specific types of enquiry comfort, appearance and cleanliness of offices cleanliness of communal areas inside blocks care and maintenance of external grounds. There are 3 main ways that tenant mystery shoppers usually test services: 1. By phone to the main customer contact number 2. By visiting housing offices 3. By inspecting internal and external communal areas. In most cases where mystery shoppers are testing staff responses, they use fictional scenarios. These are made up enquiries or questions, which can be dealt with by staff in one inter-action (that is, the enquiry does not need to be logged in the organisation s system and does not require follow-up). However, some social landlords do ask mystery shoppers to use real life enquiries. The down-side of these is that they cannot be predicted, and it may not be possible to complete enough real-life mystery shops, over a particular period of time, to yield information that is comparable and useful. On the other hand, if mystery shoppers accurately record what happens with real-life enquiries, they can test the providers responses in some depth. 8
Examples of scenarios Scenarios are the fictional enquiries that mystery shoppers make to staff / officers. Ideally they should be completed in one transaction (i.e. not lead to additional, unnecessary work by the member of staff), and should not require the mystery shopper to identify him- or herself. They must also be credible you want the officer to believe the enquiry is for real. Scenario Enquiry about office opening-times Making a complaint on behalf of someone else Reporting a neighbour nuisance problem Requesting a repair that is not a landlord responsibility Applying for a home after a relationship breakdown Requesting a transfer Enquiring about adaptations Verbal enquiry Can you tell me what times your office is open? I m phoning up on behalf of my neighbour, who is disabled. I want to find how I can make a complaint on her behalf? I m phoning because I m sick and tired of all the music my neighbours are blasting at all hours of the night and day. How can you help me? I m phoning on behalf of my Mum, who s a tenant of yours. The toilet seat is broken can you fix it for her? I ve got a joint tenancy with my partner. We ve split up and I need to find out what will happy to our place as a result. I m a housing association tenant living in... I ve been offered a new job in (your area) and want to know if I can transfer to one of your properties. My uncle s a tenant in one of your properties. He s had some health problems, and I was wondering how he d go about getting some adaptations to his flat? 9
Recording information Mystery shopping uses specially designed forms and check-lists on which mystery shoppers must record information about the service they are testing. These forms are needed to encourage mystery shoppers to look out for specific things test things which the housing provider considers particularly important (e.g. aspects of the service for which there are specific standards) ensure the information collected by different mystery shoppers is comparable. For example, a form used for recording what happens during a phonecall might ask: how quickly the call was answered whether the member of staff gave their name how the caller was greeted whether the member of staff was polite and courteous whether the caller was put on hold and, if so, for how long how the caller s request or enquiry was dealt with how the call was ended. An example of a form used for an office visit is given at the end of this pack, as an Appendix. When a mystery shopper is dealing with staff, on the phone or in person, they will not be able to complete the form during the interaction itself. And so it is important that the mystery shopper reads the form in advance and is aware of what to look out for. They should then complete the form immediately after the inter-action, while it is still fresh in their memory. When recording response times (e.g. the speed with which a phone-call is answered), the mystery shopper will need to have a stop-watch or timer to hand. Besides capturing just what happened, some housing providers like to get mystery shoppers ideas about what could be done better. It can be a good idea to leave space, at the end of forms, to enable mystery shoppers to give their ideas and suggestions. 10
Inspections Some resident mystery shoppers carry out physical inspections of communal areas, both inside and outside blocks. These can supplement formal estate inspections, which social landlords might carry out as part of their standard monitoring of contracts / services. Mystery shopping inspections have the advantage of: being random the shoppers inspections are not predictable and so can check cleanliness etc. at different times and places reflecting residents own perspectives and insights the shoppers may pick up things, or evaluate the level of cleaning and maintenance, in ways that are different than staff s. It s usually a good idea for mystery shoppers to inspect neighbourhoods where they don t live, to ensure they bring a fresh perspective. Inspections are often carried out in pairs or small groups. Things that mystery shoppers might look out for during inspections include: cleanliness and maintenance inside blocks of floors, walls, stairs, windows, entrances, lifts evidence of litter and rubbish inside and outside blocks the state of litter-bins whether notice-boards have correct information and are well maintained the condition of external areas for example, evidence of flytipping graffiti grass-cutting weeding of flower-beds and pruning of shrubs, hedges care of trees the condition of play areas and play equipment state of pavements and walkways signage. 11
What s needed from mystery shoppers? As the information given in this pack has demonstrated, mystery shopping is a skilled activity. Mystery shoppers need, in most cases, to be: attentive observant good at listening able to express themselves clearly impartial and objective good at taking notes and recording information accurately well-organised aware of equality issues and respectful of diverse people. If using fictional scenarios, mystery shoppers must be able to stick to the set question or enquiry, and rebuff staff s attempts to elicit other information. For example, if it is necessary for the mystery shopper to remain anonymous, they must be able to say I don t want to give my name. I just want to know if, Most social landlords will ask mystery shoppers to abide by a code of conduct. This will include a requirement that any information gathered by the mystery shopper in the course of their work should be treated as confidential that is, it should only be used for the purposes of the mystery shopping exercise. Housing organisations need to support mystery shoppers by: offering training and on-going support providing the necessary equipment and transport giving a clear point of contact providing good forms and check-lists providing a schedule of mystery shops for each shopper taking account of health and safety issues feeding back to mystery shoppers the results of their work and any service improvements arising from it. 12
Appendix Sample form for mystery shop of a housing office 1. Was the entrance easily accessible? (Please think particularly about how easy it would be for a person with disabilities to enter.) Yes No If you ticked no please explain why you said that: 2. Was the approach to the office and surrounding area of the building clean, tidy and presentable? Yes No If you ticked no please explain why you said that: 3. On entry, was the reception / welcome desk (Please tick one only): 1. Clearly visible / well signed 2. Visible but obscured 3. Difficult to find 4. Not found If you ticked 2, 3, or 4, please say why: 4. Was the reception / welcome desk staffed when you arrived? Yes No 13
5. On arrival, how busy was the office? (Please tick one box only) Quiet, not many other people Average Busy 6. Was there a clear process for queuing or waiting? Yes No If no, please explain why you say this: 7. From the time you arrived, how long did you have to wait until you were seen? Minutes 8. Thinking about the waiting area of the office, was there (please tick all that apply): 1. A seating area 2. Enough seats provided for everyone 3. Leaflets or other information to read 4. Something to keep children occupied 5. None of the above 14
9. When you were greeted by a staff member, which of the following occurred (please tick all that apply): 1. They just looked at me 2. They gave positive eye contact / warm expression 3. There was a comment (e.g. How can I help you? ) 4. None of the above What did they say? Please comment on your impressions based on this greeting: 10. Was this staff member wearing a visible name badge? (Please tick one answer only) 1. Yes, clearly visible 2. Yes, but obscured (e.g. by hair, on jacket on back of chair) 3. No badge seen 11. From the time you entered the building, how long did you have to wait before your enquiry was dealt with? mins. 15