Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

Transcription

1 Summary Report Oct 08 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach Jointly published byconsumer Focus Scotland and the Improvement Service

2 About Consumer Focus Scotland Consumer Focus Scotland started work in October Consumer Focus Scotland was formed through the merger of three organisations the Scottish Consumer Council, energywatch Scotland, and Postwatch Scotland. Consumer Focus Scotland works to secure a fair deal for consumers in both private markets and public services, by promoting fairer markets, greater value for money, and improved customer service. While producers of goods and services are usually well-organised and articulate when protecting their own interests, individual consumers very often are not. The people whose interests we represent are consumers of all kinds: they may be patients, tenants, parents, solicitors clients, public transport users, or shoppers in a supermarket. We have a commitment to work on behalf of vulnerable consumers, particularly in the energy and post sectors, and a duty to work on issues of sustainable development. About the Improvement Service The Improvement Service was started up in 2005 to meet a need for advice, consultancy and programme support to assist in the delivery of the improvement agenda for local government in Scotland. Focused on councils and their partners, we work to support their drive for better accountability, quality and efficiency through learning, sharing and delivering improvement solutions. Statement of Purpose for To support councils and their partners to improve the health, quality of life and opportunities of all people in Scotland through community leadership, strong local governance and the delivery of high quality, efficient local services. About this report This publication summarises and comments on research carried out by Tetlow Associates for the Improvement Service and Scottish Consumer Council (now Consumer Focus Scotland). The full report is published on the Improvement Service and Consumer Focus Scotland websites as Improving the understanding of customer satisfaction and experience in Scottish local government (October 2008).

3 Contents Preface 1 Foreword 2 Acknowledgements 3 Section One: Introduction 4 Section Two: Definitions and background 5 Section Three: Key findings 8 Section Four: Moving forward 12 Section Five: Recommendations 14

4 Preface Most users of local government services are disinterested in local government structures and processes. But what does interest them and matters to them is the responsiveness of their local council to their needs and whether their experience of interacting with their council has been a positive one. Are they treated with courtesy and politeness?, are they listened to?, and above all are they made to feel that they matter as individuals? Councils undertake satisfaction surveys; they are useful but they are limited. Why? Because most public services users don t have a choice as the council is the sole provider. Thus any assessment of satisfaction must be seen and set in this context. But just because there is no choice does not mean there is no customer equation. And it s because councils are in most cases the monopoly provider that it is doubly important that they try to make the user experience as positive as possible. This joint project between Consumer Focus Scotland and the Improvement Service supported by SOLACE will not only deliver a more cost effective and collaborative approach to measuring customer experience and satisfaction it will help in the continuing journey of making councils Customer First organisations. Douglas Sinclair Chair Consumer Focus Scotland 1 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

5 Foreword Local government is undergoing considerable change. There is on-going debate about the role and functions of local government, the way it engages with local communities and its relationship with central government. With the establishment of the Concordat between the Scottish Government and local government, the sector now has a real opportunity to take even greater control of its own development. Councils have the capacity to lead their own improvement by demonstrating ambition, being open to challenge, innovation and scrutiny and by committing to learning and sharing good practice. The major challenge for councils and their partners over this period is fully embedding an outcome focus in their governance, planning, performance management and resourcing processes. Interest is also likely to focus on whether councils and partnerships have the systems, processes and capacity in place both to deliver on outcome commitments and to know they have delivered on them. While a more proportional, streamlined and integrated approach to external scrutiny might reduce the burden for councils and partnerships, there will be higher expectations of rigour and clarity in self-assessment. Customer Relationship Management is a one of a number of critical areas where local government, in recent years, has made significant investments. Understanding and evidencing that these investments are filtering through to affect customer satisfaction and experience levels is of interest to local government and its customers. This study has confirmed widespread agreement that local government in Scotland might benefit from a more collaborative approach to customer satisfaction and experience measurement. The adoption of a number of the report s recommendations should provide an opportunity for local government to make these assessments more easily, more consistently and with a greater degree of rigour. Critically, it should provide an opportunity to underpin other policy initiatives and developments, such as Single Outcome Agreements and selfassessment tools including the Public Sector Improvement Framework (PSIF). As the project moves into its development phase, the clear expectation is that it should be pursued on a collaborative basis between local government and its partners. The Improvement Service welcomes Consumer Focus Scotland s support to date and, importantly, its commitment to continue its involvement moving forward into the development phase. Colin Mair Chief Executive The Improvement Service Summary Report Oct 08 2

6 Acknowledgements The Improvement Service and Consumer Focus Scotland would like to thank: Members of the project Advisory Group: Mary Antoniewiez, Perth and Kinross Council; Paul Davison, Stirling Council; Brian Duncan, East Lothian Council; Peter Fidgitt, Moray Council; Sarah Gadsden, Improvement Service; Jon Harris, COSLA; Graham Hobson, Aberdeenshire Council; Alex Linkston; West Lothian Council; Lesley McGiffen, Audit Scotland; Jim McIvor, West Lothian Council; Louise Scott, Scottish Government; Kathryn Tinney, South Lanarkshire Council. Interviewees and participants at the workshops and seminar who freely gave the information and opinions on which the research was based. Mary Tetlow for carrying out the research on which this summary is based. The project team included staff from both the Improvement Service and Consumer Focus Scotland: Martin Brown (IS); Claire Lightowler (IS) and Jennifer Wallace (Consumer Focus Scotland). This summary was prepared by Jennifer Wallace (Consumer Focus Scotland) and Martin Brown (IS) and is based on the full report written by Mary Tetlow, available from both the Improvement Service and Consumer Focus Scotland websites. 3 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

7 section one Introduction Since devolution there has been an increasing interest in how local government can improve its customer service. There have been a number of important developments, not least the Customer First programme, based at the Improvement Service. Through the work of Customer First and a number of independent reviews and reports, there is an increasing awareness that the current diverse methods and mechanisms for gathering information on consumer satisfaction could be improved. This project explored the need and appetite for a common approach to measuring customer satisfaction with Scottish local government. It follows an Audit Scotland report that concluded that there was significant variation in survey methods, scope and objectives and, critically, in the wide range of satisfaction levels recorded 1. More recently the issue was highlighted by the Independent Review of Inspection, Audit, Regulation and Complaints Handling of Public Services in Scotland (the Crerar Review) 2. Councils themselves asked the Improvement Service to explore this further. Due to a shared interest in improving customer satisfaction and experience, the Scottish Consumer Council (now Consumer Focus Scotland) and the Improvement Service agreed to jointly commission research into this area. 1 Audit Scotland (2005) Improving Customer Service Through Better Customer Contact Edinburgh: Audit Scotland ( customer_services.pdf) 2 Scottish Government (2007) The Independent Review of Inspection, Audit, Regulation and Complaints Handling of Public Services in Scotland Edinburgh: Scottish Government ( Doc/82980/ pdf) The project was carried out against the backdrop of a changing relationship between local and central government in Scotland. The context is a complex one, in which the relationship forged through the Concordat means that local authorities feel justifiably proud of their autonomy. On the other hand, the requirements of Single Outcome Agreements, Best Value and other inspection regimes mean that there are common national challenges within the governance system which make getting better at measuring customer satisfaction and experience a priority for Scottish local government. In order to explore the need and appetite for a common approach, the project took a consultative approach. It aimed to work with local government to understand more about how customer satisfaction and experience are currently measured and what, if anything, could improve practice. The project included: interviews with 31 stakeholders, including representatives from the Scottish Government, scrutiny bodies and local authorities; four workshops with local government staff, including service managers, researchers and senior managers to explore current practice; in total, 149 stakeholders attended these events; and an Emerging Findings seminar bringing together over 50 national and local stakeholders to discuss the findings and potential ways forward. Overall, 29 out of the 32 Scottish local authorities were engaged with the project through one of the above methods. Summary Report Oct 08 4

8 section two Definitions and background 2.1 Who are the customers? This project explores customer satisfaction and experience measurement. By customer we mean someone who experiences a service. This can include indirect customers (such as parents of children at school) and reluctant customers (for example, prisoners). Customers are often also called service users 3. For the purposes of this research we excluded residents and citizens, though we know there are a wide range of surveys into public perception of services. We excluded these groups because the perspectives of those who use services and those who do not can be significantly different. In general, we know that those who use services tend to have a more positive view of them than those who do not 4. Measuring residents views can lead to an unrealistically negative view of the quality of services. If we are trying to understand the customer experience, and use it to develop service improvements, it is important that the right group is heard. 2.2 What do we mean by satisfaction and experience? Satisfaction is a difficult term to work with from a research perspective for a number of reasons. It relates to the evaluation by customers of the quality of the experience of the service. Questions typically use a five-point scale to ask customers whether they are satisfied with the service received. A five-point scale asks customers if they are very satisfied, satisfied, neither satisfied nor dissatisfied, dissatisfied or very dissatisfied. This allows for a balance between positive and negative responses. The main difficulty is that it is inherently subjective and may be related more to expectations than the quality of the service. Some groups in society seem to be much more satisfied with services they receive than others, which is often considered to be due to different expectations 5. Simple satisfaction ratings from a survey can produce frustratingly unhelpful results as it is difficult to interpret what they mean. A person may or may not be satisfied with an experience knowing this fact on its own tells us nothing about what he or she liked or disliked about it or what could be done to make him or her more satisfied. Measuring satisfaction on its own, therefore, does not necessarily lead to better understanding of the customer s experience. Experience, on the other hand, describes the detail of what happened to the customer from their own perspective. Description of the service experience can give useful detail to build up a picture of what works and what doesn t from the customer s perspective. 3 For a discussion of the use of consumer, as opposed to customer see SCC (2004) What s in a Name Glasgow: SCC (available from Consumer Focus Scotland) 4 See for example, Scottish Consumer Council (2006) Building on Success Glasgow: SCC (available from Consumer Focus Scotland) 5 Office of Public Services Reform (2002) Public Service Reform: Measuring and Understanding Customer Satisfaction London: OPSR 5 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

9 Customer experience surveys are often based on the Drivers of Satisfaction, elements of delivery which are known to impact on overall satisfaction. The Customer Service Excellence standard (CSE), successor to the Charter Mark scheme, is based on the following drivers: customer insight, the customer of the organisation, information and access, delivery, timeliness and quality of service 6. For example, surveys on experience would ask questions such as did the staff greet you in a friendly way? or did staff give you the information you needed?. By focusing on detail, the impact of expectations on satisfaction can be reduced. 2.3 Why is measuring customer satisfaction and experience important? Good quality customer satisfaction and experience measurement is important for several reasons: It empowers high quality, responsive and intelligent decision making, eliminating any necessity for guesswork and assumption about likely customer responses. Developing a customer focused culture is good for staff morale and the psychological well-being of the organisation 7, and having good customer satisfaction and experience data is a good place to start. It has the potential to provide assurance to inspectors, which could lead to a lighter touch approach. It facilitates a truly responsive approach at the local level by providing accurate information to frontline staff and managers. 2.4 Why now? During 2008, local government and central government in Scotland embarked on a new relationship. The Concordat is based on mutual respect and partnership, producing amongst other things a performance relationship based on Single Outcome Agreements (SOAs). In reviewing the draft SOA, the project team noted that at present many councils are relying on generalised data (such as the Scottish Household Survey) for Outcome 15: Our public services are high quality, continually improving, efficient and responsive to local people s needs. 8 Few are able to present primary evidence or locally gathered data. Information from national surveys, such as the Scottish Household Survey, is often too general to provide reliable information on customer views on specific services JE Dutton, JM Dukerich: Keeping an eye on the mirror: Image and identity in organizational adaptation. Academy of Management Journal, For more information on SOAs see improvementservice.org.uk/core-programmes/singleoutcome-agreements Summary Report Oct 08 6

10 At the same time, two further developments impact on the measurement of customer satisfaction and experience: The Accounts Commission is developing its approach for the next round of Best Value audits. These are likely to place considerable emphasis on how councils are capturing and making use of the views of citizens and customers. There is also likely to be interest in the extent to which councils achieve value for money in understanding their customers. The Independent Review of Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland (the Crerar Review, 2007) emphasised the need for more effective public participation in scrutiny and the importance of service providers responsibility for gathering data on users views 9. Given these developments it is timely to consider developing collaboration and a common approach to customer satisfaction and experience measurement. 9 Scottish Government (2007) The Independent Review of Inspection, Audit, Regulation and Complaints Handling of Public Services in Scotland Edinburgh: Scottish Government ( Doc/82980/ pdf) 7 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

11 section three Key findings 3.1 What is happening now? Representatives from local authorities were asked, at the workshops and in interviews, to describe how they currently measure customer satisfaction and experience. From their responses there is clearly a great deal going on across Scotland, including large authority-wide or neighbourhood surveys, Customer Relationship Management systems, citizens panels, mystery shopping, and many different kinds of small-scale surveys and qualitative exercises. While these tools gather useful information on satisfaction, they did not appear in general to gather the type of detail that allows authorities to build up a picture of the customer experience. There was a general sense that there is insufficient co-ordination and poor sharing of knowledge within authorities. One participant summed up the current approach as one of chaos, duplication and lack of strategy another noted that getting a corporate approach to gauging customer/user views can be difficult. Aligned to this was a concern that services and departments were starting from scratch for each customer satisfaction survey rather than building on previous work and good practice. In some authorities there is a perception that managers think they already know what is best for customers and do not feel the need to commission or conduct research, in others there appears to be an over-reliance on informal face-to-face customer feedback. There were concerns raised about staff capacity, particularly in relation to technical issues such as segmentation. Segmentation is when the responses are explored to see whether one group of customers experience services significantly differently than others (for example, comparing responses from disabled and non-disabled people). It can be of particular use in exploring whether or not authorities are meeting equality duties. At present it does not appear to be widely used by authorities. 3.2 What are the problems with the current approach? The current situation is varied, with much effort and good practice in evidence but many disadvantages. Current problems include: an inconsistency of practice, meaning that different surveys can t be compared, between services and departments in authorities and between authorities; a lack of expertise, particularly in relation to both survey design and analysis; a tendency not to use data to develop service improvements. One workshop participant commented [We] don t always do anything with the results ; a perception of high costs, though the cost of current practice is very difficult to establish it is almost certainly high; and feedback to customers is often inadequate, contributing to survey fatigue. Summary Report Oct 08 8

12 3.3 Is there an appetite for co-operation or a common approach? In general there is a very strong appetite for both greater co-operation between authorities and a common approach to measuring customer satisfaction and experience. In part, this is based on a desire to improve internal practice on measuring customer satisfaction and experience. Stakeholders from the local government community felt that improving practice would lead to: the collection of information in a more timely way; the development of outcome-focused performance measurements rather than process focused measurements. This would help to put the customer at the centre; and improved evidence-based policy but providing data on what works and what doesn t from the customer s perspective. The desire to collaborate is based on the widespread perception that money is being wasted on inefficient approaches to customer satisfaction and experience measurement. It is unlikely that many Chief Executives or elected members are aware of the total their authority spends trying to understand customers - and finding out, for most, would be difficult. This is clearly not a satisfactory situation. Collaboration was seen as an efficient mechanism for improving internal practice. The desire for a common approach builds on these arguments but goes a step further to ensure the best possible value from the data collected. A common approach would consist of councils using the same methodologies and research tools. Using a common approach would allow for: improved sharing of internal data, between services and departments to build a stronger picture of the customer experience; the development of trend data to monitor improvements over time; the development of external benchmarking to allow local authorities to compare their performance; the development of a Scotland-wide picture of customer responses to facilitate national policy decisions; and the provision of reliable data to inspectorates to reduce the burdens created by inspection. However, some participants raised concerns about a common approach. Many of these centred around the possibility of developing a one size fits all approach where the needs of individual services and localities are inadequately understood and reflected. Local authorities wanted to ensure that local flexibility was retained in any common system. Some, but by no means all, local authority staff raised concerns about the negative aspects of benchmarking, such as league tables. 3.4 What would greater co-operation and a common approach consist of? Councils expressed a desire for a range of forms of support, ranging from cooperation to a common approach. Each of the mechanisms below were seen as potentially improving practice within local authorities. 9 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

13 Potential routes to greater co-operation included: 1. Good practice toolkits. Mirroring concerns raised about the capacity within local authorities to carry out robust customer surveys, participants were keen to see good practice toolkits developed. These would include advice on gathering and analysing data. This would aim to support staff and improve capacity within individual local authorities. 2. Workshops and training support. Many participants at the workshops for this project spoke of the benefit in bringing together staff from across Scotland to share experience of measuring customer satisfaction and experience. They welcomed the suggestion of more workshops to network and training sessions to develop practice. 3. Developing shared capacity for specialist technical advice. This would allow local authorities to access technical advice in an efficient manner. Such advice might include how to choose a cost-effective sample size, advice on commissioning research or how to use segmentation accurately. Potential elements of a common approach include: 1. A question bank of tested questions. The research uncovered concerns about the quality of satisfaction surveys currently being conducted at local level. A question bank of tested survey questions would allow local government to select questions that have been developed by specialists and tested with customers, reducing the burden of developing surveys from scratch. This tool could also assist local authorities in improving their understanding of how to measure customer experience, rather than limiting surveys to basic satisfaction questions A data standard for keeping information in a common format. At present, the different ways of collecting data about customers mean that it can be difficult to compare both within local authorities and between them. This limits the effectiveness of data. A data standard would allow local authorities to use the same categories for recording demographic information 11. Not all categories would be necessary for all surveys but using a common format allows data to be compared between surveys, either within individual local authorities or across different authorities A data warehouse of survey information. A data warehouse would store anonymous, locally gathered information in a central location. This would allow local authorities to view and share information gathered in other areas, encouraging the sharing of good practice and facilitating internal benchmarking. 10 For further information on question banks used in Canada see Schmidt, F. and Strickland, T. (1998) Client Satisfaction Surveying: Common Measurements Tool Canada: Canadian Centre for Management Development 11 For example, geographical location, date of birth, gender and ethnic group, disability, rural/urban location, carer status, employment status, income, sexual preference and religion 12 For further information on the development of a data standard in England see Tetlow, M, Taylor, R. and Salter, M. (2006) Developing Measures of Satisfaction for Local Government Services London: NCC/Local Government Association Summary Report Oct 08 10

14

15 section four Moving forward It is clear that there is widespread agreement that local authorities in Scotland could benefit from working together on customer satisfaction and experience measurement. The Improvement Service and Consumer Focus Scotland are aware that any development project must work with local government if it is to succeed. In order to do so, the development project must have the following characteristics. 4.1 Voluntary Feedback from workshops, the Emerging Findings seminar and interviews suggested that a development project would need to balance flexibility at a local level with the desire to co-operate and use common approaches. Services and tools need to be relevant to specific localities and for specific services meaning that a one size fits all approach is unlikely to work. A voluntary approach would support the spirit of the Concordat by ensuring that councils participate in any common scheme because they wish to rather than because it is required. 4.2 Owned by local government Local authorities themselves must have ownership of the implementation project. This ownership needs to include a wide spectrum of the local government community. Senior management and elected members need to be engaged in seeking the greatest benefit from customer satisfaction and experience measurement alongside research and customer service teams. In addition to local government ownership, there is benefit in having partners from outwith the local government community to ensure that the process and outputs have external credibility. Consumer Focus Scotland is committed to continuing its involvement with this project as it moves into the development phase. 4.3 Embedded While local authorities support the development of specialist advice and other services, there is a danger that a standalone service will be seen as an additional burden rather than a method to reduce burdens. In order to minimise this risk, the project must be embedded in performance management and scrutiny arrangements. Internally, local authorities should seek to use the data gathered from customer satisfaction and experience surveys to monitor performance. Data should be presented to elected members to assist them in decision making. Externally, the support of the Scottish Government and scrutiny bodies is required to ensure their systems support and encourage the use of a common approach to customer experience and satisfaction. Summary Report Oct 08 12

16 4.4 Efficient There is currently little information on the overall costs of customer research and many local authorities felt that this would be very difficult to establish. Any implementation project needs to ensure that greater co-operation and a common approach represent better value for money. It needs to establish approaches which are demonstrably better value for money than the status quo. 4.5 Resourced It needs adequate resources to make a difference. While there are clear opportunities for making savings in the medium term, the project will require funding at an adequate level. Resources should be focused on developing tools and support services for local authorities and ensuring the project is linked to other policy developments, such as systems for external scrutiny. 4.6 Evaluated The project itself needs clear objectives and to be rigorously evaluated. There are concerns that without well-structured objectives and clear evaluation criteria the project will lose momentum. 13 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

17 section five Recommendations 1. Locating the Project s Delivery That the location for the project s governance arrangements should rest within the Improvement Service s Customer First Programme and, specifically, within the National Customer Relationship Management (CRM) Programme, a partnership between the Scottish Government and Scottish local government. The rationale is three-fold: firstly, as a direct response to the need for local government to have ownership of the project if it is to succeed; secondly, past evidence suggests that stand-alone projects tend to have a higher incidence of failure and a more holistic and a systematic approach is now necessary; and thirdly, and critically, local government has both an interest and a duty to examine and evidence that their significant investments recently in Customer Relationship Management generally and CRM systems specifically (to manage and track customers service requests, information requests and entitlements) are making a difference to customer satisfaction and experience. To strengthen further local government ownership, it is recommended that the endorsement of both CoSLA and SOLACE (Scotland) should be sought on the preferred location of the delivery phase. 2. Resourcing and Funding the Project s Delivery For the project to be taken forward in a way that will result in a difference, it is an imperative that the project should be properly resourced, funded and empowered. As well as dedicated project resources, the project should seek to mobilise a range of analytical support and expertise, including those within the Scottish Government s Analytical Services Unit, Audit Scotland, Consumer Focus Scotland and others, to allow much better sharing of practice, opportunities and capacity. 3. Project Scheduling That the project s delivery phase, involving a development group of willing councils and other partners, should be initiated immediately to begin the development of: a Question Bank of tested questions; a Data Standard for holding information in a common format and to increase the effectiveness of data; and Good Practice Toolkits for practitioners including a standard approach to segmentation and a common approach to qualitative methodologies. A collaborative approach involving local government and other partners, at minimum, for developing these core deliverables should not only be useful in its own right, it should offer both value for money and a foundation for subsequent collaborative work. Summary Report Oct 08 14

18 4. Aligning with Other Policy Initiatives & Developments The project s resources should rightly be focused on developing the necessary toolkits and support services for local government. The delivery phase, however, should be mindful to ensure that the project s deliverables are aligned closely to, can support and underpin other policy initiatives and developments including, for example, Single Outcome Agreements and self-assessment tools including the Public Sector Improvement Framework (PSIF) and other EFQM-based approaches. In doing so, this should allow for the application of the outputs to be embedded within internal and external scrutiny arrangements. 5. Evaluating Outputs That, with the support of a group of willing councils and involving other key partners, the project s deliverables should be fully tested and rigorously evaluated against a clearly defined set of objectives and criteria. This will serve the purposes of demonstrating to local government and others that: the outputs can be replicated to meet the needs of the wider local authority community as a whole; differences in councils size, operating contexts, management and organisational structures have been respected; and the benefits of applying the project outputs locally. 15 Improving the understanding of customer satisfaction and experience in Scottish local government: Towards a collaborative and common approach

19 Consumer Focus Scotland Royal Exchange House 100 Queen Street Glasgow G1 3DN Tel Fax improvement Service Westerton House East Mains Industrial Estate Broxburn EH52 5AU Tel October 2008 ISBN kelso graphics colour printing