Evidence for Success. The guide to getting evidence and using it August 2014

Size: px
Start display at page:

Download "Evidence for Success. The guide to getting evidence and using it August 2014"

Transcription

1 Evidence for Success The guide to getting evidence and using it August 2014

2 Contents: Foreword 03 Introduction 04 Part 1: Generating useful evidence 06 Part 2: Using evidence 16 Section 1: Using evidence 18 to influence internal policy and practice Section 2: Using evidence 22 to influence external policy and practice Section 3: Using evidence 34 to influence funding and commissioning decisions Section 4: 10 Top tips to 39 use evidence for success Part 3: Additional resources 45 Resources for Part 1 46 Resources for Part 2 47 Useful websites 48 Acknowledgements 51 Glossary of terms 52 References 53

3 Foreword The Knowledge Translation Network (KTN) was established in 2012 in response to an identified need to support the third sector to maximise the impact of the knowledge and evidence they generate by ensuring it reaches Scotland s service providers and decision-makers. Running parallel to the Scottish Third Sector Research Forum, the KTN has two key aims: (i) to share learning about effective knowledge translations and (ii) to promote the use of evidence in decision-making. This guide responds to a demand amongst third sector organisations for guidance on how to use evidence to influence policy and practice. In summer 2013, the KTN conducted an online survey amongst a range of organisations across Scotland to better understand how they currently generate and use evidence. Of the 67 organisations who responded to the survey, 53 noted that they need more support with using evidence to influence policy and practice, and highlighted that having an easy-to-use guide that identifies and supports organisations through the key steps of doing so would be of value to the sector. The learning in this guide is based on the findings from this survey and from interviews that were conducted with 25 of the key stakeholders that are currently involved in this area of work in Scotland, including third sector service providers, funders, policy makers and researchers. A full list of these organisations has been included in the Acknowledgements section of this Guide. This is the first in a series of resources that will be produced by the KTN going forward and aims to provide organisations with the knowledge, skills and confidence to maximise the impact that evidence can have on policy and practice. We would very much welcome your views on both this guide and on other resources that would be useful for the sector. Joanna McLaughlin Patty Lozano-Casal Mark Meiklejohn Cath Logan Andrew Paterson Kirsten Thomlinson Evidence for Success The guide to getting evidence and using it 3

4 Introduction Evidence has become a valuable currency. Karen Indoo, Barnardo s Scotland Evidence can be a powerful tool for influencing change and is one of the key features of effective policy-making 1. The Oxford English Dictionary defines evidence as the available body of facts or information indicating whether a belief or proposition is true or valid. This term can therefore cover a wide spectrum, from the results of randomised control trials to qualitative feedback from service-users. What counts as useful evidence depends on its purpose; the question the evidence is trying to answer; how it is going to be used, and in what environment (Nutley, 2013). Good evidence is useful evidence! It is information that can help a charity make better decisions, provide better services and raise standards. Dr Jonathan Sharples, University of York In a time of reduced public sector budgets and increased demand for services, there is a growing need to ensure that an evidencebased approach is taken to developing policy and practice. The third sector plays a key role in delivering services that meet the needs of vulnerable communities and can make a valuable contribution to developing the evidence-base about what works and why when working with these groups. Therefore, it is vital that these organisations have the knowledge, skills and resources they need to generate useful evidence about the work they do and use this evidence to inform internal and external policy and practice. 1 The Cabinet Office identify nine key features of effective policy-making, which include being: (i) forward looking; (ii) outward looking; (iii) innovative and creative; (iv) evidence-based; (v) inclusive; (vi) joined up; (vii) regularly reviewed; (viii) regularly evaluated, and (ix) willing to learn from experiences of what works and what does not. (Strategic Policy Making Team, Cabinet Office) 4 Evidence for Success The guide to getting evidence and using it

5 Aims of the Guide This guide aims to provide easy to follow, step-by-step guidance and resources to support organisations to use evidence to influence policy and practice. The guide is split into three parts: Part 1 outlines how you can generate evidence that is relevant, robust and persuasive to the stakeholders that you are trying to influence. Although this guide is primarily about using evidence, this part of the guide helps to ensure you have the quality of evidence you need to maximise your chances of influencing policy and practice. Part 2 explores how this evidence can then be used to influence (i) internal policy and practice; (ii) external policy and practice, and (iii) future funding and commissioning decisions. It concludes by providing 10 top tips for using evidence for success. Who is this Guide for? This guide is for anyone who wants to use evidence to improve policy and practice, regardless of the level of experience they have in doing so. Therefore, it is intended that this guide will also be of value to a wide range of stakeholders including: practitioners service managers funders and commissioners policy makers and planners. Most importantly, the guide is aimed at learning organisations who are committed to both sharing their evidence about what works, what doesn t work and why, and to amending their own practices on the basis of this evidence. Part 3 identifies a range of resources that you can access to help you achieve your goals. How you choose to use this guide will depend on what you want to use evidence to achieve and the stage you are in this process. While the guide can be read as a whole, each of the sections can also be read independently. At the end of each section there is also a tailored checklist that you can print out and consult. This guide also touches on the important area of evaluation. There is already a range of resources dedicated to these topics so we have focussed our advice specifically on how evaluation relates to the use of evidence. If you feel you would benefit from additional support around evaluation, please see the Resources section of this guide or contact Evaluation Support Scotland. Evidence for Success The guide to getting evidence and using it 5

6 Part 1: Generating useful evidence This part of the guide explores how you can generate evidence that is relevant, robust and persuasive to the stakeholders that you are trying to influence. By the end of this part, you will know how to: Gather data about your service Analyse the data you have collected Gather secondary data and evidence about your area of work

7 Most organisations have a wealth of data about what they do and who they work with. However, gathering that data and turning it into the type of evidence that is persuasive to policy-makers, funders and practitioners can sometimes be challenging. It is important to think about the type and quality of evidence you ll need to influence policy and practice from an early stage and put a plan together to allow you to gather this. To have an impact on policy and practice you have to start from the theory of change; say what you re trying to achieve, and leave strategic and systemic change for the end. You must start from the point where you gather the evidence and ask yourself, what evidence do I need to gather now to answer that final question and what is next? If you do not have that from the onset the whole thing falters. Jennifer Wallace, Carnegie UK Trust To maximise your chances of influencing policy and practice, it is important that this evidence: is robust, relevant and solves a problem uses an appropriate balance of quantitative and qualitative data draws from a wide range of available data is up to date, timely and makes use of current data demonstrates the efficacy of your approach is clear, reasonable and doesn t overclaim is honest about its limitations. Step 1: Gather data about your work The last 10 years has seen a growth in self-evaluation amongst third sector organisations. Many third sector organisations are now routinely gathering data about their outcomes so they can measure, understand and demonstrate their impact. Steven Marwick, Evaluation Support Scotland In the KTN s survey, 48 out of 57 organisations (82%) highlighted that they face challenges when gathering data and evidence about their service. The most common reasons cited for this included: Staff not having enough time/ capacity to do so; Service users/ stakeholders not having enough time to provide this data, and Gathering data not being seen as a priority by the organisation or its staff. As this is not a guide to evaluation, we can t address all of these challenges here but we have included a list of resources where you can access support with these issues in Part 3. However, despite these challenges, it is important to remember that organisations can and do gather data and evidence about their work and use it to influence policy and practice. If we are going to interfere in peoples lives through new services, we need a clear evidence base to show that it will be to their benefit. Producing a statement of challenges or needs is not a reason to take any particular action; we need a good explanation of how the proposed approach will work including the delivery mechanism and practical steps leading to change and improvement. Also, it needs to take account of local factors as all change is local. Geoff Huggins, Scottish Government Evidence for Success The guide to getting evidence and using it 7

8 How many people visited the maze last month? 30,576 Fantastic result! Ask the right questions Identifying what data you need to collect Before collecting data about your service, make sure you are clear about the questions you are trying to answer and what data you will need to collect to allow you to do so. Thinking about this at an early stage will help to ensure that the process is as easy as possible and that you are able to collect the data you need. Begin by considering what situation your service aims to address and then identify the inputs, outputs and outcomes that will be required to achieve the desired change. Using a planning tool, such as a logic model, can provide a useful way to work through each of these stages and can also help you to understand if and how each of these stages link together. As an organisation we have a strong theory of change and this provides a really beneficial framework for examining the evidence needs for each of our programmes in more depth. As a result, we are able to focus on a number of key questions and provide learning that is specifically focused on these areas rather than undertaking a wide variety of general research. Neil Mathers, Save the Children Once you have your plan, it will be easier to identify the types of data that you ll need to gather to understand and demonstrate if the service is achieving its desired outcome. For each input, output and outcome you have identified in your planning tool, ask yourself how you will know these have taken place and what data you will need to collect to evidence this. These are called your indicators. By approaching data collection in this way, you will also ensure that all the data you collect (i.e. primary data) will be used to answer specific questions, and that you are not wasting time and resources gathering data that has no useful purpose. 8 Evidence for Success The guide to getting evidence and using it

9 Make sure you collect data about your outcomes as well as your inputs and activities. For example, if you want to improve the lives of young people with multiple support needs, it is insufficient to only record the number of young people who engage with your service. Instead, you should aim to collect data on some or all of the following: the number of young people who engage with the service; the support needs of each young person; the activities your service undertook to address these needs; how much they cost; any outcomes achieved as a result of these activities, and how you know these were achieved. Outputs are great - to have reams and reams of evidence on what somebody s done - but what we re ultimately interested in is how effective these outputs have been. You need to be able to evidence that and use your data to demonstrate the benefit that your service has had. That s the sort of thing we re looking for. David Berry, Scottish Government The best reports are clear and concise and have a good mix of qualitative and quantitative data. A real mix of the hard evidence with case studies but all focused on the impact that you ve had. Kevin Geddes, Health and Social Care Alliance Scotland (The ALLIANCE) Gathering this data can have implications on your time and budget, so it is important that you plan ahead and allocate sufficient resources. Think about the range of methods you will need to use to collect this data including surveys or face-to-face feedback. If you are undertaking this work internally, you may want to consider whether your organisation needs any training to do so. Most organisations who responded to the KTN survey reported gathering a mixture of both quantitative data and qualitative data: Quantitative data tells us about what happened, where and when, and to whom. It can include information about inputs (e.g. the cost of your service), activities (e.g. numbers of training courses delivered) or outcomes (e.g. the number of people who stop smoking). Qualitative data helps us to identify the factors or reasons affecting behaviour the how and why. The term qualitative is also used to describe information relating to soft outcomes, such as increased confidence. Evidence for Success The guide to getting evidence and using it 9

10 ... and every time I dig 10 metres, someone wins the lottery! Make sure you can claim attribution Step 2: Analyse the data you have collected In order to understand what your data means you need to analyse and interpret it. It may be helpful to go back to the plan you created at the start of your project and check whether the data you have collected indicates that your service s outputs and outcomes have taken place as planned. Regardless of whether they have or they have not, the next step is to understand why this is the case. Analysis is a vital stage in helping your organisation to understand which parts of your service have worked, which parts have not worked and most importantly, why. Identifying the internal and external factors that may have impacted on your outcomes While your planning tool will help you to understand how your service s inputs and outputs link to your intended outcomes, it is important to realise that there may also be inputs and outputs outwith your service that have impacted on these outcomes. If I m looking for an evidence base to undertake a type of programme or an activity, I will want to understand the model for change that shows the relation between the inputs and the outcome. There will be other co-occurring factors in play in any test of change and its important to be able to demonstrate that we are seeing causality not just correlation. Geoff Huggins, Scottish Government While it may not be possible to control or isolate these external factors, before attributing a specific outcome to your service a good evaluation should seek to (i) identify any relevant external factors and (ii) consider what impact they may have had on your outcomes. Asking a critical friend to look at your findings can help with this process. Outside of a science laboratory, it is very rare that anything ever happens in a vacuum. Therefore, when analysing your data, it is critical to consider what factors might have impacted on your outcomes. 10 Evidence for Success The guide to getting evidence and using it

11 Getting external help to analyse data In the KTN survey, the majority of organisations stated that they feel confident in analysing the data that they collect but 17 out of 54 organisations (33%) highlighted that they can find this process challenging. Key reasons cited for this include: time constraints lack of skills in this area lack of support and advice about how to analyse data. 30 out of 49 organisations (61%) also stated that they don t believe the evidence from self-evaluations are seen as robust or as persuasive to funders and policy-makers as evidence from external evaluations. However, discussions with funders and policy-makers reveal that, provided they are properly undertaken, evidence from self-evaluations is seen to be as valuable and persuasive as evidence from external evaluations. Have a little bit of faith in the work you re producing and the work you re doing. I completely understand that people want that external validation but actually if you re doing good evaluation, using solid tools, and are able to justify the results that you get, then you should also be confident about the impact that you re making. Kevin Geddes, The ALLIANCE There s an element of training which is required for this - it is a specialised task. There are a lot of academics who have spent years training in evaluation to understand what the evidence says. Yet we re expecting people to do it. They either need to buy expertise or you need to give them a lot of training. Catherine Bisset, Scottish Government To help you with this, you may want to use an external agency to gather data on a specific project you have undertaken or to help you analyse the data you have gathered. Another option might be to consider whether an intern could be used to undertake this piece of work. However, as noted above, evidence from external evaluations is not necessarily any more robust or persuasive than evidence from internal evaluations so if you decide to commission an organisation to undertake this work on your behalf, it is important that you fully research them first and are confident that they have the necessary skills and experience to do so. It is also important that you identify the specific research questions that you want the evaluation to address and communicate these to the organisation you commission to undertake this work. You may find it helpful to go to ESS s webpages on self-evaluation and online course on how to get the best from external evaluation. While conducting an evaluation requires a certain level of time, skills and resources, any third sector organisation can do selfevaluation well with some support. However, generating evidence from evaluations that you can use to influence policy and practice can sometimes be more difficult. Evidence for Success The guide to getting evidence and using it 11

12 External evaluations can also be expensive so it is important you plan for this and where appropriate, speak to your funders about building these costs into your service s budget. Like you, funders want to know the difference you are making as a result of your work. Supporting you to collect relevant information to evidence that difference is important to them. Depending on your capacity, you may also want to consider developing relationships with academics and working in partnership with them to analyse your data. Universities get assessed every 5 years or so and part of this assessment now looks at the wider impact that university research is making, which means that there s a real incentive for researchers to work with external organisations and individuals, including in the third sector. In this sense, it s a good time for people working in the third sector who are seeking to draw on academic support in gathering or analysing evidence - if you can find someone in a local university who has an interest in the issue your organisation is concerned with then they may well be willing to help. Dr Katherine Smith, University of Edinburgh Step 3: Gather secondary data/ evidence about your area of work Secondary data is any data that has been generated by someone other than yourself or your organisation. Most organisations will struggle to generate enough data themselves to draw robust conclusions about the impact of their work. It is therefore important that you consider how your findings relate and compare to any other relevant data/ evidence. If your evidence is supported by a wider body of evidence it will also carry more weight when you come to use it, and will be more likely to help you to influence policy and practice. Clearly, if you can link your individual findings to what others have found there ll be more weight behind them and, as you ll be talking about more than one study, they will be more difficult for people to dismiss. It s about showing how your findings fit with an existing body of work. Prof Sandra Nutley, University of St Andrews Even where your organisation generates its own evidence, you might find it helpful to consider evidence that has been produced by other organisations if you want to maximise your chances of influencing policy and practice. By gathering secondary data you will keep up to date with new statistics or studies about your area of work, learn what other organisations have found to work, and gain a better understanding of the context in which your own work is taking place. Depending on the policy area and the context in which you are working you might find it helpful to gather secondary data produced in other countries other than Scotland and the rest of the UK. 12 Evidence for Success The guide to getting evidence and using it

13 Many organisations find accessing secondary data challenging. While, on some occasions, this may be because the type of data they are looking for either does not exist or has not been made available to the public, the main reasons organisations gave for this in the KTN survey were: They don t know where to look; Gathering a wide range of data can be expensive and time-consuming, and It is difficult to tell which secondary data is the most reliable/ appropriate. It depends on the purpose of the research as to what data we ll use. We make use of a range of primary research methods, including surveys, interviews and focus group type discussions, to gather evidence directly from practitioners about what works. We also use evidence from elsewhere (e.g. evaluations that have been published, inspection reports, literature reviews and other pieces of research) to inform briefing papers and consultation responses. Laura Mulcahy, Criminal Justice Voluntary Sector Forum Accessing secondary data There are a wide range of sources of secondary data including statistical bulletins, academic studies, online journals, project evaluations and reports published by third sector and public sector organisations. While you may need a subscription or membership to access certain sources of secondary data (e.g. academic journals) the vast majority of sources can be accessed for free. Other third sector organisations can be a key source of secondary data and you shouldn t underestimate the value of the evidence that you can get from your peers, or the impact that this evidence can have when used collaboratively. While the best resources to access will depend on your area of work and the type of data you are looking for, a few websites in the Resources section might help. Don t think you have to go to a library and search the evidence base yourself because that s a big job to do. Try and find other people who ve done reviews in the area you re interested in and use intermediaries like our organisation. Dr Sarah Morton, Centre for Research on Families and Relationships Evidence for Success The guide to getting evidence and using it 13

14 Drawing data from a wide range of sources While it can sometimes be tempting to only focus on data that supports your own findings, try to draw secondary data from as wide a range of sources as possible. This will help to increase your knowledge of your area of interest as well as enabling you to make better judgements about whether certain data is reliable or not. For example, it will help you to identify where one source of data is very different from other data you have read and consider why this might be the case. Secondary data is crucial and I suppose you can say it s a gut judgement on what s reliable. If you re trying to prove that a particular area is deprived, national statistics like census statistics are wonderful because they re done by the chief statistician, making them very, very hard to refute. David Griffiths, Ecas It is also important to ensure that the secondary data you are accessing is as up to date as possible so you are not basing decisions on data that is now out of date. Accordingly, it is important that your organisation allocates sufficient time to reading and keeping up to data with relevant secondary data that is being published. work closely with Community Planning Partnerships and other stakeholders over the next 3 years to find, create, evaluate and communicate the evidence of what works in delivering the Scottish model of public service delivery. For more information visit: What Works Centres are quite a new initiative, even in England. I think that there are probably pockets of information around that need to be addressed. In England they ve probably pulled together of all of the evidence produced by different sectors into a very clear place for people to go to for evidence, the what works centres. It will be good to have that in Scotland too. Carolyn Sawers, Big Lottery Fund Scotland A useful way of both identifying other sources of secondary data and determining how reliable and relevant they are to your needs is to engage with other stakeholders and discuss as a group. Networks and forums can offer a valuable space for discussions and provide an opportunity for organisations to discuss both primary and secondary data in an open and collaborative environment. The Economic and Social Research Council (ESRC) funded What Works Centres are one of the key places to go for secondary evidence, and one of their core functions is to produce and apply a common currency for comparing the effectiveness of different interventions. A number of centres have been setup across the UK as part of this programme of work including What Works Scotland, which was launched in June Led by the Universities of Glasgow and Edinburgh, What Works Scotland will 14 Evidence for Success The guide to getting evidence and using it

15 Checklist Here is a quick checklist for you to consider when you want to generate useful evidence that will be persuasive to policy makers and practitioners: Identify the question you want to use your evidence to answer. Create a plan that outlines your service s inputs, outputs and outcomes. Consider how you will measure outcomes and identify the data that you ll collect at each stage to understand if these are taking place as planned. Analyse your data and identify any trends/ patterns. Consider whether any external factors may have impacted on your outcomes. Consider how your evidence links into other evidence people have produced. Draw secondary data from a wide evidence base. Look at a number of robust studies, which have found very strong and consistent results. Consider all evidence not only those findings which support your argument. Engage with other stakeholders and discuss your findings. Evidence for Success The guide to getting evidence and using it 15

16 Part 2: How to use evidence to influence policy and practice This part of the guide combines advice from third sector service providers, policy-makers, academics, funders and commissioners about how to use the evidence you have generated to influence policy and practice. It is split into four sections: Section 1: Using evidence to influence internal policy and practice Section 2: Using evidence to influence external policy and practice Section 3: Using evidence to influence funding and commissioning decisions Section 4: 10 tops tips for using evidence for success

17 53 out of 54 organisations (98%) who responded to the KTN survey stated that policy and practice should be based on evidence to ensure that the best use of available resources is made and that future service provision is informed by learning from what works, for whom and why. The organisations reported using evidence in a number of ways, including: (i) to improve their own services; (ii) to influence future decisions, and (iii) to inform future policy and practice within the area they work. The successful implementation of an outcome focussed approach to public services needs good data and information to improve decision making, reduce the likelihood of wasteful expenditure and improve the evaluation and learning of which approaches are effective. SOLACE Scotland The sets of steps needed to achieve these goals are outlined in the first three sections of this part of the guide. The final section offers some final tips from stakeholders who have had success in this area. Funders Policy makers Time to share my evidence Public Sharing evidence Evidence for Success The guide to getting evidence and using it 17

18 Section 1: How to use evidence to influence internal policy and practice This section of the guide explores how to use evidence to influence policy and practice within your own organisation. By the end of this section, you will know how to: Identify the problem you want to address within your organisation and how evidence can help you to do so Identify and communicate with your internal stakeholders Understand when there is a need to try something new and innovative

19 In the KTN survey, one of the most common reasons organisations gave for using evidence was to better understand the impact that their services are having so they can ensure they are of maximum benefit to their service users. This section explores how you can use evidence to influence your internal stakeholders and improve policy and practice within your own organisation. Step 1: Identify the problem you want to address In order to use evidence to improve internal policy and practice, you first have to identify the problem that you want to address. For example, does your service have a low engagement or high drop-out rate? Are some service users achieving better outcomes than others? Is your organisation making the impact that it wants to make in your local community? It works best when it starts from a practical need, when it is led by an issue that people want something to be done about and not done for the sake of it, like they are told to. It works with the practical issues that people are faced with. There are a lot of options and evidence can be used in a number of things. Therefore, it can be a really useful thing to inform decision making, in any organisation. Dr Jonathan Sharples, University of York Once you have identified this problem you then need to ask yourself: 1. Does my evidence indicate how this problem can be addressed? 2. Is there enough capacity within the organisation to implement these changes? 3. Do others within my team have the right skills to apply the evidence into practice? 4. Can I communicate this evidence to other people within my organisation in a way that is relevant and persuasive? 5. Have I made clear what changes are needed within my organisation based on this evidence and made a convincing case for doing so? 6. Are there tools in place to assess and evaluate the amended service? It is also important to remember that adapting your service based on what evidence suggests will work will not automatically give you improved outcomes. Other internal and external factors can influence whether or not the service is successful, such as the environment in which the service is being embedded. Before you use evidence to adapt your services, it is therefore important to think about the challenges you might face and discuss these with your colleagues. Evidence for Success The guide to getting evidence and using it 19

20 Step 2: Communicate your evidence to internal stakeholders If you want to influence change within your organisation it is vital that you are able to identify those internal stakeholders who have the power to influence or implement this change and communicate your evidence in a way that is relevant to them. These stakeholders are likely to include your service users; other members of staff and volunteers; your senior management team, and your trustees. It can sometimes be difficult for staff who work directly with service users - and see the progress they are making first-hand - to look objectively at their service and question whether it can be improved. Similarly, management staff, who are a step removed from the delivery side of the service, may find it difficult to understand why changes are needed if the service is generally producing positive outcomes. Collecting and using evidence about what is working, what is not working and why can help you to address both of these challenges. If you re involved with running a service, you re there on the ground and are immersed in what the service does and how it does it. However, we also need to get our colleagues across the board to see what our service providers are seeing. To do this we have to support them to gather information in a formalised way and help staff to understand the benefits of doing so. Alison McIntyre, Barnardo s Scotland Step 3: Identify when you need to try something new In the KTN s survey, a number of organisations noted that there is not always conclusive evidence about what works when addressing the needs of vulnerable groups. On these occasions, organisations highlight that there may be a need to take an innovative opposed to purely evidencebased approach when developing and delivering new services. This may include piloting a service within their own community that has had positive results in another part of the world, or designing a new service based on feedback from service users. By building evaluation into any new services they pilot, organisations also highlight that this approach can help to develop the evidence base about what works and what doesn t work in an area. When developing a new and innovative service, it is advisable to: 1. Identify what the existing evidence suggests might work and might not work in this area. 2. Speak to your service users about what they think is needed. 3. Develop a pilot based on the evidence you have gathered from your service users and external sources. 4. Test your pilot on a small scale and build in monitoring and evaluation so you know whether it achieves its intended outcomes. Remember if something doesn t work then this is useful learning too as long as you explore why. 5. Explore how successful elements of the service can be expanded upon or rolled out to other areas or groups of service users. 20 Evidence for Success The guide to getting evidence and using it

21 Checklist Here is a quick checklist for you to consider when you want to use evidence to influence internal policy and practice: Identify the problem/ situation within your organisation that you want to address. Consider how the evidence that you ve gathered suggests that this problem can be addressed. Create a plan for making these changes. This should identify what changes are needed within your organisation, what resources will be required and what difference they ll make. Identify and engage with the stakeholders within your organisation who will need to authorise and implement these changes. Make sure they understand the benefits and any potential risks of doing so. If there is not conclusive evidence about what works, explore if there is a need to develop a small pilot to test out a new approach. Evaluate any changes you implement to understand if they have achieved their intended outcomes and identify whether any further changes are needed. Evidence for Success The guide to getting evidence and using it 21

22 Section 2: How to use evidence to influence external policy and practice This section of the guide explores how to use evidence to influence external policy and practice at local and national level. The steps outlined below draw from the knowledge and experience of decision and policy-makers, funders, practitioners and academics who have influenced the policy and practice landscape using evidence of what works and what doesn t effectively. By the end of this section, you will know how to: Understand the policy cycle and the evidence needs at difference stages Plan to influence the policy cycle Find different mechanisms to influence the policy cycle Be aware of what helps and what gets in the way

23 53 of the 54 organisations (98%) who responded to the KTN survey agreed that evidence should be used to influence policy and practice. Some of the reasons for this included: Using evidence in this manner contributes to ensuring that policy and practice remain rooted in reality. While working values, the political agenda and other considerations need to be taken in to account the evidence base should also be a part of the picture. Anonymous survey quote Step 1: Understand the policy cycle and its evidence needs If you want to influence policy you need to know how the system works; how to present your evidence really concisely, and what the most effective way to work through the system in Government and Parliament is. David Berry, Scottish Government It is important to use the learning from a variety of arenas to help shape the direction and to stop re-inventing new things. Elaine Wilson, Lloyds TSB Foundation for Scotland So it is clear that we should use evidence to influence external policy and practice but, as another respondent pointed out: The Welcome Trust, a global charitable foundation, highlights that policy is closely linked to a problem and the strategies needed to solve it. The policy process is often characterised as a cycle of activities, where the final stage of one move through the cycle becomes the starting point for the next. This is what is known as the policy cycle. It s relatively easy to influence internal practice, but much more difficult to influence wider policy - especially as a small organisation with few resources and relatively few contacts. Anonymous survey quote The policy cycle has a number of stages that can be summarised as: 1) Agenda setting; 2) Policy design; 3) Policy implementation, and 4) Monitoring and Evaluation. The Scottish Government uses the illustration below to depict the policy cycle. The following steps should help you use your evidence effectively, within the capacity of your organisation. The policy cycle Agenda setting Monitoring and Evaluation Policy Implementation Policy design Evidence for Success The guide to getting evidence and using it 23

24 The table below gives an overview of the activities that are often involved at the different stages of the policy cycle, as well as evidence needs at each stage [adapted from Pollard and Court (2005)]. Evidence should flow through the whole policy process to ensure that decisions are based on what works. This means that there are points in the policy process when you can use your evidence to influence decisions. But you need to anticipate at what stages and when your evidence could be most useful. You can find more information about the policy cycle in the Resources section at the end of the guide. Evidence needs of the policy cycle Stage of the policy process Description Evidence needs Agenda setting A problem is identified The causal relationship between different factors is explored Policies are researched and analysed Identify new problems Build up evidence of the magnitude of the problem so that relevant policy-makers are aware that the problem is important Key factors: Credibility of evidence How evidence is communicated Policy design Possible solution and alternatives are developed The potential impact of these solutions is analysed Consultation The policy proposal is developed and revised accordingly The policy proposal is adopted Support policy-makers to ensure their understanding of the specific situation and the different options is as detailed and comprehensive as possible so they can make informed decisions Key factors: Links between activities and outcomes; expected cost and impact of intervention Quantity and credibility of evidence Policy implementation The policy objectives are translated in concrete activities Operational evidence to improve the effectiveness of initiatves Key factors: Evidence is relevant across different context Monitoring and Evaluation The implementation data is used to assess whether the policy is being implemented as planned, and is achieving the expected objectives Develop M&E to determine the effectiveness of an implemented policy Key factors: Evidence is objective, thorough and relevant, and communicated successfully into the continuing policy process 24 Evidence for Success The guide to getting evidence and using it

H.I.P.P. HAVE INFLUENCE ON PUBLIC POLICY A MANUAL AND TOOL KIT ON HOW VOLUNTARY ORGANIZATIONS CAN INFLUENCE PUBLIC POLICY

H.I.P.P. HAVE INFLUENCE ON PUBLIC POLICY A MANUAL AND TOOL KIT ON HOW VOLUNTARY ORGANIZATIONS CAN INFLUENCE PUBLIC POLICY Be H.I.P.P. HAVE INFLUENCE ON PUBLIC POLICY A MANUAL AND TOOL KIT ON HOW VOLUNTARY ORGANIZATIONS CAN INFLUENCE PUBLIC POLICY Prepared by YMCA Canada with the assistance of Human Resources Development Canada

More information

of Scotland s looked after children & young people.

of Scotland s looked after children & young people. JANUARY 2007 looked after children & young people: Working together to build improvement in the educational outcomes of Scotland s looked after children & young people. looked after children & young people:

More information

Guidance on partnership working between allied health professions and education

Guidance on partnership working between allied health professions and education Guidance on partnership working between allied health professions and education INCLUDED HEALTHY WORKING TOGETHER CONFIDENT Guidance on partnership working between allied health professions and education

More information

Making Connections. Using a theory of change to develop planning and evaluation

Making Connections. Using a theory of change to develop planning and evaluation Making Connections Using a theory of change to develop planning and evaluation By Jean Ellis, Diana Parkinson and Avan Wadia for Charities Evaluation Services February 2011 Background to this guide This

More information

A simple guide to improving services

A simple guide to improving services NHS CANCER DIAGNOSTICS HEART LUNG STROKE NHS Improvement First steps towards quality improvement: A simple guide to improving services IMPROVEMENT. PEOPLE. QUALITY. STAFF. DATA. STEPS. LEAN. PATIENTS.

More information

Advocacy Matters: Helping children change their world. An International Save the Children Alliance guide to advocacy. Participant s manual

Advocacy Matters: Helping children change their world. An International Save the Children Alliance guide to advocacy. Participant s manual Advocacy Matters: Helping children change their world An International Save the Children Alliance guide to advocacy The International Save the Children Alliance is the world s leading independent children

More information

User involvement in health and social care in Northern Ireland

User involvement in health and social care in Northern Ireland STAKEHOLDER PARTICIPATION REPORT 18 Looking out from the middle: User involvement in health and social care in Northern Ireland STAKEHOLDER STAKEHOLDER PARTICIPATION PARTICIPATION REPORT 18 Looking out

More information

Monitoring and Evaluation

Monitoring and Evaluation OVERVIEW Brief description This toolkit deals with the nuts and bolts (the basics) of setting up and using a monitoring and evaluation system for a project or an organisation. It clarifies what monitoring

More information

Think Family: Improving the life chances of families at risk

Think Family: Improving the life chances of families at risk Social Exclusion Task Force Think Family: Improving the life chances of families at risk Social Exclusion Task Force Ministerial Foreword Ministerial Foreword For over ten years now, the Social Exclusion

More information

These Are Our Bairns. a guide for community planning partnerships on being a good corporate parent. looked after children & young people:

These Are Our Bairns. a guide for community planning partnerships on being a good corporate parent. looked after children & young people: a guide for community planning partnerships on being a good corporate parent looked after children & young people: we can and must do better a guide for community planning partnerships on being a good

More information

Measuring what matters. National Statistician s Reflections on the National Debate on Measuring National Well-being

Measuring what matters. National Statistician s Reflections on the National Debate on Measuring National Well-being Measuring what matters National Statistician s Reflections on the National Debate on Measuring National Well-being National Statistician's reflections on the National Debate on Measuring National Well-being

More information

THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS

THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS THE CHILD AT THE CENTRE SELF-EVALUATION IN THE EARLY YEARS Crown copyright 2007 ISBN: 978-0-7053-1109-0 HM Inspectorate of Education Denholm House

More information

TRANSFORMING PARTICIPATION IN HEALTH AND CARE

TRANSFORMING PARTICIPATION IN HEALTH AND CARE www.england.nhs.uk @NHSEngland 0113 825 0861 england.nhs.participation@nhs.net TRANSFORMING PARTICIPATION IN HEALTH AND CARE The NHS belongs to us all SEPTEMBER 2013 Patients and Information Directorate,

More information

engagement between business and community organisations

engagement between business and community organisations engagement between business and community organisations A summary of research into businesses current practices, needs, motivations and experience around supporting community organisations. À À Practical

More information

together we can change our world

together we can change our world together we can change our world A Toolkit on Advocacy Supporting and encouraging young people to speak out, educate, and take action This publication has been realized with the financial support of the

More information

Report of the 21st Century Social Work Review. changing. lives

Report of the 21st Century Social Work Review. changing. lives Report of the 21st Century Social Work Review changing lives Report of the 21st Century Social Work Review lives changing Scottish Executive, Edinburgh 2006 Crown copyright 2006 ISBN: 0-7559-4824-6 Scottish

More information

What s important to me. A Review of Choice in End of Life Care

What s important to me. A Review of Choice in End of Life Care What s important to me. A Review of Choice in End of Life Care Published by The Choice in End of Life Care Programme Board February 2015 1 Page About this review The Choice in End of Life Care Programme

More information

HOW GOOD IS OUR SCHOOL?

HOW GOOD IS OUR SCHOOL? HOW GOOD ARE WE NOW? HOW GOOD IS OUR SCHOOL? HOW GOOD CAN WE BE? THE JOURNEY TO EXCELLENCE: PART 3 HOW GOOD ARE WE NOW? HOW GOOD IS OUR SCHOOL? HOW GOOD CAN WE BE? THE JOURNEY TO EXCELLENCE: PART 3 Please

More information

Guide 33. Fair Access to Care Services (FACS): prioritising eligibility for care and support

Guide 33. Fair Access to Care Services (FACS): prioritising eligibility for care and support Guide 33 Fair Access to Care Services (FACS): prioritising eligibility for care and support Fair Access to Care Services (FACS): prioritising eligibility for care and support The Social Care Institute

More information

Guidance for working with other organisations

Guidance for working with other organisations Guidance for working with other organisations This document was made possible by the generous support of the American people through the United States Agency for International Development (USAID). The

More information

Close to home. An inquiry into older people and human rights in home care

Close to home. An inquiry into older people and human rights in home care Close to home An inquiry into older people and human rights in home care Inquiry into older people and human rights in home care Acknowledgements This has been a collaborative inquiry, significantly enhanced

More information

MY VOICE HAS TO BE HEARD. Research on outcomes for young people leaving care in North Dublin. empowering people in care

MY VOICE HAS TO BE HEARD. Research on outcomes for young people leaving care in North Dublin. empowering people in care 1 MY VOICE HAS TO BE HEARD Research on outcomes for young people leaving care in North Dublin empowering people in care MY VOICE HAS TO BE HEARD Research on outcomes for young people leaving care in North

More information

Turning Strategies into Action

Turning Strategies into Action Turning Strategies into Action Economic Development Planning Guide for Communities Turning Strategies into Action Table of Contents Introduction 2 Steps to taking action 6 1. Identify a leader to drive

More information

A review of international evidence on interagency working, to inform the development of Children s Services Committees in Ireland

A review of international evidence on interagency working, to inform the development of Children s Services Committees in Ireland A review of international evidence on interagency working, to inform the development of Children s Services Committees in Ireland AUGUST 2011 DEPARTMENT OF CHILDREN AND YOUTH AFFAIRS The author of this

More information

Community planning. Turning ambition into action

Community planning. Turning ambition into action Community planning Turning ambition into action Prepared by Audit Scotland November 2014 The Accounts Commission The Accounts Commission is the public spending watchdog for local government. We hold councils

More information

Setting Up a Community-Managed Not-For-Profit Organisation. A guide to the steps and the benefits

Setting Up a Community-Managed Not-For-Profit Organisation. A guide to the steps and the benefits Setting Up a Community-Managed Not-For-Profit Organisation A guide to the steps and the benefits The Office for the Community Sector (OCS) was established as part of the Department of Planning and Community

More information

Guidelines for Research with Children and Young People. Catherine Shaw, Louca-Mai Brady and Ciara Davey

Guidelines for Research with Children and Young People. Catherine Shaw, Louca-Mai Brady and Ciara Davey Guidelines for Research with Children and Young People [Blank page] Guidelines for Research with Children and Young People NCB s vision is a society in which children and young people are valued, their

More information

How to manage performance. booklet

How to manage performance. booklet How to manage performance booklet We inform, advise, train and work with you Every year Acas helps employers and employees from thousands of workplaces. That means we keep right up-to-date with today s

More information

ADULTS SERVICES: SCIE GUIDE 51. Co-production in social care: What it is and how to do it

ADULTS SERVICES: SCIE GUIDE 51. Co-production in social care: What it is and how to do it ADULTS SERVICES: SCIE GUIDE 51 Co-production in social care: What it is and how to do it A guide to co-production in social care and how to develop co-productive approaches to working with people who use

More information