How to monitor and evaluate your community project

Transcription

1 How to monitor and evaluate your community project So you've set up your project. But how can you tell if it's going well? This guide tells you all you need to know about monitoring and evaluating what you're doing - from why it's important to what tools you can use. What is monitoring and evaluation? Monitoring means regularly collecting information on your project and analysing it to see how things are going. To do this, you can look at quantitative information, hard facts such as how much money and time you've spent on the project and how many homes you've reached. You can also consider qualitative information - the less measurable things, like whether the project is running smoothly and if people involved are satisfied. Proper monitoring will make it easy to evaluate your project when it ends. It will also give you a basis to make decisions on how the project should go forward and to make changes along the way if you find something isn't going well. Evaluation is a review of the whole project, looking at its overall value and effect. It usually takes place when the project is complete. You can use the information you've collected during monitoring. Doing an evaluation will help you understand how and where your project has been successful. It will let you see how well funds have been used, and what the benefits of your project have been to those involved. You can use this information to improve your future projects. Why is monitoring and evaluation important? There are several reasons why monitoring and evaluating your project is particularly important for community based projects: Accountability When you win funding from an organisation, they will usually make it compulsory for you to report back on what you're doing and what you're spending their money on. You might have agreed targets with them, such as the number of people your project will reach within a given time. You'll need to monitor your project regularly to collect this information. To secure future funding Funding bodies will want to see concrete evidence of the effect your group has to help them decide whether to give you money. Ongoing monitoring and evaluating of your projects will provide this. It can provide evidence of what a project has achieved and what might be achieved as this project continues in the future.

2 To check the project's progress against your original plans It can be easy to lose sight of the original aims you had for your project. Monitoring it on a monthly, quarterly or yearly basis and checking whether you're still on course to achieve your goals will prevent this. It will also help you keep an eye on whether you're keeping to your timescales and budgets. To learn from your experience Proper monitoring and evaluation of your project means you can see what has worked well and what hasn't. You can then use this information to improve future projects or funding applications. To motivate staff and volunteers Showing your volunteers and staff real, concrete evidence of the impact their hard work has had will make them feel good. It can also encourage them to continue working with your group, or to take on new sustainability projects. It may also spur them on to keep changing their own behaviour. How can you monitor your project's progress? Before you begin monitoring, it's important to think about exactly what you want your project to achieve. Then you can look at how you're going to monitor each of those areas. Doing this thinking upfront should mean that you won't miss opportunities to gather information along the way, and you'll be able to do a complete evaluation at the end. A lot of your monitoring will focus on the hard facts: how much is being spent, and how many homes or people your project is reaching. However, the qualitative side is important too. Are the expectations of the people involved in your project being met? What would they like to see done better? There are many ways you can gather qualitative information: Questionnaire or survey These are simple to put together and distribute. You can reach a lot of people quickly and easily. You can have questionnaires with multiple choice boxes people can tick, and you can ask them to give you more detailed replies. The questionnaire can be anonymous if you think you'll get better responses. However, you need to make the questions as clear as possible so that you get exactly the information you want. You'll need to analyse the information once you've collected all the questionnaires to get an overall picture of what people think. Questionnaires can be especially useful for monitoring attitudes or behaviour change. Feedback forms If you're holding an event, why not ask people to fill in a feedback form at the end? The event will be fresh in their mind, so you should get accurate responses. However, some people might rush the form if they're in a hurry to leave, so do offer them the option of ing it back to you. You can use their feedback to improve your future events. Interviews Although they can be time consuming, interviews carried out face-to-face or on the phone are a great way to get detailed information on what people think of your project. Focus groups Getting a group of people together to discuss your project is an easy way to gather information. Bringing a group together, rather than just interviewing individuals, means people can spark new ideas in each other, and you often get better responses. However, you need to think carefully about the information you want to collect from your focus group, and continually steer the conversation in that direction. It's easy to let the discussion drift off track.

3 How can you evaluate your project? Asking yourself the following questions before beginning your evaluation will help you do it well: What is the purpose of your evaluation? What do you want your evaluation to demonstrate? Who are you evaluating for? Is it for your funders, the community, your organisation? What are you going to evaluate? Which elements of the project do you intend to look at? How are you going to evaluate? What methods are you going to use to carry out your evaluation? You should use a range, and collect both quantitative and qualitative data. How will you collect the data you need? Is the data available? If you need data from across the project, have you set up monitoring to capture it? What will you do with the data you collect? Data by itself is just data; you'll need to analyse it to come to a conclusion about what it shows. You'll also need to consider how you're going to present the analysis and your conclusions. How will you act on your analysis? All too often the findings of monitoring and evaluation end up gathering dust on a shelf. Make sure your findings see the light of day. Use them to plan for the future. Important points to remember Before starting your project, consider what you'll want to evaluate at the end and what you'll need to monitor to do this. Don't just focus on statistics in your monitoring and analysis. It's just as important to get qualitative information. Case studies All over the country, community groups are monitoring and evaluating their projects using the tools and ideas above. Here's how three groups have done it: Warmer Improved Somerset Homes Making sure older people get what they're entitled to. That's what the Warmer Improved Somerset Homes (WISH) project is all about. WISH helps older people to apply for Government benefits to which they are entitled in order to make their homes more energy efficient. For example, everyone over 70 is entitled to free loft and wall insulation. WISH also carry out some home improvements. Monitoring and evaluation was important to WISH from the start. They immediately decided what quantitative data they were going to collect, including the number of people given benefits advice and the number of repairs done to homes. This helped WISH get funding from the Big Lottery Fund, who ask for concrete evidence of the effects their funded projects have. They knew WISH would be able to provide it, so were happy to give them funding. WISH also collects qualitative information from the people the project helps. They want to find out things like whether people's homes feel warmer. They gather data using two questionnaires: one which they give out when they first make contact with the older person and one after they help them. WISH spends time building a relationship with the older people, this helps to get them to be honest and open in the questionnaires. That way, the information gathered is as useful as possible. Be realistic about how long it's going to take to see effects from your project. Don't expect instant results. Monitoring and evaluation should be seen as something positive by everyone involved in the project, not something to worry about.

4 Hampshire Solar Challenge If you're looking for a really imaginative approach to teaching children about renewable energy, look no further than Hampshire Solar Challenge. Through the project, children learn about solar power by building and racing their own solar-powered cars. The race has been an annual event in the county since Monitoring and evaluating the event not only means that it gets better every year, but positive feedback also helps the organisers to make a strong case for funding. Hampshire Solar Challenge collects quantitative data about the number of children and schools involved and the number of people who come along to the race. They also give a detailed four-page evaluation form to the schools involved at the end of the project, so they can point out any problems. Hampshire Solar Challenge collects qualitative information too, on whether the children enjoyed the project and whether it raised their awareness of renewable energy. This is done through open discussions in meetings attended by everyone involved in the project. Hampshire Solar Challenge then uses the information to look at how they can improve the project for the next year. Working Herts Environmental and training charity, Working Herts, helps the environment and the local community in Hertfordshire and South Bedfordshire. They fit energy and water saving equipment in homes, and they train local unemployed young people to do the work, giving them a job and skills. Quality is very important to the organisation, so monitoring customer satisfaction is crucial. Working Herts uses the information they collect to shape how they train the young people involved. Working Herts gives the owner of each property they work on a customer form asking for their opinion on every part of the service. The form comes with a return envelope with a rubber stamped address and postage stamp, rather than a printed address and freepost mark. Working Herts do this to make respondents feel like they're sending their form back to real people, rather than a machine. This `humanisation' has helped them achieve an impressive 60% return rate. The forms are then evaluated and used to plan for the future. Every form (1,000 are returned each year) is read by the chief executive. He then passes them on to the trainees, who read and discuss them at a weekly meeting. The trainees find this very useful, and are able to use the responses to improve the way they work - the feedback forms have become an important training tool.

5 Support from Green Communities If you'd like more information or help with monitoring and evaluation, the Green Communities team is here to help: We can give you information about and put you in contact with other community projects with experience of monitoring and evaluation. Our Community Carbon Footprint Tool lets you measure your community s carbon footprint and see how it changes over the course of your project. Measuring your footprint at the beginning can help you decide what your priorities should be to reduce it. It can also help you set achievable targets. You can apply for support from your local Energy Saving Trust advice centre. Our expert advisors can help you work out your community's collective carbon footprint and put together and implement a plan to reduce it. To speak to your nearest Energy Saving Trust advice centre, call For more information on Green Communities, visit or call Energy Saving Trust 21 Dartmouth Street London SW1H 9BP Tel energysavingtrust.org.uk GC312 Energy Saving Trust October E&EO To get in touch call or greencommunities@est.org.uk