Monitoring and evaluation plan example Protecting our Places

Transcription

1 Monitoring and evaluation plan example Protecting our Places This example is designed to show you the type of information expected in a plan. Your M&E Plan should relate to the objectives you have set for your project. Grant reference number: title: Name of organisation: Contact person: 2011/PP/EXAMPLE Rehabilitation and education of culturally significant land to the Gundungurra Nation in the Blue Mountains Blue Mountains Local Aboriginal Land Council Contact number: (02) Grants under Restoration and Rehabilitation; Environmental Education; and Protecting our Places Programs must complete and submit the Monitoring and Evaluation Plan along with the completed Schedule C Measures with the signed Grant Agreement to the Environmental Trust within 30 working days of receiving the Grant Agreement. The first instalment of the grant will not be paid until a satisfactory Schedule C Measures and Monitoring and Evaluation plan is submitted. This form is designed to help you develop an M&E Plan for your project. Your M&E Plan is the major tool to help you achieve the outcomes of your project and it also aids in implementing an adaptive management approach. Monitoring your program means checking that it is reaching its expected level of performance (as defined by your objectives and expected outputs) over time. This plan will also help you assess any risks that may have an impact on you meeting your objectives and think about how you could mitigate the risks. This template also asks you to plan how you are going to communicate with your stakeholders and the wider public. Communicating about projects or elements of projects is often poorly done and this can often represent a missed opportunity to get others involved with your project. The M&E Plan will not only assist you in planning your project but it will help you to demonstrate how successful your project is. You will put a large amount of effort and time into planning and delivering your project so it is important for you to know what worked [and why], and what did not work and how to adapt your strategies to ensure you have a successful result. When you monitor and evaluate your project, you can learn from your experiences and adjust your project for the future. Evaluation provides a feedback mechanism for continuous improvement of your effort. It should contribute to decision-making at every stage of the project and it should encourage you to reflect on your outcomes so that you can consider future actions. * For a description of what this means please refer to the glossary. This is an example Monitoring and Evaluation Plan based on a hypothetical project 1

2 Evaluation is the systematic process of collecting credible data and using it to make judgements about the worth of a product, service or process at any point in a projects life cycle. Ideally, evaluation starts from the moment you design your project and runs through to the completion of the project. Knowing that your program has made a difference is crucial for everyone who is conducting the project with you and it encourages continual improvement. Some of the information requested in the M&E Plan has already been provided in the Grant Application. Therefore it is advised that you refer to your Grant Application before completing this plan. In addition, you will need Schedule C Measures. You will need to refer to this document when completing section 3, plan. Schedule C Measures is a set of common indicators developed by the Environmental Trust to enable data to be collated and reported at a project, program and state-wide level. If you would prefer to print a version of this document you can download a text only version from the Environmental Trust website. Grant Application Details the project objectives Development of a Plan Outlines to the Technical Committee the feasibility and relevance of the proposed work Schedule C Measures Identifies the relevant Environmental Trust project measures needed to achieve objectives as stated in the Grant Application Provides projected outputs for identified project measures Monitoring and Evaluation Plan Based on the Plan identifies how progress will be monitored and how success will be evaluated Reporting Reports on the implementation and ongoing application of the Monitoring and Evaluation Plan Compares projected outputs to actual outputs (what was achieved) Evaluates the success of the project Figure 1: Linking the Grant Application, Schedule C Measures, M&E Plan and Reporting. If at any stage assistance is required please contact the Environmental Trust s Monitoring, Evaluation and Reporting Liaison Officer on (02)

3 1. objectives Please list your project objectives as provided in your Grant Application. However, you may wish to revise these objectives to remove ambiguity and/or overlap. Generally your project should have a maximum of 5-6 objectives. The objectives should clearly identify what the program is intending to achieve in a measurable form. Information on what makes up a good objective can be found on the Trust s website. 1. To rehabilitate 10,000m² of culturally significant land of the Gundungurra Nation in the Blue Mountains. 2. To increase awareness and educate the local Aboriginal and non-aboriginal community about the culturally significant land of the Gundungurra Nation in the Blue Mountains Stakeholder relationships Stakeholders can be defined as those who are interested in, concerned about, affected by, have a vested interest in, or are involved in some way with, the issue. Because of their involvement and ownership of the issue many stakeholders will have knowledge, networks and resources which can add significant value to the project. This section requires you to identify your project stakeholders and their interest. Following the completion of this section, you should consider and where appropriate incorporate stakeholder relationships (as activities) when completing the Plan (Section 3). Please list your project stakeholders including the project team, contractors, consultants, volunteers and any other individuals/organisations involved in the project. Organisation or group Local Aboriginal and non-aboriginal Community Blue Mountains Landcare Group Manager Contractors Blue Mountains Nursery Hawkesbury-Nepean Catchment Management Authority Interest or views Potential use of the area and maintaining the cultural significance Provide volunteer assistance in rehabilitating the area and preventing the spread of weeds to adjacent sites Ensuring the project meets objectives and coordinate contractors and volunteers Undertake on-ground works Provision of tubestock to be planted in areas where natural regeneration fails to occur Liaison to ensure works are inline with CMA activities 3. plan Please use one table for each of your objectives as indicated above. 3

4 Information from these tables will be used when you are reporting on your project. The M&E Plan will be a major component of progress and final reports. The reporting template is available in the Monitoring, Evaluation and Reporting section of the Environmental Trust website. If you require additional rows for activities and/or project measures in the tables below, please use the Table tab from the Microsoft Word menu to insert additional rows. If you require assistance please contact Trust administration. Please see the glossary at the end of the document for explanations for each of the headings in the following tables. 4

5 OBJECTIVE 1: Activity* Engage contractors through targeted tender and volunteers through current networks to undertake onground works Conduct a baseline site assessment to establish the current condition of the site and develop a site To rehabilitate 10,000m² of culturally significant land to the Gundungurra Nation in the Blue Mountains. measures* Total How will When will What tools/ (as per Schedule C projected progress be progress be resources are Measures + any other outputs* measured* measured* required to additional project measures that are relevant to your measure project but not listed in progress* Schedule C Measures) Number of contractors 2 engaged Number of volunteers engaged Number of photo 10 reference points Number of 5 transect/quadrats established Species density and 3 diversity survey work plan Work plan developed 1 Undertake primary and secondary weeding Area regenerated Area of culturally significant land rehabilitated Area weeded 20 10,000m² 10,000m² 6,000m² Suitable contractor and volunteers engaged Establish photo reference points and transects/ quadrats and complete initial field assessment; Work plan completed; species list completed for the site Undertake transect and quadrat survey and compare to Completion of engaging contractors and volunteers Completion of site assessment and work plan Every 12 months at the end of the weed growth season; Contractor * For a description of what this means please refer to the glossary. Contracts and volunteer attendance lists Stakes; Tags; Tape measure; Field survey sheets; Digital camera Stakes; Tags; Tape measure; Field survey sheets; Digital camera; Who is responsible for measuring progress* Activity completion date(s)* Contractors and volunteers recruited by March 2012; Work to be completed October Work plan completed March 2012; Baseline assessment April 2012 Primary weeding January ; Secondary 5

6 Undertake progress and final site assessments to compare to baseline data and to monitor and evaluate change Revegetate areas not showing signs of natural regeneration and develop a watering pl;an for those areas Number of contractors 2 Number of contractor 500 hours Number of volunteers 20 Number of volunteer 800 hours Number of photo reference points monitored Number of transect/quadrats established Species density and diversity survey 10 Number of plantings 1,000 Survival rate of plantings 90% Area revegetated 2,000m² 5 3 baseline survey; Record area and weeds removed; Record contractor and volunteer hours via attendance sheets Establish photo reference points and transects/ quadrats and complete initial field assessment; Work plan completed; Species list completed for the site Prepare a watering plan and assess plant health regularly, particularly monitor signs of plants showing water stress; Record number and hours monitored weekly; Volunteer hours monitored at each working day Completion of site assessment and work plan Baseline then every 6 months (other than summer months when it will occur monthly) then at project completion Attendance sheets Stakes; Tags; Tape measure; Field survey sheets; Digital camera Stakes; Tags; Tape measure; Field survey sheets; Digital camera weeding October Work plan completed March 2012; Baseline assessment 2012 October 6

7 type of plants planted; Undertake transect and quadrat survey and compare to baseline survey How are you going to collect baseline data? Baseline data refers to the information collected before the project commences. Baseline data provides information on the current condition, it is used to provide a comparison between the condition before the project commences and the condition as a result of the work undertaken by your project. An initial baseline assessment will be conducted including; undertaking transect and quadrat surveys; undertaking a species diversity and density survey; and the establishment of 10 photo reference points. What would success look like if you achieve this objective? It is important to consider what the overall success (expected outcomes) of this objective would look like and how you know that you have reached it. For weed species in the area to be reduced to less than 5% and the area showing signs of natural regeneration. Areas planted showing 90% survivability. OBJECTIVE 2: Activity Form a committee to steer the project and develop the key message to be promoted To increase awareness and educate the local Aboriginal and non-aboriginal community about the culturally significant land of the Gundungurra Nation in the Blue Mountains. measures Total How will When will What tools/ Who is Activity (as per Schedule C projected progress be progress be resources are responsible completion Measures + any other outputs measured measured required to for measuring date(s) additional project measures that are relevant to your measure progress project but not listed in progress Schedule C Measures) Meetings held 2 Key messaged developed 5 Attendance records Following each meeting Attendance records; Meeting minutes First meeting April 2012; Second meeting April 7

8 Promote the project through media including local newspaper and newsletter Develop and install signage that details the cultural and environmental significance of the area Develop content, presentation, promote and conduct field day Develop a questionnaire and conduct a baseline and follow-up survey Number of individuals potentially reached Number of articles written Number of media releases Number of signs installed 1 Number of educational products developed Number of awareness raising events Number of attendees at awareness raising events Number of people sharing, giving or receiving traditional Aboriginal knowledge and Caring for Country 1, Questionnaire developed 1 Number of people 20 participating in initial surveyed Number of people 20 Record the readership of newspapers/ media targeted; Record the number of articles written and printed Signage developed and installed Field day agenda developed; Presentation developed; Record the number of people attending field day; A survey will be conducted to determine effectiveness Number of people participating in the surveys and the level of increased Following each media release track the papers that ran the release Following installation Agenda and filed day to be approved by September ; At the field day Following the initial survey; Following the follow-up survey The readership of each media resource; Internet; Telephone Digital camera Attendance sheet; Survey Survey responses of the local Ongoing April 2012; August 2012; December 2012; April ; October October Field day July ; Presentatio n September ; Field day November Initial survey conducted April 2012; Follow-up survey conducted 8

9 community to determine increased awareness participating in follow-up survey awareness November How are you going to collect baseline data? Baseline data refers to the information collected before the project commences. Baseline data provides information on the current condition, it is used to provide a comparison between the condition before the project commences and the condition as a result of the work undertaken by your project. A random telephone survey prior to media, signage and field day activities will be conducted to benchmark cultural awareness of the site. Then the survey will be repeated to evaluate the knowledge and education gained. What would success look like if you achieve this objective? It is important to consider what the overall success (expected outcomes) of this objective would look like and how you know that you have reached it. 50% of people surveyed at the field day and 25% of people surveyed via the telephone questionnaire have increased knowledge and education about the culturally significant site and how to protect it. 9

10 4. Risk management Risk management is the identification, assessment and prioritisation of risks that may impact on the successful delivery of the project. Although some risks have the potential to have a positive influence, the majority of risks create a degree of uncertainty and can impact on the ability to meet the projects objectives. You would have already identified a number of risks in your grant application. Here we go a step further and assess the degree of risk posed on your objectives. This process requires you to identify risks, determine the likelihood of those risks occurring, determine the impact if those risks occurred and identify actions/treatments to mitigate the risk. When completing this section it is important to consider each of your objectives and more specifically their associated activities that have been identified in the Plan (Section 3). The likelihood scale uses very likely, likely or unlikely to determine the probability of a risk occurring. A description of the likelihood scale is provided in the following table: Likelihood scale Very likely Almost certain, it will probably occur several times a year Likely High probability, it will happen once a year Unlikely Unlikely, but not impossible The impact scale uses major, moderate or minor to determine the consequences if a risk occurred. A description of the impact scale is provided in the following table: Impact scale Major Moderate Minor Huge consequences Moderate level consequences Minor or negligible consequences of effects Risk evaluation combines the likelihood together with the level of impact to evaluate the potential severity of risk. Risk evaluation is categorised by extreme, high, medium and low. For example if a risk was determined as likely under the likelihood scale and moderate on the impact scale then the risk would be evaluated as medium. Risk evaluation scale Very likely Medium High Extreme Likely Low Medium High LIKELIHOOD Unlikely Low Low Medium Minor Moderate Major IMPACT The last step in the risk assessment process is to identify treatments/actions based on the risk evaluation. The treatment/action should reflect the level of risk. Obviously the higher the risk evaluation the more priority should be given to ensuring the risk is mitigated. * For a description of what this means please refer to the glossary. This is an example Monitoring and Evaluation Plan based on a hypothetical project 10

11 Please follow the steps below in completing this section. This section will help you identify potential risks associated with your project, rate those risks according to their likelihood of occurring and their impact if they occurred along with treatment/action you will implement to reduce the risk. Step 1: Identify potential risks to your project and complete Step 1 in the table below. Depending on the grant program risks may include environmental, economic and/or social factors. You may want to include and/or expand on risks provided in your Grant Application. Step 2: Identify the likelihood of each potential risk occurring using the likelihood scale provided above and select the appropriate rating in Step 2 in the table below. Step 3: Identify the impact of each potential risk using the impact scale provided above and select the appropriate rating in Step 3 in the table below. Step 4: Identify the level of risk using the risk evaluation scale provided above and select the appropriate rating in Step 4 in the table below. Step 5: Identify the treatment/action or mitigation option you will implement to reduce and/or manage the risk and complete Step 5 in the table below. Potential risk Step 1 Contaminated matter found Drought conditions manager, or any key project staff resigns Likelihood Step 2 Impact Step 3 Score Step 4 Treatment/action/mitigation options Step 5 Unlikely Moderate Low Seek assistance from Office of Environment and Heritage regarding the removal of contaminated matter. Unlikely Moderate Low Arrange hand watering of vegetation. Unlikely Moderate Low Have more than one person familiar with/responsible for the project s *aims and *objectives who is kept up-to-date on progress. Sorry business Unlikely Moderate Low Have more than one person familiar with/responsible for the project s *aims and *objectives who is kept up-to-date on progress, in the event that the project manager is unavailable to work on the project for a period of time. Plantings not surviving Media releases not picked up by media Unable to find suitable contractors Lack of volunteers to Likely Moderate Medium During hot dry periods undertake weekly inspections of plantings and deploy watering truck if needed. Unlikely Moderate Low Call media outlets to engage them personally about the importance of the project and provide photos. Unlikely Moderate Low Liaise with Office of Environment and Heritage to seek advice on sourcing suitable contractors. Unlikely Moderate Low Liaise with Blue Mountains Landcare Group. Also, 11

12 work on the project Vandalism of the site or signage Lack of interest in the field day approach Council and obtain contact information for other local volunteer land care groups and/or letters to local neighbours/media release/letter box drop. Unlikely Moderate Low If vandalism becomes an issue look at preventing access to the area until works have been completed. Unlikely Moderate Low Ensure media outlets are engaged and commit to media releases. If there is a lack of interest look at other forms of getting the message out there such as accessing Council networks. Now that you have identified all the risks and actions to mitigate them make sure you revisit this part of your Monitoring and Evaluation Plan regularly. 5. Communication strategy A communication strategy aims to assist you in getting your message out about to the appropriate audience and ensures stakeholders and the community know about the project. Here we provide a basic template to develop a simple communication strategy, which encompasses: Who is your target audience? For example community, stakeholders etc. What is the message you want to convey? For example promote the work of your organisation, increase awareness of an issue etc How you are going to communicate that message to the stakeholders? For example local newspaper, newsletter, flyer etc. When do you propose to release the communication? How are you going to know that your communication has been effective? When completing this section it is important that you reflect on both the Stakeholder Relationships that were identified and the objectives and associated activities provided in the Plan and consider how you are going to measure whether you effectively communicated your message to the intended audience. Please use the table below to outline your communication strategy. Target audience Target message Communication method Local Community Local Community Members that use the area Promote the project and get the community to come along to the field day Increase awareness of the cultural significance of the area Local newspaper and newsletter Signage Evaluation method to determine effectiveness Number of people potentially reached; Number of people at the field day The degree to which adverse human impacts have been Proposed date of release April 2012; August 2012; December 2012; April ; October October 12

13 Environmental Trust Report on project progress and completion Written report eliminated from the area Progress and written reports deemed satisfactory by the Trust December 2012; September 6. Evaluation The final step in developing your M&E Plan is to consider how you will periodically evaluate your project. evaluation requires you to critically analyse your activities and outputs and determine the extent to which your objectives are being met or have been met during the project and at project completion. The Trust has developed a number of key evaluation questions that relate to all projects and should be used to evaluate your project (please see below). Also, the Trust encourages you to develop additional key evaluation questions which specifically relate to your project. You will be required to reflect on your key evaluation questions in your progress and final reports to the Trust. Here are some other examples you may use: To what extent did you achieve your objectives? How effectively and efficiently did you achieve your objectives? Will the project s outcomes continue once the project has been completed? If your outcomes are different to those planned, what impacted on the results? What of those were within your control? What of those were outside your influence? What would you do differently? Was the project cost effective? Did you make the best use of available resources? Did the methods chosen to implement the project ensure success? Did the right people attend training or know about the project? Who did not get involved in the project and why? Did you identify all of the risks and were they appropriately managed? Did you communicate with all appropriate stakeholders? Was the communication fit for purpose? How do you intend to periodically evaluate your project? Please use the key evaluation questions below and develop your own that relate to your specific project. 1. To what extent are the objectives being achieved? 2. Was the project being delivered within budget? 3. Were the methods and activities chosen appropriate? 4. What can we do differently? 5. Have all the risks been appropriately identified and managed? 6. Is the community more aware and educated on the cultural significance of the site?

14 Glossary Objectives Objectives define what is going to be done to achieve the overall aim of the project. These are more specific than aims. They are clear, realistic and measurable intended results of a project. Activities Are significant steps or actions that are needed to produce the identified projected outputs. They relate to dates by which specific actions will be achieved. They map out the key steps for the project and set dates for ensuring the programs objectives are met. measures measures is a standardised list of indicators (outputs) with units of measure that the Trust uses to report on projects across its programs. They show the efficiency and effectiveness with which outcomes are being achieved against the objectives of the project. They are used to judge the impact of a project. measures may be quantitative (numerical How much did we do?) or qualitative (descriptive How well did we do it?). Refer to Schedule C Measures for more information. Total projected outputs The specific amount you project to achieve. These amounts would be the same as provided in the Total ed column in Schedule C Measures (Excel document). Outcomes The outcome is the actual change in the environment or behaviour as a result of the project activities. What you expect to achieve. How will progress be measured The methods used to monitor progress of an activity. For example; quadrats to measure success of regeneration or a survey to measure increase in awareness and understanding. Monitoring techniques should adhere to best practices and/or proven methods, where possible. When will progress be measured Identifies when monitoring will be undertaken. Monitoring may occur more than once for a particular activity e.g. monthly, quarterly etc. What tools/resources are required Tools/resources that are required to monitor progress of an activity. Tools/resources must be sufficient and relevant to the monitoring techniques. For example; stakes and tape may be tools required to establish quadrats or a developed questionnaire may be required to conduct a survey. Who is responsible for measuring The person responsible for undertaking monitoring. Identify the person with the skill and experience to be responsible for monitoring the activity. Activity completion date The realistic and achievable date by which an activity is expected to be completed. NSW Environmental Trust PO Box 644, PARRAMATTA 2124 Phone: (02) info@environmentaltrust.nsw.gov.au ISBN OEH 2011/0845 Published December