Evaluation Framework and Data Collection Templates for Objective 1

Transcription

1 Evaluation Framework and Data Collection Templates for Objective 1 At the completion of the TFER Strategy, there will be an increase in the number of organisations in the EMR that have established systems that promote gender equality. STRATEGIC DIRECTION Lead and achieve change GOAL Organisations will lead initiatives that promote equal and respectful relationships and prioritise the prevention of violence against women in their plans, policies and practices (both internally and externally). Key evaluation question(s) To what extent has there been an increase in the number of organisations in the EMR with gender equitable policies, procedures and practices that reflect best practice? (impact) What organisational characteristics and contexts were facilitative or limiting of the development of systems to promote gender equality and why? (process) How will this be evaluated? A pre/post evaluation design will be adopted (where realistically achievable), using mixed methods. Individual partner organisations will share data on the above indicators with the Regional EWG. The latter will undertake quantitative data analysis (e.g. paired samples t-tests) and thematic analysis of qualitative data, using a combined regional de-identified data set. It is acknowledged that for many partners only one time-point may be completed within the timeframe of the Strategy. Whilst we encourage two time-points, this is not a strict expectation and your contribution to the regional evaluation will be no less valuable where only one gender audit has been completed. The gender audit tool is designed to assist your organisation to identify opportunities and gaps in your approach to and can be implemented periodically to enable you to track your organisation s progress over time towards promoting gender equality. It is important to maximise the opportunities afforded through the gender audit process and we therefore encourage organisations to implement it in a considered, meaningful way, with planned repeated audits ideally taking place within an ongoing monitoring cycle every two to four years, for example. Participating Partners 1 Women s Health East 2 Carrington Health 3 EACH/KSCH 4 Monash City Council 5 Whitehorse City Council 6 MonashLink Community Health Service 7 Knox City Council 8 Maroondah City Council 9 Shire of Yarra Ranges 10 Booroondara City Council 11 Inner East Community Health Service 12 Manningham Community Health Service 13 Inner East Primary Care Partnership 14 Outer East Primary Care Partnership 1

2 Proxy impact indicators Proportion of TFER partners with a policy OR a written commitment to gender equality in the workplace. Proportion of TFER partners with a procedure. Proportion of TFER organisations with policies that meet all seven quality criteria outlined in the EMR organisational tool. Proportion of TFER partner organisations that use sex disaggregated data for planning purposes. Interventions/ Strategies 1.1 Develop and adopt an organisational statement on 1.2 Develop policies relevant to Process indicators The barriers for TFER organisations to adopting an organisational statement on. The enablers for TFER organisations to adopt an organisational statement on. The barriers for TFER organisations to developing a policy on. The enablers for TFER How will data be collected? Organisational Gender Equity Audit Tool and accompanying Self Assessment Tool Data can be collected using this tool through a combination of methods (document analysis, staff survey and/or internal consultations through focus group discussions, for example). Various indicators can be measured using a range of methods it will be thus up to each participating organisation to determine which ones suit their situation most of all. We strongly encourage organisations to complete the 10 shared items as a minimum. A Self Assessment tool with the ten shared indicators is available to guide your audit process. You will be invited to share the data you report on in this template with the Regional EWG for regional level analysis. You may also be requested to contribute summaries of your consultations for thematic analysis. By whom and by when? The tool will (ideally) be conducted by organisations at two separate time points. The first time point being prior to the implementation of the actions listed below under interventions/strategies or as early on in the implementation phase as possible. The second data collection time point should be a minimum of 12 months later. Regional Level The Regional Evaluation Working Group (EWG) will conduct regional level analysis of data provided by individual organisations, on the 10 shared indicators from the Gender Audit Tool (those in buff colour). How will data be collected and by whom and when? The Audit process may capture some data around these issues, particularly where focus group discussions are carried out for consultation with staff. The participants could be project workers, implementers, management and those identified by project workers that are key decision makers for the adoption of policies. Two sample theme-lists are available for organisations to adapt to suit their own organisational needs/contexts one for focus groups with managers and senior-level staff as well as one for other staff. The focus groups should be audio-recorded where possible. Regional Level Individual organisations may be requested to provide a summary analysis 2

3 Interventions/ Strategies 1.3 Foster organisational change agenda for 1.4 Conduct organisational audit Interventions/ Strategies 1.5 Work with private enterprise to promote organisations to developing a policy on. Process indicators that allowed organisations to foster change in relation to workplace. Contextual factors that acted as barriers to fostering change in relation to workplace. The barriers experienced by TFER organisations when conducting the audit. The factors that enabled TFER organisations to conduct the audit. Process indicators that allowed TFER organisations to work with private enterprise to promote including brief discussion of the barriers and enablers for example (and transcripts in some instances) of any focus groups that were conducted internally to the Regional EWG to be used as part of the regional data set. Questions to guide your analysis are provided in the Self-assessment tool. Additionally, the Regional EWG will facilitate focus groups (n=2-3) with representatives from the above mentioned organisations (between January and March 2017) to explore the experiences and perspectives of organisational change contexts and to provide insight into the stated indicators. The regional evaluation working group will not conduct analysis on individual organisational level data - this will be the responsibility of the individual organisations. How will data be collected and by whom and when? As above. How will data be collected and by whom and when? The Regional EWG will conduct a focus group discussion with representatives from the partner organisations involved in this work with external agencies, to explore various contextual factors and how they acted as barriers and/or facilitators, from the perspectives and experiences of TFER partner staff. If your organisation is involved in this work, you may be invited to participate in this 3

4 in the workplace 1.6 Work with early childhood sector in respectful relationships and gender equity workplace. that acted as barriers for TFER partner organisations when working with private enterprise to promote workplace. that allowed TFER organisations to work with the early childhood sector around respectful relationships and. that acted as barriers for TFER organisations when working with the early childhood sector around respectful relationships and. focus group discussion. Analysis and Interpretation Analysis of quantitative data derived from the gender audit (based on the shared indicators only) will be completed using inferential statistical methods of analysis, such as paired t-tests. Interpretation of quantitative data will be based on the degree of change that has occurred across the regions organisations based on the key measures contained within the gender audit tool (those shaded). These data will be considered in the context of the qualitative data generated by the audit tool and broader focus group discussions facilitated by the Regional EWG, as well as data collected in association with other related objectives, in order provide insights into the why or why not certain changes may or may not have occurred. Qualitative data collected for Objectives 5 and 6, for example, will enable a cross or comparative analysis of themes to inform a brief case study report on organisational experiences of capacity building activities and efforts. How will the findings be disseminated? 4

5 Findings will be shared with partners in the form of an evaluation report (and summary report) and disseminated more broadly via the TFER website, contributions to the regional Family Violence paper on The Well, TFER forums, newsletters and networks and/or conference presentations, and journal publications (where relevant). Interim findings will also be reported back to partners on an annual basis. What support is available? The audit tool comes with a set of guidelines for use. Training on the use of the tool will be provided at the Regional TFER Forum in March The Regional EWG will meet with partner organisations prior to the Forum to discuss the tool and to disseminate some resources to partner organisations including an Executive Summary of the Evaluation Framework; some ethical guidelines for data collection and storage; etc. the Regional EWG can also provide theme-lists for FGs. Checklist for implementation Preparing for the Audit: Have you identified who should have responsibility for carrying out the Audit? You could form a Gender Audit Working Group for example. Perhaps there is an existing committee or working group who could take on this role? Aspects this group should consider may include what resources, time and level of political will/support there is for the work. What is the perceived level of organisational readiness for the Gender Equity audit, for instance? Also, how will you document the process for undertaking the audit? Have you decided upon your data collection methods for the audit? Is there consensus amongst the Working Group on the approach to be taken for optimal impact? Do you have the required tools and resources for data collection? Ie. Staff willing and able to conduct focus groups? Will you audio-record the consultations? If conducting a survey, will you use a paper-based and/or surveymonkey version? Has appropriate approval for conducting the audit been sought? What additional support is needed, etc.? Have the dates for data collection been determined? Are rooms booked as needed? Ethics: Has there been clear communication to all staff about what their participation in the audit means/involves? Has it been made clear to staff that all information that they provide will remain confidential and that their participation is voluntary? Has consideration been given to issues of informed consent? How will consent be sought from staff? 5

6 Data analysis and sharing: Has the PRE organisational development data collection template (provided) been completed and submitted to the Regional EWG? Has the data been analysed by the Working group and results reported back to the organisation in a timely fashion through appropriate channels and mechanisms? Action planning: Have recommendations been translated into an organisational capacity building Action Plan (with timelines, budget and responsibilities)? Has a follow up audit timeframe been determined? Who will be involved in conducting the follow up audit? 6