Cwm Taf Regional Collaboration Board Joint Consultation Project

Transcription

1 Cwm Taf Regional Collaboration Board Joint Consultation Project Evaluation Report Final Report Opinion Research Services January 2015 Opinion Research Services The Strand Swansea SA1 1AF

2 Opinion Research Services The Strand Swansea SA1 1AF As with all our studies, findings from this survey are subject to Opinion Research Services Standard Terms and Conditions of Contract. Any press release or publication of the findings of this survey requires the advance approval of ORS. Such approval will only be refused on the grounds of inaccuracy or misrepresentation Copyright January 2015 Opinion Research Services The Strand Swansea SA1 1AF

3 Contents Acknowledgements Executive Summary... 8 Resources to develop and co-ordinate the Project... 9 Cwm Taf Hub... 9 Feedback... 9 Joint Strategy... 9 Viewpoint Survey Consultation Toolkit Citizens Panel Training Stakeholder Database Introduction Background Evaluating Success Methodology Revisiting the Baseline Survey Core Project Team Workshop Stakeholder Interviews Case Studies Baseline Survey Background Current processes Providing Feedback Engaging with hard to reach groups Social Media Working in Partnership Citizens Panel Online Consultation/Engagement Training Views on the Collaboration Board Joint Consultation and Engagement project Overall views on the Project

4 4. Core Project Team Workshop Overview Coordinate activities via the Cwm Taf Hub Improved feedback to residents & stakeholders Viewpoint young person s survey Joint Consultation Toolkit Joint Citizens Panel Identify training needs & develop a training plan Monitor levels of engagement Collaborative working with Communities First Investigate the use of social media Stakeholder Interviews Introduction Overall Findings Understanding of the Project Involvement with the Project Meeting the fourteen objectives Put in place appropriate resources to develop and coordinate the Project Coordinate activities via the Cwm Taf Hub Improve feedback to residents and stakeholders Joint Consultation and Engagement Strategy Joint Citizens Panel Identify training needs and develop a training plan Set up a Local Service Board Steering Group Collaborative working with and support and guidance for Communities First Investigating the use of social media Overall Impact of the Project Case Studies Context Case Study 1 Service Change Proposals Case Study 2 Joint Quality of Life Survey Case Study 3 All Wales Consultation Hub Meeting Project Objectives Context Resources to develop and co-ordinate the Project Cwm Taf Hub Feedback Joint Strategy

5 Viewpoint Survey Consultation Toolkit Citizens Panel Training Stakeholder Database Conclusions Context Overall Areas for Improvement

6 The ORS Project Team Project Design, Management and Reporting Steve Jarman Claire Thomas Ciara Small Rhys Elliott Fieldwork Claire Thomas 6

7 Acknowledgements ORS would like to thank Christopher Davies (Corporate Policy & Consultation Manager) from Rhondda Cynon Taf Council and Idris Evans (ESF Consultation Coordinator) for their help and assistance in developing the project. We would also like to thank the other stakeholders who took part in the evaluation, without whose valuable input the research would not have been possible. 7

8 1. Executive Summary 1.1 Opinion Research Services (ORS) were appointed by the Cwm Taf Regional Collaboration Board (the Collaboration Board) to undertake an evaluation of their ESF funded Joint Consultation and Engagement Project. This sought to establish a joint consultation and engagement function that would enable a coordinated partnership approach to consultation and engagement activities across and between Local Service Boards (LSBs) in Rhondda Cynon Taf and Merthyr Tydfil. 1.2 The main purposes of the evaluation were to look at how successful the Project had been in meeting its 14 key objectives. 1.3 The evaluation programme included both quantitative and qualitative methodological approaches that ORS felt would enable the identification of how well the project objectives had been met. These included the following:» To revisit a baseline survey that was undertaken with key stakeholders at the start of the project to identify any positive and/or negative changes that have taken place as a result of the project.» To facilitate a Workshop with core project team members to articulate and discuss the key assumptions behind the project.» Completing in-depth telephone interviews with key stakeholders that have been involved in the project.» Completing a number of Case Studies looking at key elements of the project. 1.4 Looking at the impact and future sustainability of the staffing resources that were introduced as part of the project. While this Executive Summary seeks to give a balanced assessment of the discussion outcomes, readers are referred to the detail in the full report for a more comprehensive account of the views expressed in particular, for an account of people s priorities, assumptions and reasons for these views. 8

9 Main Findings Key Objectives Resources to develop and co-ordinate the Project 1.5 This objective has been largely achieved through the award of the ESF Grant and the support of existing consultation and engagement officers which have enabled a number of project resources to be put in place. These have included the establishment of 2 new posts and associated support costs; the establishment of the Joint Project Team and the holding of initial project meetings; and an audit of consultation and engagement activities across partner organisations. 1.6 It was generally felt by those who were interviewed that the Project has been well coordinated, managed and resourced. Improving links between the organisations involved in the Project was also considered to be a success by those who were interviewed. Cwm Taf Hub 1.7 The Cwm Taf Hub can be seen as one of the major successes of the Project to date and provides a comprehensive online resource for consultation and engagement activities for residents and service providers in the Cwm Taf area. 1.8 During the stakeholder interviews those who thought highly of the Cwm Taf Hub did so on the basis that it had contributed to meeting the overarching aim of the project to reduce duplication, increase coordination and to reach a wider audience. 1.9 Overall those who were interviewed supported the continuation of the Cwm Taf Hub and suggested that they would continue to use it as a valuable resource to undertake consultation in the future Ongoing commitment to support the Cwm Taf Hub from Collaboration Board partners has also been given through agreement to fund it for at least 12 months following the end of the ESF funding In addition as part of the development of the Cwm Taf Hub, a Young Persons Hub was developed with young people across the Cwm Taf area. This will be developed further and rolled out with youth services in the future. Feedback 1.12 This was largely to be achieved through a You Said We Did feature on the Cwm Taf Hub, as well as through the use of other printed media and individual organisations websites. Joint Strategy 1.13 The Strategy has been completed and is also seen as one of the main achievements of the Project to date as it provides a consistent framework for organisations to follow when planning and 9

10 undertaking consultation and engagement activities. The Strategy has also been formally adopted by all Collaboration Board partners and the Action Plan is currently being developed. Viewpoint Survey 1.14 Despite a substantial amount of preparation work that was undertaken by the Project Team a number of factors coincided that meant that it has not yet been possible to begin the data collection. These included the Education Service being put into special measures, a Wales-wide survey of school pupils by Sport Wales and the running of a PAS Survey Work has continued by members of the Project Team to engage with the new 6 th Form College which opened in September 2014 and it is hoped that data collection can commence in early In addition the Project Team are exploring opportunities to roll out the survey through the Youth Service, Scouts, Guides and other local clubs and organisations for young people in Merthyr Tydfil. Consultation Toolkit 1.16 The Toolkit has been completed and distributed across partner organisations to assist them in their consultation and engagement activities. Citizens Panel 1.17 A Panel was recruited consisting of approximately 1,600 members (1,000 from Rhondda Cynon Taf and 600 from Merthyr Tydfil) Despite recognition that there have been limited opportunities to make use of the Panel since it was established stakeholders who were interviewed did realise the potential of the Panel and the Panel has been on a number of occasions since it was established. Training 1.19 An Audit of Training Needs was undertaken and a training package was developed in partnership with Participation Cymru. A total of 14 training courses were delivered covering 8 subject areas and of 70 people attended one or more of the courses Feedback from those who completed the training courses was positive and indicated that the training had been successful, with the majority of those saying that their understanding had improved as a result of attending the courses However there were a number of issues raised in relation to the training courses during the stakeholder interviews and those who had been involved in setting up the training felt that getting buy-in from some of the other organisations involved in the project had been challenging at times and acknowledged that the sessions were primarily attended by council staff even though open invitations were sent out to all partners. 10

11 Stakeholder Database 1.22 Members of the Project Team worked with other project partners to establish a Stakeholder Database and a list of Forums that could be used to target specific consultation exercises. This is currently held by Rhondda Cynon Taf and comprises a list of community groups, town councils, public agencies that can be consulted with. Monitoring Levels of Engagement 1.23 Whilst half of the respondents to the follow-up survey said that they did not monitor levels of engagement, those that did said that they did so by asking respondents demographic questions during their consultations. However it is unclear how this information has been used to improve future consultation and engagement exercises and target specific groups of the population. Steering Group 1.24 A Steering Group was set up to guide the project, share best practice, join up survey activity and commission joint consultation projects. Membership of the Steering Group comprised representatives from Rhondda Cynon Taf County Borough Council, Merthyr Tydfil County Borough Council, Cwm Taf University Health Board, South Wales Police, Voluntary Action Merthyr Tydfil, Interlink RCT, Communities First, the Merthyr Tydfil Partnership Team and the RCT Young Persons Partnership. A total of 6 Quarterly Steering Group meetings were held. However issues were raised during the stakeholder interviews about difficulties at times engaging with all partners A new Joint Cwm Taf Public Engagement Group has been established to replace the Steering Group once the funded period of the Project is over and this is seen as a positive outcome in terms of sustaining the work that has been completed to date and continuing the support for multi-agency working. Communities First 1.26 Whilst it was accepted by a number of the stakeholders who were interviewed that this objective had been difficult to achieve - largely due to the fact that Communities First have recently gone through a major restructure - most agreed that there are now much improved working relationships in place to enable greater liaison to take place and to include consultation and engagement proposals in the development of Community Engagement Plans. Social Media 1.27 The use of Social Media was seen by the Project as being a means of promoting consultation and engagement activities to residents and service users. It was agreed at the Workshop that the use of social media can help to promote consultation and engagement activities but a number of practical issues were identified around restrictions that are placed on public service providers on allowing staff to access social media Improvements are now being made in relation to the use of social media and it was agreed that continuing to push for social media is to be used to promote consultation and engagement activities was an important part of the Project. 11

12 2. Introduction Evaluation Overview Background 2.1 Opinion Research Services (ORS) were appointed by the Cwm Taf Regional Collaboration Board (the Collaboration Board) to undertake an evaluation of their ESF funded Joint Consultation and Engagement Project. This sought to establish a joint consultation and engagement function that would enable a coordinated partnership approach to consultation and engagement activities across and between Collaboration Board partners. 2.2 The Collaboration Board applied to the ESF for funding for the Joint Consultation Project. They were awarded a total of 282,481 to fund the project up until December Overall the project set out to:» Develop improved consultation & engagement capacity across both areas, led and overseen by the existing mechanisms in place in Rhondda Cynon Taf;» Share best practice and encourage shared learning and expertise including the improvement of effective consultation with marginalised and under-represented groups;» Develop joined up consultation initiatives, to improve the mechanisms for consultation in both LSBs;» Prevent survey overload and duplication; and» Provide cost savings and achieving economies of scale. 2.4 It aimed to do this by putting in place support and resources to provide an effective and efficient approach to consultation and engagement across the partners of the Collaboration Board. By working together it was hoped that the approach would realise cost savings and lead to a more joined up, continuous and impactful approach to consultation and engagement that would benefit the citizen, by providing the partners and services with evidence that can be used to shape and improve service delivery and thereby helping to ensure that publicly funded services are more accountable and responsive to community need. 2.5 The main objectives of the project are as follows;» To put in place an appropriate resource to develop and co-ordinate an approach to take the project forward. This would include better use of and links with the existing consultation resources that exist across partnerships.» To coordinate consultation and engagement activity across local services from initial planning through to feedback to the citizen. This Consultation Hub would 12

13 provide an online presence for upcoming, ongoing and completed consultations, to allow the Citizen to establish how and when to get involved and what happened as a result.» To ensure that the LSB partners provide feedback to citizens on the outcomes of consultation & engagement activity, as well as evaluating the effectiveness of the work undertaken to ensure that the information collected is used to improve services.» To develop a joint LSB consultation and engagement strategy and action plan, linked to the National Principles for Public Engagement in Wales, endorsed by the Welsh Government in March This could include joint projects, such as a Resident Survey, providing cost savings and preventing survey overload and duplication.» To develop a joint approach to the Viewpoint Young Persons Survey that has been so successful in RCT, consulting with over 8, 000 young people.» To develop a joint Consultation Toolkit that provides partners with guidance and best practice on consultation and engagement activity.» To develop and manage a joint Citizen Panel, including refreshment and recruitment, developing the Panel so it can be utilised by partners across the LSBs.» To identify partner training needs and offer a training plan for staff and elected members in terms of citizen and community engagement.» To establish a stakeholder database that will ensure all consultations are inclusive.» To monitor the levels of engagement in consultation processes, track representation of citizens from areas of deprivation and determine what works in terms of effective participation with people experiencing poverty.» To set up a Rhondda Cynon Taf and Merthyr Tydfil Consultation & Engagement LSB Steering Group to guide the project, share best practice, join up survey activity and commission joint consultation projects.» To set up a mechanism for new collaborative working between members of the Partnership Support Network for Communities First in Merthyr Tydfil and the Consultation and Engagement LSB Steering Group.» To provide support and guidance to Communities First areas in the development of the Community Engagement Plans, which are required as part of the next phase of the Communities First programme.» To investigate the use of social media as a method to engage and consult with the community 13

14 Evaluating Success 2.6 In order to evaluate the success of the project the Collaboration Board requested that a number of workstreams be undertaken. In summary these were: Methodology» To revisit a baseline survey that was undertaken with key stakeholders at the start of the project to identify any positive and/or negative changes that have taken place as a result of the project.» To facilitate a Workshop with core project team members to articulate and discuss the key assumptions behind the project.» Completing in-depth telephone interviews with key stakeholders that have been involved in the project.» Completing a number of Case Studies looking at key elements of the project.» Looking at the impact and future sustainability of the staffing resources that were introduced as part of the project. Revisiting the Baseline Survey 2.7 Between May and June 2013 a Baseline Survey was completed with consultation and engagement staff working in LSB partners including Rhondda Cynon Taf County Borough Council, Merthyr Tydfil County Borough Council, Cwm Taf University Health Board (UHB), South Wales Police (SWP), Voluntary Action Merthyr Tydfil, Interlink RCT and Communities First. 2.8 The aim of the Baseline Survey was to establish the current position in terms of how consultation and engagement activities were carried out by members of staff across the partner organisations. 2.9 The consultation was conducted in-house by Consultation Officers from Rhondda Cynon Taf County Borough Council and the Consultation Coordinator that was appointed as part of the Project. This involved the distribution of links to an online survey to key consultation and engagement contacts across the partners. A paper based version of the survey was also made available for those who did not have access to the internet A total of 30 responses were received The survey sought view on a wide range of consultation and engagement practices including:» How consultation and engagement currently works in each organisation.» Strengths and weaknesses of current approaches.» The use of consultation and engagement guidance.» Providing feedback.» Engaging with hard-to-reach groups.» Citizens Panels.» Online consultation and engagement.» Training. 14

15 » Use of Social Media» Partnership working ORS worked with the Project s Consultation and Engagement team to identify the contacts that were sent the link to the survey in 2013, together with any new contacts that had started work in the previous 12 months. This resulted in a total of 143 contacts. They were ed a link to an online survey containing the same questions that were asked in 2013, together with some additional questions to assess the success of the project. A number of reminders were sent out and a total of 40 useable responses were received These have been analysed and compared with the findings from the 2013 study to identify in any changes have been identified as the project has been taken forward. Core Project Team Workshop 2.14 The main purpose of the Core Project Team Workshop to identify some of outcomes and activities that were required to make the project a success and to consider how and why the changes needed were expected to come about. The workshop was based on a process called Theory of Change but did not follow through the full methodology Theory of Change is about identifying long-term goals and outcomes that relate to a given project and then seeking to identify the requirements needed to achieve these goals. It looks to identify any interventions that will need to happen to create the changes, and helps to identify indicators that can be used to measure the success of the outcomes The workshop was held in June 2014 and was attended by representatives from the following partners:» Rhondda Cynon Taf County Borough Council.» Merthyr Tydfil County Borough Council.» Communities First.» Cwm Taf University Health Board Discussions took place around the following core objectives of the project:» Coordinate activities via the online Consultation Hub.» Improve feedback to residents & stakeholders.» Viewpoint young person s survey.» Joint Consultation Toolkit.» Joint Citizens Panel.» Identify training needs & develop a training plan.» Monitor levels of engagement.» Collaborative working with Communities First.» Investigate the use of social media. 15

16 Stakeholder Interviews 2.18 ORS completed a total of 17 in-depth interviews with members of core project group and with wider stakeholders. These included individuals from the following organisations:» Rhondda Cynon Taf County Borough Council» Merthyr Tydfil County Borough Council» Cwm Taf University Health Board» South Wales Police» Voluntary Action Merthyr Tydfil» Merthyr Tydfil Communities First» RCT Communities First» Interlink RCT» Fframwaith RCT Children and Young Peoples Partnership 2.19 An interview script was developed by ORS in conjunction with the Project s Consultation and Engagement Team which covered the following issues and the interviews were completed by ORS trained interview staff during July and August 2014:» Awareness of the Consultation and Engagement project» Involvement with the consultation and engagement project» Meeting the 14 objectives» What worked well/not so well» How could things be improved in the future» Impact of the consultation and engagement project» Future of the consultation and engagement project 2.20 The kind of in-depth engagement used for the interviews can be properly rigorous, inclusive and representative in the sense of involving diverse groups of people; but it should not be understood as achieving a statistical sample of individuals. Likewise the outcomes from depth-interviews cannot be certified as statistically representative samples of opinion. But the variety of representatives who took part in the interviews were given ample opportunity to comment in detail on the issues related to the project and we were able to learn a considerable amount about the issues at the heart of this study. Case Studies 2.21 Following discussions with members of the core project team at the Workshop the following Case Studies were identified:» Phase 1 Service Change Proposals Consultation in Rhondda Cynon Taf and Merthyr Tydfil County Borough Councils. 16

17 » Joint Quality of Life Survey involving Age Connects Morgannwg, RCTCBC Older Persons Strategy, RCT Carers Support project and Cwm Taf UHB.» Working towards an all Wales Engagement Hub with the Welsh Local Government Association. 17

18 3. Baseline Survey Background 3.1 ORS worked with the Project s Consultation and Engagement Team to identify the contacts that were sent the link to the survey in 2013, together with any new contacts that had started work in the previous 12 months. This resulted in a total of 143 contacts. They were ed a link to an online survey containing the same questions that were asked in 2013, together with some additional questions to assess the success of the project. A number of reminders were sent out and a total of 40 useable 1 responses were received. Please note that some respondents did not provide a response to all of the questions. 3.2 These have been analysed and compared with the findings from the 2013 study to identify in any changes have been identified as the project has been taken forward. Summary of Findings Current processes 3.3 Respondents were asked what method(s) they currently use to undertake consultations and engagement. The chart overleaf shows the responses received. The most common methods of engagement were workshops/public meetings, online, events and questionnaires. When compared to the baseline survey that was undertaken in 2013 the main change is that the number of respondents who use online questionnaires has almost doubled. 1 Some respondents who had not completed all of the questionnaire 18

19 Figure 1: What methods do you currently use for consultation or engagement activities? Base: All respondents (number of respondents shown on each bar) Workshops/Public Meetings 27 Online Event - Stakeholders Event Questionnaire - Online Questionnaire - Face to Face Questionnaire - Postal 19 Interview - Face to Face Citizens Panels Questionnaire - Telephone 8 Other - please specify Other responses included: Have also used focus groups Number of respondents Active Theatre / Forum Theatre techniques User specific user groups i.e. Disability groups, minority groups etc. Community consultations, project evaluations, cluster boards, local focus groups End of course evaluation forms and class discussions; suggestion boxes at facilities; hard copy survey every 3 years in libraries as required by the 4th Library Framework. 3.5 When asked how they would rate their current organisational process for undertaking consultation and engagement, almost half of respondents (19 respondents) rated this as either excellent or good. 16 respondents rated this as fair. Just 2 respondents rated it as poor. This is comparable with the findings from the 2013 study, but with more respondents now rating their processes as excellent. Table 1: How would you rate your current organisational process for undertaking consultation/engagement? Base: All respondents How would you rate your current organisational process for undertaking consultation/engagement? Number of respondents Excellent 5 Good 14 Fair 16 Poor 2 Very poor 0 19

20 3.6 Respondents were asked about what they felt the strengths of their current process for undertaking consultations and engagement were. Many respondents felt that a major strength was that a large variety of people were consulted including young and hard to reach groups: A strength is ensuring the young peoples voices are heard to make a difference to their lives. Engaging with hard to reach and vulnerable people through theatre processes. We are on the ground so we deal with the hardest to reach. 3.7 Some stakeholders found that their consultation and engagement was advantageous as it was done face to face: Face to face meetings with the Public and engagement events prior to the consultation. We aim to engage face to face in consultation. 3.8 Respondents were also asked on what they felt was weak regarding their current process for undertaking consultation and engagement. One thing that some viewed as a disadvantage was a lack of co-ordination between the consultation and engagement projects of different departments which leads to duplication: We don't necessarily join up the opportunities as well as we could. There is no co-ordination of consultation taking place, each department does their own consultation exercises this can result in duplication. 3.9 Some respondents felt that the timing of their consultation and engagement processes were weak. Typical responses are highlighted below: One problem we have is not giving enough time for evaluation. We are often then constrained by time Other perceived weaknesses included: a lack of social media use; a lack of appreciation for engagement and consultation by senior managers; and that consultation and engagement projects are not far reaching enough Respondents were asked whether they used any particular consultation/engagement guidance. 22 respondents reported doing so. Table 2: When undertaking consultations do you use any particular consultation/engagement guidance? Base: All respondents When undertaking consultations do you use any particular consultation/engagement guidance? Number of respondents Yes 22 No Out of the respondents that took consultation/engagement guidance, many got it from Participation Cymru or from the National Principals for Public Engagement in Wales. Others got their guidance from the Independent Theatre Council, the Equality and Diversity team at their workplace or from SNAP. 20

21 3.13 Respondents were also asked whether they follow the National Principles for Public Engagement when undertaking consultation/engagement; 25 respondents reported doing so. Table 3: When undertaking consultation/engagement do you follow the National Principles for Public Engagement? Base: All respondents When undertaking consultation/engagement do you follow the National Principles for Public Engagement? Number of respondents Yes 25 No 6 Providing Feedback 3.14 Stakeholders were asked how they provide feedback to the public following consultation or engagement. The most common method was via their website (18 respondents), followed by events (13 respondents) and magazines/letters (11 respondents). When compared to the 2013 survey it can be seen that there has been a noticeable increase in the number of respondents who are now using the internet to provide feedback. Figure 2: How do you provide feedback to the public following consultation or engagement? Base: All respondents (number of respondents shown on each bar) Via the website 18 Events 13 Magazine/letter 12 Other 10 Via the citizens panel 5 We do not provide feedback Other forms of feedback included: We have liaison with specific groups that we will also use for feedback. Sometimes it is by personal, individual letters to service users I have provided all feedback to all consultations past and present I have undertaken. I organise programmes and we also advise each new programme participants of changes made from previous consultations. Face to face, public forums, newsletter, s etc. Copy of consultation report sent Number of respondents 21

22 On an individual basis if relevant Via Strategic documentation e.g. Corporate Plan or Delivery Report Through discussion at public event Via project delivery, face to face, local meetings with local groups and local boards Public Reports Via leaflets and posters distributed amongst centre customers Engaging with hard to reach groups 3.16 When asked how they engage with hard to reach groups when they consult, the most common method that stakeholders reported were via the website (16 respondents), via Communities First groups (16 respondents) and via the voluntary sector (16 respondents). Differences from the 2013 survey include an increase in the number of respondents who use their website, an increase in those who work with the voluntary sector, and a decrease in the number of respondents who use the Youth Forum and school councils. Figure 3: How do you engage with hard to reach groups when you consult? Base: All respondents (number of respondents shown on each bar) Via the website Via Community First Groups Via the voluntary sector Via citizens panel 10 Youth Forum 7 Via the Community Voice Programme School Councils 5 5 Other - please specify 9 We do not specifically engage with hard to reach groups Number of respondents 3.17 Other forms of engagement included: In SSD all nearly all our service users are deemed hard to reach so we constantly do this through normal processes and ad hoc ones where this is necessary Through Community Services Day Centres, Meals on Wheels and the mobile library service We organise programmes to help young people who are the hardest to reach or multiple barriers to engagement, we link in, invite them on the programme and listen to their views on past experiences 22

23 Disability Forum / Older peoples groups 50+ Forums, day centres, public buildings, council buildings, council staff If appropriate by using the Social care database They come to us during the course of our work and we then consult By going to where those hard to reach people are. By the very nature Communities First has to work with those groups, however more joined up approach to carry out consultations with older people, those who are housebound would be beneficial 3.18 Respondents were asked if their organisation/department monitors the level of engagement achieved with hard to reach groups. Similar numbers of respondents reported doing this as those who do not do it. This is comparable with the findings of the 2013 study. Please see Table 4 below. Table 4: Does your organisation/department monitor the level of engagement achieved with hard to reach groups (Young people, disabled groups, people experiencing poverty etc.)? Base: All respondents Does your organisation/department monitor the level of engagement achieved with hard to reach groups (Young people, disabled groups, people experiencing poverty etc.)? Number of respondents Yes 12 No Many of those who worked for organisations/departments that did monitor this, said that they did so by asking respondents demographic questions during their consultations. Indeed, some participants expressed that this was the responsibility of their consultation and engagement teams. One participant mentioned a particularly rigorous process of monitoring engagement/consultation with hard to reach groups: We monitor by using a register, feedback forms, photographic and digital media records, questionnaires, discussions, workshops and debriefs of projects with target group and stakeholders Respondents were asked what worked well in their most recent consultation/engagement programme. Many felt that face-to-face consultations worked particularly well. Some respondents also saw value when they included a range of consultation methods including online surveys: We had numerous staff undertaking both paper questionnaires and also online surveys. I think giving people the option of completing a questionnaire online worked well. One advantage was the mixture of methods used Some also felt the following worked well in their most recent consultation engagement: going into schools and conducting surveys to reach young people; and ensuring that respondents actually cared about the consultation. 23

24 3.22 Respondents were also asked what could have worked better at their most recent consultation/engagement project. One thing that some respondents felt could have been improved upon was the complicated language used in consultations and consultation information: Some of the questions were difficult for people to respond to. I'm not sure it gave us meaningful information. The consultation could have benefited from better use of language that the public can relate to Moreover, some felt that there was a poor response to their consultations; and that this could be improved on by offering respondents an incentive to take part. Other ways that respondents felt that their most recent consultation could have been improved included: engagement with the Polish community; the use of focus groups; better time management; and improved questionnaire design Respondents were also asked if their organisation/department monitors any equality and diversity information as part of the consultation process. The majority of respondents who answered this question reported that their organisation/department does do this. This is comparable with the findings of the 2013 study. Please see Table 5 below. Table 5: Does your organisation/department monitor any equality and diversity information as part of the consultation process? Base: All respondents Does your organisation/department monitor any equality and diversity information as part of the consultation process? Number of respondents Yes 18 No Stakeholders whose organisations/departments did monitor any equality and information data were then asked what equality and diversity information their organisation/department collects. A few respondents were unaware of any specific information, but those who were mainly listed the following: gender, age, disability, religion and sexual orientation. Other information included whether their respondents spoke Welsh and whether they had access to art provisions Social Media 3.26 Of the respondents who answered this question, 14 respondents reported that their organisation uses social media when consulting with the public. A similar proportion (11 respondents) said that their organisation does not use social media for this purpose. This is comparable with the findings of the 2013 study. Table 6: Does your organisation/department monitor any equality and diversity information as part of the consultation process? Does your organisation use social media at all when consulting with the public? Number of respondents Yes 14 No 11 24

25 Base: All respondents 3.27 Those who reported that their organisations used social media were asked what sites they use. They unanimously responded with Facebook and/or Twitter Respondents whose organisations do not use any social media sites were asked why this is the case. Many explained that it is due to an organisational policy; while others viewed that it was inappropriate for their type of consultation. Other responses included that older respondents do not use such sites and that their organisation is new to social media. Working in Partnership 3.29 Of the respondents who answered this question, 11 reported that they have been involved in partnership working across the Cwm Taf region (RCT & Merthyr Tydfil areas) when consulting or engaging. A similar proportion (13 respondents) said that they had not been involved in partnership working. This is comparable with the findings of the 2013 study. Table 7: Have you ever been involved in partnership working across the Cwm Taf region (RCT & Merthyr Tydfil areas) when consulting or engaging? Base: All respondents Have you ever been involved in partnership working across the Cwm Taf region (RCT & Merthyr Tydfil areas) when consulting or engaging? Number of respondents Yes 11 No Respondents who had worked in partnership with others were asked who these partners were and what projects they collaborated on. A few organisations that came up included: the Local Authority, South Wales Police and the Cwm Taf University Health Board. Types of projects included changes to service provision, the redesign of a hospital and a quality of life survey When asked how they would rate the success of this partnership consultation exercise, the majority of respondents (9) rated it as either excellent or good. 2 respondents rated it as fair. This is comparable with the findings of the 2013 study. Table 8: How would you rate the success of this partnership consultation exercise? Base: 11 respondents How would you rate the success of this partnership consultation exercise? Number of respondents Excellent 4 Good 5 Fair 2 Poor 0 Very poor Respondents who had worked on a partnership consultation exercise were then asked what worked well in their respective collaborative exercises. Some respondents saw a strength in the 25

26 sharing of both knowledge and physical resources. One respondent valued how their collaboration meant that more events could be attended by staff These respondents were also asked what could have worked better in their collaborative consultation exercise. The majority felt that nothing could have gone better in their collaborations. However, others felt that their projects could have: reached more people; had improved co-ordination; and had more suitable dates for public consultation. Citizens Panel 3.34 Respondents were given a brief description of a Citizens Panel. They were then asked whether the ability to access a Citizens Panel would help their work. Of the respondents who answered the question, - 11 stated that a Citizens Panel would help their work, 12 stated that they already use a Citizens Panel in their work and 3 thought that it would not help their work. When compared to the 2013 study it can be seen that the number of people that already use a Panel has doubled and the number of people who do not think one could help their work has halved. Table 9: Would the ability to access a Citizens Panel help you with your work? Base: All respondents Would the ability to access a Citizens Panel help your work? Number of respondents Yes 11 No 3 Already use Respondents were asked about their awareness of the Joint Citizens Panel. The majority of those that answered the question (20 out of 25 respondents) were aware of it. This question was not asked in 2013 as the Panel had not been set up but this does show a high level of awareness of the new Panel. Table 10: Are you aware of the Joint Citizens Panel? Base: All respondents Are you aware of the Joint Citizens Panel? Number of respondents Yes 20 No 5 Online Consultation/Engagement 3.36 Respondents were asked to rate their organisation's ability to undertake online consultation/engagement. 18 respondents rated their organisation s ability as either excellent or good. 5 respondents rated it as fair and 3 rated it as being poor. See Table 11 below. When compared to the findings of the 2013 study this shows an improvement in how respondents rated their organisations ability to undertake online consultation. This is also reflected in the increase in the number of respondents to the follow-up survey who are using online surveys to undertake consultation. 26

27 Table 11: From your experience, how well would you rate your organisation's ability to undertake online consultation/engagement? Base: All respondents From your experience, how well would you rate your organisation's ability to undertake online consultation/engagement? Number of respondents Excellent 6 Good 12 Fair 5 Poor 3 Very poor 0 Training 3.37 Respondents were asked if they or their team had received any training on consultation and engagement in the last two years. 5 respondents reported that they not had any training, while 21 said that they had. This is a significant increase compared to the 2013 study when more than half of respondents had not received any training. Table 12: Have you/your team received any training on consultation and engagement in the last two years? Base: All respondents Have you/your team received any training on consultation and engagement in the last two years? Number of respondents Yes 21 No Out of the respondents that received any training on consultation and engagement in the last two years, the majority received it from Participation Cymru. One respondent received training from their company s corporate team, whilst the remaining respondents actually made mention to their training on the Cwm Taf Consultation Hub and the Joint Consultation Project Respondents who said that they or their team had not received any training on consultation and engagement in the last two years were asked if they were aware that training was available. All stated that they were not aware of this. Table 13: Were you aware that training was available? Base: All respondents Were you aware that training was available? Number of respondents Yes 0 No Stakeholders were asked what sort of training they would find useful. This showed that the most popular training respondents would find useful are public engagement methods, consultation language and style and designing effective surveys. When compared to the 2013 study the main 27

28 differences are a decrease in the number of people wanting training on online consultation processes. This suggests that people may have now completed this training and are now more confident using online consultation methods. Figure 14: Which of the following training would you find useful? Base: All respondents (number of respondents shown on each bar) Public engagement methods Consultation language and style Designing Effective Surveys Online consultation processes Effective feedback Using a citizens panel Other - please specify Other forms of training that respondents would find useful included: Analysing the results Number of respondents I don't require any further training at the moment but I think training needs to be aimed at more senior members of staff to understand the merits of public engagement in times of austerity. Engaging with Gypsies and Travellers (to help fulfil the new requirements of the Housing Bill for Wales) More respondents said that they had not received training as a result of the project compared to those who said they had received training as a result of the project. This question was not asked in Table 145: Have you received any training as a result of the Project? Base: All respondents Have you received any training as a result of the Project? Number of respondents Yes 10 No Some of those who received training as a result of the project received it from Participation Cymru, while others mentioned training on the following: engagement methods, using the Hub; and designing effective questionnaires. Views on the Collaboration Board Joint Consultation and Engagement project out of the 25 respondents who answered the question reported that they had heard of the Project before they were contacted about this survey. 28

29 Table 15: Had you heard of the Project before you were contacted about this survey? Base: All respondents Had you heard of the Project before you were contacted about this survey? Number of respondents Yes 20 No The 20 respondents that had heard of the Project prior to the survey had done so in the following ways: through word of mouth; through talks, presentations and s; and through being directly involved with the development and implementation of the project Out of the 25 respondents who were asked if they had heard about the Project, there was an even split of stakeholders that had been involved (13 respondents) or not involved (12 respondents) in the project. Table 167: Have you been involved in the Project? Base: All respondents Have you been involved in the Project? Number of respondents Yes 13 No The 13 stakeholders that had been involved in the Project were asked how they had been involved. These respondents had been involved by being a part of the Project Group or by using the Consultation Hub for joint engagement and consultation The majority of respondents (16 respondents) who answered the question reported using the Cwm Taf Consultation Hub to promote their consultation and engagement activities. Table 178: Do you use the Cwm Taf Consultation Hub to promote your consultation and engagement activities? Base: All respondents Do you use the Cwm Taf Consultation Hub to promote your consultation and engagement activities? Number of respondents Yes 16 No Stakeholders who use the Cwm Taf Consultation Hub were then asked what consultation and engagement activities they promote using it. Typical responses included: online surveys; focus groups; and corporate budget setting exercises Those who did not use the Cwm Taf Hub were asked why this was the case. Respondents were either unaware of the Cwm Taf Hub, or were not undertaking any consultation. 29

30 Overall views on the Project 3.51 Respondents were asked how well overall - they thought the Project has improved consultation and engagement activities in RCT and Merthyr Tydfil. The table below shows the responses. Table 189: Overall how well do you think the Project has improved consultation and engagement activities in RCT and Merthyr Tydfil? Base: All respondents Overall how well do you think the Project has improved consultation and engagement activities in RCT and Merthyr Tydfil? Number of respondents Very well 7 Fairly well 15 Not very well 2 Not at all well Stakeholders were asked if they could give any examples of how the Project has improved consultation and engagement activities in RCT and Merthyr Tydfil. Some respondents mentioned that the Project enabled them to access more experienced staff, while others mentioned that the Project had improved collaborative working between different agencies and therefore decreased duplication. The following comment represents a typical response: I think the Project has raised the profile of engagement to senior staff members... I think the project has broken down barriers between organisations, reduced duplication and made it easier to reach a wide range of people. 30

31 Summary Baseline Survey» When compared to the baseline survey the use of online questionnaires has almost doubled.» Almost half of respondents (19 respondents) rated their current consultation and engagement processes as either excellent or good.» 25 respondents follow the National Principles for Public Engagement when undertaking consultation/engagement.» When compared to the baseline survey there has been a noticeable increase in the number of respondents who are now using the internet to provide feedback.» When asked why they do not use social media many respondents explained that it is due to an organisational policy; while others viewed that it was inappropriate for their type of consultation.» 11 respondents reported that they have been involved in partnership working and the majority (9) rated it as either excellent or good.» When compared to the baseline study the number of people that already use a Panel has doubled.» The majority (20 out of 25 respondents) were aware of the Citizens Citizens Panel.» When compared to the baseline study there has been an improvement in how respondents rated their ability to undertake online consultation.» The majority of respondents (21) said they had received training. This is a significant increase compared to the baseline study.» The majority of respondents stated that they were aware of the Project and that they had heard about it through word of mouth; through talks, presentations and s; and through being directly involved with the development and implementation of the project.» The majority of those who had heard of the Project reported using the Cwm Taf Hub to promote their consultation and engagement activities including online surveys; focus groups; and corporate budget setting exercises. Those who had not used the Cwm Taf Hub said that they were either unaware of it or were not undertaking any consultation.» When asked overall how well the Project has improved consultation and engagement activities the majority of those reported with very well (7) or fairly well (15) with just 3 respondents stating not well.» Examples of how the Project has improved consultation and engagement activities included that the project enabled them to access more experienced staff; and that the Project had improved collaborative working between different agencies and therefore decreased duplication. 31