Community Survey Tool:

Transcription

1 Community Survey Tool: Template and Guidance Support for councils completing Template 1 Prepared by Jetty Research for the NSW Office of Local Government April 2015

2 Table of Contents DISCLAIMER & COPYRIGHT... 3 EXECUTIVE SUMMARY... 4 Table i: Recommended Standard Survey Instrument... 4 Table ii: Recommended Standard Methodology... 6 Guidelines: How to implement and interpret the results... 7 PART 1: INTRODUCTION ABOUT THIS GUIDE BACKGROUND Why is understanding of the community response important? Should I conduct a community opinion poll? What information do I need to gather from the community?... 8 PART 2: STANDARD SURVEY TOOL Table 2.1: Standard Survey Tool PART 3: STANDARD SURVEY METHODOLOGY SAMPLE SPECIFICATION AND SCREENING EXCLUSIONS AND QUOTAS METHOD FOR CONTACTING RESIDENTS Table 3.1: Methodology types and advantages/disadvantages SAMPLE SIZE Graph 3.1: Sampling error variation by population size Table 3.2: Respondents needed for 95 per cent confidence level TIMEFRAME Table 3.4: Timeframe for survey implementation ANALYSIS APPENDIX A: REQUEST FOR QUOTE Community Opinion Standard Survey Tool Page 2

3 Disclaimer While all care and diligence has been exercised in the preparation of this report, Jetty Research Pty. Ltd. does not warrant the accuracy of the information contained within and accepts no liability for any loss or damage that may be suffered as a result of reliance on this information, whether or not there has been any error, omission or negligence on the part of Jetty Research Pty. Ltd. or its employees. Copyright & appropriate usage The NSW Office of Local Government owns the intellectual property for this survey tool and the accompanying guidance material. NSW councils may use this survey to inform the preparation of Template 1 for the purpose of Fit for the Future proposals. Councils should only amend this survey where it is indicated (eg inserting the Council name and merger options under consideration in the relevant places). Any other amendments may invalidate the legitimacy of the survey results. Community Opinion Standard Survey Tool Page 3

4 Executive summary Understanding the views of the community is important for councils considering a merger. This document outlines a recommended standard survey tool and standard methodology for the collection of community opinion to support the completion of Template 1. Table i below outlines a summary of the standard survey tool and Table ii outlines a summary of the standard methodology. Comprehensive outlines of the Standard Survey Instrument and Standard Methodology are available in Parts 2 and 3 respectively. Note: The table below of standard survey tool questions is intended to act as a brief summary of the questions. The full survey tool - including questions and answer codes - is available in Part 2. Table i: Recommended Standard Survey Instrument Category Screening and demographics Question 1 Do you live within the XXX Local Government Area? 2 And in what capacity are you answering? (Tick any that apply) 3 OPTIONAL: Are you a local Councillor or Council employee within this Local Government area? 4 OPTIONAL: And have you lived in [INSERT AREA] for at least the last 6 months? 5 And into which of the following age ranges do you fall? 6 Please confirm your gender for the record 7 And which suburb/town/village/locality do you live in? 8 OPTIONAL: How many people live in your household? 9 And do you rent or own your home? 9 Do you speak a language other than English at home? Drivers of local identity and sense of community Involvement with local council Current satisfaction with local council 10 Are you a ratepayer within the Council area? 11 What do you like MOST about living where you live? 12 I d now like to ask you about the types of activities you undertake in your local area. Can you tell me in which suburb you undertake each of the following; Grocery shopping Other shopping Work Leisure activities (cinema/dining) Recreation activities (exercise /sport) Health services Education 13 Can you tell me the name of your local Mayor? 14 Frequency of visitation to Council Chambers or Administrative Centre 15 Have you contacted your Council over the past 12 months for any reason apart from paying rates? 16 And have you contacted any of your elected councillors or the mayor over the past 12 months? 17 Evaluate Council services: Using a scale of 1-5, where 1 is very poor and 5 is excellent, how would you rate your local Council on the following? The quality of services it delivers to local residents Its financial strength Community Opinion Standard Survey Tool Page 4

5 Awareness and opinion regarding local government reform 18 Its maintenance of local infrastructure The way it consults with the community The way it plans and delivers services for the local community The way it lobbies for the local region The way it works with other levels of government to plan and deliver services for the local community The way it plans and provides for the residents of the future The way it carries out its regulatory functions, for example processing development applications, parking controls, cat and dog management The way it works with businesses in the LGA 19 (optional) Do you feel your Council currently plays an important role in fostering the economic development of your Local Government Area? 20 (optional) And do you believe this should be a Council responsibility? 21 (optional) If the Council were to merge with x/y, do you think this would improve economic development opportunities for business in the region? 22 (optional) And If the Council were to merge with x/y do you think this would reduce the impact of regulation on business in the region? 23 As you may know, the NSW State Government is looking at ways that councils can better serve their communities and be sustainable for the long term. This includes asking councils to consider voluntary mergers and other reforms. Prior to this call, were you aware of this local government reform program? 24 If YES, where did you hear about it? 25 And how interested are you personally in the subject of Council reform? 26 [INSERT COUNCIL] has a number of options available including [Outline the options available to your LGA] Which is your preferred option? Option 1 Merger option 1 with council/s x and y Option 2- Merger option 2 merging only with council x (etc. etc.) Not merging with another Council Really don t care 27 And what is the main reason why that is your preferred option? 28 And which is your least preferred option? Option Benefits and concerns regarding a merger Communications regarding a possible merger 29 And what is the main reason why that is your least preferred option? 30 (IF not merging or don t care to Q26) If the decision was made to merge with another Council, which option would you prefer? Option 1 Merger option 1 with council/s x and y Option 2- Merger option 2 merging only with council x (etc. etc.) 31 What do you see as the major advantages of your council merging with one or more neighbouring councils? (Unprompted/Open answer- allow up to 3) 32 And what do you see as the major disadvantages? (Unprompted/open answer allow up to 3) 33 You have stated [1], [2] and [3] as the major disadvantages of a merger. Please rank those three issues according to priority. Which of those three do you believe is the greatest concern? 34 To what extent would you agree or disagree with the following statements relating to the creation of a new council through a merger? A new Council would deliver a broader range and better services A new Council would make it harder to adequately represent residents A new Council would be more financially sustainable A new Council would change the character of my suburb/town/village/locality 35 (optional) What would you see as the single biggest benefit arising from having a new Council? 36 (optional) And what would be your single biggest concern about having a new Council? 37 (optional) We are looking to understand the ways in which the community prefers to be involved in community consultation over the issue of a possible merger. How would you prefer to be consulted about this issue? 38 (optional) What do you believe would be the best ways for [INSERT COUNCIL] to communicate information about these local government reforms to residents? Community Opinion Standard Survey Tool Page 5

6 Table ii: Recommended Standard Methodology Methodology component Sample Recommendation LGA residents as per postcodes, wards etc. Not Council representative (councillor) Over 18 years old Representative of LGA by age and gender 1 Sample size Where population <10,000 = 400 Where population >10,000 = 500 (To achieve a sampling error of <5%) Contact method Questionnaire length Time frame Computer-assisted telephone interviewing (CATI) 15 mins when applied in full but dependant on options considered (may be reduced to mins) 4-5 weeks from contract signing to report delivery 1 Or at least aiming for a robust sample by age and gender, which can then be post-weighted to match ABS 2011 Census data for the LGA Community Opinion Standard Survey Tool Page 6

7 Guidelines: How to implement and interpret the results 1 Do 2 3 Appoint 4 you need to conduct a community opinion poll? Are you completing Template 1? If yes, you need to explain community perceptions of the benefits and concerns regarding a merger Decide on your survey parameters and issue an RFQ Pick your sample size (see Table 3.2) and define your sample (based on your LGA and the possible merger LGAs) Issue your RFQ to research providers (see for a list of providers) your research supplier Talk to your procurement specialist about how to select and appoint a research provider to undertake the data collection for your community opinion survey Agree the research specifications with your research provider Consider the cost and the degree of sampling error you are willing to bear based on Table 3.2) Finalise the sample size and demographic quotas with your research provider 5 Tailor your questionnaire Determine which optional questions you would like to include and talk to your research provider about how the addition of optional questions will impact the timing and cost of your data collection 6 7 Leave your research provider to it Your research provider is skilled at turning a questionnaire into meaningful insights They will give you frequent updates on how the research is going Interpret the results Your research provider will tell you what the community insights mean for your merger Incorporate the findings into Template 1 and consider what the results mean for your communications Community Opinion Standard Survey Tool Page 7

8 Part 1: Introduction 1.1 About this guide This guide will assist you to implement a survey instrument to support the completion of Part 3 of Template 1. Specifically, those Councils intending to merge with another Council are asked to gather an understanding of community opinion regarding the benefits and concerns around a merger. This guide outlines a standard survey tool which can be tailored to your Council, the methodology required for gathering the survey data in a statistically reliable and cost-effective way, and guidance material which outlines all of the information you need to undertake the project. 1.2 Background Why is understanding of the community response important? Understanding how your community will respond to a merger is important for a number of reasons. First, it will assist you to put processes or communications strategies in place to address any concerns. Second, knowing your community will assist you to tailor your communication and education material to suit them. Finally, it is important that the community feel heard and involved in the merger process. By consulting with them and communicating an understanding of their opinion, your community will feel more open, engaged and supportive of your decisions rather than feeling they are having something done to them. Template 1 asks councils to research the community response to a possible merger. A methodologically sound, primary research project which will gather intelligence about the community response to a possible merger to inform the councils Fit for the Future proposal. This document outlines a standard survey tool, methodology and guidance material to assist you to implement a best practice community opinion poll Should I conduct a community opinion poll? Before you work though this guide it is important to be clear about your need to conduct a community opinion poll. To help you decide, see Section 3 of the Template 1 Guidance material on the Fit for the Future website What information do I need to gather from the community? Part three of Template 1 asks you to demonstrate your understanding of the community s perception of the benefits a merger may offer and concerns regarding a merger. In addition, it is of value to understand a number of other aspects including their level of engagement with the community and Council, baseline ratings of Council service provision and aspects important for driving a sense of community. Community Opinion Standard Survey Tool Page 8

9 Overall, the standard survey tool addresses the following research objectives: Assess awareness of, and attitudes towards the local Council s position regarding a merger; Understand drivers of local identity and sense of community and impact of local Council on these aspects; Determine confidence in your Council s scale and capacity to effectively advocate for the local region and engage with other levels of government to plan and deliver outcomes for the local community; Evaluate perceptions of satisfaction towards local Council s; a) provision of services, b) maintenance of local infrastructure; c) quality of community engagement; and d) planning and providing for its residents in the future; Scenario test the benefits and costs of a merger for example, would the community look more favorably on a merger if it meant that local services, infrastructure and planning could be delivered to a higher standard or at a reduced cost? Determine current engagement and communication with Council and perceptions regarding how this engagement and communication might change as a result of a merger; and Understand support for/opposition to mergers vis. a vis. other specified options. Community Opinion Standard Survey Tool Page 9

10 Part 2: Standard Survey Tool The following section outlines the standard survey tool. The tool allows for the exploration of community opinion regarding a merger with one or more Councils. It is all-encompassing and outlines all questions relevant to the topic including some optional questions which are noted as such. It is appropriate to use the full survey tool and it is designed to run in the question order presented below to avoid any sequencing bias. There is opportunity to tailor some elements of the survey to make them applicable to your Council situation (for example, the number and names of Councils being considered for a merger). However, most questions should remain as is. The length of the survey will depend on the merger options being considered, and the number of optional questions included. When all of the questions below are applied, the survey will take approximately 15 minutes to complete. However, this may be reduced to minutes if some questions are deemed unnecessary in your specific circumstance. The standard survey is designed for residents. In some circumstances, the sample will also capture people who have businesses in the LGA. In these cases, some business-specific questions are relevant. Community Opinion Standard Survey Tool Page 10

11 Table 2.1: Standard Survey Tool Category Question List options Answer options Rationale Screening 1 Do you live within the XXX Local Government Area? Yes 2 And in what capacity are you answering? (Tick any that apply) 3 OPTIONAL: I just have a few quick qualifying questions. Firstly, are you a local Councillor or permanent Council employee within this Local Government area? 4 OPTIONAL: And have you lived in [INSERT AREA] for at least the last 6 months? 5 And into which of the following age ranges do you fall? No I am a resident I own or manage a business I work in the Local Government area I am a volunteer Other Yes No Yes No plus Useful cross-tab, plus provides opportunity to ask additional questions of business owners/managers later in survey May wish to exclude these May wish to exclude new residents Useful cross-tab 6 Please confirm your gender for the record Male Female 7 And which suburb/town/village/locality do you live in? List of possible suburbs/towns/villages/localities in the LGA 8 OPTIONAL: How many people live in your household? Number 9 And do you rent or own your home? Rent Own / part own Rather not say 10 Do you speak a language other than English at No home? Yes (please specify) 11 Are you a ratepayer within the Council area? Yes No Useful cross-tab May be of interest to councillors/managers, and may allow cross-tabs by ward Useful cross-tab (e.g. sole occupant/multiple person) Useful cross-tab Useful cross-tab Community Opinion Standard Survey Tool Page 11

12 Drivers of local identify and sense of community 12 What do you like MOST about living where you live? (unprompted probe for up to THREE. Research company to pre-code likely options plus other ) Pre-coded list examples might include: Natural environment Sense of community/friendly place A safe place to live Good place for families and children Lots of community events Sport and recreation opportunities Balanced lifestyle Active lifestyle Parks, playgrounds, walking trails and open spaces Access to natural parks Access to rural/peri-urban lifestyle Public transport Great council Friends and family OTHER Enables local and state government to understand exactly what residents value about their local area, and (hence) which are and aren't LGA-specific Involvement with local council 13 I d now like to ask you about the types of activities you undertake in your local area. Can you tell me in which suburb you undertake each of the following; 14 Can you tell me the name of your local mayor? (open ended 15 How often, on average, would you visit your local Council Chamber or service centre? 16 Have you contacted your Council over the past 12 months for any reason apart from paying rates? 17 And have you contacted any of your elected councillors or the mayor over the past 12 months? Grocery shopping Other shopping Work Leisure activities (cinema/dining) Recreation activities (exercise /sport) Health services Education List to include the suburbs in the LGA plus City and other Answer correct Answer incorrect Unsure/don t know Monthly or more Three to four times a year A couple of times a year Once a year or less Yes No Unsure Yes No Unsure Understand local engagement and involvement Designed to understand level of engagement with local government Designed to understand importance of having such a centre nearby Designed to understand level of involvement with local council. Also a potentially useful cross-tab Designed to understand level of involvement with local council. Also a potentially useful cross-tab Community Opinion Standard Survey Tool Page 12

13 Current satisfaction with local Council 18 Using a scale of 1-5, where 1 is very poor, 3 is neutral or unsure and 5 is very good, how would you rate your local Council on the following? The quality of services it delivers to local residents Its financial strength Its maintenance of local infrastructure The way it consults with the community The way it plans and delivers services for the local community 1 Very poor 2 Poor 3 Neutral/unsure 4 Good 5 Very good Provides broad measures of satisfaction with different aspects of current council. 19 (Optional) Do you feel your Council currently plays an important role in fostering the economic development of your Local Government Area? 20 (Optional) And do you believe this should be a Council responsibility? 21 (optional) If the Council were to merge with x/y, do you think this would improve economic development opportunities for business in the region? 22 (optional) And If the Council were to merge with x/y do you think this would reduce the impact of regulation on business in the region? How? The way it lobbies for the local region The way it works with other levels of government to plan and deliver services for the local community The way it plans and provides for the residents of the future The way it carries out its regulatory functions, for example processing development applications, parking controls, cat and dog management. The way it works with businesses in the LGA Yes No Unsure Yes No Unsure Yes No Unsure Number of inspections Permits required General red tape Other For those councils running economic development activities, understanding business response to this role For those councils running economic development activities, understanding business response to this role Understand how business perceive that the business orientated Council service might change as a result of a merger Community Opinion Standard Survey Tool Page 13

14 Awareness and opinion regarding Council reform Awareness and opinion regarding Council reform 23 As you may know, the NSW State Government is looking at ways that councils can better serve their communities and be sustainable for the long term. This includes asking councils to consider voluntary mergers and other reforms. Prior to this call, were you aware of this local government reform program? 24 If YES, where did you hear about it? Newspaper Radio Social Media Community notice board Friends / neighbours / word of mouth Council Website Other 25 And how interested are you personally in the subject of Council reform? 26 [INSERT COUNCIL] has a number of options available including [Outline the options available to your LGA] Which is your preferred option? (prompted except unsure or don t care ) 27 And what is the main reason why that is your preferred option? (unprompted) List likely options, plus "other" Yes No Very interested Quite interested Not very interested Not at all interested Option 1 Merger option 1 with council/s x and y Option 2- Merger option 2 merging only with council x (etc. etc.) Not merging with another Council Unsure Don t care 28 And which is your least preferred option? Option 1 Stay as is Option 2 Merger option 1 with council/s x and y Option 3- Merger option 2 merging only with council x (etc etc) Really don t care 29 And what is the main reason why that is your least List likely options, plus "other" preferred option? (unprompted) Measure awareness of FFF. Potentially useful cross-tab Understand how locals are receiving information regarding mergers Measure interest in FFF. Potentially useful cross-tab (prob. x 2: very/quite vs. not very/not at all) Understand preference for various models Understand reasons for preferences Understand preference for various models Understand reasons for preferences Community Opinion Standard Survey Tool Page 14

15 Option Benefits and Concerns regarding a merger Option Benefits and Concerns regarding a merger 30 [if Q26 = Not merging or Don t care ) If the decision was made to merge with another Council, which option would you prefer? (prompted except unsure or don t care ) 31 If one of the merging options were chosen, what would you see as the major advantages of your council merging with one or more neighbouring councils? (Unprompted/Open answer- allow up to 3) 32 And what would you see as the major disadvantages? (Unprompted/open answer allow up to 3) It will unify our region Lower rates More influence to attract funding or attention from State and Federal governments Less bureaucracy/waste Simplify or reduce the layers of government Better facilities and/or services for the area Fewer politicians/councillors Increased community diversity None/can't think of any Other Benefits unknown/can't see benefits/unnecessary A bigger council will be less efficient/too bureaucratic Not needed/happy with the way things are Make Council performance worse Doubt about cost savings/waste of time and/or money Delays or inefficiencies during the transition Local issues will overlooked/loss of local identity Loss of representation/loss of voice Mergers have not worked in other places Resource shifting -our rates being spent elsewhere None/can't think of any Other Option 1 Merger option 1 with council/s x and y Option 2- Merger option 2 merging only with council x (etc. etc.) Unsure/don t care Allows measurement of unprompted/top-of-mind assessment of the possible advantages of council mergers Allows measurement of unprompted/top-of-mind assessment of the possible disadvantages of council mergers Community Opinion Standard Survey Tool Page 15

16 33 You have stated [1], [2] and [3] as the major disadvantages of a merger. Which of those three do you believe is the greatest concern? Benefits unknown/can't see benefits/unnecessary A bigger council will be less efficient/too bureaucratic Not needed/happy with the way things are Make Council performance worse Doubt about cost savings/waste of time and/or money Delays or inefficiencies during the transition Local issues will overlooked/loss of local identity Loss of representation/loss of voice Mergers have not worked in other places Resource shifting -our rates being spent elsewhere None/can't think of any Option Benefits and Concerns regarding a merger 34 To what extent would you agree or disagree with the following statements relating to the creation of a new council through a merger? In each case we'll use a 5 point sliding scale, where 1 means you strongly disagree, 3 is neutral or unsure, and 5 means you strongly agree. A new Council would deliver a broader range and better services A new Council would make it harder to adequately represent residents A new Council would be more financially sustainable A new Council would change the character of my suburb/town/village/locality 1 - Strongly disagree Neutral/unsure Strongly agree Allows respondent to judge potential specific outcomes of a merged council Community Opinion Standard Survey Tool Page 16

17 Option Benefits and Concerns regarding a merger 35 (optional) What would you see as the single biggest benefit arising from having a new Council? (Unprompted code likely responses plus other ) 36 (optional) And what would be your single biggest concern about having a new Council? (Unprompted code likely responses plus other ) Reduced costs More efficient/better use of resources and services Better facilities and/or services for the region It makes sense Regional approach to issues (health, crime, public transport etc.) Reduce bureaucracy and reduce number of staff Less politicians/councillors Mergers have worked in other places It will unify the local area. Lower rates More influence to attract funding or attention from State and Federal governments Simplify or reduce the layers of government None/can't think of any Other Benefits unknown/can't see benefits A bigger Council will be less efficient/too bureaucratic Not needed/happy with the way things are Make Council performance worse Doubt about cost savings/waste of time and/or money Delays or inefficiencies during the transition Local issues will be overlooked/loss of local identity Loss of representation/loss of voice Mergers have not worked in other places Resource shifting -our rates being spent elsewhere None/can't think of Other Understand perceived personal benefits arising from a merger Questions explore perceived advantages and disadvantages of a merger at a broad level which is a requirement of Template 1. Questions are optional as they provide for further explorations of perceived benefits and concerns regarding a merger at a more personal level (thus are beyond the requirements of Template 1 but should be included where you a) have time in the questionnaire and b) wish to explore community opinion in more depth). Understand perceived personal concerns arising from a merger Community Opinion Standard Survey Tool Page 17

18 Communications regarding a possible merger 37 (optional) We are looking to understand the ways in which the community prefers to be involved in community consultation over the issue of a possible merger. How would you prefer to be consulted about this issue? (Unprompted code likely responses plus other ) Community surveys Council meetings Community workshops / focus groups Online discussion forums Talking with others in the community Contacting Councillors Writing a submission to Council Communicating through community group leaders Joining an advisory committee Other None Allows council to understand how community wishes communication and consultation to take place, and how this varies by factors such as age 38 (optional) What do you believe would be the best ways for [INSERT COUNCIL] to communicate information about these local government reforms to residents? (Unprompted code likely responses plus other ) Letterbox drop - flyers or brochures Personalised mail Local newspaper ads s Council newsletter Council website Social media Displays Rates notice Mayor's column Other Allows council to understand how community wishes communication and consultation to take place, and how this varies by factors such as age Community Opinion Standard Survey Tool Page 18

19 Part 3: Standard Survey Methodology This section outlines the recommended methodology to be applied when collecting the standard survey data to address the core research objectives. This methodology is based on best practice market research principles and considers the requirements for statistical reliability and robustness of the results. 3.1 Sample specification and screening The nature of the survey (and as outlined in Section 14 of the Local Government Act) indicates that the interested parties are the electors. Thus the sample includes any person who is living within the LGA of interest. Screening questions and quotas should be applied to ensure that the sample is representative of the population and reduces any potential bias Exclusions and quotas Surveying should be conducted on a purely random basis. However attempts should be made to ensure a robust mix of ages and genders, and that all parts of the LGA are adequately sampled. Residents should be asked a number of demographic questions including age, gender, household composition and whether languages other than English are commonly spoken in the home. These demographic characteristics should be monitored to ensure the sample collected is at least partially representative of your LGA. Any person likely to have a bias regarding the subject matter should be excluded to ensure the survey results are reflective of the residents opinions. Thus elected members are to be excluded. You may also decide to exclude those who have only recently moved into the area (e.g. have lived there for less than six months) as they may not yet have enough knowledge to provide an informed opinion. 3.3 Method for contacting residents There are a number of options typically used for contacting respondents regarding a community opinion survey. These options include; telephone, online, paper; face-to-face (street intercept or door-to-door) or by mail. This survey requires a geographically specific sample a representative sample of residents of a defined LGA. Table 3.1 below outlines the potential of each survey method for contacting residents. 2 Note that in practice this can be either (at best) cost prohibitive and (at worst) unrealistic given time and budget constraints. In most instances it would be acceptable to seek a robust mix of ages and genders, with the aim being to post-weight results to match ABS 2011 Census age and gender profiles for your LGA. Community Opinion Standard Survey Tool Page 19

20 Table 3.1: Methodology types and advantages/disadvantages TYPE Advantages Drawbacks Costs Data Quality Random CATI (telephone) Due to time constraints, does not allow considered thought High Highest (equal) Random face-toface Paperbased Online survey Results can be extrapolated to the wider community to a predefined margin of sampling error Sample size can be guaranteed Sub-sample size (e.g. by ward) can also be guaranteed Best at capturing the middle ground Same advantages as CATI Better than CATI at targeting younger residents (i.e ), CALD residents etc. Very inclusive everyone literate can have a say Enables visual cues/long lists of options Includes businesses No rush - allows considered responses Easier for people with English as a second language to take part Same benefits as paperbased, above Cheaper than paper Higher data quality than paper Can be interactive e.g. YouTube videos, links to background information, or graphics Not suitable for visual surveys or long lists of options Hard/expensive to reach residents due to low incidence of fixed line phones Will not cover businesses Not everyone can have a say Biased against non-english speakers Subject to location bias Hard to stop people for the minutes required May be subject to unconscious interviewer bias Can be difficult/expensive to gain interview permission in many ideal locations (e.g. train stations, shopping centres) Door-to-door may raise OH&S issues No guarantee on response rate (either overall or for specific wards) Tend to be completed only by those with strong views Results not representative due to self-selecting sample Data quality tends to be poor Expensive due to printing, mailing and data entry costs May bias against older/lower socio-economic residents (unless paper or phone option added) No guarantee on response rate Tend to be completed only by those with strong views Results not representative due to self-selecting sample Highest Medium Lowest Highest (equal) Lowest High Community Opinion Standard Survey Tool Page 20

21 Telephone and face-to-face interviewing are the only alternatives which allow a geographically specific, unbiased and random sample to be selected. Both online and paper/mail surveys are selfselecting (i.e. opt-in ) and, for this reason, not representative of the population in question. Faceto-face surveys tend to be extremely expensive due to the interviewer time required to both administer and re-visit residences. Therefore, to address the research objectives, residents should ideally be surveyed randomly by CATI (computer-assisted telephone interviewing). This allows a geographically specific, representative sample to be selected in a cost effective way 3. Phone number selection should be discussed with your research supplier typical sources include E-White Pages/commercial residential phone lists, random (suffix) number generation, commercial suppliers of (computer-proven) random valid phone numbers, or a combination of these. 3.4 Sample Size Sample sizes are typically chosen based on a number of considerations including; How large the target population is; How similar or different respondents/responses are likely to be; Whether you want to analyse data according to smaller sub-groups within the larger target population; The acceptable degree of random sampling error in your final results; What looks credible to an external stakeholder; and Budget and time. Sampling error refers to the degree to which the results differ to the actual but unknown value of the population on account of observing a sample rather than the real population (i.e. conducting a census). By minimising the sampling error, we ensure the results are within an acceptable margin of error to the actual but unknown value of the population. As graph 3.1 (next page) shows, margin of error falls as sample size rises. Hence sub-groups within the overall sample will typically create much higher margins for error. For example, considering a population size of 50,000, a sample size of 500 exhibits a margin for error of +/- 4.4% (at the 95% confidence level4). 3 The standard survey is designed for residents. In some circumstances, the sample will also capture people who have businesses in the LGA. In these cases, some business-specific questions are relevant. 4 Confidence level is the degree of confidence we have regarding the survey sample behaviours and opinions reflecting the overall population. A 95% confidence level means that if we conducted a similar poll twenty times, results should reflect the views and behaviour of the overall survey population to within a +/-x% margin in 19 of those 20 surveys.) Community Opinion Standard Survey Tool Page 21

22 Sampling error (at 95% confidence) Graph 3.1: Sampling error variation by population size 10% 9% 8% 7% How random sampling error varies with population size Jetty Research 2008 Pop = 5,000 Pop = 50,000 Pop = 20m 6% 5% 4% 3% 2% Sample size It is helpful to consult a sampling chart to estimate how many individuals to include to be confident your sample represents the target population. Typically, ± 5% is considered acceptable. Table 3.2 outlines the acceptable sample size based on different population bases. For population sizes over 5,000, the typical sample size required is between 400 and 600 depending on whether you are willing to accept a sampling error of ± 4% or ± 5%. To achieve a sampling error of ± 3% is often not cost effective the cost of collecting the required sample to achieve a sampling error of ± 3% is too high given the marginal benefit of ± 1%. Table 3.2: Respondents needed for 95 per cent confidence level Population Size ± 3% Sampling Error ± 4% Sampling Error ± 5% Sampling Error 2, , , ,000 1, ,000 1, ,000 1, ,000,000 1, NOTE: How to read this table ± 10% Sampling Error A. Determine how many people are in your target population and find the closest number (rounding up) in column 1, Population Size. B. Determine the level of sampling error you can tolerate: 3%, 5% or 10%. Sampling error indicates how much confidence you can have that your sample is representative of the larger population. Generally a 5% sampling error is selected. In other words, 95 times out of 100 this sample is representative of the larger population. C. Find the cell in which your population size and sampling error meet. For example: if there are 5,000 in your LGA you would need a sample size of 357 in order to be 95% confident in your sample. If there are 50,000 people in your LGA you would need a sample size of 381 in order to be 95% confident in your sample. Community Opinion Standard Survey Tool Page 22

23 You should consider the population base of all of the LGAs of interest. For example, if you are wishing to gain the opinion of a number of LGAs (in the case of a merger), you must collect a sample in each of the LGAs to ensure each LGA can be analysed and compared to the others with statistical reliability. In the case of a population size of less than or equal to 10,000, we recommend a sample size of n=400. In the case of a population size of greater than 10,000 we recommend a sample size of Timeframe This is subject to discussion between council and research supplier. However for CATI (telephone) surveys, polling is typically conducted over 5-6 nights (depending on call centre capacity and sample size selected). Residents are generally phoned from p.m. each weekday evening (possibly excluding Fridays), and (if necessary) on weekends from midday to 5pm. To maximise response rates, potential respondents should be called up to five times at different times of the afternoon and evening. Within reason, callbacks can usually be programmed to suit resident wishes. Table 3.4 outlines the likely timeframe for implementation of the survey. Table 3.4: Timeframe for survey implementation WHAT WHO HOW LONG Decide survey parameters (in particular sample size and likely survey length) and issue RFQ Council Determined by Council Appoint research supplier Council Determined by Council Define sample and agree sample size (post Council and Supplier 1 day codes and demographic makeup based on 2011 Census profile) Tailor questionnaire Council and Supplier 2-3 days Script CATI and source relevant numbers Supplier 1 day Fieldwork Supplier 5-6 days Analysis Supplier 5 days Reporting Supplier 5-8 days As outlined above, a survey will typically take approx. four to five weeks (20-25 business days) from contract signing to delivery of a report. This can vary. 3.6 Analysis As noted above, results are typically post-weighted to match the LGA s age and gender profile as at the 2011 ABS Census (Usual Resident Profile.) Analysis is generally conducted in SPSS, Q, or other statistical database software which allows detailed cross-tabulations and identification of significant differences (e.g. between LGAs or genders). Analysis should include comparisons to the other LGAs involved in the merger and should consider any meaningful differences between subgroups of interests (such as different demographic groups). Open-ended questions should be coded in an attempt to quantify any themes. Reports should include an executive summary and recommendations, designed not to exceed two pages. Community Opinion Standard Survey Tool Page 23

24 APPENDIX A: Request for Quote The following outlines the details required by a research provider within a request for quote to conduct data collection. Methodology component Sample Standard Requirement Council to make decision (will vary by Council area) Which LGAs to survey? LGA residents as per postcodes, wards etc. Not Council representative (councillor) Over 18 years old Representative of LGA by age and gender 5 Sample size Where population <10,000 = 400 Where population >10,000 = 500 (To achieve a sampling error of <5%) Contact method Questionnaire length Requirements CATI 15 mins when applied in full but dependant on options considering (may be reduced to mins) Data collection Reporting (no questionnaire design a standard instrument will be used) Consider optional questions do you need them? 5 Or at least aiming for a robust sample by age and gender, which can then be post-weighted to match ABS 2011 Census data for the LGA Community Opinion Standard Survey Tool Page 24