Conducting a Tenant Satisfaction Survey Key Issues

Transcription

1 Conducting a Tenant Satisfaction Survey Key Issues Produced by PPCR Associates

2 2 Introduction I have set out below the major key issues which we would normally include in a tender for conducting a resident satisfaction survey. Typical Terms of Reference 1. The survey should provide a clear opportunity for residents to identify key issues affecting service delivery, particularly quality and performance. 2. The survey needs to be more than just a popularity poll and should examine the attitudes, perceptions and experiences of residents and identify priorities for future service policies and improvements. 3. Specifically, the survey needs to focus on obtaining views about key service areas which will cover at least the following: attitudes and opinions about the quality and effectiveness of housing management and specific services such as repairs, major works, tenancy matters, financial matters, advice services and any services provided through service charges views on residents perceptions of value for money can be determined for individual services, as well as for the service as a whole

3 3 views on staff helpfulness and efficiency in dealing with telephone enquiries, reception enquiries, response of staff and action taken the quality and usefulness of information levels of unmet need and future aspirations identify specific departments/sections or areas of service delivery that residents think underperform general satisfaction levels with the performance of the organisation as a landlord 4. Assessment of resident views can also highlight whether current standards reflect the requirements that customers have of services and also evaluate preferences for particular services. Conjoint analysis techniques can be used to provide a measure of the relative importance of one service as opposed to another. 5. The survey findings should establish a benchmark against which it would be possible to draw up and introduce measures for monitoring service performance. 6. The survey sample will need to be representative of the resident population; the range of housing eg. range of housing types; and the spread of housing stock.

4 4 Key Issues The successful achievement of the underlying survey requirements requires that a number of key issues must be addressed in the design, organisation and implementation of the survey approach. The survey should involve effective contributions by residents in assessing the quality and performance of services and identifying priorities for future improvements. The survey should be representative of the resident population: by age, gender, ethnicity, household type, property type, disability, and any geographical variables. Questions on demography will achieve two purposes: to allow for details on the main subgroups identified through the questionnaire to be analysed separately in the survey findings. to ensure that the survey returns are effectively monitored so that the return samples are representative. The survey approach should be designed in such a way that it seeks to inform as well as elicit opinions and views. The survey processes must avoid raising expectations which are later dashed and further create undue delay between identifying issues for action and their implementation.

5 5 General Approach Resident Consultation PPCR s approach to conducting research in the public and social housing sectors is outlined briefly below: Quantitative Consultation In our view there is no off the peg model of quantitative consultation and the methods used depend upon the type and detail of information required, sample size, design and the selection and range of services to be covered. A variety of questionnaire survey methods can be considered to obtain quantifiable data. These include: postal questionnaire survey self-completion questionnaire survey with assisted callback personal interview survey Our approach to these methodologies is outlined below: Postal Questionnaire Survey A good response rate is vital if the results are to be representative and worthy of analysis. Traditionally the response rate to postal questionnaire surveys is in the region of 30%-35%. However, with careful consideration given to certain key factors this can be increased to 50%-60%. Appropriate questionnaire design is the initial issue to be addressed.

6 6 Careful use of reminders and follow-ups should increase response rates. The questionnaire needs to look attractive and interesting, be easy to follow and to complete. Other factors that will influence responses include: - the relevance and interest of the subject matter. - first impression given by the initial mailing. - persuasiveness of the accompanying letter. - reminders and follow-ups. Self-completion questionnaire survey with assisted call-back PPCR has developed this unique method of conducting quantitative studies. We call this Interactive Selected Assistance (isa). It is particularly useful in obtaining information from a large number of residents. The method requires personal distribution and collection of questionnaires by trained fieldwork staff and allows for some degree of interaction between the fieldwork team and the respondents. Design and construction of the questionnaire are crucial factors in this method, as the majority of questionnaires are completed by respondents themselves. In order to ensure that the response is representative, returns are monitored and selective call-backs or booster samples of particular groups can be carried out. In addition, further information, assistance with completion or translation services can be provided on call-backs. This method enables an inter-active element to be included in the

7 7 quantitative research the questionnaire itself provides respondents with information upon which they can make informed choices and express informed opinions. The method produces results that are as accurate as a personal interview type survey. Personal interview survey This approach is particularly useful when a large amount of detail is required from respondents. Respondents within the sample are interviewed by trained fieldwork staff. Prior to conducting any interview each person/household within the sample should be contacted to inform them that they have been included in the sample and that a fieldworker will be calling to conduct an interview PPCR are experienced in achieving required response rates. In the recent personal interview survey conducted for Thames Valley Housing Association a sample survey involving 1,000 tenants and leaseholders was used, a response rate of 70% was achieved within a total of four weeks fieldwork. Sampling One of the first questions to be addressed in any quantitative survey is the sampling methodology, viz, who should be interviewed and how are such people selected. Some of the issues in the questionnaire will relate to an individual, whereas others will cover household details demographic data being an example of the latter. Many questions will be opinion

8 8 orientated and subgroup views will be gathered. For these and other reasons it is vital to achieve a representative sample of all different types of residents so that aggregate opinion mirrors that of the respondent population as a whole. Selecting the sample also has to take account of the need to have large enough sub-samples to examine the views of particular groups. To generate such a sample one has basically two options: a quota sample or a random sample. Each of these is now considered in turn: Quota sample This methodology is based on the principle of setting interviewers targets for their quota of respondents based on demographic and other factors. If properly applied such an approach can generate a reliable sample in a cost-efficient manner. The adoption of a quota sample will depend on the nature and accessibility of recorded information held by client organisation. Random sample As the term implies, this approach dictates that the household and individual for interview are selected entirely at random. Households and individuals are selected by a random start equal interval number system from relevant data. In this manner a representative sample is ensured, provided a good response rate is achieved. Whichever sampling technique is used to draw a sample from tenancy records we would carry out a minimum of four call

9 9 backs at each address to attempt an interview. Sample size It is essential that the sample size should be such that generalisations about the respective sample populations are meaningful. The number of respondents to be included should be decided on the basis of achieving statistically valid returns for all subgroup variables identified in the brief. We normally want to express results at a 95% confidence level and therefore avoid wide margins of error which occur when main or sub sample returns are less than 100 responses. Confidence levels/margins of error Margins of error in any survey findings concern the comparison in numerical terms, between the survey population and the size and selection of the sample. The key questions here concern defining the survey population; eg. age range, geographic location, ethnicity, class etc, constructing a representative sample; and type of sampling to be used. Once these questions have been answered, it is possible to assess confidence levels or margins of error in making a comparison between the sample and the survey population. The margins of error are calculated on the basis of a 95% confidence level using the following table:

10 10 Range of error ( or-) for 95% confidence level Percentage found by survey Sample 5%or95% 10%or90% 20%or80% 30%or70% 50% size , , This table can be used to calculate the margins of error in the sample as a whole and for the subgroup analysis eg. at a 95% confidence level we can be sure that for a sample value of 30% in a random sample of 500 we can say that the margin of error is approximately within + or - 4.1%, i.e. between 26% and 34%. Quality Control In conducting any survey we ensure that all households/individuals within the sample would be visited a minimum of four times in order to obtain an interview or to make contact. In addition to day time calls they would also take place in the early evening and at weekends. Records are kept of all attempted visits and also of those refusing to participate in the survey. If possible, our fieldwork team would obtain a reason for any person refusing to take part, as these can often be included as an element of the consumer s view.

11 11 Fieldwork Resources As a research consultancy that conducts assignments throughout the country we have experienced regional fieldwork teams in certain regions. For London and the surrounding area, we have an experienced mixed gender team of 20 personal interviewers. Interviewer training for each assignment includes: familiarisation with the subject matter confidentiality monitoring and administration written instructions presentation and appearance equal opportunities Confidentiality It is important that the confidentiality of individual respondents is respected. In all our surveys we enable responses to be identified by area but not by specific addresses. PPCR is registered under the Data Protection Act. All PPCR qualitative and quantitative research assignments are conducted under the auspices of the Social Research Association Code of Ethics. Equal Opportunities it is fundamental to our approach that any survey process must obtain

12 12 views from all sectors of the community. All PPCR fieldwork teams are constructed on the basis of matching the population to be surveyed. If required, PPCR can provide translation and interpretation facilities. Questionnaire Design and Length It is inherent to our approach that the design and construction of the questionnaire should be carried out in consultation with the client. Our expertise in questionnaire design will be used to produce a workable final questionnaire based on the range of issues to be examined. Questionnaire length and type of question are the crucial elements in assessing the appropriate methodology to be adopted and the costing of an assignment. In considering the possible subject areas supplied by the client we would design an appropriate questionnaire and recommend a preferred methodology. Qualitative Consultation (Optional) We would welcome the opportunity of conducting focus group meetings both before and after the quantitative phase. Our overall approach to conducting qualitative research is outlined below: Qualitative research is increasingly recognised as a powerful research tool in its own right. With careful selection of small valid samples of service providers or consumers and the use of

13 13 structured and semi-structured methods of conducting interviews or focus group meetings, it is possible to carry out a detailed information gathering exercise into issues relevant to a geographic area or range of services. A number of objectives can be achieved. Qualitative research can examine in detail, individual attitudes and perceptions, so important in identifying group concerns and barriers to quality services. This process involves a participative dimension by ordinary service users; providing a resident input into questionnaire construction and enabling resident views on solutions and priorities for action to be identified from the survey findings. Ron Houston, PPCR