Top Ten Tips Series: Online Surveys

Transcription

1 Top Ten Tips Series: Online Surveys Definition of a good online survey: A good quality online survey is well structured, provides representative, actionable, clear and concise information that will guide your professional membership organisation or association offering. The response rate will be typically high and so will the quality of data generated. Advice Sheet Content: Introduction Online Surveys Our Top Ten Tips Which online provider should my organisation use? MemberWise Online Survey Service Outsourcing online surveys to independent research companies Introduction The format and delivery of membership surveys has changed dramatically over recent years. Let s go back in time for a moment Less than ten years ago professional membership organisations and associations sent out hardcopy surveys to their members/stakeholders. The process included: Drafting and (sometimes professionally) printing the survey Posting to members (sometimes with a pre-paid envelope to increase the response rate) A sometimes lengthy wait for members to respond Scanning of completed surveys with often clunky software Interrogating the data manually Compiling the results and drawing up of graphs Publishing a formal survey results document The activity included at least seven separate processes that were often costly in terms of time and money. This resulted in many organisations only administering surveys once every year or two. Impact: Results sometimes redundant before being published Level of engagement limited by logistics Sample size limited due to financial cost of delivery Quality of results sometimes dependent on allocated budget Survey software came next and for a number years this made the process easier, however included expensive and clunky software that required consultancy and was sometimes not that dependable. Times have changed thank goodness!

2 A new era of online data gathering and member engagement Survey creation and delivery has changed dramatically with the introduction of new online survey (or e-survey) tools. It is now quick, easy, and more cost effective to regularly gather member, non-member and stakeholder opinion and the margin of error has also decreased. Live results can be accessed and interrogated via a few clicks, questions can be changed if needs be and results can be published live online with simple plug-in html modules. We calculate that the processes involved in creating, user-testing, delivering and reporting results have been reduced by 50%, total time taken to administer and report has been reduced by more than 75% and financial costs have been reduced by over 90%. Although this all sounds great you should remember that online surveys are only good if your members have easy access to a computer. Online Surveys Our Top Ten Tips Tip No 1: Clearly define the purpose of your online survey What is the main purpose of the survey? What information do you wish to extract from the results? We recommend that you compile a draft version of your survey in Word format and circulate to appropriate colleagues to ensure questions are relevant, well structured and fit-for-purpose. Consider asking all main department/function heads for their top questions and ensure they discuss with their respective teams. Tip No 2: Keep your online survey as short as possible Try not to use more than 15 questions as a rule and consider (if appropriate) incentivising responses. The use of matrices will help you to condense similar questions if needs be. If it looks as though your survey is turning into a bit of beast (i.e. too many questions) perhaps a stand-alone survey is required to address a particular issue. E.g. competitor analysis, readership survey or new product/service user-testing feedback. Tip No 3: Try to use as many closed questions as possible - This will ensure results are easily quantifiable and will save you time when evaluating the results! Closed answer questions by their very nature are the easier to answer than open ones. The results are easy to interpret and communicate from live and final results too. E.g. 85% of members want to continue to receive a hardcopy version of the monthly magazine as opposed to an e-journal. We recommend that you communicate results in graphical format as many online survey providers offer export functionality in Excel, PowerPoint and PDF format. Results will be easy to understand and should give a clear indication of opinion. Qualitative results take longer to analyse, however can provide a deeper understanding of what your members are thinking. We strongly recommend having at least one open-ended question (perhaps at the end of the survey) to enable members to convey their own personal thoughts, feelings and

3 suggestions. These answers can sometimes be priceless and can help improve member engagement. Some online survey tool providers enable the tagging of open answer questions to allow easier analysis. Tip No 4: Keep you questions simple and on message Nobody likes reading questions that are full of waffle. Nobody likes reading questions that are full of waffle, so get straight to the point and don t ask questions for the sake of it. Tip No 5: If you decide to use a rating scale be sure to use the same scale throughout If you use the same scale throughout your survey it will enable members to complete the questions quicker and so will lessen the likelihood of non-completion. Some websites do recommend changing the scale length and order to keep respondents alert, however we believe that member surveys are not the same as cold customer surveys. Tip No 6: Keep the structure of your survey logical and personalise as much as possible Structure surveys in a progressive manner. For example, if you are surveying new members about customer experience we recommend that you start by asking for the member s initial reason(s) for wanting to join, progress to questions that relate to their membership application experience asking for feedback on the literature received, and then finally ask if expectations have been met or exceeded. Most reputable survey providers enable personalisation in terms of fonts and branding. Make sure you personalise as much as possible and try using a colour scheme that is sympathetic to your corporate style. Tip No 7 - User-test, user-test and user-test again before your send out your survey! There is nothing worse than knowing that you have sent something out that is either inaccurate, incomplete or it simply doesn t make sense. Not only will your members find this disengaging but you will not get fit-for-purpose results to analyse. We recommend that you a draft survey to your colleagues before launch. Tip No 8 - Ensure the survey is relevant to your audience and fully inclusive. Your survey should be relevant to your audience you are sending the survey out to. Consider either using skip logic or sending out multiple surveys. Different segments are likely to include Student Members, Practicing Members, Retired Members, members with special interests and non-members. You can ensure the survey is fully inclusive by asking colleagues to take on the role of different member profiles when user-testing. E.g. student member, practicing member, retired member. Tip No 9 If you can send out the survey via the tool you should do so Make the most of the functionality of your online survey tool - after all you have paid for it! This functionality will enable you to send out reminders and track responses if needs be. It will also provide the opportunity for members to opt-out. We recommend that you give members at least three weeks to complete the survey. Reminders should be sent in weeks 2 and 3, and it goes without saying that you should communicate the closing date prominently in all correspondence.

4 Tip No 10 - Consider offering an incentive This tip is slightly subjective in that you would hope that your membership would willingly respond to a survey, however if you are surveying non-members or have purchased contact addresses of non-members you are likely to get a better response. We recommend that incentives should appeal to the profile of your audience. A popular incentive is high-street vouchers. E.g. M&S vouchers. Which online provider should my organisation use? We have user-tested three of the most widely known suppliers: SurveyMonkey - the most well-established provider that is used by a large number of well-known membership organisations and associations in the UK and overseas. Result: Good value for money. Zoomerang! - the lesser known competitor that is used by a smaller number of membership organisations and associations in the UK and overseas. Result: Good value for money, increased functionality compared to SurveyMonkey, and the option to send mobile phone surveys. QuestionPro a well established provider that is used by a wide range of organisations (including academic institutions) Result: The most cost effective (i.e. free) in return for a mention on your homepage See the Online Survey Tools section of the MemberWise website for a comprehensive list of providers. MemberWise Online Survey Service MemberWise subscribers can take advantage of our independent online survey service. Our MemberWise subscriber-only service includes: Plus: Aims, objectives and question generation consultancy Survey compilation, branding and user-testing Results analysis and reporting A one-year free Zoomerang! online survey account (worth approx. 100) ½ day training session enabling you and your organisation to get to grips with the tool This MemberWise service is available from only 480 (including VAT) To find out more surveys@memberwise.org.uk.

5 Outsourcing online surveys to independent research companies We have negotiated discounts with a range of reputable research companies and online survey tool providers. See the Research Company section of the MemberWise website for further information. Notes: All MemberWise Advice Sheets are compiled with the help of professional membership organisation and association professionals, and are constantly evolving. If you would like to submit amendment or additions to this advice sheet please submit to myadvice@memberwise.org.uk