How To Be A SurveyMonkey Audience Pro at Writing Surveys

Transcription

1 How To Be A SurveyMonkey Audience Pro at Writing Surveys Impress your friends, colleagues and clients with your survey skills look no further for all the tips and best practices to create a great SurveyMonkey Audience survey

2 Table of Contents Survey Writing Guide Ask yourself WHY Stay relevant Stay focused Write the right question Speak their language Tell them when Stay balanced Keep it personal Be direct Write the right answers Use closed-ended questions Use your words Avoid yes / no questions Measuring pricing Measuring how much Provide a full set of answers Watch your ranges Matrix / Grid questions Specific Settings Guidelines Require all questions Randomize answer choices when possible or relevant Skip logic Titles Progress bars Audience Pro Preferences For Multiple Choice (Multiple answer) questions When similarly worded questions are asked, highlighting the keywords that make them different helps keep respondents focused. There are a few easy ways to do this: Use sentence case for all answer choices Survey Formatting Guide Formatting Key References

3 Ask yourself WHY The first question you should ask yourself is why you are running a survey to begin with. Asking yourself why you re running a survey will help you come up with one clear goal for your survey, like a decision you need to make or an insight you re trying to get. This goal will guide you through the whole survey writing process and will give you a place to refer back to if you get stuck. Stay relevant Once you have that big picture why of your survey, you ll need to break this main goal up into subtopics. Focus on key data needs, not nice to have information. This keeps the survey creation process simpler and makes data analysis easier. Make sure your goal is very clear and your subtopics are measurable. Stay focused Write out the questions to answer your goal and subtopics directly. Working from your broad goal down to the evidence you need to support it will make sure that the specific questions you ask relate directly back to the reason you re asking in the first place. Example: Question for Subtopic 1: Are you currently subscribed to Netflix? Questions for Subtopic 2: Overall, how much do you enjoy the content available on Netflix? How likely are you renew your Netflix subscription at the end of your billing cycle? Question for Subtopic : What other service(s) do you use to watch streaming video? Tip from the Audience Pro: Don t know how to ask your questions? Use the SurveyMonkey Question Library to find commonly used survey questions that are simply worded and easy to analyze. Tip from the Audience Pro: This will streamline the number of questions you need to ask, and speed up your analysis since you can focus on the things that matter most. Example: Goal: What does Netflix consumer usage look like? Subtopic 1: How many people subscribe to Netflix? Subtopic 2: How satisfied are subscribers with Netflix? Subtopic : What are people using instead of Netflix?

4 Write the right question Speak their language Be careful when throwing around terminology that may be familiar to only you or your industry, or complex ideas that other people may not understand. Remember, you need to make sure the people who are taking your survey understand what you re talking about. If they don t understand your question, they might answer it incorrectly, randomly, or not at all! Try to write your survey in as plain language as possible. If you have to leave in a complex or obscure word, providing a definition or some examples of what you re talking about can be helpful. d Avoid: Do you own a tablet PC? 2 Use: Do you own a tablet PC (e.g. ipad, Android tablet)? Tell them when Limiting the scope of a question to a specific time period ensures that all of your survey takers are thinking about the same scope of time when they re answering your question. If you don t do this, then you have no way of knowing whether someone is thinking about the past day, the past week, or the past year. Moreover, specifying a specific time period will make recall of past behaviors and projections of future behaviors more accurate. For example: Yesterday, did you In the past week, have you How many times in the past 0 days have you In the next 12 months, will you If you re not sure what time period you re most interested in, or you want an overall picture, you can begin questions with words such as In general, Overall, or Typically, to ensure that your respondents are thinking about their average behaviors. A word of caution, however, that this is difficult for respondents to think about and the answers you get back may not be a true average of their feelings. d Avoid: How much do you usually spend on groceries? 2 Use: In the past 7 days, about how much have you spent on groceries? 4

5 Write the right question Stay balanced When writing a question, try not to bias the respondent s answer by leading them in a certain direction. If possible, try to incorporate both sides of an opinion within the question. (For example, Do you like spinach, or not? ) This will make sure that respondents are comfortable expressing their true opinions. When it s not possible to balance within a question, try to balance the survey overall. Ask questions that are framed in both a positive and negative manner so that the tone of the survey is balanced as a whole. If you don t balance within questions or within the survey, respondents are likely to think that you have a particular bias and this will impact how they answer the questions. (Some may try to tell you what they think you want to hear and others may be reactive, trying to tell you the opposite of what they think you want to hear.) d Avoid: How much have you increased your usage of Netflix in the past months? Keep it personal Have you ever heard the expression, There s no such thing as a stupid question? Well, you need to reassure survey respondents that there s no such thing as a stupid answer. Sometimes respondents can feel a little intimidated by the idea of giving their opinions on issues they don t feel qualified to talk about. Make the questions feel more approachable by keeping them personal. Use cues like do you think, do you feel, in your opinion to signal to respondents that you re not looking for an absolute judgment here and that there s no right answer all you want is their perspective on things. d Avoid: Is the price for a yearly Netflix subscription too high, too low, or about right? 2 Use: Do you feel that the price for a yearly Netflix subscription is too high, too low, or about right? 2 Use: Has your usage of Netflix increased, decreased, or stayed the same in the past months? 5

6 Write the right question Be direct The simplest question is usually the best. Don t beat around the bush get straight to the point! Asking direct questions that are written clearly and concisely will ensure that survey respondents understand what you mean. This will help lower your dropout rate and make sure that the data you get back is good data. d Avoid: Why did you decide to cancel your Netflix subscription? 2 Use: What was the main reason you canceled your Netflix subscription? Price was too high Didn t like the content Don t have time to watch Netflix content anymore Other (please specify) d Avoid: What was the state of the cleanliness of the room? 2 Use: How clean was the room? Write the right answers Use closed-ended questions When writing a survey question, think about your main goal of the question. If you want to know the main reason behind respondents actions, or their topmost opinion on something, allow only one answer. Open text questions are helpful to get a thorough list of keywords that people think of about certain topics. However, using single-response, closed-ended questions will make your questions easier to answer and your analysis process quicker and easier. NOTE: We do offer open-ended questions and select all that apply options for all of our questions. However, use these question types with caution, as the data you get can be difficult to analyze and interpret. Therefore, as a general rule of thumb, we recommend against allowing multiple answers. 6

7 Write the right answers Use your words People tend to think in words, not numbers so you should ask them to rate their opinions on scales that have words not numbers. Rating scales (also known as Likert scales) should have 5 points if they are unipolar or 7 points if they are bipolar. You should use a unipolar scale as your default, and a bipolar scale ONLY if a clear midpoint exists and the adjective at the top is a clear and direct opposite of the adjective at the bottom. For example Unipolar: the response can be thought of as having a five-point scale (1 to 5). Extremely Quite Moderately Slightly Not at all Bipolar: the response can be thought of as having a seven point scale (- to +). Use this with ratings that are exact opposites of each other (e.g., short and tall, hot and cold). d Avoid: Please rate 1 to 5 how much you enjoy the content available on Netflix. 1 - Enjoy a lot Enjoy a little 2 Use: How enjoyable is the content available on Netflix? Extremely enjoyable Quite enjoyable Moderately enjoyable Slightly enjoyable Not at all enjoyable Much more Moderately more Slightly more Neither more nor less Slightly less Moderately less Much less 7

8 Write the right answers Avoid yes / no questions Yes/no questions demand that your survey takers think in black and white but in the real world, most opinions are shades of gray. Sure, a yes/no question can be valuable if you need a straightforward answer to filter responses by, or if you want to split your respondents into different groups in order to disqualify certain respondents. However, in most cases, offering survey takers a range of options will make them feel more comfortable selecting an answer choice that accurately reflects their opinion. It will also make your analysis easier and more powerful to have a variety of answers for respondents to choose from. d Avoid: Will you be upgrading to the iphone 5 in the next month? Yes No 2 Use: How likely are you to upgrade to the iphone 5 in the next month? Extremely likely Quite likely Moderately likely Slightly likely Not at all likely Measuring pricing Some of the most frequently asked questions for our Audience customers are questions about the pricing of their product or service. For these questions, follow the same rules we ve talked about above and focus on giving people a range of responses, and think about whether you want more objective information (such as: How much would you be willing to pay for a video subscription? ) or more subjective information such as: Do you think our prices are too high, too low, or about right? ). Take questions that only give a respondent two options and turn them into a range that allows people to give you a more nuanced answer. d Avoid: What do you think of the current price for a Netflix subscription? I like it I don t like it 2 Use: Do you think $100 for a yearly Netflix subscription is high, low, or about right? Extremely high Quite high Somewhat high About right Somewhat low Quite low Extremely low 8

9 Write the right answers Measuring how much Want to know how much respondents spend their time, money, or energy on something? Just ask. There are many different ways to ask how frequent a certain behavior is. You can measure their subjective perceptions ( How often do you go to the gym? ) or their objective actions ( How many times a week do you go to the gym? ). One may be more useful than the other depending on your research goal so think about which one you choose. If you choose to measure subjective perceptions, use a rating scale like we mentioned above; if you choose to measure objective actions, make sure you limit your response options to no more than 6. Also, providing ranges for each answer option will help people with fuzzy memories find a response that feels right for them. Example (subjective): About how much of your marketing budget do you spend on online advertising? All of my marketing budget Most of my marketing budget About half of my marketing budget Some of my marketing budget None of my marketing budget Example (objective): About how much of your marketing budget do you spend on online advertising? 0-20% 21-40% 41-60% 61-80% % 9

10 Write the right answers Provide a full set of answers When providing answer options, make sure your list is exhaustive and has all possible answer choices that a respondent could choose. But, in the rare case that you think you might not have allanswer choices down or cannot remember all applicable options, providing an alternative for respondents who cannot find a suitable answer choice is a good idea. An answer choice that can be helpful in accomplishing inclusivity is: Other (please specify) d Avoid: In the past 0 days, what site have you used most to watch streaming TV shows? Netflix Amazon Prime Hulu Plus 2 Use: In the past 0 days, what site have you used most to watch streaming TV shows? Netflix Amazon Prime Hulu Plus Other (please specify) A lot of our customers add a none of the above option. We recommend against including options such as none of the above, not applicable, and don t know as they provide an easy way out for respondents not to answer the question regardless of whether they have an answer. Instead, we recommend using skip logic! Skip logic keeps respondents honest and engaged in your questions, and ensures that your data quality stays high. d Avoid: 2 Use 2 questions: In the past 0 days, what site have you used most to watch streaming TV shows? Netflix Amazon Prime Hulu Plus None of the above In the past 0 days, have you watched streaming TV shows? Yes [if yes, display next question] No [if no, skip next question] In the past 0 days, what site have you used most to watch streaming TV shows? Netflix Amazon Prime Hulu Plus Other (please specify) 10

11 Write the right answers Watch your ranges When using ranges in a multiple-choice (single answer) question, there are three main things to remember. First, make sure that they don t overlap. Overlapping ranges will confuse respondents, decreasing data quality. Next, keep the range intervals the same size. This will make your data analysis go more smoothly as you can use the midpoint of the range to look at trends in your data. In general, keeping intervals constant makes any statistical analyses you run that much stronger. Finally, make sure your list of response options spans the whole possible range of responses. Again, as we talked about before, 5-7 response choices are ideal depending on what is helpful for you. d Avoid: In a typical day, how much time do you spend watching content on Netflix? 1-2 hours 2-5 hours 5-10 hours 2 Use: In a typical day, how much time do you spend watching content on Netflix? Less than 2 hours 2-4 hours 5-7 hours 8-10 hours More than 10 hours Matrix / Grid questions Organizing similar questions into a matrix is visually appealing and easy. However, displaying questions in this manner actually hurts your data quality in two ways. First, having questions in a matrix allows respondents to get lazy and sloppy, clicking quickly down a column of responses without actually engaging with the questions you re asking. The repetitive nature of a matrix question in a survey gets respondents bored fast. This also increases dropout rate, which is bad news for you. Additionally, not only do matrix questions hurt your data quality because your respondents aren t engaged, it also hurts your data quality because you, as the survey creator, are not as engaged. Matrix questions allow you not to think critically enough about how you re asking the questions. Most questions shouldn t have identical response options. What do you do instead? We recommend breaking down your matrix question into a separate question for each row with customized responses. We charge you by the row for a matrix question anyway, so it won t cost you a penny more, and it will keep your data quality and your response rate high! 11

12 Specific Settings Guidelines Require all questions Survey respondents might skip a question by mistake, or click the next page button too early, so requiring answers to questions can help avoid missing key data. Randomize answer choices when possible or relevant Remove order bias by randomizing answer choices in multiple choice questions. Flip the order of scales. Whenever possible randomize questions on a page and even pages in the survey. Skip logic Skip logic is a great way to customize your respondents survey experience and make sure that they only see questions that are relevant to them. Make it easy on yourself label all pages involved in skip logic paths to be sure you re skipping your respondents to the right places. Titles If including survey titles and page titles would bias responses they should be hidden, however, if possible, try to include them. Titles help to organize the narrative and flow of your survey, and provide a quick sense of order and orientation for your respondents. Progress bars We recommend against including progress bars as they are distracting for respondents. You want to keep respondents focused on the questions you re asking, not how fast (or slow) they re going. Audience Pro Preferences The following tips are SurveyMonkey Audience Survey Expert preferences for consistency purposes. They are internal guidelines and not necessarily needed to ensure methodological soundness, but can lead to better respondent experience. For Multiple Choice (Multiple answer) questions At the end of the question, always add: (Please select all that apply.) When similarly worded questions are asked, highlighting the keywords that make them different helps keep respondents focused. There are a few easy ways to do this: Underlining sticks out So does bold ALL CAPS work too Use sentence case for all answer choices Capitalize only first letter of each option This does not apply in the case of proper nouns 12

13 Specific Settings Guidelines Survey Formatting Guide Tip from the Audience Pro: If you are programming your survey offline, use this formatting guide to make it easy for you or anyone else to understand and upload your survey into SurveyMonkey. This is not necessarily to ensure Methodological soundness, but will be helpful when you program your survey into your SurveyMoneky account. Formatting Key Items in [BRACKETS] are instructions for inputting the survey into SurveyMonkey, examples include: [randomize] = randomize responses [skip to PAGE TITLE] = skip that answer choice to PAGE TITLE [skip to DISQUALIFY] = skip that answer choice to DISQUALIFY o if [skip to DISQUALIFY] is used on a PAGE TITLE, use page logic to skip all to DISQUALIFY [open text] = use an open text or paragraph question type Items on their own line, in ALL CAPS, indicate that a new page should be added, and the page title of the new page SPLIT PAGE indicates to add a new page or use the split page button Numbered items (e.g., 1. Lorem ipsum...) indicate questions Specific formatting guidelines When a question ends with (Select all that apply.), this question should be a multiple choice, multiple answer question type When [randomize] is used, if the last answer choice, or second to last answer choice when an Other (please specify) option is used, should not be randomized (this is an option in the relevant question types) If words in a question or answer choice use underline or bold formatting, use HTML in the question for the designated words or phrases o for question language, use underline, since all questions are bolded: <u>word or phrase</u> o for answer language, use bold: <b>word or phrase</b> All questions should be required unless otherwise noted Hide asterisks for required questions Hide all page titles Hide all question numbering References How to add Other as an answer choice How to require questions How to hide page titles and question numbers How to enable HTML in your account When needed, the add Other as an answer choice option should be checked while inputting the survey into SurveyMonkey 1