ESOMAR 28 QUESTIONS TO HELP BUYERS OF ONLINE SAMPLES

Transcription

1 ESOMAR 28 QUESTIONS TO HELP BUYERS OF ONLINE SAMPLES SEPTEMBER ESOMAR. All rights reserved. No part of this publication may be reproduced or copied in any form or by any means, or translated, without the prior permission in writing of ESOMAR. ESOMAR codes, guidelines and similar documents are drafted in English and the English texts are the definitive versions.

2 INTRODUCTION The primary aim of these 28 Questions is to increase transparency and raise awareness of the key issues for researchers to consider when deciding whether an online sampling approach is fit for their purpose. Put another way, the aim is to help researchers to ensure that what they receive meets their expectations. The questions are also designed to introduce consistent terminology for providers to state how they maintain quality, to enable buyers to compare the services of different sample suppliers. Notes on the context of the questions explain why the questions should be asked and which issues researchers should expect to be covered in the answer. These new questions replace ESOMAR s 26 Questions to help Research Buyers of Online Samples. ESOMAR has updated the text to recognize the ongoing development of techniques. While some of the questions remain constant, new questions have been added to incorporate new techniques and new technology in this area. In particular, this revision recognises the broad trend within the industry to build online samples from multiple sources rather than relying on a single panel. It should be noted that these 28 Questions focus on the questions that need to be asked by those buying online samples. If the sample provider is also hosting the data collection you will need to ask additional questions to ensure that your project is carried out in a way that satisfies your quality requirements. The 28 Questions complement ESOMAR s Guideline to Online Research which was revised in 2011 to add updated legal and ethical guidance and new sections on privacy notices, cookies, downloadable technology and interactive mobile. 2

3 COMPANY PROFILE 1. What experience does your company have in providing online samples for market research? Branded Research is a provider of online sample. Its executive team and sales group have over 25 years of online market research experience, sampling, and building and managing online panel. The company handles online panel exclusively and relies on pioneering social practices and worldwide strategic partnerships to deliver superior panel quality. SAMPLE SOURCES AND RECRUITMENT 2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples? Our sample is provided from our proprietary online research community MintVine.com, and through our global partner network. MintVine is our primary source of actively managed online panel-a membership community of people who are interested in participating in surveys, product and service offers, polls, general entertainment, and sharing these activities with one another and their outside communities. We do leverage social networks, most notably Facebook, for our online sample. All community members/panellists are acquired through a double-opted in, multi-stage verification process. Panellists acquired through our partner network are vetted and verified under the same processes. 3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources? BR Inc. leverages proprietary technology, such as Spot Check, to ensure the validity of blended responses from MintVine and any partner panels. Spot Check addresses any overlap when using multiple sample sources, helps to identify and eliminate potential fraudulent activity, and acts as a deduping mechanism, using a scored digital fingerprint system. BR Inc. also takes into consideration source bias and works with the client to ensure balanced sample are deployed for each wave. 4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for? No, not exclusively. BR Inc. panels are built around market research as the main purpose, and ultimately that is what they become. However, we employ different marketing techniques in customer acquisition in an effort to not only defray the amount of acquisition costs (these are significantly larger expenses for us now as we are a new company) but also as the core strategic marketing practices needed to sign up quality panel online. None of our marketing efforts conflict with our core business of market research. 3

4 5. How do you source groups that may be hard to reach on the internet? The tactics we employ to build our proprietary panels are designed solely for this purpose, and our abilities as a first-rate panel company very much depend on this. For BR Inc, this process begins with our customer acquisition efforts and is a continual process of refinement, data organization, and metric analysis. The end result is a very representative panel and improvement of panel in this regard and broadening of our panel to meet higher expectations of the hard-to-reach for our clients is a continual process. 6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third party provider? Partner selection is based on capabilities, feasibility, client specifications and budget. Our goal is to provide our clients the best sample seamlessly from our vetted list of approved partners. Our project team works closely with the client to make sure that the selected partners meet all requirements. In the event that new additional partners are needed, the client is notified and is included the decision making process and there is a transparency from start to project finish. SAMPLING AND PROJECT MANAGEMENT 7. What steps do you take to achieve a representative sample of the target population? Context: BR Inc. considers attributes of the target population, considers the audience, and makes recommendations for the best approach. We may use quotas or suggest specific weighting. These options are discussed at length with the analyst to make sure we are meeting all needs for the particular project. Further, each sample send is randomized prior to send based on the available targeted sample to ensure a true representative data set. 8. Do you employ a survey router? BR Inc. does have survey routing capabilities and we recommend that approach when appropriate. For example: A market segmentation would not be a good candidate for our router, but a niche target such as users of a specific product might be more efficiently fielded with the use of the router. Some of the key factors are based on how the data will be used and what type of analysis the client plans to run. 9. If you use a router: Please describe the allocation process within your router. How do you decide which surveys might be considered for a respondent? On what priority basis are respondents allocated to surveys? Our system allows us to define and continually change priorities by sample source, audience and study. We also limit the number of survey opportunities available during any session. There is always a balance struck between providing the best user experience for our community members while maintaining the highest level of data and sample integrity for our clients. 4

5 10. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias? BR Inc. s employs tools that allow us the ability to define participation/invitation rules. When a respondent completes a study, they can still be presented with an invitation to participate in another one. The entire management of this process is entirely customizable and updated based myriad factors, including number and type of surveys leveraging the router. 11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers? The COO sets larger and overreaching rules around participation. Individual project managers are given the ability to setup and manage their own projects and establish the project specific parameters sources; i.e.- include/exclude, quotas, and balancing. Historical parameter data is stored and accessible for future studies. 12. What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low incidence projects dealt with? Demographic profiles as described above are collected and stored in our database. In addition we also collect profile information on Finance, Employment, Autos, Technology, Parenting, Phone usage, Travel and Leisure, Military, Health, Gambling, Food, Sports, Beauty, Beverage, Pets, Video Games and Culture. This list will and does increase in scope at times to meet the needs of specific market research and includes hundreds of data points. All profile and demographic information on our members is available to them at all times and they are able to update and/or edit at will. We encourage and incentivize all members to complete as much profile/ demographic information as they can and to keep all such information as current and accurate as possible. Our entire profiling and querying system works intelligently with our continual panel management efforts and offers certain complexities that allow us to deal with lr projects carefully. 13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of s. Our community members/panellists are invited to take part in surveys in several ways: Through html correspondence to their addresses on file. Notifications are sent to their MintVine inbox. Notifications are sent via Facebook to panelists who are registered through Facebook Open Graph. Twitter notifications Voluntary logging-in to MintVine.com 5

6 A typical invitation includes: Html graphic A unique URL that provides access to the survey. Survey information including topic and est. time to complete. Opt-out information Member support address Link to privacy policy Our community members are told, at a minimum, of the subject matter of the survey-typically deducted from the Survey title, the point reward (if there is one) associated with completing that survey, and estimated time of completion, (LOI) 14. Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics? Our panellists are rewarded points (equal to cash) for multiple actions and achievements, including completing surveys and offers, polls, signing in, sharing, achieving certain point thresholds, and participation events. Each survey is given a point/cash value based on the specifics of that survey and the overall bounty associated with that survey. Lengthier, more specific surveys typically offer larger point rewards. Panellists are able to cash out via PayPal once their account achieves certain minimum balances. 15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources? Some of the basic elements we need are: Target audience Number of completes needed Estimated Incidence Rate (if available) Survey length Type of study Time in field Quotas 16. Do you measure respondent satisfaction? Is this information made available to clients? Respondent satisfaction is a top priority and focus of ongoing efforts and communication with our community members. We retain an open dialogue and adhere to general policy of transparency with them. We also encourage open communication amongst our community members themselves. This information is not readily accessible to our clients but such needs are dealt with on a case-by-case basis. 17. What information do you provide to debrief your client after the project has finished? We provide any and all information we can upon Client request, including response rates, open rates, term rates, and demographic information. 6

7 DATA QUALITY AND VALIDATION 18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviours, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. Don t Know ) or (d) speeding (too rapid survey completion)? Please describe these procedures. BR Inc. and MintVine apply strict security protocols to ensure the integrity of our data quality. We rely on Spot Check to eliminate most fraudulent behavior and to prevent duplicate respondents. We employ measure to identify speeders, identify illogical or inconsistent responses, as well as all straight line responses. In the event we are responsible for programming and hosting, we employ are own proactive measures, including managing certain logic flow, utilization of open-ended responses, and consistent quality analysis. 19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources? Strict polices have been implemented to monitor and track many variables associated with our panel members, including frequency of surveys issued and participation history. We generate various metrics resulting from this data. We also limit participation by survey topic or category. These are metrics that are constantly monitored and change slightly as dictated by our growth and panel makeup. Generally speaking, we generally limit invitations to 2x per week and 8x per month and actual responses are limited to 4-5 in a 30-day period. We, like other community/panel managers, have to measure balanced, quality data with the overall user experience. 20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods? Participation rules are discussed above. With regard to management, BR Inc s system is flexible and has the capabilities to filter by category, survey, time and any profile or system data point. The filters include invitations, responses or completion. The past participation rules may be customized per project depending on client requirements. 21. Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual level data? Yes, all panellist data is individual and stored accordingly in our database. We provide to clients per their individual requests. 22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any? BR Inc. uses its proprietary technology, Spot Check, which addresses any overlap when using multiple sample sources, helps to identify and eliminate potential fraudulent activity, and acts as a deduping mechanism, using a scored digital fingerprint system, employing multiple measure across IP addresses and browser specific data. We employ the same procedures with B2B samples. 7

8 POLICIES AND COMPLIANCE 23. Please describe the opt-in for market research processes for all your online sample sources. Our initial registration takes two forms-basic registration and Facebook registration. Basic information includes Name, Country, Language, Postal Address, D.O.B., Gender, address, and Username. Facebook registration includes ID, Name, Picture, Gender, Locale, Age Range, Birthday, Education, Relationship, Religion & Politics, Work History, and . In both cases, users are sent confirmation s where they double opt-in to our site 24. Please provide a link to your Privacy Policy. How is your Privacy Policy provided to your respondents? Our Privacy Policy can be found in multiple places around the MintVine site and we encourage all of our community members to read, particularly in our left hand margin MV Guidebook Please describe the measures you take to ensure data protection and data security. BR Inc. and MintVine apply strict security protocols to ensure the integrity of its panels and provide bestin-breed security to our panellists data. More specific information that outlines all of our quality control measures can be delivered upon request. From an enterprise level, we adhere to strict security measures as it relates to our servers and databases and our entire platform. 26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents? All of our clients are advised that, even with the best security measures, information presented online is not always entirely secure. As long as the client is willing to accept the potential risk, we employ a number of measures to reduce the risk. We also include terms in our member Terms & Conditions to make sure members understand that information shared through surveys and our systems are proprietary and protected. 27. Are you certified to any specific quality system? If so, which one(s)? Our security and quality measures are proprietary, including Spot Check. 28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example COPPA in the United States, do you comply with? We strictly adhere to each of ESOMAR, CASRO, and COPPA laws and employ a strict practice of obtaining parental consent prior to engaging any minors in market research activities. 8

9 GUIDANCE ON PROFESSIONAL STANDARDS Maintaining consumer trust is integral to effective market, social and opinion research. ESOMAR through its codes and guidelines promotes the highest ethical and professional standards for researchers around the world. The ICC/ESOMAR Code on Market and Social Research, which was developed jointly with the International Chamber of Commerce, sets out global fundamentals for self-regulation for researchers. It has been undersigned by all ESOMAR members and adopted or endorsed by more than 60 national market research associations worldwide. The ESOMAR Guideline on Guideline on Online Research is of particular relevance to researchers using online panels and should be read in conjunction with these questions for more explanation of the legal and professional responsibilities of researchers who are collecting and analysing online research data. In addition, ESOMAR has issued the following guidelines to provide more detailed advice on how to address the legal, ethical, and practical considerations of conducting specific areas of research: Guideline on Social media research Guideline for Conducting mobile research Guideline on Online research Guideline on Distinguishing market research from other data collection activities Guideline on Passive data collection, observation and recording Guideline on Interviewing children and young people Guideline on Customer satisfaction studies Guideline on Mystery shopping Guideline on How to commission research ESOMAR/WAPOR Guide to opinion polls There are also two ISO standards which relate to market research: ISO Standard Access panels in market, opinion and social research - Vocabulary and service requirements ISO Market, opinion and social research - Vocabulary and service requirements These documents contain useful definitions and explain process quality standards for access panels and research projects. They explain good practice and the information that should be provided to clients. They can be purchased from the International Standards Organisation ESOMAR World Association for Social, Opinion and Market Research Eurocenter 2,11th Floor Barbara Strozzilaan HN Amsterdam The Netherlands Tel: For more information contact professional.standards@esomar.org 9

10 San Diego, CA London, UK Panchkula, India (888) (619)